View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0002962 | OpenFOAM | Bug | public | 2018-05-29 10:26 | 2018-06-28 09:06 |
Reporter | shildenbrand | Assigned To | will | ||
Priority | normal | Severity | minor | Reproducibility | always |
Status | resolved | Resolution | fixed | ||
Platform | HP Z640 | OS | Debian | OS Version | 9.4 |
Product Version | dev | ||||
Fixed in Version | dev | ||||
Summary | 0002962: Refresh button in Paraview (using paraFoam) does not work | ||||
Description | When postprocessing with Paraview (using paraFoam with OpenFOAM reader) pressing the refresh button does not load newly created timesteps. This functionality existed in a prior version of the OpenFOAM reader and is very useful when postprocessing a big transient case. Without this funtionality paraview has to be closed an re-opened which can be very time-consuming. | ||||
Steps To Reproduce | Run tutorial case, e.g., ./tutorials/incompressible/simpleFoam/windAroundBuildings, start paraFoam after time=50 has been written and press reload after time=100 has been written. The new timestep is not read in. | ||||
Tags | No tags attached. | ||||
|
I use the "Refresh Times" button several time a day and just used it now without any problems. |
|
I just ran the example with windAroundBuildings using: - openfoam-dev pack, version 20180527 - paraviewopenfoam54, version 0-3 It worked fine, I cannot reproduce the issue. |
|
I can reproduce the issue if I start paraFoam BEFORE time=50 has been written and press reload after time=50 has been written. |
|
I cannot reproduce the problem even if I start paraFoam BEFORE time=50 has been written and press reload after time=50 has been written; time=50 loads fine. |
|
I use locally compiled version of both. I'll check the compilations again to see if there are any errors. |
|
Go to "View > Animation View", and set "Mode" in the new panel to "Snap To TimeSteps". Does that fix it? |
|
My compilation of the PVReader shows many warnings (see file attached) but I am not sure if this is connected to my problem. If I provide "stdbool.h" manually the problem of not being able to refresh persists. In "View > Animation View", "Mode" was already set to "Snap To TimeSteps", so this doesn't fix it. |
|
Additional information: If I leave Paraview open, close the loaded case, and re-open it (when there are more timesteps available), I still only see the number of timesteps available at the first start of Paraview. When using the builtin-reader (.foam extension), refresh works fine |
|
The warnings generated when compiling the reader module are a result of paraview's code. They are irritating, but they do not affect functionality. I have built on debian 9.4 in a virtual machine, and I cannot reproduce your issue. |
|
Thank you for testing this in my environment. I made some more tests this morning. Could you please try to reproduce the issue when you set fileHandler collated; in ./OpenFOAM-dev/etc/controlDict and do a wmRefresh? We have set collated as standard, I tried this morning the uncollated filehandler and now the refresh button works again. |
|
OK, the fact that you are using collated file handling was the crucial bit of information. I can reproduce this now. The file handler is keeping a cache of the times and this cache is not being rebuilt when you hit refresh. If the first half of the if statement in masterUncollatedFileOperation::findTimes is removed then the problem goes away. That's not a solution, though. @MattijsJ could you please advise how this should be fixed? |
|
This is not easily fixed. The masterUncollated and collated file handlers cache times to avoid excessive directory reading. They internally keep track of any additional time directories being added. The paraFoam is external to this so does not know that time directories have been added. For now as a workaround run paraFoam with uncollated. In bash: FOAM_FILEHANDLER=uncollated paraFoam |
|
The workaround works for me. Thank you. |
|
We can't expect users to know of this limitation and modify their execution of paraFoam accordingly. I would have thought that we need to either (a) allow paraFoam/vtkPVFoam to disable the caching and live with the additional read-times, or (b) add a method so that paraFoam/vtkPVFoam can clear the cache when the user hits refresh. @Mattijs, what is your preference, or do you have an alternative? |
|
A similar (but different) issue lives inside e.g. decomposePar (and that thermo field writing business) where we need to switch off the threading. In fact we don't need to switch off threading - we just need to be able to wait for everything to be written. Similar to your suggestion I would add a 'clearOut' or 'flush' method which would clear the cache and guarantee any files to be written. This would cover both the thread business and time caching issues. |
|
flush.patch (15,350 bytes)
diff --git a/applications/solvers/multiphase/compressibleInterFoam/twoPhaseMixtureThermo/twoPhaseMixtureThermo.C b/applications/solvers/multiphase/compressibleInterFoam/twoPhaseMixtureThermo/twoPhaseMixtureThermo.C index d2a3ca2c2..df79d50c0 100644 --- a/applications/solvers/multiphase/compressibleInterFoam/twoPhaseMixtureThermo/twoPhaseMixtureThermo.C +++ b/applications/solvers/multiphase/compressibleInterFoam/twoPhaseMixtureThermo/twoPhaseMixtureThermo.C @@ -50,12 +50,6 @@ Foam::twoPhaseMixtureThermo::twoPhaseMixtureThermo thermo1_(nullptr), thermo2_(nullptr) { - // Note: we're writing files to be read in immediately afterwards. - // Avoid any thread-writing problems. - float bufSz = - fileOperations::collatedFileOperation::maxThreadFileBufferSize; - fileOperations::collatedFileOperation::maxThreadFileBufferSize = 0; - { volScalarField T1 ( @@ -86,9 +80,9 @@ Foam::twoPhaseMixtureThermo::twoPhaseMixtureThermo T2.write(); } - fileOperations::collatedFileOperation::maxThreadFileBufferSize = - bufSz; - + // Note: we're writing files to be read in immediately afterwards. + // Avoid any thread-writing problems. + fileHandler().flush(); thermo1_ = rhoThermo::New(U.mesh(), phase1Name()); thermo2_ = rhoThermo::New(U.mesh(), phase2Name()); diff --git a/applications/utilities/mesh/manipulation/setSet/setSet.C b/applications/utilities/mesh/manipulation/setSet/setSet.C index f425a201f..70d8d3bc8 100644 --- a/applications/utilities/mesh/manipulation/setSet/setSet.C +++ b/applications/utilities/mesh/manipulation/setSet/setSet.C @@ -48,7 +48,6 @@ Description #include "faceZoneSet.H" #include "pointZoneSet.H" #include "timeSelector.H" -#include "collatedFileOperation.H" #include <stdio.h> @@ -349,6 +348,8 @@ void removeZone zones.setSize(zones.size()-1); zones.clearAddressing(); zones.write(); + // Force flushing so we know it has finished writing + fileHandler().flush(); } } @@ -603,6 +604,8 @@ bool doCommand currentSet.instance() = mesh.time().timeName(); } currentSet.write(); + // Make sure writing is finished + fileHandler().flush(); } } } @@ -809,8 +812,6 @@ int main(int argc, char *argv[]) // Specific to topoSet/setSet: quite often we want to block upon writing // a set so we can immediately re-read it. So avoid use of threading // for set writing. - fileOperations::collatedFileOperation::maxThreadFileBufferSize = 0; - timeSelector::addOptions(true, false); #include "addRegionOption.H" argList::addBoolOption("noVTK", "do not write VTK files"); diff --git a/applications/utilities/mesh/manipulation/topoSet/topoSet.C b/applications/utilities/mesh/manipulation/topoSet/topoSet.C index e4aa0eb77..10e1718fb 100644 --- a/applications/utilities/mesh/manipulation/topoSet/topoSet.C +++ b/applications/utilities/mesh/manipulation/topoSet/topoSet.C @@ -40,7 +40,6 @@ Description #include "faceZoneSet.H" #include "pointZoneSet.H" #include "IOdictionary.H" -#include "collatedFileOperation.H" using namespace Foam; @@ -85,6 +84,7 @@ void removeZone zones.setSize(zones.size()-1); zones.clearAddressing(); zones.write(); + fileHandler().flush(); } } @@ -193,11 +193,6 @@ polyMesh::readUpdateState meshReadUpdate(polyMesh& mesh) int main(int argc, char *argv[]) { - // Specific to topoSet/setSet: quite often we want to block upon writing - // a set so we can immediately re-read it. So avoid use of threading - // for set writing. - fileOperations::collatedFileOperation::maxThreadFileBufferSize = 0; - timeSelector::addOptions(true, false); #include "addDictOption.H" #include "addRegionOption.H" @@ -302,6 +297,7 @@ int main(int argc, char *argv[]) // Synchronize for coupled patches. if (!noSync) currentSet().sync(mesh); currentSet().write(); + fileHandler().flush(); } break; @@ -336,6 +332,7 @@ int main(int argc, char *argv[]) // Synchronize for coupled patches. if (!noSync) currentSet().sync(mesh); currentSet().write(); + fileHandler().flush(); } break; @@ -343,12 +340,14 @@ int main(int argc, char *argv[]) Info<< " Clearing " << currentSet().type() << endl; currentSet().clear(); currentSet().write(); + fileHandler().flush(); break; case topoSetSource::INVERT: Info<< " Inverting " << currentSet().type() << endl; currentSet().invert(currentSet().maxSize(mesh)); currentSet().write(); + fileHandler().flush(); break; case topoSetSource::REMOVE: diff --git a/applications/utilities/parallelProcessing/decomposePar/decomposePar.C b/applications/utilities/parallelProcessing/decomposePar/decomposePar.C index 92e2950f5..642e8ccb8 100644 --- a/applications/utilities/parallelProcessing/decomposePar/decomposePar.C +++ b/applications/utilities/parallelProcessing/decomposePar/decomposePar.C @@ -104,7 +104,6 @@ Usage #include "pointFieldDecomposer.H" #include "lagrangianFieldDecomposer.H" #include "decompositionModel.H" -#include "collatedFileOperation.H" // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // @@ -459,12 +458,6 @@ int main(int argc, char *argv[]) // Decompose the mesh if (!decomposeFieldsOnly) { - // Disable buffering when writing mesh since we need to read - // it later on when decomposing the fields - float bufSz = - fileOperations::collatedFileOperation::maxThreadFileBufferSize; - fileOperations::collatedFileOperation::maxThreadFileBufferSize = 0; - mesh.decomposeMesh(dictIO.objectPath()); mesh.writeDecomposition(decomposeSets); @@ -521,8 +514,7 @@ int main(int argc, char *argv[]) << endl; } - fileOperations::collatedFileOperation::maxThreadFileBufferSize = - bufSz; + fileHandler().flush(); } diff --git a/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoam.C b/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoam.C index 6a24a189c..684fab66d 100644 --- a/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoam.C +++ b/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoam.C @@ -97,7 +97,8 @@ int Foam::vtkPVFoam::setTime(int nRequest, const double requestTimes[]) { Time& runTime = dbPtr_(); - // Get times list + // Get times list. Flush first to force refresh. + fileHandler().flush(); instantList Times = runTime.times(); int nearestIndex = timeIndex_; @@ -248,11 +249,6 @@ Foam::vtkPVFoam::vtkPVFoam fileName FileName(vtkFileName); - // Make sure not to use the threaded version - it does not like - // being loaded as a shared library - static cleanup order is problematic. - // For now just disable the threaded writer. - fileOperations::collatedFileOperation::maxThreadFileBufferSize = 0; - // avoid argList and get rootPath/caseName directly from the file fileName fullCasePath(FileName.path()); @@ -575,6 +571,8 @@ double* Foam::vtkPVFoam::findTimes(int& nTimeSteps) if (dbPtr_.valid()) { Time& runTime = dbPtr_(); + // Get times list. Flush first to force refresh. + fileHandler().flush(); instantList timeLst = runTime.times(); // find the first time for which this mesh appears to exist diff --git a/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoamUpdateInfo.C b/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoamUpdateInfo.C index 2f1c0add6..dba80d9a0 100644 --- a/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoamUpdateInfo.C +++ b/applications/utilities/postProcessing/graphics/PVReaders/vtkPVFoam/vtkPVFoamUpdateInfo.C @@ -192,6 +192,9 @@ void Foam::vtkPVFoam::updateInfoLagrangian // Generate a list of lagrangian clouds across all times HashSet<fileName> cloudDirs; + + // Get times list. Flush first to force refresh. + fileHandler().flush(); instantList times = dbPtr_().times(); forAll(times, timei) { @@ -706,6 +709,8 @@ void Foam::vtkPVFoam::updateInfoLagrangianFields() // set. ParaView will display "(partial)" after field names that only apply // to some of the clouds. const arrayRange& range = arrayRangeLagrangian_; + + fileHandler().flush(); for (label partId = range.start(); partId < range.end(); ++ partId) { const instantList times = dbPtr_().times(); diff --git a/applications/utilities/postProcessing/graphics/PVReaders/vtkPVblockMesh/vtkPVblockMesh.C b/applications/utilities/postProcessing/graphics/PVReaders/vtkPVblockMesh/vtkPVblockMesh.C index d79398601..69b76444b 100644 --- a/applications/utilities/postProcessing/graphics/PVReaders/vtkPVblockMesh/vtkPVblockMesh.C +++ b/applications/utilities/postProcessing/graphics/PVReaders/vtkPVblockMesh/vtkPVblockMesh.C @@ -31,7 +31,6 @@ License #include "Time.H" #include "patchZones.H" #include "OStringStream.H" -#include "collatedFileOperation.H" // VTK includes #include "vtkDataArraySelection.h" @@ -171,12 +170,6 @@ Foam::vtkPVblockMesh::vtkPVblockMesh << FileName << endl; } - // Make sure not to use the threaded version - it does not like - // being loaded as a shared library - static cleanup order is problematic. - // For now just disable the threaded writer. - fileOperations::collatedFileOperation::maxThreadFileBufferSize = 0; - - // avoid argList and get rootPath/caseName directly from the file fileName fullCasePath(fileName(FileName).path()); diff --git a/etc/controlDict b/etc/controlDict index d927ea413..4179091ae 100644 --- a/etc/controlDict +++ b/etc/controlDict @@ -58,7 +58,7 @@ OptimisationSwitches //- Parallel IO file handler // uncollated (default), collated or masterUncollated - fileHandler uncollated; + fileHandler collated; //- collated: thread buffer size for queued file writes. // If set to 0 or not sufficient for the file size threading is not used. diff --git a/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.C b/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.C index 614e846af..5abaf3c70 100644 --- a/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.C +++ b/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.C @@ -608,6 +608,18 @@ bool Foam::fileOperations::collatedFileOperation::writeObject } } +void Foam::fileOperations::collatedFileOperation::flush() const +{ + if (debug) + { + Pout<< "collatedFileOperation::flush : clearing and waiting for thread" + << endl; + } + masterUncollatedFileOperation::flush(); + // Wait for thread to finish (note: also removes thread) + writer_.waitAll(); +} + Foam::word Foam::fileOperations::collatedFileOperation::processorsDir ( diff --git a/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.H b/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.H index f1acda477..4e65454ce 100644 --- a/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.H +++ b/src/OpenFOAM/global/fileOperations/collatedFileOperation/collatedFileOperation.H @@ -155,6 +155,9 @@ public: // Other + //- Forcibly wait until all output done. Flush any cached data + virtual void flush() const; + //- Actual name of processors dir virtual word processorsDir(const IOobject&) const; diff --git a/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.C b/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.C index 8c8640ff0..65b39843d 100644 --- a/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.C +++ b/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.C @@ -982,6 +982,17 @@ Foam::label Foam::fileOperation::nProcs } +void Foam::fileOperation::flush() const +{ + if (debug) + { + Pout<< "fileOperation::flush : clearing processor directories cache" + << endl; + } + procsDirs_.clear(); +} + + Foam::fileName Foam::fileOperation::processorsCasePath ( const IOobject& io, diff --git a/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.H b/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.H index 2ea2832a7..84c928b57 100644 --- a/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.H +++ b/src/OpenFOAM/global/fileOperations/fileOperation/fileOperation.H @@ -485,6 +485,9 @@ public: virtual void setTime(const Time&) const {} + //- Forcibly wait until all output done. Flush any cached data + virtual void flush() const; + //- Generate path (like io.path) from root+casename with any // 'processorXXX' replaced by procDir (usually 'processsors') fileName processorsCasePath diff --git a/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C b/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C index 25fc3c811..0f0675e30 100644 --- a/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C +++ b/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C @@ -2532,6 +2532,13 @@ Foam::fileOperations::masterUncollatedFileOperation::NewOFstream } +void Foam::fileOperations::masterUncollatedFileOperation::flush() const +{ + fileOperation::flush(); + times_.clear(); +} + + Foam::label Foam::fileOperations::masterUncollatedFileOperation::addWatch ( const fileName& fName diff --git a/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H b/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H index 15ee260a0..93862fc2a 100644 --- a/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H +++ b/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H @@ -752,6 +752,9 @@ public: //- Callback for time change virtual void setTime(const Time&) const; + //- Forcibly wait until all output done. Flush any cached data + virtual void flush() const; + //- Return cached times const HashPtrTable<instantList>& times() const { |
|
Attached a patch for dev which adds a 'flush' method to all fileOperations. It calls this from the PV* readers and replaces all instances of maxThreadXXX manipulation. The only testing I've done so far is the collated+RefreshTimes button so it needs some more testing. |
|
Thanks for the patch @MattijsJ. The file handler tests run fine with this, the whole test loop runs collated, and I've run a couple of additional cases successfully, too, so I'm happy it's not causing additional issues. Pushed as commit 025e4837. |
Date Modified | Username | Field | Change |
---|---|---|---|
2018-05-29 10:26 | shildenbrand | New Issue | |
2018-05-29 11:17 | henry | Note Added: 0009665 | |
2018-05-29 11:21 | chris | Note Added: 0009666 | |
2018-05-29 11:44 | chris | Note Added: 0009667 | |
2018-05-29 11:48 | henry | Note Added: 0009668 | |
2018-05-29 12:02 | shildenbrand | Note Added: 0009669 | |
2018-05-29 18:19 | will | Note Added: 0009670 | |
2018-05-30 07:28 | shildenbrand | File Added: compile_PVReader | |
2018-05-30 07:28 | shildenbrand | Note Added: 0009671 | |
2018-05-30 07:33 | shildenbrand | Note Added: 0009672 | |
2018-06-14 16:23 | will | Note Added: 0009766 | |
2018-06-15 06:44 | shildenbrand | Note Added: 0009771 | |
2018-06-15 10:19 | will | Note Added: 0009775 | |
2018-06-19 11:52 | MattijsJ | Note Added: 0009795 | |
2018-06-19 12:15 | shildenbrand | Note Added: 0009796 | |
2018-06-20 12:46 | will | Note Added: 0009805 | |
2018-06-22 10:43 | MattijsJ | Note Added: 0009810 | |
2018-06-26 20:49 | MattijsJ | File Added: flush.patch | |
2018-06-26 20:49 | MattijsJ | Note Added: 0009832 | |
2018-06-28 09:06 | will | Assigned To | => will |
2018-06-28 09:06 | will | Status | new => resolved |
2018-06-28 09:06 | will | Resolution | open => fixed |
2018-06-28 09:06 | will | Fixed in Version | => dev |
2018-06-28 09:06 | will | Note Added: 0009840 |