View Issue Details

IDProjectCategoryView StatusLast Update
0003295OpenFOAMBugpublic2019-06-27 16:06
Reportergerry_kan Assigned Tohenry  
PrioritynormalSeverityblockReproducibilityalways
Status closedResolutionno change required 
PlatformLinuxOSCentOSOS Version7
Summary0003295: externalCoupled boundary patches mixed up in parallel run
DescriptionFor runs using the externalCoupled boundary condition type, the boundary patches appear to have lost their original spatial correspondence prescribed by patchPoints and patchFaces when the invoking OpenFOAM solver runs in parallel. The degree of spatial disparity is dependent on the manner of domain decomposition. This issue does not affect serial run, however.

An example code, modified from the externalCoupledCavity tutorial, demonstrates the issue. A new external solver (externalSolver), rewritten in Python 3 instead of Shell, provides spatial and temporal temperature distributions on the hot and cold wall boundaries. Said spatial distribution is determined geometrically using the patchPoints and patchFaces files at the beginning of the external solver call.

The attached images show the spatial temperature distribution of the hot wall boundary after one iteration for (a) a single core run, (b) a parallel run with 4 cores, and (c) a parallel run with 16 cores. While the externalCoupled boundary worked correctly for the single core, it was mixed up in both parallel runs.

Two observation could be made from this exercise. First, the difference in the results between the two parallel runs suggests this issue depends on domain decomposition. Second, the temperature range (reported by Paraview) also indicates this mix-up is localized within the same boundary.
Steps To Reproduce1) Unpack attached test case (20190620-externalCoupledCavity.tar.gz)
2) Edit Shebang (#!) line in externalSolver to use python3 on test system
3a) Call Allrun to obtain results for a serial run
3b) Call Allrun-parallel to obtain results for a parallel run
4) View output on either hot or cold wall and note the difference
Additional InformationAfter some initial albeit superficial investigation, it appears that the OpenFOAM correctly read in the patches belonging to each domain. The mix up could take place from this point forward.

I personally do not know if this behavior is by design (i.e., externalCoupled was meant to be used in serial). However, in the meantime, I am looking into the relevant code to see what could be the issue.
TagsNo tags attached.

Activities

gerry_kan

2019-06-21 09:42

reporter  

gerry_kan

2019-06-27 07:50

reporter   ~0010533

To whom it may concern:

   I finally realized what was wrong. The documentation (or the lack thereof) has led me to believe that:

   1) createExternalCoupledPatchGeometry must be executed before decomposePar
   2) the -parallel option of createExternalCoupledPatchGeometry is for parallelization of the application alone

   Running createExternalCoupledPatchGeometry in parallel following decomposePar, with both execution running on the same number of processors in mpirun, results in the correct temperature distribution.

   Fortunately code change is no longer necessary to solve this problem. Attached is a modified version of the externalCoupledCavity tutorial that, in addition to introducing spatial and temporal temperature variations, also works in both serial and parallel. If you find it suitable, please feel free to incorporate this into your existing collection of standard tutorials.

   Otherwise please mark this ticket as resolved.

Thanks again, Gerry.

Issue History

Date Modified Username Field Change
2019-06-21 09:42 gerry_kan New Issue
2019-06-21 09:42 gerry_kan File Added: 20190620-externalCoupledCavity.tar.gz
2019-06-21 09:42 gerry_kan File Added: externalCoupledCavity_wall_hot_parallel_np_04.png
2019-06-21 09:42 gerry_kan File Added: externalCoupledCavity_wall_hot_parallel_np_16.png
2019-06-21 09:42 gerry_kan File Added: externalCoupledCavity_wall_hot_serial.png
2019-06-27 07:50 gerry_kan File Added: 20190627-externalCoupledCavity.tar.gz
2019-06-27 07:50 gerry_kan Note Added: 0010533
2019-06-27 16:06 henry Assigned To => henry
2019-06-27 16:06 henry Status new => closed
2019-06-27 16:06 henry Resolution open => no change required