View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000334OpenFOAMBugpublic2011-11-09 16:202011-11-23 10:45
Reporteruser286Assigned Touser4 
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
PlatformIntel Xeon X5680OSRed Hat Enterprise Linux Server OS Version5.7 (Tikanga)
Summary0000334: Problem with redistributeMeshPar
DescriptionI am trying to build a mesh with local refinement. I start by making a simple uniform mesh with blockMesh. I then use decomposePar to split up to processors. Then I use topoSet and refineHexMesh in parallel to do the refinement. The mesh is now load-unbalanced, so I use redistributeMeshPar in parallel.

redistributeMeshPar fails if one of my solution variables has a "fixedGradient" boundary condition on a boundary that gets manipulated by redistributeMeshPar in parallel. If the boundary is changed to something like "fixedValue" it works fine. The failure message is as follows:

/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: 2.0.x |
| \\ / A nd | Web: www.OpenFOAM.org |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : 2.0.x-03233a7696f0
Exec : redistributeMeshPar -parallel -overwrite
Date : Nov 09 2011
Time : 08:52:48
Host : wind-las10.nrel.gov
PID : 11696
Case : /data1/home/mchurchf/cases/AIAA_ASM_2012/test
nProcs : 4
Slaves :
3
(
wind-las10.nrel.gov.11697
wind-las10.nrel.gov.11698
wind-las10.nrel.gov.11699
)

Pstream initialized with:
    floatTransfer : 0
    nProcsSimpleSum : 0
    commsType : nonBlocking
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Disallowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Using mesh subdirectory "polyMesh"

Found points in "/data1/home/mchurchf/cases/AIAA_ASM_2012/test/processor0/constant/polyMesh"

Per processor mesh availability : 4{1}
    Boundary definition OK.
Selecting decompositionMethod ptscotch
Merge tolerance : 1e-06
Write tolerance : 1e-12
Overall meshes bounding box : (0 0 0) (4000 4000 1000)
Relative tolerance : 1e-06
Absolute matching distance : 0.00574456264654

[1]
[1]
[1] --> FOAM FATAL IO ERROR:
[1] Expected a '(' or a '{' while reading List, found on line 0 the label 0
[1]
[1] file: IOstream at line 0.
[1]
[1] From function Istream::readBeginList(const char*)
[1] in file db/IOstreams/IOstreams/Istream.C at line 138.
[1]
FOAM parallel run exiting
[1]
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun has exited due to process rank 1 with PID 11697 on
node wind-las10.nrel.gov exiting improperly. There are two reasons this could occur:

1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits







Steps To ReproduceI have uploaded a test case to try this on. It works on 4 processors. Simply untar and unzip the case, then use "Allrun" to run it. The variable T has a fixedGradient top boundary, which you can manipulate in the 0/T file.
TagsNo tags attached.

Activities

user286

2011-11-09 16:20

 

test.tar.gz (13,414 bytes)

user4

2011-11-23 10:45

  ~0000818

This is fixed in the next version.

(the problem was that the fixedGradient b.c. does an evaluate upon construction from dictionary which triggers rebuilding of weighting factors which triggers parallel communication. This behaviour is now disabled specifically during redistribution)

Thanks for reporting.

Issue History

Date Modified Username Field Change
2011-11-09 16:20 user286 New Issue
2011-11-09 16:20 user286 File Added: test.tar.gz
2011-11-23 10:45 user4 Note Added: 0000818
2011-11-23 10:45 user4 Status new => resolved
2011-11-23 10:45 user4 Fixed in Version => Other
2011-11-23 10:45 user4 Resolution open => fixed
2011-11-23 10:45 user4 Assigned To => user4