View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0003967 | OpenFOAM | Bug | public | 2023-03-22 14:07 | 2023-04-12 12:26 |
| Reporter | rrossi | Assigned To | will | ||
| Priority | high | Severity | crash | Reproducibility | always |
| Status | resolved | Resolution | fixed | ||
| Platform | GNU/Linux | OS | Centos | OS Version | 7 |
| Product Version | 10 | ||||
| Fixed in Version | 10 | ||||
| Summary | 0003967: Case with NCC crashses in parallel | ||||
| Description | I'm running a case based on the compressibleInterFoam solver which worked just as fine with any number of processor without NCC. Using NCC, the case runs in parallel only with 2 cores using scotch for decomposition and crashes with larger numbers of cores (4, 48, 96 tested). Also, using 2 cores but the simple method for decomposition the case crashes. Can't attach case because even compressed and with the mesh only it's 4 MB in size. Can I use an external link instead? | ||||
| Steps To Reproduce | Simply run the compressibleInterFoam solver with scotch method on any number of cores larger than 2 | ||||
| Tags | No tags attached. | ||||
|
|
Why is the case so large? Is the mesh included? If so, is it possible to reproduce this with a case that we can generate the mesh for? Could you upload a log of the failure? Maybe we can figure it out without needing the case. Have you tried OpenFOAM-dev? If all that fails, then yes, we will provide a mechanism for you to transfer the files to us. |
|
|
yes the mesh is include. Being a moving mesh case, it would be not straightfoward to create the meshes, merge them, set the initial condition etc etc Attached is the log file and err files I haven't tried the dev yet runFoam.log (9,055 bytes)
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 10
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
Build : 10-8213cb4a3f81
Exec : setFields -dict system/setFieldsDict
Date : Mar 22 2023
Time : 14:57:56
Host : "r520c01n02"
PID : 52651
I/O : uncollated
Case : /g100_work/rfd_prod2021/medical/Chiesi/projects/devices/pMDI/testing/device/discharge/actualGeometry/runs/test13
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Create mesh for time = 0
Reading "system/setFieldsDict"
Setting field default values
Setting internal values of volScalarField p_rgh
Setting internal values of volScalarField p
Setting internal values of volScalarField alpha.liquid
Setting field region values
Adding all cells of cellZone fixedVolume ...
Found matching zone fixedVolume with 47551 cells.
Setting internal values of volScalarField p_rgh
Setting internal values of volScalarField p
Setting internal values of volScalarField alpha.liquid
End
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 10
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
Build : 10-8213cb4a3f81
Exec : decomposePar -latestTime
Date : Mar 22 2023
Time : 14:57:57
Host : "r520c01n02"
PID : 52676
I/O : uncollated
Case : /g100_work/rfd_prod2021/medical/Chiesi/projects/devices/pMDI/testing/device/discharge/actualGeometry/runs/test13
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Decomposing mesh region0
Create mesh
Calculating distribution of cells
Selecting decomposer scotch
Finished decomposition in 0.146063 s
Calculating original mesh data
Distributing cells to processors
Distributing faces to processors
Distributing points to processors
Constructing processor meshes
Processor 0
Number of cells = 21994
Number of faces shared with processor 1 = 339
Number of faces shared with processor 1 = 0
Number of processor patches = 2
Number of processor faces = 339
Number of boundary faces = 4276
Processor 1
Number of cells = 22431
Number of faces shared with processor 0 = 339
Number of faces shared with processor 0 = 0
Number of faces shared with processor 2 = 338
Number of processor patches = 3
Number of processor faces = 677
Number of boundary faces = 5807
Processor 2
Number of cells = 22301
Number of faces shared with processor 1 = 338
Number of faces shared with processor 3 = 525
Number of processor patches = 2
Number of processor faces = 863
Number of boundary faces = 4716
Processor 3
Number of cells = 22125
Number of faces shared with processor 2 = 525
Number of processor patches = 1
Number of processor faces = 525
Number of boundary faces = 5758
Number of processor faces = 1202
Max number of cells = 22431 (0.982544% above average 22212.8)
Max number of processor patches = 3 (50% above average 2)
Max number of faces between processors = 863 (43.594% above average 601)
Time = 0s
Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer
End
MPI startup(): Warning: I_MPI_PMI_LIBRARY will be ignored since the hydra process manager was found
MPI startup(): Warning: I_MPI_PMI_LIBRARY will be ignored since the hydra process manager was found
MPI startup(): Warning: I_MPI_PMI_LIBRARY will be ignored since the hydra process manager was found
MPI startup(): Warning: I_MPI_PMI_LIBRARY will be ignored since the hydra process manager was found
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 10
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
Build : 10-8213cb4a3f81
Exec : /g100_work/rfd_prod2021/medical/Chiesi/projects/devices/pMDI/solver/compressibleInterFoam/platforms/linux64IcxDPInt32OptCSL/bin/compressibleInterFoam -parallel
Date : Mar 22 2023
Time : 14:57:59
Host : "r520c01n02"
PID : 52825
I/O : uncollated
Case : /g100_work/rfd_prod2021/medical/Chiesi/projects/devices/pMDI/testing/device/discharge/actualGeometry/runs/test13
nProcs : 4
Slaves :
3
(
"r520c01n02.52826"
"r520c01n02.52827"
"r520c01n02.52828"
)
Pstream initialised with:
floatTransfer : 0
nProcsSimpleSum : 0
commsType : nonBlocking
polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Create mesh for time = 0
patchToPatch: Calculating couplings between 370 source faces and 370 target faces
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 0 PID 52825 RUNNING AT r520c01n02
= KILLED BY SIGNAL: 11 (Segmentation fault)
===================================================================================
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 1 PID 52826 RUNNING AT r520c01n02
= KILLED BY SIGNAL: 11 (Segmentation fault)
===================================================================================
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 2 PID 52827 RUNNING AT r520c01n02
= KILLED BY SIGNAL: 11 (Segmentation fault)
===================================================================================
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= RANK 3 PID 52828 RUNNING AT r520c01n02
= KILLED BY SIGNAL: 11 (Segmentation fault)
===================================================================================
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 10
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
Build : 10-8213cb4a3f81
Exec : reconstructPar
Date : Mar 22 2023
Time : 14:58:00
Host : "r520c01n02"
PID : 53005
I/O : uncollated
Case : /g100_work/rfd_prod2021/medical/Chiesi/projects/devices/pMDI/testing/device/discharge/actualGeometry/runs/test13
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
--> FOAM Warning :
From function static Foam::instantList Foam::timeSelector::select0(Foam::Time &, const Foam::argList &)
in file db/Time/timeSelector.C at line 269
No time specified or available, selecting 'constant'
Reconstructing fields for mesh region0
Time = 0s
Reconstructing FV fields
No FV fields
Reconstructing point fields
No point fields
No lagrangian fields
End
Converting cut planes files...
Error converting cut planes: postProcessing/cutPlanes does not exist...
Converting cut planes files...
Error converting cut planes: postProcessing/sampledSurfaces does not exist...
Updating computing budget...
Done.
|
|
|
> Being a moving mesh case, it would be not straightfoward to create the meshes, merge them, set the initial condition etc etc That's what an ./Allrun script is for. Can you create a script that I (or anyone else) can run that does these pre-processing steps? |
|
|
Without having access to the actual case, I'd make a wild guess that this would work in recent OF-dev version as a bit similar parallel problem I had got fixed by commits some time ago. |
|
|
@peksa Yes, that seems likely. The hash in the log indicates that the version of OpenFOAM-10 is from shortly after release, and hasn't been updated for bug fixes either. Closing. This is probably fixed, and the reporter hasn't responded for two weeks now. If the problem is encountered on an up-to-date build of dev or version 10 then a new report can be opened. But a case must also be made available (either via a file sharing service, or make it small enough to upload here). If there is no case I will close again as not reproducible. |
|
|
A very similar crash came up in compiler testing, so whilst I can't be completely sure, I think this is resolved now. See the following commits: https://github.com/OpenFOAM/OpenFOAM-dev/commit/71979accf5d0830de3fa5360b20fa9d37576cbd4 https://github.com/OpenFOAM/OpenFOAM-10/commit/2dce8f82fa4864d242c8f7865dd52aba53e16fda |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2023-03-22 14:07 | rrossi | New Issue | |
| 2023-03-22 14:33 | will | Note Added: 0012971 | |
| 2023-03-22 15:51 | rrossi | Note Added: 0012972 | |
| 2023-03-22 15:51 | rrossi | File Added: FOAM.err | |
| 2023-03-22 15:51 | rrossi | File Added: runFoam.log | |
| 2023-03-23 09:08 | will | Note Added: 0012973 | |
| 2023-04-06 08:59 | peksa | Note Added: 0012984 | |
| 2023-04-06 12:21 | will | Assigned To | => will |
| 2023-04-06 12:21 | will | Status | new => closed |
| 2023-04-06 12:21 | will | Resolution | open => no change required |
| 2023-04-06 12:21 | will | Note Added: 0012985 | |
| 2023-04-12 12:25 | will | Status | closed => resolved |
| 2023-04-12 12:25 | will | Fixed in Version | => 10 |
| 2023-04-12 12:25 | will | Note Added: 0012986 | |
| 2023-04-12 12:26 | will | Resolution | no change required => fixed |
