View Issue Details
ID  Project  Category  View Status  Date Submitted  Last Update 

0003173  OpenFOAM  Bug  public  20190219 15:52  20190301 15:46 
Reporter  jherb  Assigned To  will  
Priority  normal  Severity  major  Reproducibility  always 
Status  resolved  Resolution  fixed  
Platform  SUSE Linux Enterprise Server 11  OS  SUSE Linux Enterprise Server 11  OS Version  (please specify) 
Product Version  6  
Fixed in Version  6  
Summary  0003173: buoyantPimpleFoam with nOuterCorrectors crashes (regression from OpenFOAM 5)  
Description  For different cases buoyantPimpleFoam crashes, if nOuterCorrectors are used: PIMPLE: Iteration 7 DILUPBiCGStab: Solving for Ux, Initial residual = 0.0026433, Final residual = 6.27222e06, No Iterations 1 DILUPBiCGStab: Solving for Uy, Initial residual = 0.00255987, Final residual = 6.41521e06, No Iterations 1 DILUPBiCGStab: Solving for Uz, Initial residual = 0.0026433, Final residual = 6.27221e06, No Iterations 1 DILUPBiCGStab: Solving for h, Initial residual = 0.00181503, Final residual = 8.62371e06, No Iterations 1 DICPCG: Solving for p_rgh, Initial residual = 0.00737213, Final residual = 5.90448e05, No Iterations 35 DICPCG: Solving for p_rgh, Initial residual = 5.906e05, Final residual = 5.01722e07, No Iterations 10 DICPCG: Solving for p_rgh, Initial residual = 5.03358e07, Final residual = 4.33555e09, No Iterations 16 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 time step continuity errors : sum local = 2.42194e07, global = 2.42194e07, cumulative = 0.000212064 DICPCG: Solving for p_rgh, Initial residual = 0.00283376, Final residual = 2.0804e05, No Iterations 35 DICPCG: Solving for p_rgh, Initial residual = 2.0808e05, Final residual = 1.8005e07, No Iterations 10 DICPCG: Solving for p_rgh, Initial residual = 1.82666e07, Final residual = 6.11662e09, No Iterations 14 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 time step continuity errors : sum local = 2.4238e07, global = 2.4238e07, cumulative = 0.000212306 #0 Foam::error::printStack(Foam::Ostream&) at ??:? #1 Foam::sigFpe::sigHandler(int) at ??:? #2 ? in "/lib64/libc.so.6" #3 Foam::convergenceControl::getInitialResiduals(Foam::fvMesh const&, Foam::word const&, int, Foam::ITstream&, double&, double&) at ??:? Waiting for data... (interrupt to abort)/home/hej/OpenFOAM/OpenFOAM6/bin/tools/RunFunctions: line 53: 9825 Floating point exception$APP_RUN "$@" > log.$LOG_SUFFIX 2>&1 #4 Foam::singleRegionCorrectorConvergenceControl::corrCriteriaSatisfied() const at ??:? #5 Foam::pimpleLoop::loop(Foam::correctorConvergenceControl&) at ??:? #6 Foam::pimpleControl::loop() at ??:? #7 ? at ??:? #8 __libc_start_main in "/lib64/libc.so.6" #9 ? at /usr/src/packages/BUILD/glibc2.11.3/csu/../sysdeps/x86_64/elf/start.S:116 The crashes are reproducible at the same iteration, if the same number of CPUs (one ore x with MPI) are used. These crashes seem not to occur with OpenFOAM 5.x Also they occur with a case with completely different physics, geometry, mesh, ...  
Steps To Reproduce  Start the attached case with is based on the tutorial OpenFOAM6/tutorials/heatTransfer/buoyantPimpleFoam/hotRoom Just the fvSolutions file was modified: PIMPLE { momentumPredictor yes; nOuterCorrectors 30; nCorrectors 2; nNonOrthogonalCorrectors 2; // OF 6 outerCorrectorResidualControl { "(U.*p.*)" { relTol 0; tolerance 0.0001; } "(hkepsilon)" { relTol 0; tolerance 0.001; } } /* // OF 5 residualControl { "(hkepsilon)" { relTol 0; tolerance 1e3; } "(U.*pp_rgh)" { relTol 0; tolerance 1e4; } */ } relaxationFactors { fields { p_rgh 0.6; p_rghFinal 1; } equations { "(TkomegaR)" 0.6; "U(xyz)*" 0.3; h 0.4; "(hU.*TkomegaR)Final" 1; } }  
Additional Information  Additional observation: The simulation with OpenFOAM 5.x seems to converge much fast per time step.  
Tags  No tags attached.  

hotRoom.nOuter.tar.gz (2,950 bytes) log.buoyantPimpleFoam5.gz (552,494 bytes) log.buoyantPimpleFoam6.gz (74,676 bytes) 

I also run the test case on a different computer using the latest OpenFOAMdev and also OpenFOAM6. Both simulations crashed. But at different points: OpenFOAM6: Courant Number mean: 0.100328 max: 0.75168 Time = 204 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 PIMPLE: Iteration 1 ... PIMPLE: Iteration 11 DILUPBiCGStab: Solving for Ux, Initial residual = 0.000190101, Final residual = 9.03535e08, No Iterations 1 DILUPBiCGStab: Solving for Uy, Initial residual = 0.000253735, Final residual = 1.92472e07, No Iterations 1 DILUPBiCGStab: Solving for Uz, Initial residual = 0.000190101, Final residual = 9.03528e08, No Iterations 1 DILUPBiCGStab: Solving for h, Initial residual = 0.000113681, Final residual = 1.25367e07, No Iterations 1 DICPCG: Solving for p_rgh, Initial residual = 0.00141397, Final residual = 1.30006e05, No Iterations 4 DICPCG: Solving for p_rgh, Initial residual = 1.30033e05, Final residual = 6.46205e08, No Iterations 35 DICPCG: Solving for p_rgh, Initial residual = 7.67564e08, Final residual = 5.73888e09, No Iterations 4 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 time step continuity errors : sum local = 1.93598e08, global = 1.93598e08, cumulative = 0.000519603 DICPCG: Solving for p_rgh, Initial residual = 0.000559505, Final residual = 4.68605e06, No Iterations 4 DICPCG: Solving for p_rgh, Initial residual = 4.68644e06, Final residual = 2.32746e08, No Iterations 35 DICPCG: Solving for p_rgh, Initial residual = 4.89851e08, Final residual = 8.79882e09, No Iterations 2 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 time step continuity errors : sum local = 1.93718e08, global = 1.93718e08, cumulative = 0.000519622 #0 Foam::error::printStack(Foam::Ostream&) at ??:? #1 Foam::sigFpe::sigHandler(int) at ??:? #2 ? in "/lib64/libc.so.6" #3 Foam::convergenceControl::getInitialResiduals(Foam::fvMesh const&, Foam::word const&, int, Foam::ITstream&, double&, double&) at ??:? #4 Foam::singleRegionCorrectorConvergenceControl::corrCriteriaSatisfied() const at ??:? #5 Foam::pimpleLoop::loop(Foam::correctorConvergenceControl&) at ??:? #6 Foam::pimpleControl::loop() at ??:? #7 ? at ??:? #8 __libc_start_main in "/lib64/libc.so.6" #9 ? at /home/abuild/rpmbuild/BUILD/glibc2.18/csu/../sysdeps/x86_64/start.S:125 OpenFOAMdev: Courant Number mean: 0.110577 max: 0.758001 Time = 358 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 PIMPLE: Iteration 1 ... PIMPLE: Iteration 9 DILUPBiCGStab: Solving for Ux, Initial residual = 0.000159993, Final residual = 4.0151e08, No Iterations 1 DILUPBiCGStab: Solving for Uy, Initial residual = 0.000305932, Final residual = 1.38454e07, No Iterations 1 DILUPBiCGStab: Solving for Uz, Initial residual = 0.000159993, Final residual = 4.01516e08, No Iterations 1 DILUPBiCGStab: Solving for h, Initial residual = 0.000127525, Final residual = 7.34138e08, No Iterations 1 DICPCG: Solving for p_rgh, Initial residual = 0.000974706, Final residual = 7.24818e06, No Iterations 3 DICPCG: Solving for p_rgh, Initial residual = 7.24994e06, Final residual = 4.7008e08, No Iterations 35 DICPCG: Solving for p_rgh, Initial residual = 6.23827e08, Final residual = 8.32687e09, No Iterations 3 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 time step continuity errors : sum local = 1.02877e08, global = 1.02877e08, cumulative = 0.000202414 DICPCG: Solving for p_rgh, Initial residual = 0.000385133, Final residual = 2.90124e06, No Iterations 3 DICPCG: Solving for p_rgh, Initial residual = 2.90174e06, Final residual = 1.85987e08, No Iterations 35 DICPCG: Solving for p_rgh, Initial residual = 4.44122e08, Final residual = 8.46067e09, No Iterations 2 diagonal: Solving for rho, Initial residual = 0, Final residual = 0, No Iterations 0 time step continuity errors : sum local = 2.37139e10, global = 9.16484e12, cumulative = 0.000202414 #0 Foam::error::printStack(Foam::Ostream&) at ??:? #1 Foam::sigFpe::sigHandler(int) at ??:? #2 ? in "/lib64/libc.so.6" #3 Foam::convergenceControl::getInitialResiduals(Foam::fvMesh const&, Foam::word const&, int, Foam::ITstream&, double&, double&) at ??:? #4 Foam::singleRegionCorrectorConvergenceControl::corrCriteriaSatisfied() const at ??:? #5 Foam::pimpleLoop::loop(Foam::correctorConvergenceControl&) at ??:? #6 Foam::pimpleControl::loop() at ??:? #7 ? at ??:? #8 __libc_start_main in "/lib64/libc.so.6" #9 ? at /home/abuild/rpmbuild/BUILD/glibc2.18/csu/../sysdeps/x86_64/start.S:125 

@jherb: Quick question: Which compiler and which version of it did you use? This is just in case you've used a compiler that is too old for the more recent code in OpenFOAM, which could explain some of the symptoms you are describing. 

On the first computer, Intel 18.0.2 was used, on the second computer GCC 7.2.0 

It's not a compiler issue. I can reproduce it. The list of solver residuals is being indexed out of bounds. Either the index is incorrect or the list isn't being filled properly or is being reset. 

With a debug build, you get: > FOAM FATAL ERROR: index 6 out of range 0 ... 1 From function void Foam::UList<T>::checkIndex(Foam::label) const [with T = Foam::SolverPerformance<Foam::Vector<double> >; Foam::label = int] in file /home/bender2/will/OpenFOAM/OpenFOAMdev/src/OpenFOAM/lnInclude/UListI.H at line 106. FOAM aborting (FOAM_ABORT set) #0 Foam::error::printStack(Foam::Ostream&) at ??:? #1 Foam::error::abort() at ??:? #2 Foam::Ostream& Foam::operator<< <Foam::error>(Foam::Ostream&, Foam::errorManip<Foam::error>) at ~/OpenFOAM/OpenFOAMdev/src/OpenFOAM/lnInclude/errorManip.H:85 (discriminator 4) #3 Foam::UList<Foam::SolverPerformance<Foam::Vector<double> > >::checkIndex(int) const at ~/OpenFOAM/OpenFOAMdev/src/OpenFOAM/lnInclude/UListI.H:108 #4 Foam::UList<Foam::SolverPerformance<Foam::Vector<double> > >::operator[](int) const at ~/OpenFOAM/OpenFOAMdev/src/OpenFOAM/lnInclude/UListI.H:201 #5 void Foam::convergenceControl::getInitialTypeResiduals<Foam::Vector<double> >(Foam::fvMesh const&, Foam::word const&, int, Foam::ITstream&, double&, double&) at ~/OpenFOAM/OpenFOAMdev/src/finiteVolume/cfdTools/general/solutionControl/convergenceControl/convergenceControl/convergenceControlTemplates.C:69 (discriminator 2) #6 ... 

The iteration index is being held perentry in the outerCorrectorResidualControl subdictionary. When the field name is given as a regular expression and relates to multiple fields, and those fields have different iteration sequences (e.g., p and U), this can result in the wrong iteration index being used. The index needs to be held perfield. I'll make this change, but might take a bit of time. It's not a simple fix. In the meantime, you can run the case by separating the entries for p and U; e.g., "(U.*)" { relTol 0; tolerance 0.0001; } "(p.*)" { relTol 0; tolerance 0.0001; } 

Thank you for finding the root cause of the problem. The temporary fix to not use regular expressions is OK for now. 

Despite what I said before, it actually wasn't that complicated. Fixed in version 6 and in dev. Commits below. https://github.com/OpenFOAM/OpenFOAM6/commit/e29811f5dff86b792974a11f01d4155acccb69c0 https://github.com/OpenFOAM/OpenFOAMdev/commit/d7a5456aae71aef41cbd8cb0de7b94a530644a04 
Date Modified  Username  Field  Change 

20190219 15:52  jherb  New Issue  
20190219 15:52  jherb  File Added: hotRoom.nOuter.tar.gz  
20190219 15:52  jherb  File Added: log.buoyantPimpleFoam5.gz  
20190219 15:52  jherb  File Added: log.buoyantPimpleFoam6.gz  
20190219 16:04  jherb  Note Added: 0010300  
20190220 09:18  wyldckat  Note Added: 0010301  
20190220 11:22  jherb  Note Added: 0010302  
20190220 11:33  will  Note Added: 0010304  
20190220 11:34  will  Note Added: 0010305  
20190220 12:03  will  Note Added: 0010306  
20190220 13:37  jherb  Note Added: 0010308  
20190220 14:46  will  Assigned To  => will 
20190220 14:46  will  Status  new => resolved 
20190220 14:46  will  Resolution  open => fixed 
20190220 14:46  will  Fixed in Version  => 6 
20190220 14:46  will  Note Added: 0010309  
20190301 15:46  will  Relationship added  related to 0003182 