#### View Issue Details

ID Project Category View Status Date Submitted Last Update 0002516 OpenFOAM Bug public 2017-03-29 14:52 2019-04-04 19:16 Shorty henry normal tweak always resolved fixed GNU/Linux Ubuntu 14.04 dev 0002516: The more outer loops, the slower the outer loop calculation gets Hey everybody, I am not sure if this is a bug or something that can be explained in a mathematical point of view. However, based on the studies that I did the last few days I will just let you know that there might be a big problem for solid mechanics calculation in particular the speed of the outer loops. In fact, I build my own solver in which I realized this problem but I can reproduce it with the normal solidDisplacementFoam too (the simple case is attached). What did I realized: ----------------------------------------------------------------------------- I realized that the more outer loops I have, the slower the outer loop calculation gets. In particular, I found that solving the system is the point where the solver needs more effort from outer loop to outer loop. To analyze it, I turned off the solver performance and added the cpu clock time before and after the DEqn.solve() function and output the result for each 50 outer loop. I performed more than 25000 outer loops. The time to calculate 50 outer loops increased linear. This lead to a parabolic profile in the whole calculation duration. However, if we would stop after 100 outer loops and go on in time, this behavior cannot be analyzed/realized well. Example given: ----------------------------------------------------------------------------- The first 50 outer loops of one particular time step are performed with 0.1s Further 50 outer loops (after 20000 finished outer loops) took almost 6.8s. I would expect that 50 outer loops at the beginning should take the same amount than 50 outer loops after 10000 outer loops. Of course, if the linear system would need more and more inner iterations then the time would increase to a special threshold after we reach 1000 inner iterations. I could understand this based on the fact that the matrix gets (maybe) poorer and thus more inner iterations are needed. However this is not the case. In order to prevent this, I limited the solver to 100 inner iterations (maxIter). I could not find the particular code in which we need more time. I would be very interested what causes this behavior. Tobi - insert the following line into the solidDisplacementFoam:                 initialResidual = DEqn.solve().max().initialResidual();                 if (show == 50)                 {                     Info<< iCorr << " "                         << runTime.elapsedCpuTime() << " "                         << runTime.elapsedCpuTime() - oldCpu << " "                         << initialResidual                         << endl;                     show = 0;                     oldCpu = runTime.elapsedCpuTime();                 }                                    show++; - Recompile - Run the attached solidDisplacementFoam case I think I realized this behavior of outer loop slow down even in normal solver. No tags attached.

#### Activities

 2017-03-29 14:52 reporter 2DBeam.tar.gz (228,550 bytes) 2017-03-29 14:53 reporter outerCorrectiondt.eps (37,949 bytes)    outerCorrectiondt.eps (37,949 bytes) 2017-03-29 14:53 reporter outerCorrectionCPUTime.png (5,300 bytes)    outerCorrectionCPUTime.png (5,300 bytes) 2017-03-29 14:55 reporter   ~0007992 Maybe a few words to the two pictures: outerCorrectiondt.eps ----------------------------------------------------------------------- It shows the time needed to solve 50 outer loops. It increases linearly The more outer loops the slower is the outer loop calculation outerCorrectionCPUTime.png ----------------------------------------------------------------------- It is the accumulated time for all outer loops. It is clear that if the single outer loop calculation increases linearly that the global time increase parabolic 2017-04-01 10:28 reporter PimpleFoamOuterCorrections.pdf (20,166 bytes) 2017-04-01 10:28 reporter PimpleFoamOuterCorrectionsPressure.pdf (1,741,938 bytes) 2017-04-01 10:30 reporter pitzDaily.tar.gz (187,315 bytes) 2017-04-01 10:33 reporter   ~0008012 Hey together, I made the same analyze now with the pimpleFoam case and got the same behavior. Okay I might check it with a bit under-relaxation but I guess it will give me the same linear increase of CPU time for the outer corrections. The residuals should be better I guess. Any glue why this problem exist? 2017-04-01 18:40 manager   ~0008013 I am unable to reproduce this behaviour: when I run pimlpeFoam on the pitzDaily case and change the number of outer correctors from 1 to 128 the time taken increases slightly less than linearly as expected. This is a user-support request. 2017-04-01 19:19 reporter   ~0008014 If you want to reproduce it you have to use 50000 outer loops. You may ask why we need 50.000 ... not for fluid mechanics but for solid mechanics it can happen. I refer to Philip Cardiff and his work. If you are not interested, you can close it again. 2017-04-03 20:35 reporter   ~0008016 Hey henry, I got the point. Compiled FOAM for profiling and the result is: (just the interesting thing):  26.38 9.20 9.20 20010 0.46 0.46 Foam::List::~List()  19.59 16.03 6.83 60028 0.11 0.11 Foam::List > >::setSize(int) Based on the fact that we store each iteration in the solverPerformanceDict (dataTemplates.C) the function that appends the new result (List.C -> setSize(int)) is very slow and extreme limiting (no USEMEMCPY activated). I will check how it behaves with usememcpy activated. 2017-04-03 21:00 reporter   ~0008017 Okay, after activating USEMEMCPY, everything is very fast! Thats what I expected. So no bug, no tweak, nothing needed. Sorry for bothering you. 2017-06-30 12:43 manager   ~0008312 Reporter says there is no issue 2019-04-04 19:16 manager   ~0010408 Resolved by commit 2f3d47ad7e54a537758666a561809a02946d34f2

#### Issue History

2017-03-29 14:52 Shorty New Issue
2017-03-29 14:52 Shorty File Added: 2DBeam.tar.gz
2017-03-29 14:52 Shorty Tag Attached: outer loops slow down
2017-03-29 14:53 Shorty File Added: outerCorrectiondt.eps
2017-03-29 14:53 Shorty File Added: outerCorrectionCPUTime.png
2017-03-29 14:55 Shorty Note Added: 0007992
2017-04-01 10:28 Shorty File Added: PimpleFoamOuterCorrections.pdf
2017-04-01 10:28 Shorty File Added: PimpleFoamOuterCorrectionsPressure.pdf
2017-04-01 10:30 Shorty File Added: pitzDaily.tar.gz
2017-04-01 10:33 Shorty Note Added: 0008012
2017-04-01 18:40 henry Assigned To => henry
2017-04-01 18:40 henry Status new => closed
2017-04-01 18:40 henry Resolution open => no change required
2017-04-01 18:40 henry Note Added: 0008013
2017-04-01 19:19 Shorty Status closed => feedback
2017-04-01 19:19 Shorty Resolution no change required => reopened
2017-04-01 19:19 Shorty Note Added: 0008014
2017-04-03 20:35 Shorty Note Added: 0008016
2017-04-03 20:35 Shorty Status feedback => assigned
2017-04-03 21:00 Shorty Note Added: 0008017
2017-06-30 12:43 chris Status assigned => closed
2017-06-30 12:43 chris Resolution reopened => fixed
2017-06-30 12:43 chris Note Added: 0008312
2018-07-10 11:20 administrator Tag Detached: outer loops slow down
2019-04-04 19:16 henry Status closed => resolved
2019-04-04 19:16 henry Fixed in Version => dev
2019-04-04 19:16 henry Note Added: 0010408