View Issue Details

IDProjectCategoryView StatusLast Update
0002324OpenFOAM[All Projects] Bugpublic2016-11-25 14:43
ReporterjoegiAssigned Tohenry 
PrioritynormalSeverityminorReproducibilityrandom
Status resolvedResolutionfixed 
PlatformcentosOScentosOS Versioncentos
Product Version4.x 
Fixed in Versiondev 
Summary0002324: writeDictionary + timeActivatedFileUpdate
DescriptionUse of both function objects together will stalled the simulation, sometimes after the pc is in iddle for a long time the simulation will restart. Have seen this behavior specially when running with many cores.
Steps To ReproduceTry to use a combination of these function objects in parallel.
TagsNo tags attached.

Activities

wyldckat

2016-11-08 23:01

updater   ~0007127

@joegi: Can you please provide more specific details?
There are probably few dozens of possible combinations, given the number of possible dictionary files that can be written and replaced... maybe even if no dictionary file is given?


> will stalled the simulation, sometimes after the pc is in iddle for a long time the simulation will restart

This could potentially be linked to the use of NFS or another filesystem sharing mechanism. Did you make any changes to the settings in "etc/controlDict", specifically within the block "OptimisationSwitches"?

joegi

2016-11-08 23:27

reporter   ~0007129

I see this problem in distributed and shared memory computers.

With timeActivatedFileUpdate I am changing fvSchemes and fvSolution. A wild guess is that each function object is trying to access the same dictionary at the same time.

This issue can be sensitive when running in cluster as you are burning time while the simulation is stalled. In any case, the combination of writeDictionary + timeActivatedFileUpdate is something that not everybody will use.

wyldckat

2016-11-09 11:13

updater   ~0007130

I still have to ask you, to be certain of what needs to be tested:

  1. Which dictionaries are selected to be written by "writeDictionary"?

  2. Does it fail in either order, namely "writeDictionary + timeActivatedFileUpdate" and "timeActivatedFileUpdate + writeDictionary" ?

joegi

2016-11-10 08:53

reporter   ~0007131

The order is indifferent.

The dictionaries are fvSolution, fvSchemes, controlDict, thermophysicalPropertires, turbulenceProperties

MattijsJ

2016-11-17 13:36

reporter   ~0007224

If you use e.g. timeStampMaster then it will trigger a re-read if the new modification time is bigger than the old one + fileModificationSkew (in etc/controlDict). If your filing system is slower than that the reading might overlap with the writing.

This is a generic problem but I would imagine that this might happen more frequently with the timeActivatedFileUpdate functionObject since it is executed at the start of the time loop as if the file-modification checking.

You could try increasing the fileModificationSkew.

MattijsJ

2016-11-22 20:23

reporter   ~0007290

An additional problem is that the copy of the file can be quite slow. Attached a drop-in replacement for src/functionObjects/utilities/timeActivatedFileUpdate/timeActivatedFileUpdate.C that uses a move instead so much much less likely to see your problem.

timeActivatedFileUpdate.C (3,916 bytes)

henry

2016-11-25 14:43

manager   ~0007328

Resolved in OpenFOAM-dev by commit b2d5dca4882e7e460146c092f32c384615ba55b8

Issue History

Date Modified Username Field Change
2016-11-08 22:38 joegi New Issue
2016-11-08 23:01 wyldckat Note Added: 0007127
2016-11-08 23:27 joegi Note Added: 0007129
2016-11-09 11:13 wyldckat Note Added: 0007130
2016-11-10 08:53 joegi Note Added: 0007131
2016-11-17 13:36 MattijsJ Note Added: 0007224
2016-11-22 20:23 MattijsJ File Added: timeActivatedFileUpdate.C
2016-11-22 20:23 MattijsJ Note Added: 0007290
2016-11-25 14:43 henry Assigned To => henry
2016-11-25 14:43 henry Status new => resolved
2016-11-25 14:43 henry Resolution open => fixed
2016-11-25 14:43 henry Fixed in Version => dev
2016-11-25 14:43 henry Note Added: 0007328