2017-08-21 07:29 BST

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0002401OpenFOAM[All Projects] Bugpublic2016-12-19 16:25
Reporterkevnor 
Assigned Tohenry 
PrioritynormalSeverityminorReproducibilityalways
StatusresolvedResolutionfixed 
Product Version4.x 
Target VersionFixed in Version 
Summary0002401: Bad parallel performance of volRegion function objects
DescriptionThe performance of the volRegion function object on large parallel cases is bad (and in extreme cases can lead to out-of-memory issues), due to the reconstruction of the entire field on the master process before processing is performed.

A better solution would be for the relevant processing to be performed on the separate processors and then combined as appropriate onto the master.
Steps To ReproduceConstruct any large case with e.g. > 100M cells and perform parallel decomposition.

Run parallel post-processing with any volRegion function object and note that evaluation of function object is expensive!
TagsfunctionObject, volRegion
Attached Files
  • patch file icon volRegionTemplates.patch (3,027 bytes) 2016-12-16 15:26 -
    diff --git src/functionObjects/field/fieldValues/volRegion/volRegionTemplates.C src/functionObjects/field/fieldValues/volRegion/volRegionTemplates.C
    index d3741f7..616cac7 100644
    --- src/functionObjects/field/fieldValues/volRegion/volRegionTemplates.C
    +++ src/functionObjects/field/fieldValues/volRegion/volRegionTemplates.C
    @@ -84,52 +84,52 @@ Type Foam::functionObjects::fieldValues::volRegion::processValues
         {
             case opSum:
             {
    -            result = sum(values);
    +            result = gSum(values);
                 break;
             }
             case opSumMag:
             {
    -            result = sum(cmptMag(values));
    +            result = gSum(cmptMag(values));
                 break;
             }
             case opAverage:
             {
    -            result = sum(values)/values.size();
    +	  result = gSum(values)/nCells_;
                 break;
             }
             case opWeightedAverage:
             {
    -            result = sum(weightField*values)/sum(weightField);
    +            result = gSum(weightField*values)/gSum(weightField);
                 break;
             }
             case opVolAverage:
             {
    -            result = sum(V*values)/sum(V);
    +            result = gSum(V*values)/volume_;
                 break;
             }
             case opWeightedVolAverage:
             {
    -            result = sum(weightField*V*values)/sum(weightField*V);
    +            result = gSum(weightField*V*values)/gSum(weightField*V);
                 break;
             }
             case opVolIntegrate:
             {
    -            result = sum(V*values);
    +            result = gSum(V*values);
                 break;
             }
             case opMin:
             {
    -            result = min(values);
    +            result = gMin(values);
                 break;
             }
             case opMax:
             {
    -            result = max(values);
    +            result = gMax(values);
                 break;
             }
             case opCoV:
             {
    -            Type meanValue = sum(values*V)/sum(V);
    +            Type meanValue = gSum(values*V)/volume_;
     
                 const label nComp = pTraits<Type>::nComponents;
     
    @@ -139,7 +139,7 @@ Type Foam::functionObjects::fieldValues::volRegion::processValues
                     scalar mean = component(meanValue, d);
                     scalar& res = setComponent(result, d);
     
    -                res = sqrt(sum(V*sqr(vals - mean))/sum(V))/mean;
    +                res = sqrt(gSum(V*sqr(vals - mean))/volume_)/mean;
                 }
     
                 break;
    @@ -173,14 +173,10 @@ bool Foam::functionObjects::fieldValues::volRegion::writeValues
                 weightField = setFieldValues<scalar>(weightFieldName_, true);
             }
     
    -        // Combine onto master
    -        combineFields(values);
    -        combineFields(V);
    -        combineFields(weightField);
    +	Type result = processValues(values, V, weightField);
     
             if (Pstream::master())
             {
    -            Type result = processValues(values, V, weightField);
     
                 // Add to result dictionary, over-writing any previous entry
                 resultDict_.add(fieldName, result, true);
    
    patch file icon volRegionTemplates.patch (3,027 bytes) 2016-12-16 15:26 +

-Relationships
+Relationships

-Notes

~0007495

henry (manager)

I agree that the current way 'volRegion' works is inefficient running in parallel and should be re-written. Can you provide a patch for this?

~0007503

kevnor (reporter)

Here's a potential patch. I've tested it with all available operations on a small number of cases in both serial and parallel and it appears to give identical results to the previous implementation, but functions significantly faster in parallel.

I guess that something similar should be done for the surfaceRegion functionObject as well, though there's less of an efficiency issue there due to the lower dimensionality.

~0007531

henry (manager)

Thanks for the patch. I have updated it to conform to
http://openfoam.org/dev/coding-style-guide/

and applied it to OpenFOAM-4.x:

commit a7c2bd6cacf12110af6587378c96b98f003645d5

However it cannot be applied to OpenFOAM-dev due to significant changes in the handling of volRegion. I will make the corresponding changes shortly.
+Notes

-Issue History
Date Modified Username Field Change
2016-12-16 09:21 kevnor New Issue
2016-12-16 09:21 kevnor Tag Attached: functionObject
2016-12-16 09:21 kevnor Tag Attached: volRegion
2016-12-16 09:55 henry Note Added: 0007495
2016-12-16 15:26 kevnor File Added: volRegionTemplates.patch
2016-12-16 15:26 kevnor Note Added: 0007503
2016-12-19 16:25 henry Assigned To => henry
2016-12-19 16:25 henry Status new => resolved
2016-12-19 16:25 henry Resolution open => fixed
2016-12-19 16:25 henry Note Added: 0007531
+Issue History