View Issue Details

IDProjectCategoryView StatusLast Update
0003748OpenFOAMBugpublic2021-10-29 10:49
Reportermboesi Assigned Tohenry  
PrioritynormalSeveritytweakReproducibilityalways
Status resolvedResolutionfixed 
PlatformUnixOSUbuntuOS Version18.04
Summary0003748: Significant MPI overhead when solving chemistry if mechanism reduction is not active
DescriptionThe active species are synced between the different MPI processors at the end of the solve method in TDACChemistryModel:

    if (Pstream::parRun())
    {
        List<bool> active(composition.active());
        Pstream::listCombineGather(active, orEqOp<bool>());
        Pstream::listCombineScatter(active);

        forAll(active, i)
        {
            if (active[i])
            {
                composition.setActive(i);
            }
        }
    }

If the mechanism reduction is not active, the active species should never change and, thus, the sync should not be required.
I've done some code profiling in parallel mode which showed that typically more than 30% of the computational costs arise from that MPI communication. I'm still running some tests on this.

To prevent this excessive and, in cases without mechanism reduction, unnecessary MPI communication, the synchronization of the active species should only be done when mechanism reduction is actvite:

OF9: if (Pstream::parRun() && reduced)

OFdev if (Pstream::parRun() && mechRedActive_)

TagsNo tags attached.

Activities

henry

2021-10-29 10:49

manager   ~0012261

Resolved in OpenFOAM-9 by commit bafd78b3f46220b574bc6792b5a495bfca32da30
Resolved in OpenFOAM-dev by commit f376c1b08792892d225729e40925f6d3ff804035

Issue History

Date Modified Username Field Change
2021-10-29 09:23 mboesi New Issue
2021-10-29 10:49 henry Assigned To => henry
2021-10-29 10:49 henry Status new => resolved
2021-10-29 10:49 henry Resolution open => fixed
2021-10-29 10:49 henry Fixed in Version => 9
2021-10-29 10:49 henry Note Added: 0012261