View Issue Details

IDProjectCategoryView StatusLast Update
0001494OpenFOAMBugpublic2016-02-03 21:03
ReportermatteoL Assigned Tohenry  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
PlatformcentosOSOtherOS Version(please specify)
Summary0001494: mirrorMesh does not work in parallel
DescriptionHello,
it seems mirrorMesh does not work in parallel even if it has the -parallel flag.

When I run it in parallel I get:
Create time

--> FOAM Warning :
    From function Time::operator++()
    in file db/Time/Time.C at line 1055
    Increased the timePrecision from 7 to 100 to distinguish between timeNames at time 0
--> FOAM Warning :
    From function Time::operator++()
    in file db/Time/Time.C at line 1064
    Current time name 0 is the old as the previous one 0
    This might result in overwriting old results.
[node291.cluster.racing.lc:77034] 7 more processes have sent help message help-mpi-btl-sm.txt / knem fail open
[node291.cluster.racing.lc:77034] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
Mirroring points. Old points: 190025 New points: 380050
Mirroring faces. Old faces: 998867--> FOAM Warning :
    From function mirrorFvMesh::mirrorFvMesh(const IOobject&)
    in file mirrorFvMesh.C at line 158
    Found coupled patch procBoundary0to1
    Mirroring faces on coupled patches destroys the ordering. This might be fixed by running a dummy createPatch afterwards.
--> FOAM Warning :
    From function mirrorFvMesh::mirrorFvMesh(const IOobject&)
    in file mirrorFvMesh.C at line 158
    Found coupled patch procBoundary0to8
    Mirroring faces on coupled patches destroys the ordering. This might be fixed by running a dummy createPatch afterwards.
--> FOAM Warning :
    From function mirrorFvMesh::mirrorFvMesh(const IOobject&)
    in file mirrorFvMesh.C at line 158
    Found coupled patch procBoundary0to32
    Mirroring faces on coupled patches destroys the ordering. This might be fixed by running a dummy createPatch afterwards.
--> FOAM Warning :
    From function mirrorFvMesh::mirrorFvMesh(const IOobject&)
    in file mirrorFvMesh.C at line 158
    Found coupled patch procBoundary0to33
    Mirroring faces on coupled patches destroys the ordering. This might be fixed by running a dummy createPatch afterwards.
--> FOAM Warning :
    From function mirrorFvMesh::mirrorFvMesh(const IOobject&)
    in file mirrorFvMesh.C at line 158
    Found coupled patch procBoundary0to40
    Mirroring faces on coupled patches destroys the ordering. This might be fixed by running a dummy createPatch afterwards.
 New faces: 1997734
Mirroring patches. Old patches: 11 New patches: 8
Mirroring cells. Old cells: 407326 New cells: 814652
Mirroring cell shapes.

Creating new mesh
[3] [5]
[5]
[5] --> FOAM FATAL ERROR:
[5] Unknown polyPatch type processor for patch procBoundary5to4

Valid polyPatch types are :

18
(
cyclic
cyclicACMI
cyclicAMI
cyclicSlip
empty
genericPatch
mappedPatch
mappedWall
mappedWallVariableThickness
nonuniformTransformCyclic
oldCyclic
patch
regionCoupled
regionCoupledWall
symmetry
symmetryPlane
wall
wedge
)
[5]
[5]
[5] From function polyPatch::New(const word&, const word&, const label, const label, const label, const polyBoundaryMesh&)
[5] in file meshes/polyMesh/polyPatches/polyPatch/polyPatchNew.C at line 58.
[5]
FOAM parallel run exiting
[5]

[4]
[4]
[4] --> FOAM FATAL ERROR:
[4] Unknown polyPatch type processor for patch procBoundary4to3





Steps To ReproduceTake a decomposed mesh and run mirrormesh in parallel.
TagsNo tags attached.

Activities

user4

2015-03-11 12:13

  ~0004081

This is a known limitation and seems easy enough to fix. The problem is that the patches for the mirrored mesh get created from components instead of being cloned from the old mesh. We will look into this.

wyldckat

2015-11-29 21:00

updater  

mirrorFvMesh.C (12,238 bytes)   
/*---------------------------------------------------------------------------*\
  =========                 |
  \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
   \\    /   O peration     |
    \\  /    A nd           | Copyright (C) 2011-2015 OpenFOAM Foundation
     \\/     M anipulation  |
-------------------------------------------------------------------------------
License
    This file is part of OpenFOAM.

    OpenFOAM is free software: you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
    ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
    FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
    for more details.

    You should have received a copy of the GNU General Public License
    along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.

\*---------------------------------------------------------------------------*/

#include "mirrorFvMesh.H"
#include "Time.H"
#include "plane.H"

// * * * * * * * * * * * * * * * * Constructors  * * * * * * * * * * * * * * //

Foam::mirrorFvMesh::mirrorFvMesh(const IOobject& io)
:
    fvMesh(io),
    mirrorMeshDict_
    (
        IOobject
        (
            "mirrorMeshDict",
            time().system(),
            *this,
            IOobject::MUST_READ_IF_MODIFIED,
            IOobject::NO_WRITE
        )
    ),
    mirrorMeshPtr_(NULL)
{
    plane mirrorPlane(mirrorMeshDict_);

    scalar planeTolerance
    (
        readScalar(mirrorMeshDict_.lookup("planeTolerance"))
    );

    const pointField& oldPoints = points();
    const faceList& oldFaces = faces();
    const cellList& oldCells = cells();
    const label nOldInternalFaces = nInternalFaces();
    const polyPatchList& oldPatches = boundaryMesh();

    // Mirror the points
    Info<< "Mirroring points. Old points: " << oldPoints.size();

    pointField newPoints(2*oldPoints.size());
    label nNewPoints = 0;

    labelList mirrorPointLookup(oldPoints.size(), -1);

    // Grab the old points
    forAll(oldPoints, pointI)
    {
        newPoints[nNewPoints] = oldPoints[pointI];
        nNewPoints++;
    }

    forAll(oldPoints, pointI)
    {
        scalar alpha =
            mirrorPlane.normalIntersect
            (
                oldPoints[pointI],
                mirrorPlane.normal()
            );

        // Check plane on tolerance
        if (mag(alpha) > planeTolerance)
        {
            // The point gets mirrored
            newPoints[nNewPoints] =
                oldPoints[pointI] + 2.0*alpha*mirrorPlane.normal();

            // remember the point correspondence
            mirrorPointLookup[pointI] = nNewPoints;
            nNewPoints++;
        }
        else
        {
            // The point is on the plane and does not get mirrored
            // Adjust plane location
            newPoints[nNewPoints] =
                oldPoints[pointI] + alpha*mirrorPlane.normal();

            mirrorPointLookup[pointI] = pointI;
        }
    }

    // Reset the size of the point list
    Info<< " New points: " << nNewPoints << endl;
    newPoints.setSize(nNewPoints);

    Info<< "Mirroring faces. Old faces: " << oldFaces.size();

    // Algorithm:
    // During mirroring, the faces that were previously boundary faces
    // in the mirror plane may become ineternal faces. In order to
    // deal with the ordering of the faces, the algorithm is split
    // into two parts.  For original faces, the internal faces are
    // distributed to their owner cells.  Once all internal faces are
    // distributed, the boundary faces are visited and if they are in
    // the mirror plane they are added to the master cells (the future
    // boundary faces are not touched).  After the first phase, the
    // internal faces are collected in the cell order and numbering
    // information is added.  Then, the internal faces are mirrored
    // and the face numbering data is stored for the mirrored section.
    // Once all the internal faces are mirrored, the boundary faces
    // are added by mirroring the faces patch by patch.

    // Distribute internal faces
    labelListList newCellFaces(oldCells.size());

    const labelUList& oldOwnerStart = lduAddr().ownerStartAddr();

    forAll(newCellFaces, cellI)
    {
        labelList& curFaces = newCellFaces[cellI];

        const label s = oldOwnerStart[cellI];
        const label e = oldOwnerStart[cellI + 1];

        curFaces.setSize(e - s);

        forAll(curFaces, i)
        {
            curFaces[i] = s + i;
        }
    }

    // Distribute boundary faces.  Remember the faces that have been inserted
    // as internal
    boolListList insertedBouFace(oldPatches.size());

    forAll(oldPatches, patchI)
    {
        const polyPatch& curPatch = oldPatches[patchI];

        if (curPatch.coupled())
        {
            WarningInFunction
                << "Found coupled patch " << curPatch.name() << endl
                << "    Mirroring faces on coupled patches destroys"
                << " the ordering. This might be fixed by running a dummy"
                << " createPatch afterwards." << endl;
        }

        boolList& curInsBouFace = insertedBouFace[patchI];

        curInsBouFace.setSize(curPatch.size());
        curInsBouFace = false;

        // Get faceCells for face insertion
        const labelUList& curFaceCells = curPatch.faceCells();
        const label curStart = curPatch.start();

        forAll(curPatch, faceI)
        {
            // Find out if the mirrored face is identical to the
            // original.  If so, the face needs to become internal and
            // added to its owner cell
            const face& origFace = curPatch[faceI];

            face mirrorFace(origFace.size());
            forAll(mirrorFace, pointI)
            {
                mirrorFace[pointI] = mirrorPointLookup[origFace[pointI]];
            }

            if (origFace == mirrorFace)
            {
                // The mirror is identical to current face.  This will
                // become an internal face
                const label oldSize = newCellFaces[curFaceCells[faceI]].size();

                newCellFaces[curFaceCells[faceI]].setSize(oldSize + 1);
                newCellFaces[curFaceCells[faceI]][oldSize] = curStart + faceI;

                curInsBouFace[faceI] = true;
            }
        }
    }

    // Construct the new list of faces.  Boundary faces are added
    // last, cush that each patch is mirrored separately.  The
    // addressing is stored in two separate arrays: first for the
    // original cells (face order has changed) and then for the
    // mirrored cells.
    labelList masterFaceLookup(oldFaces.size(), -1);
    labelList mirrorFaceLookup(oldFaces.size(), -1);

    faceList newFaces(2*oldFaces.size());
    label nNewFaces = 0;

    // Insert original (internal) faces
    forAll(newCellFaces, cellI)
    {
        const labelList& curCellFaces = newCellFaces[cellI];

        forAll(curCellFaces, cfI)
        {
            newFaces[nNewFaces] = oldFaces[curCellFaces[cfI]];
            masterFaceLookup[curCellFaces[cfI]] = nNewFaces;

            nNewFaces++;
        }
    }

    // Mirror internal faces
    for (label faceI = 0; faceI < nOldInternalFaces; faceI++)
    {
        const face& oldFace = oldFaces[faceI];
        face& nf = newFaces[nNewFaces];
        nf.setSize(oldFace.size());

        nf[0] = mirrorPointLookup[oldFace[0]];

        for (label i = 1; i < oldFace.size(); i++)
        {
            nf[i] = mirrorPointLookup[oldFace[oldFace.size() - i]];
        }

        mirrorFaceLookup[faceI] = nNewFaces;
        nNewFaces++;
    }

    // Mirror boundary faces patch by patch


    labelList newToOldPatch(boundary().size(), -1);
    labelList newPatchSizes(boundary().size(), -1);
    labelList newPatchStarts(boundary().size(), -1);
    label nNewPatches = 0;

    forAll(boundaryMesh(), patchI)
    {
        const label curPatchSize = boundaryMesh()[patchI].size();
        const label curPatchStart = boundaryMesh()[patchI].start();
        const boolList& curInserted = insertedBouFace[patchI];

        newPatchStarts[nNewPatches] = nNewFaces;

        // Master side
        for (label faceI = 0; faceI < curPatchSize; faceI++)
        {
            // Check if the face has already been added.  If not, add it and
            // insert the numbering details.
            if (!curInserted[faceI])
            {
                newFaces[nNewFaces] = oldFaces[curPatchStart + faceI];

                masterFaceLookup[curPatchStart + faceI] = nNewFaces;
                nNewFaces++;
            }
        }

        // Mirror side
        for (label faceI = 0; faceI < curPatchSize; faceI++)
        {
            // Check if the face has already been added.  If not, add it and
            // insert the numbering details.
            if (!curInserted[faceI])
            {
                const face& oldFace = oldFaces[curPatchStart + faceI];
                face& nf = newFaces[nNewFaces];
                nf.setSize(oldFace.size());

                nf[0] = mirrorPointLookup[oldFace[0]];

                for (label i = 1; i < oldFace.size(); i++)
                {
                    nf[i] = mirrorPointLookup[oldFace[oldFace.size() - i]];
                }

                mirrorFaceLookup[curPatchStart + faceI] = nNewFaces;
                nNewFaces++;
            }
            else
            {
                // Grab the index of the master face for the mirror side
                mirrorFaceLookup[curPatchStart + faceI] =
                    masterFaceLookup[curPatchStart + faceI];
            }
        }

        // If patch exists, grab the name and type of the original patch
        if (nNewFaces > newPatchStarts[nNewPatches])
        {
            newToOldPatch[nNewPatches] = patchI;

            newPatchSizes[nNewPatches] =
                nNewFaces - newPatchStarts[nNewPatches];

            nNewPatches++;
        }
    }

    // Tidy up the lists
    newFaces.setSize(nNewFaces);
    Info<< " New faces: " << nNewFaces << endl;

    newToOldPatch.setSize(nNewPatches);
    newPatchSizes.setSize(nNewPatches);
    newPatchStarts.setSize(nNewPatches);

    Info<< "Mirroring patches. Old patches: " << boundary().size()
        << " New patches: " << nNewPatches << endl;

    Info<< "Mirroring cells.  Old cells: " << oldCells.size()
        << " New cells: " << 2*oldCells.size() << endl;

    cellList newCells(2*oldCells.size());
    label nNewCells = 0;

    // Grab the original cells.  Take care of face renumbering.
    forAll(oldCells, cellI)
    {
        const cell& oc = oldCells[cellI];

        cell& nc = newCells[nNewCells];
        nc.setSize(oc.size());

        forAll(oc, i)
        {
            nc[i] = masterFaceLookup[oc[i]];
        }

        nNewCells++;
    }

    // Mirror the cells
    forAll(oldCells, cellI)
    {
        const cell& oc = oldCells[cellI];

        cell& nc = newCells[nNewCells];
        nc.setSize(oc.size());

        forAll(oc, i)
        {
            nc[i] = mirrorFaceLookup[oc[i]];
        }

        nNewCells++;
    }

    // Mirror the cell shapes
    Info<< "Mirroring cell shapes." << endl;

    Info<< nl << "Creating new mesh" << endl;
    mirrorMeshPtr_ = new fvMesh
    (
        io,
        xferMove(newPoints),
        xferMove(newFaces),
        xferMove(newCells)
    );

    fvMesh& pMesh = *mirrorMeshPtr_;

    // Add the boundary patches
    List<polyPatch*> p(newPatchSizes.size());

    forAll(p, patchI)
    {
        p[patchI] = boundaryMesh()[newToOldPatch[patchI]].clone
        (
            pMesh.boundaryMesh(),
            patchI,
            newPatchSizes[patchI],
            newPatchStarts[patchI]
        ).ptr();
    }

    pMesh.addPatches(p);
}


// * * * * * * * * * * * * * * * * Destructor  * * * * * * * * * * * * * * * //

Foam::mirrorFvMesh::~mirrorFvMesh()
{}


// ************************************************************************* //
mirrorFvMesh.C (12,238 bytes)   

wyldckat

2015-11-29 21:02

updater  

elipsekkLOmega.tar.gz (4,698 bytes)

wyldckat

2015-11-29 21:08

updater   ~0005688

Attached are the following files:

  - elipsekkLOmega.tar.gz - It's a modified case of the tutorial "incompressible/pimpleFoam/elipsekkLOmega", adapted for testing the mesh generation with mirrorMesh in serial and in parallel. It uses it twice, so it's both the only and the almost perfect test case.

  - mirrorFvMesh.C - updated file for replacing "applications/utilities/mesh/manipulation/mirrorMesh/mirrorFvMesh.C". This is directly based on Mattijs fix that is present in commit 769abd353ca2e324b513b7c51524d12c481bc1d3 in the OpenFOAM-history repository: https://github.com/OpenCFD/OpenFOAM-history/commit/769abd353ca2e324b513b7c51524d12c481bc1d3

The attached file "mirrorFvMesh.C" is already adapted to OpenFOAM-dev, and possibly can also be used in 3.0.x.

The test was done in serial with the attached case in OpenFOAM 3.0.x without the test fix, and then in OpenFOAM-dev with the fix in serial and parallel.
The resulting "log.checkMesh" files are nearly identical of 3.0.x serial vs dev parallel.
Therefore it does seem that this fixes the reported issue and has been tested with at least this test case.


@matteoL: Any chance you still have your case that lead to the initial report, so that you can test the attached fix as well?

henry

2015-11-29 21:43

manager   ~0005690

Thanks Bruno, I have applied the change to OpenFOAM-dev for evaluation:
commit a24eec402c39fbc69dd546fffa4c0d4ea6aff182

henry

2016-02-03 21:03

manager   ~0005904

No problems with the patch have been reported so I assume this is now resolved.

Thanks for the fix Bruno.

Issue History

Date Modified Username Field Change
2015-01-20 10:29 matteoL New Issue
2015-03-11 12:13 user4 Note Added: 0004081
2015-03-24 00:17 liuhuafei Issue cloned: 0001622
2015-11-29 21:00 wyldckat File Added: mirrorFvMesh.C
2015-11-29 21:02 wyldckat File Added: elipsekkLOmega.tar.gz
2015-11-29 21:08 wyldckat Note Added: 0005688
2015-11-29 21:43 henry Note Added: 0005690
2016-02-03 21:03 henry Note Added: 0005904
2016-02-03 21:03 henry Status new => resolved
2016-02-03 21:03 henry Resolution open => fixed
2016-02-03 21:03 henry Assigned To => henry