View Issue Details

IDProjectCategoryView StatusLast Update
0002442OpenFOAMBugpublic2017-01-26 15:42
Reporterkevnor Assigned Tohenry  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
PlatformUnixOSOtherOS Version(please specify)
Fixed in Versiondev 
Summary0002442: Lagrangian particles outside mesh prevent restart
DescriptionRunning sprayFoam with a tetrahedral mesh (passes all checkMesh -allTopology tests) results in particles that lie a small distance outside of the mesh. These prevents simulation restarts due to a fatal error in determining the cell containing such a particle, e.g.

  --> FOAM FATAL ERROR:
  position (-0.003876213003 -0.006687904921 -0.002035501372)
      for requested cell 41667
      If this is a restart or reconstruction/decomposition etc. it is likely that the write precision is not sufficient.
      Either increase 'writePrecision' or set 'writeFormat' to 'binary'

      From function void Foam::particle::initCellFacePt()
      in file /remote/soft/OpenFOAM/OpenCFD/OpenFOAM-4.x/src/lagrangian/basic/lnInclude/particleI.H at line 718.

This occurs regardless of the 'writePrecision' and 'writeFormat' settings. Additionally, it appears to be due to the mesh type, since running exactly the same case with a hexahedral mesh has not been seen to give any issues.
Steps To ReproduceRun attached case to time 5e-05 using sprayFoam.
Attempt to restart simulation (note that since the particle injection is stochastic it may not always fail, though it has on the majority of test runs I've made.)
TagsLagrangian, sprayFoam

Activities

kevnor

2017-01-25 12:05

reporter  

TCSimpleTet.zip (3,147,579 bytes)

henry

2017-01-25 12:23

manager   ~0007665

Try OpenFOAM-dev.

kevnor

2017-01-25 13:45

reporter   ~0007666

OpenFOAM-dev does indeed appear to resolve the above issue.

However, I see in my initial comparison of the results from 4.x and -dev that the -dev results in striations in the particle distribution that appear to correspond to the cells of the patch used for injection (see the attached plot which shows dev results on top and 4.x on the bottom). These weren't apparent in the 4.x results.
dst_4x_dev.png (1,476,434 bytes)

henry

2017-01-25 14:21

manager   ~0007667

This might relate to the fix for http://bugs.openfoam.org/view.php?id=2286

tniemi

2017-01-25 14:43

reporter   ~0007668

The striations may be caused because after the fix the parcels are forced to stay within the cell they are created. With tetrahedrons some tets only have a corner or edge at the patch and thus those volumes will be empty. Prior the patch it might have happened that the parcels have been pushed into those cells. I'll investigate.

tniemi

2017-01-25 17:06

reporter   ~0007669

Ok, I can confirm that the change between 4.x and dev is because of the fix. If it is disabled, dev behaves the same as 4.x.

The patchInjection perturbs the particles in the direction of the patch normal and with tetrahedral cells this means that a large number of parcels will be pushed into another cell (roughly 30 % in this test case). After the fix these parcels will be relocated to a random position in the original cell, but the randomization is uniform with respect to cell volume. With tetrahedral cells it means that the probability of picking a position is larger at the middle of the cell where there is a larger volume with respect to the patch face.

I can come up with two ideas how to fix this:

1. In patchInjectionBase, if findTetFacePt fails, try findCellFacePt to see if the cell has changed. If also findCellFacePt fails, it means the parcel has been pushed out of the domain or to another processor -> randomize within cell.

2. Modify the randomization code so that the probability is uniform with respect to the area of the face on the patch.

Option 1 introduces some overhead as the cell has to be searched, but otherwise it should work ok. (It might be possible to make the search faster, as the cell has to be a neighbor of the original cell)

For option 2, I'm not sure how it should be done so that it would reliably work with different types of cells.

BTW, what is the reason for the perturbations in the first place? Is to reduce the likelihood of possible overlap with DEM particles or to ensure that the parcels don't immediatly interact with the patch?

henry

2017-01-25 17:24

manager   ~0007670

Last edited: 2017-01-25 17:37

Is the reason for purturbing the particles in the direction of the patch normal to handle the case where the particles are a usiform cloud outside the patch being transported into the domain during the time-step? Is which case the distance into the cell over which the particles are distributed would relate to the tracking time-step and the particle speed.

If this process is to be handled then the particles might not be in the cell adjacent to the patch into which they are injected and might be in neighbouring cells. To handle this correctly the injected particles whould need to be tracked from their injection faces to the cells into which they should reside at the beginning of the time-step.

tniemi

2017-01-26 09:23

reporter  

patchInjectionBase.C (8,425 bytes)   
/*---------------------------------------------------------------------------*\
  =========                 |
  \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
   \\    /   O peration     |
    \\  /    A nd           | Copyright (C) 2013-2016 OpenFOAM Foundation
     \\/     M anipulation  |
-------------------------------------------------------------------------------
License
    This file is part of OpenFOAM.

    OpenFOAM is free software: you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
    ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
    FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
    for more details.

    You should have received a copy of the GNU General Public License
    along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.

\*---------------------------------------------------------------------------*/

#include "patchInjectionBase.H"
#include "polyMesh.H"
#include "SubField.H"
#include "cachedRandom.H"
#include "triPointRef.H"
#include "volFields.H"
#include "polyMeshTetDecomposition.H"

// * * * * * * * * * * * * * * * * Constructors  * * * * * * * * * * * * * * //

Foam::patchInjectionBase::patchInjectionBase
(
    const polyMesh& mesh,
    const word& patchName
)
:
    patchName_(patchName),
    patchId_(mesh.boundaryMesh().findPatchID(patchName_)),
    patchArea_(0.0),
    patchNormal_(),
    cellOwners_(),
    triFace_(),
    triToFace_(),
    triCumulativeMagSf_(),
    sumTriMagSf_(Pstream::nProcs() + 1, 0.0)
{
    if (patchId_ < 0)
    {
        FatalErrorInFunction
            << "Requested patch " << patchName_ << " not found" << nl
            << "Available patches are: " << mesh.boundaryMesh().names() << nl
            << exit(FatalError);
    }

    updateMesh(mesh);
}


Foam::patchInjectionBase::patchInjectionBase(const patchInjectionBase& pib)
:
    patchName_(pib.patchName_),
    patchId_(pib.patchId_),
    patchArea_(pib.patchArea_),
    patchNormal_(pib.patchNormal_),
    cellOwners_(pib.cellOwners_),
    triFace_(pib.triFace_),
    triToFace_(pib.triToFace_),
    triCumulativeMagSf_(pib.triCumulativeMagSf_),
    sumTriMagSf_(pib.sumTriMagSf_)
{}


// * * * * * * * * * * * * * * * * Destructor  * * * * * * * * * * * * * * * //

Foam::patchInjectionBase::~patchInjectionBase()
{}


// * * * * * * * * * * * * * * * Member Functions  * * * * * * * * * * * * * //

void Foam::patchInjectionBase::updateMesh(const polyMesh& mesh)
{
    // Set/cache the injector cells
    const polyPatch& patch = mesh.boundaryMesh()[patchId_];
    const pointField& points = patch.points();

    cellOwners_ = patch.faceCells();

    // Triangulate the patch faces and create addressing
    DynamicList<label> triToFace(2*patch.size());
    DynamicList<scalar> triMagSf(2*patch.size());
    DynamicList<face> triFace(2*patch.size());
    DynamicList<face> tris(5);

    // Set zero value at the start of the tri area list
    triMagSf.append(0.0);

    forAll(patch, facei)
    {
        const face& f = patch[facei];

        tris.clear();
        f.triangles(points, tris);

        forAll(tris, i)
        {
            triToFace.append(facei);
            triFace.append(tris[i]);
            triMagSf.append(tris[i].mag(points));
        }
    }

    forAll(sumTriMagSf_, i)
    {
        sumTriMagSf_[i] = 0.0;
    }

    sumTriMagSf_[Pstream::myProcNo() + 1] = sum(triMagSf);

    Pstream::listCombineGather(sumTriMagSf_, maxEqOp<scalar>());
    Pstream::listCombineScatter(sumTriMagSf_);

    for (label i = 1; i < triMagSf.size(); i++)
    {
        triMagSf[i] += triMagSf[i-1];
    }

    // Transfer to persistent storage
    triFace_.transfer(triFace);
    triToFace_.transfer(triToFace);
    triCumulativeMagSf_.transfer(triMagSf);

    // Convert sumTriMagSf_ into cumulative sum of areas per proc
    for (label i = 1; i < sumTriMagSf_.size(); i++)
    {
        sumTriMagSf_[i] += sumTriMagSf_[i-1];
    }

    const scalarField magSf(mag(patch.faceAreas()));
    patchArea_ = sum(magSf);
    patchNormal_ = patch.faceAreas()/magSf;
    reduce(patchArea_, sumOp<scalar>());
}


void Foam::patchInjectionBase::setPositionAndCell
(
    const fvMesh& mesh,
    cachedRandom& rnd,
    vector& position,
    label& cellOwner,
    label& tetFacei,
    label& tetPti
)
{
    scalar areaFraction = rnd.globalPosition(scalar(0), patchArea_);

    if (cellOwners_.size() > 0)
    {
        // Determine which processor to inject from
        label proci = 0;
        forAllReverse(sumTriMagSf_, i)
        {
            if (areaFraction >= sumTriMagSf_[i])
            {
                proci = i;
                break;
            }
        }

        if (Pstream::myProcNo() == proci)
        {
            // Find corresponding decomposed face triangle
            label trii = 0;
            scalar offset = sumTriMagSf_[proci];
            forAllReverse(triCumulativeMagSf_, i)
            {
                if (areaFraction > triCumulativeMagSf_[i] + offset)
                {
                    trii = i;
                    break;
                }
            }

            // Set cellOwner
            label facei = triToFace_[trii];
            cellOwner = cellOwners_[facei];

            // Find random point in triangle
            const polyPatch& patch = mesh.boundaryMesh()[patchId_];
            const pointField& points = patch.points();
            const face& tf = triFace_[trii];
            const triPointRef tri(points[tf[0]], points[tf[1]], points[tf[2]]);
            const point pf(tri.randomPoint(rnd));

            // Position perturbed away from face (into domain)
            const scalar a = rnd.position(scalar(0.1), scalar(0.5));
            const vector& pc = mesh.cellCentres()[cellOwner];
            const vector d = mag((pf - pc) & patchNormal_[facei])*patchNormal_[facei];

            position = pf - a*d;

            // Try to find tetFacei and tetPti in the current position
            mesh.findTetFacePt(cellOwner, position, tetFacei, tetPti);
            
            // tetFacei and tetPti not found, check if the cell has changed
            if (tetFacei == -1 ||tetPti == -1) 
            {
                mesh.findCellFacePt(position, cellOwner, tetFacei, tetPti);
            }
            
            // Both searches failed, choose a random position within 
            // the original cell
            if (tetFacei == -1 ||tetPti == -1)
            {
                // Reset cellOwner
                cellOwner = cellOwners_[facei];
                const scalarField& V = mesh.V();

                // Construct cell tet indices
                const List<tetIndices> cellTetIs =
                    polyMeshTetDecomposition::cellTetIndices(mesh, cellOwner);

                // Construct cell tet volume fractions
                scalarList cTetVFrac(cellTetIs.size(), 0.0);
                for (label teti=1; teti<cellTetIs.size()-1; teti++)
                {
                    cTetVFrac[teti] =
                        cTetVFrac[teti-1]
                      + cellTetIs[teti].tet(mesh).mag()/V[cellOwner];
                }
                cTetVFrac.last() = 1;

                // Set new particle position
                const scalar volFrac = rnd.sample01<scalar>();
                label teti = 0;
                forAll(cTetVFrac, vfI)
                {
                    if (cTetVFrac[vfI] > volFrac)
                    {
                        teti = vfI;
                        break;
                    }
                }
                position = cellTetIs[teti].tet(mesh).randomPoint(rnd);
                tetFacei = cellTetIs[teti].face();
                tetPti = cellTetIs[teti].tetPt();
            }
        }
        else
        {
            cellOwner = -1;
            tetFacei = -1;
            tetPti = -1;

            // Dummy position
            position = pTraits<vector>::max;
        }
    }
    else
    {
        cellOwner = -1;
        tetFacei = -1;
        tetPti = -1;

        // Dummy position
        position = pTraits<vector>::max;
    }
}


// ************************************************************************* //
patchInjectionBase.C (8,425 bytes)   

tniemi

2017-01-26 09:24

reporter   ~0007671

Ok, but as you stated, the perturbation should be a function of particle velocity and time step, not cell size as it currently is. However, making this change would probably require larger modifications to the injection logic.

Anyway, I have attached a modified patchInjectionBase.C, which applies option 1, ie. allows the parcels to be pushed to another cell. I also modified the perturbation to be relative to the perpendicular distance from the patch face to the cell center. This reduces the likelihood of too long perturbations for flat cells, where the sideways distance can be very long compared to the thickness in the direction of the patch normal.

With these modifications about 20 % of the parcels will be pushed to new cells in the test case, but the findCellFacePt is able to find them most of the time. In my test run I had 1 parcel that needed to be randomized among 40 k injected parcels in serial. In parallel with 12 cores there were about 50 parcels that needed to be randomized, as they probably were pushed across procBoundary.

henry

2017-01-26 09:32

manager   ~0007672

> Ok, but as you stated, the perturbation should be a function of particle velocity and time step, not cell size as it currently is. However, making this change would probably require larger modifications to the injection logic.

Yes, I think it needs a bit more thought and a rewrite. We need to remove the need to arbitrarily move particles from the tracked location to somewhere else; all particle movement should be handled by the tracking only.

> ie. allows the parcels to be pushed to another cell.

How do you handle the case when the other cell is on another processor?

tniemi

2017-01-26 09:35

reporter   ~0007673

> How do you handle the case when the other cell is on another processor?

In the same way as before, ie. randomize within the original cell. I would estimate that in a typical realistic case the amount of such parcels should be very low.

henry

2017-01-26 09:55

manager   ~0007674

Last edited: 2017-01-26 10:04

> In the same way as before

My understanding is that moving the particles into cells on other processors was not handled before.

> With these modifications about 20 % of the parcels will be pushed to new cells in the test case

Does this include the possibility that the particles can be pushed to new cells on other processors? This would introduce some complicated parallel communications logic.

> as they probably were pushed across procBoundary

Are these particles removed or moved to a different cell on the current processor or put somewhere else is the current cell?

tniemi

2017-01-26 10:44

reporter   ~0007675

> My understanding is that moving the particles into cells on other processors was not handled before.

It is in such a way that these particles are re-positioned to a randomized location in the original cell. This of course will cause a slight bias as these parcels are randomized with respect to cell volume. This is not ideal, but at least the parcels are not lost.

> Does this include the possibility that the particles can be pushed to new cells on other processors? This would introduce some complicated parallel communications logic

Yes, the 20 % was taken from a serial test run where I counted the amount of times when findTetFacePt failed. Among that number might also be some other fails within the original cell. This amount largely depends on the perturbation length. If it is decreased, the probability will drop.


> Are these particles removed or moved to a different cell on the current processor or put somewhere else is the current cell?

They are put to the original cell to a randomized position on the current processor.

> This would introduce some complicated parallel communications logic.

Yes, I couldn't come up with a clean procedure how to handle the processor transfer without several syncs and such.

tniemi

2017-01-26 10:48

reporter   ~0007676

To add, something like this could be done to handle processor transfer

1. Determine the owner of the original cell
2. Owner perturbs the parcel and tries to search for it, the other processors wait
3. If the owner was successful, continue, if not, all procs search for the parcel
4. If someone found it, continue, if not randomize within the original cell

henry

2017-01-26 10:57

manager   ~0007677

Last edited: 2017-01-26 13:56

I think the patch you have provided is sufficient for now.

In the longer-term the patch injection should be rewritten to track the particles from the injection patch different distances over the injection time-step rather than inserting them directly into the cells which introduces serious issues for DEM apart from being inaccurate.

tniemi

2017-01-26 13:41

reporter   ~0007678

Last edited: 2017-01-26 13:56

> In the longer-term the patch injection should be rewritten to track the particles from the injection patch different distances over the injection time-step rather than inserting them directly into the cells which introduces serious issues for DEM apart from being inaccurate.

Yes, I agree. Also in general for DEM particles, as far as I know, no injection performs checks for possible overlap. Eg. in cellZoneInjection a random location is chosen, but no checks are made to ensure that there is sufficient space.

henry

2017-01-26 15:42

manager   ~0007679

Thanks for the patch Timo

Resolved by commit ac28d44eff27595f150b562b5c23bb9e610a4ce1

Issue History

Date Modified Username Field Change
2017-01-25 12:05 kevnor New Issue
2017-01-25 12:05 kevnor File Added: TCSimpleTet.zip
2017-01-25 12:05 kevnor Tag Attached: Lagrangian
2017-01-25 12:05 kevnor Tag Attached: sprayFoam
2017-01-25 12:23 henry Note Added: 0007665
2017-01-25 13:45 kevnor File Added: dst_4x_dev.png
2017-01-25 13:45 kevnor Note Added: 0007666
2017-01-25 14:21 henry Note Added: 0007667
2017-01-25 14:43 tniemi Note Added: 0007668
2017-01-25 17:06 tniemi Note Added: 0007669
2017-01-25 17:24 henry Note Added: 0007670
2017-01-25 17:37 henry Note Edited: 0007670
2017-01-26 09:23 tniemi File Added: patchInjectionBase.C
2017-01-26 09:24 tniemi Note Added: 0007671
2017-01-26 09:32 henry Note Added: 0007672
2017-01-26 09:35 tniemi Note Added: 0007673
2017-01-26 09:55 henry Note Added: 0007674
2017-01-26 10:04 henry Note Edited: 0007674
2017-01-26 10:44 tniemi Note Added: 0007675
2017-01-26 10:48 tniemi Note Added: 0007676
2017-01-26 10:57 henry Note Added: 0007677
2017-01-26 13:41 tniemi Note Added: 0007678
2017-01-26 13:56 henry Note Edited: 0007677
2017-01-26 13:56 henry Note Edited: 0007678
2017-01-26 15:42 henry Assigned To => henry
2017-01-26 15:42 henry Status new => resolved
2017-01-26 15:42 henry Resolution open => fixed
2017-01-26 15:42 henry Fixed in Version => dev
2017-01-26 15:42 henry Note Added: 0007679