View Issue Details

IDProjectCategoryView StatusLast Update
0002111OpenFOAMBugpublic2016-11-22 02:14
Reporterguin Assigned Tohenry  
PrioritynormalSeveritymajorReproducibilitysometimes
Status resolvedResolutionfixed 
PlatformGNU/LinuxOSUbuntuOS Version14.04
Product Versiondev 
Fixed in Versiondev 
Summary0002111: DPMFoam hangs in parallel when using patchFlowRateInjection model.
DescriptionThe case hangs in parallel runs when using DPMFoam combined with patchFlowRateInjection.

More accurately, the hang happens when calling returnReduce(parcelsAdded, sumOp<label>()) from within the function postInjectCheck(), in the file:

/src/lagrangian/intermediate/submodels/Kinematic/InjectionModel/InjectionModel/InjectionModel.C
Steps To ReproduceUsing the tutorial case MPPICFoam/injectionCase:

0. File 0/U.air: Corrected Y-component of velocity in upperInlet patch with a "minus" sign (see reported bug with ID 0002110).

1. Decompose in two subdomains using simple method with:
simpleCoeffs
{
    n (1 2 1);
    delta 0.001;
}

2. Modify constant/kinematicCloudProperties:
 2.1.Replace the injection type in both injection models from "patchInjection" to "patchFlowRateInjection":
        model1
        {
// type patchInjection;
// parcelsPerSecond 1390885;
            //-----------------------------------------------------------------
                type patchFlowRateInjection;
                phi phi.air;
                rho rho.air;
                concentration 1;
                parcelConcentration 7e8; // it leads to a similar inlet rate
            //-----------------------------------------------------------------

            [...]
        }

        model2
        {
// type patchInjection;
// parcelsPerSecond 1390885;
            //-----------------------------------------------------------------
                type patchFlowRateInjection;
                phi phi.air;
                rho rho.air;
                concentration 1;
                parcelConcentration 7e8; // it leads to a similar inlet rate
            //-----------------------------------------------------------------

            [...]
        }

 2.2. Add necessary entries required by DPMFoam:
constantProperties
{
    [...]

    // For DPMFoam (taken from DPMFoam/Goldschmidt)
    //-------------------------------------------------------------------------
        poissonsRatio 0.35;
        youngsModulus 1e5; //1e8;
    //-------------------------------------------------------------------------
}
subModels
{
    [...]

    collisionModel none; // For DPMFoam
}

3. Run the simulation in parallel using DPMFoam
Additional Information - The issue only happens in parallel runs.

 - MPPICFoam does not face such problem, neither in parallel nor with single-process simulations.

 - The issue appears to be affecting only the injector "model2", which gets assigned to the processor 0.

 - Last message from logfile is "Solving 3-D cloud kinematicCloud".
TagsNo tags attached.

Activities

guin

2016-06-06 12:05

reporter  

injectionChannel.tar.gz (396,317 bytes)

guin

2016-06-06 12:19

reporter   ~0006378

The issue is not limited to this tutorial case. I experienced similar hangs with other cases (2D- as well as 3D-ones).

tniemi

2016-06-07 08:59

reporter   ~0006399

I found the reason for this problem. In function parcelsToInject at PatchFlowRateInjection.C, the code calculates a random number to check if an additional parcel should be injected. However, each processor can create a different random number and thus some processors may think that a parcel should be injected while others do not. This creates a hang up in the injection loop in setPositionAndCell.

The solution would be to calculate a global probability for additional injection in parcelsToInject function.

tniemi

2016-06-07 09:08

reporter  

PatchFlowRateInjection.C (7,030 bytes)   
/*---------------------------------------------------------------------------*\
  =========                 |
  \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
   \\    /   O peration     |
    \\  /    A nd           | Copyright (C) 2011-2016 OpenFOAM Foundation
     \\/     M anipulation  |
-------------------------------------------------------------------------------
License
    This file is part of OpenFOAM.

    OpenFOAM is free software: you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
    ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
    FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
    for more details.

    You should have received a copy of the GNU General Public License
    along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.

\*---------------------------------------------------------------------------*/

#include "PatchFlowRateInjection.H"
#include "TimeFunction1.H"
#include "distributionModel.H"
#include "mathematicalConstants.H"
#include "surfaceFields.H"

// * * * * * * * * * * * * * * * * Constructors  * * * * * * * * * * * * * * //

template<class CloudType>
Foam::PatchFlowRateInjection<CloudType>::PatchFlowRateInjection
(
    const dictionary& dict,
    CloudType& owner,
    const word& modelName
)
:
    InjectionModel<CloudType>(dict, owner, modelName,typeName),
    patchInjectionBase(owner.mesh(), this->coeffDict().lookup("patchName")),
    phiName_(this->coeffDict().template lookupOrDefault<word>("phi", "phi")),
    rhoName_(this->coeffDict().template lookupOrDefault<word>("rho", "rho")),
    duration_(readScalar(this->coeffDict().lookup("duration"))),
    concentration_
    (
        TimeFunction1<scalar>
        (
            owner.db().time(),
            "concentration",
            this->coeffDict()
        )
    ),
    parcelConcentration_
    (
        readScalar(this->coeffDict().lookup("parcelConcentration"))
    ),
    sizeDistribution_
    (
        distributionModels::distributionModel::New
        (
            this->coeffDict().subDict("sizeDistribution"),
            owner.rndGen()
        )
    )
{
    duration_ = owner.db().time().userTimeToTime(duration_);

    patchInjectionBase::updateMesh(owner.mesh());

    // Re-initialise total mass/volume to inject to zero
    // - will be reset during each injection
    this->volumeTotal_ = 0.0;
    this->massTotal_ = 0.0;
}


template<class CloudType>
Foam::PatchFlowRateInjection<CloudType>::PatchFlowRateInjection
(
    const PatchFlowRateInjection<CloudType>& im
)
:
    InjectionModel<CloudType>(im),
    patchInjectionBase(im),
    phiName_(im.phiName_),
    rhoName_(im.rhoName_),
    duration_(im.duration_),
    concentration_(im.concentration_),
    parcelConcentration_(im.parcelConcentration_),
    sizeDistribution_(im.sizeDistribution_().clone().ptr())
{}


// * * * * * * * * * * * * * * * * Destructor  * * * * * * * * * * * * * * * //

template<class CloudType>
Foam::PatchFlowRateInjection<CloudType>::~PatchFlowRateInjection()
{}


// * * * * * * * * * * * * * * * Member Functions  * * * * * * * * * * * * * //

template<class CloudType>
void Foam::PatchFlowRateInjection<CloudType>::updateMesh()
{
    patchInjectionBase::updateMesh(this->owner().mesh());
}


template<class CloudType>
Foam::scalar Foam::PatchFlowRateInjection<CloudType>::timeEnd() const
{
    return this->SOI_ + duration_;
}


template<class CloudType>
Foam::scalar Foam::PatchFlowRateInjection<CloudType>::flowRate() const
{
   const polyMesh& mesh = this->owner().mesh();

    const surfaceScalarField& phi =
        mesh.lookupObject<surfaceScalarField>(phiName_);

    const scalarField& phip = phi.boundaryField()[patchId_];

    scalar flowRateIn = 0.0;
    if (phi.dimensions() == dimVelocity*dimArea)
    {
        flowRateIn = max(0.0, -sum(phip));
    }
    else
    {
        const volScalarField& rho =
            mesh.lookupObject<volScalarField>(rhoName_);
        const scalarField& rhop = rho.boundaryField()[patchId_];

        flowRateIn = max(0.0, -sum(phip/rhop));
    }

    reduce(flowRateIn, sumOp<scalar>());

    return flowRateIn;
}


template<class CloudType>
Foam::label Foam::PatchFlowRateInjection<CloudType>::parcelsToInject
(
    const scalar time0,
    const scalar time1
)
{
    if ((time0 >= 0.0) && (time0 < duration_))
    {
        scalar dt = time1 - time0;

        scalar c = concentration_.value(0.5*(time0 + time1));

        scalar nParcels = parcelConcentration_*c*flowRate()*dt;
        label nParcelsToInject = floor(nParcels);

        // Inject an additional parcel with a probability based on the
        // remainder after the floor function
        scalar injectProb = 0;

        if (Pstream::master())
        {
            injectProb = this->owner().rndGen().position(scalar(0), scalar(1));
        }

        Pstream::scatter(injectProb);
        
        if
        (
            nParcelsToInject > 0
         && (
               nParcels - scalar(nParcelsToInject)
             > injectProb
            )
        )
        {
            ++nParcelsToInject;
        }

        return nParcelsToInject;
    }
    else
    {
        return 0;
    }
}


template<class CloudType>
Foam::scalar Foam::PatchFlowRateInjection<CloudType>::volumeToInject
(
    const scalar time0,
    const scalar time1
)
{
    scalar volume = 0.0;

    if ((time0 >= 0.0) && (time0 < duration_))
    {
        scalar c = concentration_.value(0.5*(time0 + time1));

        volume = c*(time1 - time0)*flowRate();
    }

    this->volumeTotal_ = volume;
    this->massTotal_ = volume*this->owner().constProps().rho0();

    return volume;
}


template<class CloudType>
void Foam::PatchFlowRateInjection<CloudType>::setPositionAndCell
(
    const label,
    const label,
    const scalar,
    vector& position,
    label& cellOwner,
    label& tetFacei,
    label& tetPtI
)
{
    patchInjectionBase::setPositionAndCell
    (
        this->owner().mesh(),
        this->owner().rndGen(),
        position,
        cellOwner,
        tetFacei,
        tetPtI
    );
}


template<class CloudType>
void Foam::PatchFlowRateInjection<CloudType>::setProperties
(
    const label,
    const label,
    const scalar,
    typename CloudType::parcelType& parcel
)
{
    // set particle velocity to carrier velocity
    parcel.U() = this->owner().U()[parcel.cell()];

    // set particle diameter
    parcel.d() = sizeDistribution_->sample();
}


template<class CloudType>
bool Foam::PatchFlowRateInjection<CloudType>::fullyDescribed() const
{
    return false;
}


template<class CloudType>
bool Foam::PatchFlowRateInjection<CloudType>::validInjection(const label)
{
    return true;
}


// ************************************************************************* //
PatchFlowRateInjection.C (7,030 bytes)   

tniemi

2016-06-07 09:12

reporter   ~0006400

Last edited: 2016-06-07 09:15

I uploaded a modified PatchFlowRateInjection.C which uses global randoms. With this the test case won't hang.

Edit: actually a more elegant fix would be to just replace rndGen().position with rndGen().globalPosition

henry

2016-06-07 09:26

manager   ~0006401

Yes, I think making PatchFlowRateInjection consistent with PatchInjection (i.e. using rnd.globalPosition is the best approach. I have implemented this and testing, will push shortly.

tniemi

2016-06-07 09:34

reporter   ~0006402

Yes, I didn't realize that PatchInjection already has the same approach, ie. use globalPosition.

Related to this, the Pstream::master/scatter code in patchInjectionBase::setPositionAndCell could be changed to use globalPosition instead to make it cleaner.

henry

2016-06-07 09:40

manager   ~0006403

OK, I will make that change as well, test and push them both.

henry

2016-06-07 10:59

manager   ~0006404

Thanks for reporting and providing a simple method to reproduce the problem.

Resolved by commit 0e802bc2899c3a033e36e73c2206c91dea90f572

Issue History

Date Modified Username Field Change
2016-06-06 12:05 guin New Issue
2016-06-06 12:05 guin File Added: injectionChannel.tar.gz
2016-06-06 12:19 guin Note Added: 0006378
2016-06-07 08:59 tniemi Note Added: 0006399
2016-06-07 09:08 tniemi File Added: PatchFlowRateInjection.C
2016-06-07 09:12 tniemi Note Added: 0006400
2016-06-07 09:15 tniemi Note Edited: 0006400
2016-06-07 09:26 henry Note Added: 0006401
2016-06-07 09:34 tniemi Note Added: 0006402
2016-06-07 09:40 henry Note Added: 0006403
2016-06-07 10:59 henry Note Added: 0006404
2016-06-07 10:59 henry Status new => resolved
2016-06-07 10:59 henry Fixed in Version => dev
2016-06-07 10:59 henry Resolution open => fixed
2016-06-07 10:59 henry Assigned To => henry