View Issue Details

IDProjectCategoryView StatusLast Update
0003053OpenFOAMPatchpublic2018-09-03 18:44
ReporterJuho Assigned Tohenry  
PrioritylowSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
PlatformUnixOSOtherOS Version(please specify)
Product Versiondev 
Fixed in Versiondev 
Summary0003053: Function1: CSV reader enhancements
DescriptionI've attached modified CSV.C and CSV.H with two changes:

1. Allow reference column index to be higher than highest component column index. The previous version did not work with such setup. Change on line 115 in the attached CSV.C:
- label nEntries = max(componentColumns_);
+ label nEntries = max(refColumn_,max(componentColumns_));

2. Added optional scaling coefficients for the reference and component values. Default is 1. Possible use cases:
- Easy adjustment of function frequency or amplitude
- Unit conversions, for ex. when reading material property tables
TagsNo tags attached.

Activities

Juho

2018-08-24 12:56

reporter  

CSV.C (8,338 bytes)   
/*---------------------------------------------------------------------------*\
  =========                 |
  \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
   \\    /   O peration     | Website:  https://openfoam.org
    \\  /    A nd           | Copyright (C) 2011-2018 OpenFOAM Foundation
     \\/     M anipulation  |
-------------------------------------------------------------------------------
License
    This file is part of OpenFOAM.

    OpenFOAM is free software: you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
    ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
    FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
    for more details.

    You should have received a copy of the GNU General Public License
    along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.

\*---------------------------------------------------------------------------*/

#include "CSV.H"
#include "DynamicList.H"
//#include "IFstream.H"

// * * * * * * * * * * * * * Private Member Functions  * * * * * * * * * * * //

template<>
Foam::label Foam::Function1Types::CSV<Foam::label>::readValue
(
    const List<string>& split
)
{
    if (componentColumns_[0] >= split.size())
    {
        FatalErrorInFunction
            << "No column " << componentColumns_[0] << " in "
            << split << endl
            << exit(FatalError);
    }

    return readLabel(IStringStream(split[componentColumns_[0]])());
}


template<>
Foam::scalar Foam::Function1Types::CSV<Foam::scalar>::readValue
(
    const List<string>& split
)
{
    if (componentColumns_[0] >= split.size())
    {
        FatalErrorInFunction
            << "No column " << componentColumns_[0] << " in "
            << split << endl
            << exit(FatalError);
    }

    return readScalar(IStringStream(split[componentColumns_[0]])());
}


template<class Type>
Type Foam::Function1Types::CSV<Type>::readValue(const List<string>& split)
{
    Type result;

    for (label i = 0; i < pTraits<Type>::nComponents; i++)
    {
        if (componentColumns_[i] >= split.size())
        {
            FatalErrorInFunction
            << "No column " << componentColumns_[i] << " in "
                << split << endl
                << exit(FatalError);
        }

        result[i] =
            readScalar(IStringStream(split[componentColumns_[i]])());
    }

    return result;
}


template<class Type>
void Foam::Function1Types::CSV<Type>::read()
{
    fileName expandedFile(fName_);
    // IFstream is(expandedFile.expand());
    autoPtr<ISstream> isPtr(fileHandler().NewIFstream(expandedFile.expand()));
    ISstream& is = isPtr();

    if (!is.good())
    {
        FatalIOErrorInFunction(is)
            << "Cannot open CSV file for reading."
            << exit(FatalIOError);
    }

    DynamicList<Tuple2<scalar, Type>> values;

    // skip header
    for (label i = 0; i < nHeaderLine_; i++)
    {
        string line;
        is.getLine(line);
    }

    label nEntries = max(refColumn_,max(componentColumns_));

    // read data
    while (is.good())
    {
        string line;
        is.getLine(line);


        label n = 0;
        std::size_t pos = 0;
        DynamicList<string> split;

        if (mergeSeparators_)
        {
            std::size_t nPos = 0;

            while ((pos != std::string::npos) && (n <= nEntries))
            {
                bool found = false;
                while (!found)
                {
                    nPos = line.find(separator_, pos);

                    if ((nPos != std::string::npos) && (nPos - pos == 0))
                    {
                        pos = nPos + 1;
                    }
                    else
                    {
                        found = true;
                    }
                }

                nPos = line.find(separator_, pos);

                if (nPos == std::string::npos)
                {
                    split.append(line.substr(pos));
                    pos = nPos;
                    n++;
                }
                else
                {
                    split.append(line.substr(pos, nPos - pos));
                    pos = nPos + 1;
                    n++;
                }
            }
        }
        else
        {
            while ((pos != std::string::npos) && (n <= nEntries))
            {
                std::size_t nPos = line.find(separator_, pos);

                if (nPos == std::string::npos)
                {
                    split.append(line.substr(pos));
                    pos = nPos;
                    n++;
                }
                else
                {
                    split.append(line.substr(pos, nPos - pos));
                    pos = nPos + 1;
                    n++;
                }
            }
        }


        if (split.size() <= 1)
        {
            break;
        }

        scalar x = referenceCoeff_*readScalar(IStringStream(split[refColumn_])());
        Type value = componentCoeff_*readValue(split);

        values.append(Tuple2<scalar,Type>(x, value));
    }

    this->table_.transfer(values);
}


// * * * * * * * * * * * * * * * * Constructors  * * * * * * * * * * * * * * //

template<class Type>
Foam::Function1Types::CSV<Type>::CSV
(
    const word& entryName,
    const dictionary& dict
)
:
    TableBase<Type>(entryName, dict),
    nHeaderLine_(readLabel(dict.lookup("nHeaderLine"))),
    refColumn_(readLabel(dict.lookup("refColumn"))),
    referenceCoeff_(dict.lookupOrDefault<scalar>("referenceCoeff", 1)),
    componentColumns_(dict.lookup("componentColumns")),
    componentCoeff_(dict.lookupOrDefault<scalar>("componentCoeff", 1)),
    separator_(dict.lookupOrDefault<string>("separator", string(","))[0]),
    mergeSeparators_(readBool(dict.lookup("mergeSeparators"))),
    fName_(dict.lookup("file"))
{
    if (componentColumns_.size() != pTraits<Type>::nComponents)
    {
        FatalErrorInFunction
            << componentColumns_ << " does not have the expected length of "
            << pTraits<Type>::nComponents << endl
            << exit(FatalError);
    }
    read();

    TableBase<Type>::check();
}


template<class Type>
Foam::Function1Types::CSV<Type>::CSV(const CSV<Type>& tbl)
:
    TableBase<Type>(tbl),
    nHeaderLine_(tbl.nHeaderLine_),
    refColumn_(tbl.refColumn_),
    componentColumns_(tbl.componentColumns_),
    separator_(tbl.separator_),
    mergeSeparators_(tbl.mergeSeparators_),
    fName_(tbl.fName_)
{}


// * * * * * * * * * * * * * * * * Destructor  * * * * * * * * * * * * * * * //

template<class Type>
Foam::Function1Types::CSV<Type>::~CSV()
{}


// * * * * * * * * * * * * * * * Member Functions  * * * * * * * * * * * * * //

template<class Type>
const Foam::fileName& Foam::Function1Types::CSV<Type>::fName() const
{
    return fName_;
}


template<class Type>
void Foam::Function1Types::CSV<Type>::writeData(Ostream& os) const
{
    Function1<Type>::writeData(os);
    os  << token::END_STATEMENT << nl;
    os  << indent << word(this->name() + "Coeffs") << nl;
    os  << indent << token::BEGIN_BLOCK << incrIndent << nl;

    // Note: for TableBase write the dictionary entries it needs but not
    // the values themselves
    TableBase<Type>::writeEntries(os);

    os.writeKeyword("nHeaderLine") << nHeaderLine_ << token::END_STATEMENT
        << nl;

    os.writeKeyword("refColumn") << refColumn_ << token::END_STATEMENT << nl;
    os.writeKeyword("referenceCoeff") << referenceCoeff_ << token::END_STATEMENT << nl;

    componentColumns_.writeEntry("componentColumns", os);
    os.writeKeyword("componentCoeff") << componentCoeff_
        << token::END_STATEMENT << nl;

    os.writeKeyword("separator") << string(separator_)
        << token::END_STATEMENT << nl;
    os.writeKeyword("mergeSeparators") << mergeSeparators_
        << token::END_STATEMENT << nl;
    os.writeKeyword("file") << fName_ << token::END_STATEMENT << nl;
    os  << decrIndent << indent << token::END_BLOCK << endl;
}


// ************************************************************************* //
CSV.C (8,338 bytes)   
CSV.H (4,802 bytes)   
/*---------------------------------------------------------------------------*\
  =========                 |
  \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
   \\    /   O peration     | Website:  https://openfoam.org
    \\  /    A nd           | Copyright (C) 2011-2018 OpenFOAM Foundation
     \\/     M anipulation  |
-------------------------------------------------------------------------------
License
    This file is part of OpenFOAM.

    OpenFOAM is free software: you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
    ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
    FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
    for more details.

    You should have received a copy of the GNU General Public License
    along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.

Class
    Foam::Function1Types::CSV

Description
    Templated CSV function.

    Reference column is always a scalar, e.g. time.

    Usage:
    \verbatim
        <entryName> csvFile;
        <entryName>Coeffs
        {
            nHeaderLine         4;          // number of header lines
            refColumn           0;          // reference column index
            referenceCoeff      1;          // reference scaling coefficient
            componentColumns    (1 2 3);    // component column indices
            componentCoeff      1;          // component scaling coefficient
            separator           ",";        // optional (defaults to ",")
            mergeSeparators     no;         // merge multiple separators
            file                "fileXYZ";  // name of csv data file
            outOfBounds         clamp;      // optional out-of-bounds handling
            interpolationScheme linear;     // optional interpolation scheme
        }
    \endverbatim

SourceFiles
    CSV.C

\*---------------------------------------------------------------------------*/

#ifndef CSV_H
#define CSV_H

#include "Function1.H"
#include "TableBase.H"
#include "Tuple2.H"
#include "labelList.H"
#include "ISstream.H"

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

namespace Foam
{
namespace Function1Types
{

/*---------------------------------------------------------------------------*\
                           Class CSV Declaration
\*---------------------------------------------------------------------------*/

template<class Type>
class CSV
:
    public TableBase<Type>
{
    // Private data

        //- Number header lines
        label nHeaderLine_;

        //- Column of the reference values
        label refColumn_;

        //- Optional scaling coefficient for the reference values
        scalar referenceCoeff_;

        //- Labels of the components
        labelList componentColumns_;

        //- Optional scaling coefficient for the components
        scalar componentCoeff_;

        //- Separator character
        char separator_;

        //- Merge separators flag, e.g. ',,,' becomes ','
        bool mergeSeparators_;

        //- File name for csv table
        fileName fName_;


    // Private Member Functions

        //- Read csv data table
        void read();

        //- Read the next value from the split string
        Type readValue(const List<string>&);

        //- Disallow default bitwise assignment
        void operator=(const CSV<Type>&);


public:

    //- Runtime type information
    TypeName("csvFile");


    // Constructors

        //- Construct from entry name and dictionary
        CSV
        (
            const word& entryName,
            const dictionary& dict
        );

        //- Copy constructor
        CSV(const CSV<Type>& tbl);


    //- Destructor
    virtual ~CSV();


    // Member Functions

        //- Return const access to the file name
        virtual const fileName& fName() const;

        //- Write in dictionary format
        virtual void writeData(Ostream& os) const;
};


template<>
label CSV<label>::readValue(const List<string>& split);

template<>
Foam::scalar CSV<scalar>::readValue(const List<string>& split);


// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

} // End namespace Function1Types
} // End namespace Foam

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

#ifdef NoRepository
    #include "CSV.C"
#endif

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

#endif

// ************************************************************************* //
CSV.H (4,802 bytes)   

henry

2018-08-31 16:25

manager   ~0010019

I am concerned that you have added scaling functionality directly into the CSV class rather than providing it in a more general way; surely this would be useful to apply to any Function1? Or is there a reason why it should only be applied to CSV?

Note that there is already a "scale" Function1: Function1/Scale/Scale.H
However this only scales the result of the function rather than "x", perhaps we should extend "scale" or provide an additional Function1 which scales "x" so that it can be applied to any Function1.

henry

2018-08-31 16:50

manager   ~0010020

I have applied the first part of the patch:

commit 90c74d8c7c6959dd6e7dc00f9dce69e6acf0d116
    Function1/CSV: Allow reference column index to be higher than highest component column index

Juho

2018-08-31 17:40

reporter   ~0010022

You are correct, "scale" provides the functionality for value and thus the scaling coefficient for the result is unnecessary. I was mainly interested in scaling "x" and just added it also for the result while I was there without recognizing that there already was a more general approach for the result. Like you propose, extending "scale" to the "x" would be a more general solution.

Juho

2018-08-31 19:19

reporter   ~0010023

Although, perhaps multiplication by constant (i.e. unit conversion) is common enough task when reading tables that it would be worthwhile to include it directly to the table readers (TableBase). User interface would be much more simple than when using "scale" and the generality of the "scale" implementation is probably not needed in most cases. Scaling of the "x" is also not that relevant for many of the other function as they already include parameters to that effect.

henry

2018-08-31 20:17

manager   ~0010024

If we include scaling an one of the Function1 implementations we should include it in all of them to provide a consistent interface. I am not clean on the clutter and maintenance overhead this introduces so would rather keep the scaling operation separate.

Adding "x" scaling to "scale" or provide it in a separate Function1 should not be a big problem and then it can be used in conjunction with any of the current or future Function1 implementations; I don't see why it should be limited to tabulated data.

henry

2018-09-03 18:44

manager   ~0010036

Resolved by commit 3fc52f94d77b513927ab896f4465105db4223b09

Issue History

Date Modified Username Field Change
2018-08-24 12:56 Juho New Issue
2018-08-24 12:56 Juho File Added: CSV.C
2018-08-24 12:56 Juho File Added: CSV.H
2018-08-30 10:25 wyldckat Assigned To => henry
2018-08-30 10:25 wyldckat Status new => assigned
2018-08-31 16:25 henry Note Added: 0010019
2018-08-31 16:50 henry Note Added: 0010020
2018-08-31 17:40 Juho Note Added: 0010022
2018-08-31 19:19 Juho Note Added: 0010023
2018-08-31 20:17 henry Note Added: 0010024
2018-09-03 18:44 henry Status assigned => resolved
2018-09-03 18:44 henry Resolution open => fixed
2018-09-03 18:44 henry Fixed in Version => dev
2018-09-03 18:44 henry Note Added: 0010036