View Issue Details

IDProjectCategoryView StatusLast Update
0002040ThirdParty[All Projects] Bugpublic2016-04-03 15:26
ReporterwyldckatAssigned Tohenry 
PrioritynormalSeveritytweakReproducibilitysometimes
Status resolvedResolutionfixed 
Product Version 
Fixed in Version 
Summary0002040: The default "stop on 1st error" feature wasn't working within ThirdParty
DescriptionWhen the implementation was done for "wmake/scripts/AllwmakeParseArguments" in OpenFOAM-dev, the "ThirdParty-*/Allwmake" script was not updated accordingly. In particular, I completely forgot about this in issue #1485: http://www.openfoam.org/mantisbt/view.php?id=1485

Attached is the package "ThirdParty-stopOnError.tar.gz" which provides the following files:

  - Allwmake
    - Check if the "Makefile" exists before trying to clean up the build environment in Open-MPI and MPICH2. Otherwise it would trigger the "stop on first error".

  - AllwmakeLibccmio
    - small update to the description, since it doesn't download the package.

  - etc/tools/ThirdPartyFunctions
    - It now has the same check for "WM_CONTINUE_ON_ERROR" that "AllwmakeParseArguments" has got.
    - Switched to using upper-case letters at the start of the phrases in the comments, for consistency with other scripts.
Additional InformationI'm still looking into some related issues in "OpenFOAM-dev", because when an application is built by relying on Allwmake, it's not triggering a "stop on first error" when running in parallel.
TagsNo tags attached.

Activities

wyldckat

2016-04-03 10:03

updater  

ThirdParty-stopOnError.tar.gz (4,198 bytes)

henry

2016-04-03 10:32

manager   ~0006072

Thanks Bruno

Resolved in OpenFOAM-dev by commit 1379e08a39ef9e199307e9b6608e7ad97ed03b77

wyldckat

2016-04-03 10:41

updater   ~0006073

I was hoping you hadn't already merged it :)

Attached is an update for "makeCGAL", because the current implementation is breaking hard when "cgal_version=cgal-system".
The attached file fixes that, by skipping the build if CGAL and announce that the system one will be used. It still allows for Boost to be built, given that Boost might not be the system one, even if it looks strange...

wyldckat

2016-04-03 10:41

updater  

makeCGAL (8,471 bytes)
#!/bin/sh
#------------------------------------------------------------------------------
# =========                 |
# \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
#  \\    /   O peration     |
#   \\  /    A nd           | Copyright (C) 2012-2016 OpenFOAM Foundation
#    \\/     M anipulation  |
#------------------------------------------------------------------------------
# License
#     This file is part of OpenFOAM.
#
#     OpenFOAM is free software: you can redistribute it and/or modify it
#     under the terms of the GNU General Public License as published by
#     the Free Software Foundation, either version 3 of the License, or
#     (at your option) any later version.
#
#     OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
#     ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
#     FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
#     for more details.
#
#     You should have received a copy of the GNU General Public License
#     along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.
#
# Script
#     makeCGAL
#
# Description
#     Build script for CGAL
#
# Note
#     Normally builds against ThirdParty boost and gmp/mpfr when possible.
#     To override this behaviour (and use the system boost and/or gmp/mpfr),
#     simply specify a 'system' version. For example,
#         makeCGAL boost-system gmp-system
#
#     Mixing system and ThirdParty for gmp/mpfr is not supported.
#
#------------------------------------------------------------------------------

# Get CGAL, boost and gmp/mpfr versions
. $WM_PROJECT_DIR/etc/config.sh/functions
_foamEval SOURCE_CGAL_VERSIONS_ONLY=yes \
          $($WM_PROJECT_DIR/bin/foamEtcFile config.sh/CGAL)
_foamSource $($WM_PROJECT_DIR/bin/foamEtcFile config.sh/compiler)

set -x

cgalPACKAGE=${cgal_version:-CGAL-4.7}
boostPACKAGE=${boost_version:-boost-system}
gmpPACKAGE=${gmp_version:-gmp-system}
mpfrPACKAGE=${mpfr_version:-mpfr-system}

#------------------------------------------------------------------------------
# Run from third-party directory only
wmakeCheckPwd "$WM_THIRD_PARTY_DIR" || {
    echo "Error: Current directory is not \$WM_THIRD_PARTY_DIR"
    echo "    The environment variables are inconsistent with the installation."
    echo "    Check the OpenFOAM entries in your dot-files and source them."
    exit 1
}
. etc/tools/ThirdPartyFunctions
#------------------------------------------------------------------------------

usage() {
    exec 1>&2
    while [ "$#" -ge 1 ]; do echo "$1"; shift; done
    cat<<USAGE

usage: ${0##*/} [OPTION] [CGAL-VERSION] [boost-VERSION] [gmp-VERSION] [mpfr-VERSION]
options:
  -gcc              force g++ instead of the value from \$WM_CXX
  -help

* build CGAL with
      $cgalPACKAGE
      $boostPACKAGE
      $gmpPACKAGE
      $mpfrPACKAGE

Normally builds against ThirdParty boost and gmp/mpfr when possible.
To override this behaviour (and use the system boost and/o gmp/mpfr),
simply specify a 'system' version. For example,
    ${0##*/} boost-system gmp-system

Note: mixing system and ThirdParty for gmp/mpfr is not supported.

USAGE
    exit 1
}


# Ensure CMake gets the correct C++ compiler
[ -n "$WM_CXX" ] && export CXX="$WM_CXX"


# Parse options
while [ "$#" -gt 0 ]
do
    case "$1" in
    -h | -help)
        usage
        ;;
    -gcc)
        export CXX=g++   # use g++
        shift
        ;;
    gmp-[4-9]* | gmp-sys*)
        gmpPACKAGE="${1%%/}"
        shift
        ;;
    mpfr-[2-9]* | mpfr-sys*)
        mpfrPACKAGE="${1%%/}"
        shift
        ;;
    CGAL-[0-9]*)
        cgalPACKAGE="${1%%/}"
        shift
        ;;
    boost-[0-9]* | boost_[0-9]* | boost-sys* )
        boostPACKAGE="${1%%/}"
        shift
        ;;
    *)
        usage "unknown option/argument: '$*'"
        ;;
    esac
done


#------------------------------------------------------------------------------

#
# Build Boost
#
#   BOOST_SOURCE_DIR : location of the original sources

BOOST_ARCH_PATH=$installBASE/$boostPACKAGE
BOOST_SOURCE_DIR=$WM_THIRD_PARTY_DIR/$boostPACKAGE

if [ -d "$BOOST_ARCH_PATH" ]
then
    boostInc="$BOOST_ARCH_PATH/include"
    boostLib="$BOOST_ARCH_PATH/lib"
elif [ -d "$BOOST_SOURCE_DIR" ]
then
    boostInc="$BOOST_ARCH_PATH/include"
    boostLib="$BOOST_ARCH_PATH/lib"

    echo "Starting build: boost"
    (
        cd $BOOST_SOURCE_DIR || exit 1

        rm -rf $BOOST_ARCH_PATH

        ./bootstrap.sh \
            --prefix=$BOOST_ARCH_PATH \
            --with-libraries=thread \
            --with-libraries=system \
        && ./bjam toolset=$WM_CC -j $WM_NCOMPPROCS install \
        && echo "Built: boost"
    ) || {
        echo "Error building: boost"
        exit 1
    }
else
    boostInc="/usr/include"

    # For completeness:
    # 64-bit needs lib64, but 32-bit needs lib (not lib32)
    if [ "$WM_ARCH_OPTION" = 64 ]
    then
        boostLib="/usr/lib$WM_ARCH_OPTION"
    else
        boostLib="/usr/lib"
    fi
fi


# Retrieve boost version:
if [ -f "$boostInc/boost/version.hpp" ]
then
    BOOST_VERSION_NO=`sed -ne 's/^#define *BOOST_VERSION *\([0-9][0-9]*\).*$/\1/p' $boostInc/boost/version.hpp`
else
    echo "Boost does not appear to be installed"
    echo "stopping build"
    exit 1
fi

if [ "$cgal_version" = "cgal-system" ]
then
    echo "Using system installation of CGAL"
    exit 0
fi

#
# Build CGAL
#
#   CGAL_SOURCE_DIR : location of the original sources
#   CGAL_BINARY_DIR : location of the build
#   CGAL_DIR        : location of the installed program

CGAL_SOURCE_DIR=$WM_THIRD_PARTY_DIR/$cgalPACKAGE
CGAL_BINARY_DIR=$buildBASE/$cgalPACKAGE
CGAL_ARCH_PATH=$installBASE/$cgalPACKAGE
CGAL_DIR=$CGAL_ARCH_PATH

#
# gmp/mpfr installed without compiler name
installBASE=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH

GMP_ARCH_PATH=$installBASE/$gmpPACKAGE
MPFR_ARCH_PATH=$installBASE/$mpfrPACKAGE

(
    # Remove any existing build folder and recreate
    if [ -d $CGAL_BINARY_DIR ]
    then
        echo "removing old build directory"
        echo "    $CGAL_BINARY_DIR"
        rm -rf $CGAL_BINARY_DIR
    fi
    mkdir -p $CGAL_BINARY_DIR

    cd $CGAL_BINARY_DIR || exit 1

    unset configBoost configGmpMpfr
    echo "----"
    echo "Configuring $cgalPACKAGE with boost $BOOST_VERSION_NO"
    echo "    Source         : $CGAL_SOURCE_DIR"
    echo "    Build          : $CGAL_BINARY_DIR"
    echo "    Target         : $CGAL_DIR"


    if [ -d "$BOOST_ARCH_PATH" ]
    then
        echo "    ThirdParty     : boost"
        configBoost=$(cat <<CMAKE_OPTIONS
 -DBoost_INCLUDE_DIR=$boostInc
 -DBoost_LIBRARY_DIRS=$boostLib
 -DBoost_THREAD_LIBRARY=$boostLib/libboost_thread.so
 -DBoost_THREAD_LIBRARY_RELEASE=$boostLib/libboost_thread.so
 -DBoost_SYSTEM_LIBRARY=$boostLib/libboost_system.so
 -DBoost_SYSTEM_LIBRARY_RELEASE=$boostLib/libboost_system.so
 -DBoost_VERSION=$BOOST_VERSION_NO
CMAKE_OPTIONS
)
    else
        echo "    system         : boost"
        configBoost=$(cat <<CMAKE_OPTIONS
 -DBOOST_LIBRARYDIR=$boostLib
CMAKE_OPTIONS
)
    fi

    if [ -d "$GMP_ARCH_PATH" -a -d "$MPFR_ARCH_PATH" ]
    then
        echo "    ThirdParty     : gmp/mpfr"
        configGmpMpfr=$(cat <<CMAKE_OPTIONS
 -DGMP_INCLUDE_DIR=$GMP_ARCH_PATH/include
 -DGMP_LIBRARIES_DIR=$GMP_ARCH_PATH/lib
 -DGMP_LIBRARIES=$GMP_ARCH_PATH/lib/libgmp.so
 -DGMPXX_INCLUDE_DIR=$GMP_ARCH_PATH/include
 -DGMPXX_LIBRARIES=$GMP_ARCH_PATH/lib/libgmpxx.so
 -DMPFR_INCLUDE_DIR=$MPFR_ARCH_PATH/include
 -DMPFR_LIBRARIES_DIR=$MPFR_ARCH_PATH/lib
 -DMPFR_LIBRARIES=$MPFR_ARCH_PATH/lib/libmpfr.so
CMAKE_OPTIONS
)
    else
        echo "    system         : gmp/mpfr"
    fi

    echo "----"
    set -x
    cmake \
        -DCMAKE_INSTALL_PREFIX=$CGAL_ARCH_PATH \
        -DCMAKE_BUILD_TYPE=Release \
        $configBoost $configGmpMpfr \
        $CGAL_SOURCE_DIR \
    && make -j $WM_NCOMPPROCS \
    && make install || exit 1

    echo "----"
    echo "create '\$CGAL_ARCH_PATH/share/files'"
    echo "----"
    mkdir -p $CGAL_ARCH_PATH/share/src
    rm -f $CGAL_ARCH_PATH/share/files

    for i in assertions.cpp io.cpp MP_Float.cpp Random.cpp
    do
        if [ -e "$CGAL_SOURCE_DIR/src/CGAL/$i" ]
        then
            \cp $CGAL_SOURCE_DIR/src/CGAL/$i $CGAL_ARCH_PATH/share/src/
            echo "\${CGAL_ARCH_PATH}/share/src/$i" >> $CGAL_ARCH_PATH/share/files
        fi
    done

    echo "Done CGAL"
)

#------------------------------------------------------------------------------
makeCGAL (8,471 bytes)

wyldckat

2016-04-03 10:49

updater   ~0006074

Sorry for not providing the complete changes in a single go. I'm currently testing this on Ubuntu 16.04 (beta), which is why I caught the issue in makeCGAL.

It seems to have also broken makeParaView :( I'll have a fix for this as well in a few minutes.

wyldckat

2016-04-03 12:40

updater  

ThirdParty-stopOnError_part2.tar.gz (4,648 bytes)

wyldckat

2016-04-03 12:50

updater  

ThirdParty-stopOnError_part2_v2.tar.gz (4,642 bytes)

wyldckat

2016-04-03 13:00

updater   ~0006075

OK, hopefully I've caught them all now. Henry, please also use the following two attached files:

  - makeCGAL

    - As mentioned in a previous comment, it can now deal properly with "cgal_version=cgal-system", allowing it to build a custom Boost if so desired.


  - the package "ThirdParty-stopOnError_part2_v2.tar.gz" provides the following two files and respective changes:

    - etc/tools/ParaViewFunctions

      - Inline implicit if-blocks, such as:

            [ -n "$cmakeExe" ] || cmakeExe=cmake

        were changed into a standard if-block, otherwise it would trigger the error catching mechanism done by "set -e".

    - etc/tools/ParaView3Functions

      - Same inline if-blocks change as done in "etc/tools/ParaViewFunctions".

      - Also changed the Bash if-block check for Qt version to standard if-block, along with enforcing that only Qt 4.x can be used (now that Qt 5.x exists).



Don't mind about "ThirdParty-stopOnError_part2.tar.gz", I didn't notice a couple of issues, which are now fixed in "ThirdParty-stopOnError_part2_v2.tar.gz".

henry

2016-04-03 14:45

manager   ~0006076

Thanks Bruno
Resolved in ThirdParty-dev by commit 69dba76b3195420c71173a65b1c227b2650eaebf

Issue History

Date Modified Username Field Change
2016-04-03 10:03 wyldckat New Issue
2016-04-03 10:03 wyldckat Status new => assigned
2016-04-03 10:03 wyldckat Assigned To => henry
2016-04-03 10:03 wyldckat File Added: ThirdParty-stopOnError.tar.gz
2016-04-03 10:32 henry Note Added: 0006072
2016-04-03 10:32 henry Status assigned => resolved
2016-04-03 10:32 henry Resolution open => fixed
2016-04-03 10:41 wyldckat Note Added: 0006073
2016-04-03 10:41 wyldckat Status resolved => feedback
2016-04-03 10:41 wyldckat Resolution fixed => reopened
2016-04-03 10:41 wyldckat File Added: makeCGAL
2016-04-03 10:49 wyldckat Note Added: 0006074
2016-04-03 10:49 wyldckat Status feedback => assigned
2016-04-03 12:40 wyldckat File Added: ThirdParty-stopOnError_part2.tar.gz
2016-04-03 12:50 wyldckat File Added: ThirdParty-stopOnError_part2_v2.tar.gz
2016-04-03 13:00 wyldckat Note Added: 0006075
2016-04-03 14:45 henry Note Added: 0006076
2016-04-03 14:45 henry Status assigned => resolved
2016-04-03 14:45 henry Resolution reopened => fixed