View Issue Details

IDProjectCategoryView StatusLast Update
0002204OpenFOAMBugpublic2016-09-14 08:24
Reporterdrdaveturner Assigned Tohenry  
Status closedResolutionsuspended 
PlatformGNU/LinuxOSOtherOS Version(please specify)
Product Versiondev 
Summary0002204: Failed compilation
DescriptionGentoo Linux, but should apply to any
While compiling OpenFOAM 4.0 and ThirdParty software from the dev branch (week of August 15th), it failed to touch 2 file because the needed directories were not there. I had to do the following as a workaround:

mkdir -p platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/Pstream/mpi

mkdir -p platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/parallel/decompose/ptscotchDecomp


touch: cannot touch ‘/homes/daveturner/libs3/OpenFOAM-dev/platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/Pstream/mpi/using:openmpi-system’: No such file or directory

touch: cannot touch ‘/homes/daveturner/libs3/OpenFOAM-dev/platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/parallel/decompose/ptscotchDecomp/using:scotch_6.0.3’: No such file or directory
Steps To ReproduceStart from a clean distribution and do Allwmake.
TagsNo tags attached.


related to 0002223 resolvedhenry Problem using relative path when sourcing bashrc 



2016-08-20 08:55

manager   ~0006710

In OpenFOAM-dev/src/Pstream/Allwmake:

        wmake $targetType $libName
        touch "$whichmpi"

The 'wmake' command will create the directory in which the 'touch' creates the file. If the directory does not exist the 'wmake' must have failed.

Put 'set -x' in the 'Allwmake' script and 'wmakeMpiLib' function and trace through what is going wrong.

I have tested a clean build with the system and other mpi installations and cannot reproduce the problem you get.


2016-08-24 22:43

reporter   ~0006761

Added set -x in src/Pstream/Allwmake as well as a few echo commands. The tail of the output is below. wmake is trying to touch a file in ./platforms/Linux64GccDPInt32OptSYSTEMOPENMPI while the only directory in platforms is linux64GccDPInt32Opt.

Selene ls -l platforms/
total 0
drwxr-xr-x 1 daveturner daveturner_users 89024 Aug 24 16:12 linux64GccDPInt32Opt

+ echo 'touch whichmpi=/homes/daveturner/libs2/OpenFOAM-dev/platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/Pstream/mpi/using:openmpi-system'

touch whichmpi=/homes/daveturner/libs2/OpenFOAM-dev/platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/Pstream/mpi/using:openmpi-system

+ touch /homes/daveturner/libs2/OpenFOAM-dev/platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/Pstream/mpi/using:openmpi-system

touch: cannot touch ‘/homes/daveturner/libs2/OpenFOAM-dev/platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/Pstream/mpi/using:openmpi-system’: No such file or directory


2016-08-25 00:42

updater   ~0006762

Last edited: 2016-08-25 01:20

@drdaveturner: Please run the following commands:

   cd $FOAM_SRC/Pstream/
   ./Allwmake > log.make 2>&1

The resulting "log.make" file should provide us with the necessary information to diagnose the origin of the problem. Please attach this file, so that we can take a look. Nonetheless, do feel free to search-replace any sensitive information within the file before uploading.

Another detail I would like to ask is what the following commands give you:

   ls -l /bin/sh
   ls -l $(which sh)
   ls -l $(which bash)
   bash --version

I haven't used Gentoo in a few years now, and I only used it briefly back then, so I don't remember what's the default "sh". These 4 commands will let us know if:

  - ensure that "/bin/sh" is the same as the output for the next command;
  - the "sh" file is simply a link or an actual binary;
  - if "bash" is the same as "sh" (at least based on file size);
  - what version of "bash" you're using.

I'm asking for these details, just in case it's something to do with a specific version of "bash" or whichever is the default "sh".


2016-08-25 01:57


Allwmake.out (138,459 bytes)


2016-08-25 02:01

reporter   ~0006764

I uploaded a file Allwmake.out, generated by 'Allwmake |& tee Allwmake.out' which should be the same as you requested.

Selene ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Jan 20 2016 /bin/sh -> bash
Selene ls -l $(which sh)
lrwxrwxrwx 1 root root 7 May 7 2015 /usr/local/bin/sh -> /bin/sh
Selene ls -l $(which bash)
lrwxrwxrwx 1 root root 9 May 7 2015 /usr/local/bin/bash -> /bin/bash
Selene bash --version
GNU bash, version 4.3.42(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.


2016-08-25 02:07

reporter   ~0006765

This time I did exactly as you requested, running Allwmake from ./src/Pstream, and it created the linux64GccDPInt32OptSYSTEMOPENMPI directory in platforms as it should. When I run ./Allwmake from the base directory I get the Allwmake.out file I sent you with the failure on the touch command.


2016-08-25 02:46

updater   ~0006766

OK, I believe I've figured out the origin of the problem. It's related to how the path "/homes/daveturner/libs2/OpenFOAM-dev/" really relates to the path defined in the environment variable "WM_PROJECT_DIR".

The "Allwmake.out" file you provided tells us that the object files are being stored locally in the current source code folder, e.g. "Make/linux64GccDPInt32OptSYSTEMOPENMPI/UOPwrite.o", instead of "/homes/daveturner/libs2/OpenFOAM-dev/platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src/Pstream/mpi/UOPwrite.o" (the global path).

In order for this problem to happen only when you run from the main "Allwmake" script, it means that when the scripts are calling each other, it eventually unravels the real path from a symbolic link path. For example, you might have the path "/homes/daveturner/libs2/OpenFOAM-dev/" as the standard working path, but that path is possibly just a symbolic link to something like "/mnt/libs2/OpenFOAM-dev/". This means that the path "/mnt/libs2/OpenFOAM-dev/" is not the one defined in the environment variable "WM_PROJECT_DIR", leading to "wmake" making the mistake of using the local path instead of the global path.

The following two commands should give us the clear picture:

  realpath $WM_PROJECT_DIR

If these two commands output different paths, then it's explained.
If the paths are identical... then I'm not sure what's going on...


2016-08-25 22:22

reporter   ~0006768

Selene echo $WM_PROJECT_DIR
Selene realpath $WM_PROJECT_DIR

They are the same.

I do have symbolic links pointing to the actual OpenFOAM-dev and ThirdParty-dev directories. I don't know if that would matter.

Selene pwd
Selene ll

memory 1.0KB
lrwxrwxrwx daveturner 12 Aug 24 16:06 OpenFOAM-4.0 -> OpenFOAM-dev
drwxr-xr-x daveturner 201M Aug 24 19:56 OpenFOAM-dev
drwxr-xr-x daveturner 90M Aug 16 15:14 tarballs
lrwxrwxrwx daveturner 14 Aug 24 16:06 ThirdParty-4.0 -> ThirdParty-dev
drwxr-xr-x daveturner 293M Aug 24 16:11 ThirdParty-dev


2016-08-26 01:42

updater   ~0006770

Those symbolic links pointing from 4.0 to dev shouldn't be a problem... although I am assuming that you're sourcing the "OpenFOAM-dev/etc/bashrc" file when building?

I'm attaching the file "wmake_v1", which has a very slight change to help diagnose what happens at the critical point where the decision is made on where the build should be made. To use this file, please do the following steps:

  1. Place the attached file "wmake_v1" inside the folder "OpenFOAM-dev/wmake".

  2. Then run the following commands (with the OpenFOAM environment active):

      cd $WM_PROJECT_DIR
      cd wmake
      mv wmake wmake.orig
      mv wmake_v1 wmake
      chown +x wmake

  3. Then try again running the main "Allwmake" script at "OpenFOAM-dev", while redirecting the output to a log file.

The change made to the "wmake" file is somewhat simple: I added "set -x" and "set +x" around the critical decision point for which object directory path it should use, namely if local or global.

Because the output in the log file that you provided, namely "Allwmake.out", reveals that the following line of code in "wmake" is failing:

  if echo $PWD | grep "$WM_PROJECT_DIR"

because there is no path given before building each library. And in order for this to fail, either "$PWD" is empty or different from "$WM_PROJECT_DIR", or the latter is somehow broken.

The other request I have is if you can please provide a full list of what environment variables are loaded in your shell environment, by running:

   export > log.export

and then attaching the file "log.export"... although you may want to check for any sensitive information in there before uploading.
I ask for this because there is at least one possible variable that could be breaking wmake's work-flow and that's the "GREP_OPTIONS" variable. If not, it could be something else breaking the work-flow... but being able to see what's loaded on the shell environment should help figure this out.


2016-08-26 01:43


wmake_v1 (15,157 bytes)   
# =========                 |
# \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
#  \\    /   O peration     |
#   \\  /    A nd           | Copyright (C) 2011-2016 OpenFOAM Foundation
#    \\/     M anipulation  |
# License
#     This file is part of OpenFOAM.
#     OpenFOAM is free software: you can redistribute it and/or modify it
#     under the terms of the GNU General Public License as published by
#     the Free Software Foundation, either version 3 of the License, or
#     (at your option) any later version.
#     OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
#     ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
#     FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
#     for more details.
#     You should have received a copy of the GNU General Public License
#     along with OpenFOAM.  If not, see <>.
# Script
#     wmake
# Description
#     General, easy to use make system for multi-platform development
#     with support for local and network parallel compilation.
#     This updated wmake supports out-of-tree object and dependency files to
#     avoid the clutter which accumulates in the source-tree with the original
#     wmake system.  Now when building the OpenFOAM package both the object and
#     dependency files are now located in a tree with the same structure as the
#     source tree but in the platforms/$WM_OPTIONS sub-directory of
#     When building user libraries and applications which are not located in the
#     OpenFOAM source-tree the object and dependency files are located in a tree
#     with the same structure as the source tree but in the Make/$WM_OPTIONS
#     sub-directory.
#     The disadvantage of the out-of-tree compilation is that the dependency
#     files are harder to find but are sometimes useful to study which header
#     files are included.  For those who need access to the dependency files the
#     new wdep script is provided to locate them.  See the wdep script header or
#     run:
#         wdep -h
# See also
#     wmakeLnInclude, wmakeLnIncludeAll, wmakeQueue, wdep, wrmdep, wrmo,
#     wclean, wcleanPlatform, wcleanLnIncludeAll

# Source the wmake functions
. ${0%/*}/scripts/wmakeFunctions

usage() {
    exec 1>&2
    while [ "$#" -ge 1 ]; do echo "$1"; shift; done

Usage: $Script [OPTION] [dir]
       $Script [OPTION] target [dir [MakeDir]]

  -s | -silent      'silent' mode (does not echo commands)
  -a | -all         wmake all sub-directories, running Allwmake if present
  -q | -queue       wmakeQueue all sub-directories, running Allwmake if present
  -k or -non-stop   Compile without stopping when errors occur
  -j                Compile using all local cores/hyperthreads
  -jN or -j N       Compile using N cores/hyperthreads
  -no-scheduler     Compile without wmakeScheduler
  -update           Update lnInclude directories, dep files, remove deprecated
                    files and directories
  -h | -help        Print the usage

A general, easy-to-use make system for multi-platform development
with support for local and network parallel compilation.

The 'target' is a Makefile target:
  e.g., platforms/linux64GccDPOpt/.../fvMesh.o

or a special target:
  all               wmake all sub-directories, running Allwmake if present
  queue             wmakeQueue all sub-directories, running Allwmake if present
  exe               Compile statically linked executable
  lib               Compile statically linked archive lib (.a)
  libo              Compile statically linked lib (.o)
  libso             Compile dynamically linked lib (.so)
  dep               Compile lnInclude and dependencies only
  updatedep         Compile dependencies only (in case of broken dependencies)
  objects           Compile but not link

    exit 1

# Default make is the "make" in the path

# Set WM_NCOMPPROCS to number of cores on local machine

    if [ -r /proc/cpuinfo ]
        WM_NCOMPPROCS=$(egrep "^processor" /proc/cpuinfo | wc -l)

    export WM_NCOMPPROCS

# Parse arguments and options

# Default to compiling the local target only

while [ "$#" -gt 0 ]
    case "$1" in
        # Print help
        -h | -help)
        -s | -silent)
            make="$make -s"
        -a | -all | all)
        -q | -queue | queue)
        # Parallel compilation on all cores of local machine
            test $# -ge 2 && expr $2 + 1 > /dev/null 2>&1 \
                && shift && export WM_NCOMPPROCS=$1
            echo "Compiling enabled on $WM_NCOMPPROCS cores"
        # Parallel compilation on specified number of cores
            export WM_NCOMPPROCS=${1#-j}
            echo "Compiling enabled on $WM_NCOMPPROCS cores"
        # Non-stop compilation, ignoring errors
        -k | -non-stop)
            export WM_CONTINUE_ON_ERROR=1
        # Disable scheduled parallel compilation
            unset WM_SCHEDULER
        # Meant to be used following a pull, this will:
        # - remove dep files that depend on deleted files;
        # - remove stale dep files;
        # - update lnInclude directories;
        # - remove empty directories, along with deprecated object directories
        #   and respective binaries.
            [ -z "$all" ] && all="all"
            usage "unknown option: '$*'"

# Check environment variables


# When compiling anything but a standalone exe WM_PROJECT and WM_PROJECT_DIR
# must be set
[ "$1" = exe -o \( "$WM_PROJECT" -a "$WM_PROJECT_DIR" \) ] || {
    echo "$Script error:" 1>&2
    echo "    environment variable \$WM_PROJECT or " \
         "\$WM_PROJECT_DIR not set" 1>&2
    echo "    while building project library" 1>&2
    exit 1

# Setup parallel compilation

# Set WM_NCOMPPROCS automatically when both WM_HOSTS and WM_SCHEDULER are set
if [ -z "$WM_NCOMPPROCS" -a -n "$WM_HOSTS" -a -n "$WM_SCHEDULER" ]
    WM_NCOMPPROCS=$(wmakeScheduler -count)
    [ $? -eq 0 ] || unset WM_NCOMPPROCS

    parOpt="-j $WM_NCOMPPROCS"

    if [ "$WM_NCOMPPROCS" -gt 1 -a ! "$MAKEFLAGS" ]

        if [ -d $lockDir ]
            rm -f $lockDir/*
            mkdir -p $lockDir

        make="$make --no-print-directory $parOpt"

# Check arguments and change to the directory in which to run wmake

unset dir targetType

if [ $# -ge 1 ]
    if [ -d "$1" ]

    # Specified directory name:
    [ $# -ge 2 ] && dir=$2

    # Specified alternative name for the Make sub-directory:
    [ $# -ge 3 ] && MakeDir=$3

    if [ "$dir" ]
        cd $dir 2>/dev/null || {
            echo "$Script error: could not change to directory '$dir'" 1>&2
            exit 1

# Recurse the source tree to compile "all" targets

if [ -n "$update" ]
    wrmdep -update
    wrmdep -old
    wmakeLnIncludeAll -update $parOpt
    wclean empty

# Recurse the source tree to compile "all" targets

if [ "$all" = "all" ]
    if [ -e Allwmake ]
        ./Allwmake -fromWmake $targetType
        exit $?
        # Have to keep track of the main exit code for the call to "make"

        # Find all the sub-directories containing a 'Make' directory
                       for d in *; \
                       do [ -d "$d" -a "$d" != Optional -a "$d" != Make ] \
                          && echo "$d"; \
                       done | xargs \
        if [ ! "$FOAM_APPS" = "" ]
            # Compile all applications in sub-directories
            $make ${WM_CONTINUE_ON_ERROR:+-k} \
                  -f $WM_DIR/makefiles/apps \
                  TARGET="$targetType" FOAM_APPS="$FOAM_APPS"
        # If the current directory contains a 'Make' directory continue
        # otherwise exit, or always exit in case of error
        if [ ! -d $MakeDir -o $makeExitCode -ne 0 ]
            exit $makeExitCode

        # Clean up tracking variable
        unset makeExitCode

# Recurse the source tree to compile "all" targets using wmakeQueue

if [ "$all" = "queue" ]
    [ -n "$update" ] || wmakeLnIncludeAll $parOpt

    (                                                                          \
        WM_COLLECT_DIR=$WM_PROJECT_DIR/platforms/${WM_OPTIONS}/${PWD////_}     \
        WM_SCHEDULER=wmakeCollect                                              \
        trap '$WM_SCHEDULER -kill' TERM INT;                                   \
        $WM_SCHEDULER -clean                                                   \
     && wmake -all objects                                                     \
     && $WM_SCHEDULER                                                          \
    ) && wmake -all
    exit $?

# Search up the directory tree for the Make sub-directory,
# check the existance of the 'files' file and build there if present


# Transform options

# Transform no option to "libso" if that looks appropriate or remove it
# so that the call to make builds the application
if [ "$targetType" = "" ]
    unset targetType
    if grep -e '^ *LIB *=' "$MakeDir/files" >/dev/null 2>&1
elif grep -e '^ *EXE *=' "$MakeDir/files" >/dev/null 2>&1
    # Application. Remove any nonsense targetType
    case "$targetType" in
        unset targetType

# Spawn a sub-shell and unset MAKEFLAGS in that sub-shell to avoid
# files and options being built in parallel

set -x
if echo $PWD | grep "$WM_PROJECT_DIR"
    objectsDir=$platformPath$(echo $PWD | sed s%$WM_PROJECT_DIR%% )

set +x

    unset MAKEFLAGS
    mkdir -p $objectsDir

    # Pre-build the $WM_OPTIONS/options file
    # which is included when building the $WM_OPTIONS/files file
    $make -s -f $WM_DIR/makefiles/files MAKE_DIR=$MakeDir \
          OBJECTS_DIR=$objectsDir $objectsDir/options

    $make -s -f $WM_DIR/makefiles/files MAKE_DIR=$MakeDir \

# Check the $objectsDir/sourceFiles file was created successfully

[ -r $objectsDir/sourceFiles ] || {
    echo "$Script error: file '$objectsDir/sourceFiles'" \
         "could not be created in $PWD" 1>&2
    exit 1

# Make the dependency files

# For libraries create lnInclude ...
case "$targetType" in
    lib | libo | libso | dep )
        # ... but only if 'LIB' is declared in 'Make/files'
        if grep -e '^ *LIB *=' "$MakeDir/files" >/dev/null 2>&1
            $make -s -f $WM_DIR/makefiles/general MAKE_DIR=$MakeDir \
                  OBJECTS_DIR=$objectsDir lnInclude

# When WM_UPDATE_DEPENDENCIES is set, use forced dependency files update


    $make -f $WM_DIR/makefiles/general MAKE_DIR=$MakeDir \
        OBJECTS_DIR=$objectsDir updatedep

    if [ $makeExitCode -ne 0 ]
        exit $makeExitCode

    unset makeExitCode

# Make the dependency files or object files and link

exec $make -f $WM_DIR/makefiles/general MAKE_DIR=$MakeDir \
     OBJECTS_DIR=$objectsDir $targetType

# Cleanup local variables and functions

unset Script usage useAllCores update expandPath findTarget

wmake_v1 (15,157 bytes)   


2016-08-26 23:01

reporter   ~0006776

I believe I've figured out the problem. When I do not supply any symlinks to
OpenFOAM-dev, the code compiles fine. When I put a symlink from OpenFOAM-4.0
to OpenFOAM-dev and cd to OpenFOAM-4.0, then it bombs. Sourcing the etc/bashrc
file provided pulls all the absolute paths even if the symlink is used, but you're also
using $PWD which will use the symlink path.

To allow your code to compile when symlinks are in the path, you should use 'pwd -P' to
get the physical path rather than using $PWD. The pwd manpage on my system says
-P is assumed but each shell may have its own version of pwd, and on my system the
-P is not assumed.


2016-08-27 00:40

updater   ~0006777

Many thanks for figuring this out and reporting your findings!
Although only now did I connect the dots on what you meant when you initially wrote on the report:

> While compiling OpenFOAM 4.0 and ThirdParty software from the dev branch

And when you later on mentioned the symbolic links...
Unfortunately is wasn't clear to me that you were actually running "Allwmake" from within the "OpenFOAM-4.0" path, since the majority of the paths given out by the outputs were referring to "OpenFOAM-dev".

But hold on: then what is the version that you have defined in "OpenFOAM-dev/etc/bashrc", namely in "WM_PROJECT_VERSION"? Is it "dev" or "4.0"?


2016-08-27 00:54

reporter   ~0006778

source ~/libs3/OpenFOAM-4.0/etc/bashrc

sourcing the etc/bashrc file doesn't care whether you use the absolute or symlink
path name. It defines the project version as 'dev' because that's how it's distributed.
It isn't getting this from the path at all. See in etc/bashrc:


There is a user editable part below for when the install directory if different from the
directory you're compiling in, but this isn't the case with the symlink where it's the
same directory, just a difference between the symlink or the absolute path.

So again, if you just use pwd -P to compare to rather than $PATH that should solve
this problem and allow for easy compilation with symlinks in the path.


2016-08-30 14:54

updater   ~0006790

I've been thinking about this and there is one particular detail that is still a bit confusing for me: Why did you use the symbolic path from "OpenFOAM-4.0" to "OpenFOAM-dev", if you've built using the latest commits from OpenFOAM-dev and therefore use the version "dev"?

I ask this because there are a few possible reasons why this is useful, e.g. to have a single source folder that is managed with Git and to have binaries built on their own folders, based on architecture options.
But in order for things to be consistent, it would be necessary for the "etc/bashrc" file to be configured to use the version "4.0" and not "dev".

Because there are in fact 2 layers of changes needed, in case path consistency is to be enforced:

  - "Allwmake" scripts will have to enforce switching to the real folder.
  - the "wmake" script needs to change from "$PWD" to "$(pwd -P)".

But this still raises the question: Then why not simply place the main source code folder in something more name-agnostic, such as "~/OpenFOAM/sourcecode" instead of "~/OpenFOAM/OpenFOAM-dev"?
Because in this scenario, things change drastically and instead it would be necessary to use "pwd -L", if I'm not mistaken.


2016-09-14 08:24

manager   ~0006858

Waiting for feedback.

Issue History

Date Modified Username Field Change
2016-08-20 00:05 drdaveturner New Issue
2016-08-20 08:55 henry Note Added: 0006710
2016-08-24 22:43 drdaveturner Note Added: 0006761
2016-08-25 00:42 wyldckat Note Added: 0006762
2016-08-25 01:20 wyldckat Note Edited: 0006762
2016-08-25 01:57 drdaveturner File Added: Allwmake.out
2016-08-25 02:01 drdaveturner Note Added: 0006764
2016-08-25 02:07 drdaveturner Note Added: 0006765
2016-08-25 02:46 wyldckat Note Added: 0006766
2016-08-25 22:22 drdaveturner Note Added: 0006768
2016-08-26 01:42 wyldckat Note Added: 0006770
2016-08-26 01:43 wyldckat File Added: wmake_v1
2016-08-26 23:01 drdaveturner Note Added: 0006776
2016-08-27 00:40 wyldckat Note Added: 0006777
2016-08-27 00:54 drdaveturner Note Added: 0006778
2016-08-30 14:54 wyldckat Note Added: 0006790
2016-08-30 17:56 wyldckat Relationship added related to 0002223
2016-09-14 08:24 henry Assigned To => henry
2016-09-14 08:24 henry Status new => closed
2016-09-14 08:24 henry Resolution open => suspended
2016-09-14 08:24 henry Note Added: 0006858