View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000211 | OpenFOAM | Bug | public | 2011-06-05 00:18 | 2014-12-14 21:46 |
| Reporter | wyldckat | Assigned To | henry | ||
| Priority | low | Severity | feature | Reproducibility | N/A |
| Status | resolved | Resolution | fixed | ||
| Platform | Any | ||||
| Summary | 0000211: Update the work flow when building with wmake for modern machines, by updating aliases | ||||
| Description | Today's machines have up to 8 cores/threads in a single CPU, where some of them can probably lap around OpenCFD's good old "dm" and "noisy" a few times ;) So my suggestion is to add to the portfolio of etc/aliases.{sh,csh} the following (or similar) entries (this is for bash/sh, untested with korn shell): #wmSC - single core machine #wmMC - multi-core machine alias wmSC='unset WM_NCOMPPROCS && echo "Building enabled for a single core"' alias wmMC='export WM_NCOMPPROCS=1; test -r /proc/cpuinfo && export WM_NCOMPPROCS=$(egrep "^processor" /proc/cpuinfo | wc -l) && echo "Building enabled for $WM_NCOMPPROCS cores"' ( Based on "ThirdParty/tools/makeThirdPartyFunctions:getNCOMPPROCS", so I assume it will work with korn shell. ) Of course this is for a single machine, multi-core set-up. As for the work flow, this alias could be referenced on the chapter "Building the Sources": http://www.openfoam.com/download/source.php#x5-25000 Something like: «Go to the top-level source directory $WM_PROJECT_DIR, run wmMC and execute the top-level build script ./Allwmake.» Or refer to it after that paragraph. | ||||
| Additional Information | "wmOpt", "wmDbg" and "wmProf" would also be a nice thing to add... but I suppose you'll want to leave that to the people who are more comfortable with OpenFOAM's building system and let them tweak the environment on their own :) But multi-core is "sort-of a must" nowadays! ;) | ||||
| Tags | No tags attached. | ||||
|
|
Or, even better, create a command-line parsing shell script à la git-sh-setup which is sourced in all Allwmake scripts and in the wmake script. For one, it should know the -j flag. If it is missing, WM_NCOMPPROCS is left alone. If it is passed without an argument, WM_NCOMPPROCS is set to the value of $(($(getconf _NPROCESSORS_ONLN)+1)), otherwise it is set to the value passed as the argument. This would allow one to do $ ./Allwmake -j to get a parallel build with the optimal number of threads (_NPROCESSORS_ONLN+1 is a good idea because it makes sure that there is still some work to be done while the cores are waiting for the disk). |
|
|
Attached is the patch "patch_multi_core_aliases" that adds the two proposed aliases, along with some more aliases that can simplify the usage of multi-core single/multi-machine. I'll document their usage on the next note(s). The patch "patch_paraview_csh" is a fix for "etc/config/paraview.csh", because with csh (C shell) in Ubuntu gave me the error "Invalid null command." on that line. And by what I've read, that was the quickest and most reliable fix. @mwild: over-scheduling by 1 can be a bit annoying to those people who wish to continue using the machine for browsing and other stuff :( |
|
|
patch_multi_core_aliases (2,890 bytes)
diff --git a/etc/config/aliases.csh b/etc/config/aliases.csh
index 6573c91..c259bac 100644
--- a/etc/config/aliases.csh
+++ b/etc/config/aliases.csh
@@ -39,7 +39,7 @@ alias wmSP 'wmSET WM_PRECISION_OPTION=SP'
alias wmDP 'wmSET WM_PRECISION_OPTION=DP'
# clear env
-alias wmUNSET='source $WM_PROJECT_DIR/etc/config/unset.csh'
+alias wmUNSET 'source $WM_PROJECT_DIR/etc/config/unset.csh'
# Toggle wmakeScheduler on/off
# - also need to set WM_HOSTS
@@ -47,6 +47,22 @@ alias wmUNSET='source $WM_PROJECT_DIR/etc/config/unset.csh'
alias wmSchedON 'setenv WM_SCHEDULER $WM_PROJECT_DIR/wmake/wmakeScheduler'
alias wmSchedOFF 'unsetenv WM_SCHEDULER'
+# Set hosts for scheduler
+# - if none defined, single machine is assumed automatically
+alias wmHosts 'setenv WM_HOSTS'
+
+# Set default colours for scheduler
+alias wmColours 'black blue green cyan red magenta yellow'
+
+# Fast switching between multi-core and single-core building
+# wmSCS - single-core single-machine
+# wmMCS - multi-core single-machine
+# wmMCM - multi-core multi-machine
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+alias wmSCS 'unsetenv WM_NCOMPPROCS && echo "Building enabled for a single core"'
+alias wmMCS 'setenv WM_NCOMPPROCS 1; test -r /proc/cpuinfo && setenv WM_NCOMPPROCS `egrep "^processor" /proc/cpuinfo | wc -l` && echo "Building enabled for $WM_NCOMPPROCS cores"'
+alias wmMCM 'setenv WM_NCOMPPROCS `wmSchedON; $WM_SCHEDULER -count` && echo "Building enabled for $WM_NCOMPPROCS cores"'
+
# Change ParaView version
# ~~~~~~~~~~~~~~~~~~~~~~~
alias foamPV 'source `$WM_PROJECT_DIR/etc/config/paraview.csh` ParaView_VERSION=\!*; echo paraview-$ParaView_VERSION'
diff --git a/etc/config/aliases.sh b/etc/config/aliases.sh
index 1881597..d5b654a 100644
--- a/etc/config/aliases.sh
+++ b/etc/config/aliases.sh
@@ -47,6 +47,25 @@ alias wmUNSET='. $WM_PROJECT_DIR/etc/config/unset.sh'
alias wmSchedON='export WM_SCHEDULER=$WM_PROJECT_DIR/wmake/wmakeScheduler'
alias wmSchedOFF='unset WM_SCHEDULER'
+# Set hosts for scheduler
+# - if none defined, single machine is assumed by wmakeScheduler
+wmHosts()
+{
+ export WM_HOSTS="$*"
+}
+
+# Set default colours for scheduler
+alias wmColours='black blue green cyan red magenta yellow'
+
+# Fast switching between multi-core and single-core building
+# wmSCS - single-core single-machine
+# wmMCS - multi-core single-machine
+# wmMCM - multi-core multi-machine
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+alias wmSCS='unset WM_NCOMPPROCS && echo "Building enabled for a single core"'
+alias wmMCS='export WM_NCOMPPROCS=1; test -r /proc/cpuinfo && export WM_NCOMPPROCS=$(egrep "^processor" /proc/cpuinfo | wc -l) && echo "Building enabled for $WM_NCOMPPROCS cores"'
+alias wmMCM='export WM_NCOMPPROCS=$(wmSchedON; $WM_SCHEDULER -count) && echo "Building enabled for $WM_NCOMPPROCS core(s)"'
+
# Change ParaView version
# ~~~~~~~~~~~~~~~~~~~~~~~
unset foamPV
|
|
|
patch_paraview_csh (492 bytes)
diff --git a/etc/config/paraview.csh b/etc/config/paraview.csh
index 86a5287..7a09f1b 100644
--- a/etc/config/paraview.csh
+++ b/etc/config/paraview.csh
@@ -74,8 +74,7 @@ case "$ParaView_MAJOR".*:
case [0-9]*:
# extract major from the version
- setenv ParaView_MAJOR `echo ${ParaView_VERSION} | \
- sed -e 's/^\([0-9][0-9]*\.[0-9][0-9]*\).*$/\1/'`
+ setenv ParaView_MAJOR `echo ${ParaView_VERSION} | sed -e 's/^\([0-9][0-9]*\.[0-9][0-9]*\).*$/\1/'`
breaksw
endsw
|
|
|
On BASH/SH: wmHosts #initialize an empty WM_HOSTS. wmakeScheduler will use single machine by default. wmHosts aaa:1 bbb:4 #same as 'export WM_HOSTS="aaa:1 bbb:4"' wmColours #start up with the default colours, just like 'export WM_COLOURS="black blue green cyan red magenta yellow"' wmSCS # enable building with single-machine, single-core, but doesn't deactivate the scheduler wmMCS # enable building with single-machine, multi-core. wmMCM # activates the scheduler and building with multi-core. Implies proper previous usage of wmHosts Scenarios: Scenario 1 - single machine, multi-core - run: wmMCS Scenario 2 - multiple machines, multi-core - run: wmColours wmHosts aaa:1 bbb:4 wmMCM Scenario 3 - deactivate multi-core: wmSCS |
|
|
On CSH/TCSH: wmHosts #initialize an empty WM_HOSTS. wmakeScheduler will use single machine by default. wmHosts 'aaa:1 bbb:4' #same as 'setenv WM_HOSTS "aaa:1 bbb:4"' wmColours #start up with the default colours, just like 'setenv WM_COLOURS "black blue green cyan red magenta yellow"' wmSCS # enable building with single-machine, single-core, but doesn't deactivate the scheduler wmMCS # enable building with single-machine, multi-core. wmMCM # activates the scheduler and building with multi-core. Implies proper previous usage of wmHosts Scenarios: Scenario 1 - single machine, multi-core - run: wmMCS Scenario 2 - multiple machines, multi-core - run: wmColours wmHosts "aaa:1 bbb:4" wmMCM Scenario 3 - deactivate multi-core: wmSCS |
|
|
Resolved by commit bc5134a55ec35950a302608c81b47b8efe84798b in https://github.com/OpenFOAM/OpenFOAM-dev |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2011-06-05 00:18 | wyldckat | New Issue | |
| 2011-06-05 10:31 |
|
Note Added: 0000421 | |
| 2011-07-10 14:28 | wyldckat | Note Added: 0000532 | |
| 2011-07-10 14:29 | wyldckat | File Added: patch_multi_core_aliases | |
| 2011-07-10 14:29 | wyldckat | File Added: patch_paraview_csh | |
| 2011-07-10 14:37 | wyldckat | Note Added: 0000533 | |
| 2011-07-10 14:38 | wyldckat | Note Added: 0000534 | |
| 2011-07-10 14:39 | wyldckat | Note Edited: 0000533 | |
| 2013-11-27 13:08 |
|
File Added: bcl_HTML4.html | |
| 2013-11-27 22:22 | henry | File Deleted: bcl_HTML4.html | |
| 2014-12-14 21:46 | henry | Note Added: 0003316 | |
| 2014-12-14 21:46 | henry | Status | new => resolved |
| 2014-12-14 21:46 | henry | Resolution | open => fixed |
| 2014-12-14 21:46 | henry | Assigned To | => henry |
