- provide relativePath() for argList and for Time.
These are relative to the case globalPath().
Eg,
Info<< "output: " << runTime.relativePath(outputFile) << nl;
- for some special cases we wish to mark command-line arguments as
being optional, in order to do our own treatment. For example,
when an arbitrary number of arguments should be allowed.
Now tag this situation with argList::noMandatoryArgs().
The argList::argsMandatory() query can then be used in any further
logic, including the standard default argument checking.
- with the new default check, can consolidate the special-purpose
"setRootCaseNonMandatoryArgs.H"
into the regular
"setRootCase.H"
- revert to a simple "setRootCase.H" and move all the listing related
bits to a "setRootCaseLists.H" file. This leaves the information
available for solvers, or whoever else wishes, without being
introduced everywhere.
- add include guards and scoping to the listing files and rename to
something less generic.
listOptions.H -> setRootCaseListOptions.H
listOutput.H -> setRootCaseListOutput.H
- was PREFIX/site, now PROJECT/site
This avoids several issues when installing OpenFOAM in clusters
without an intermediate OpenFOAM-specific installation prefix.
The 'site' directory may have a reserved meaning in these situations
and it is undesirable to 'leak' upwards into the parent directory to
look for configuration files.
Placing the default within the project directory avoids this.
Alternative locations can be given via the WM_PROJECT_SITE variable.
- use std::string instead of c-string for the string constants
- centralize some definitions of resources into foamVersion.H
Now expose some of the hard-coded values used in foamEtcFiles()
so that they can be known or even overridden as required.
Relocate to src/OpenFOAM/include as a constant location.
- foamVersion.H now includes stdFoam.H for consistent use
of compatibility compiler defines.
- move forward declaration of Detail, Module namespaces to stdFoam.H
- doxygen documentation of Foam namespace in stdFoam.H
- with the 'cwd' optimization switch it is possible to select the
preferred behaviour for the cwd() function.
A value of 0 causes cwd() to return the physical directory,
which is what getcwd() and `pwd -P` return.
Until now, this was always the standard behaviour.
With a value of 1, cwd() instead returns the logical directory,
which what $PWD contains and `pwd -L` returns.
If any of the sanity checks fail (eg, PWD points to something other
than ".", etc), a warning is emitted and the physical cwd() is
returned instead.
Apart from the optical difference in the output, this additional
control helps workaround file systems with whitespace or other
characters in the directory that normally cause OpenFOAM to balk.
Using a cleaner symlink elsewhere should skirt this issue.
Eg,
cd $HOME
ln -s "/mounted volume/user/workdir" workdir
cd workdir
# start working with OpenFOAM
- centralizes IOobject handling and treatment of alternative locations.
If an alternative file location is specified, it will be used instead.
- provide decompositionMethod::canonicalName instead of using
"decomposeParDict" in various places.
General:
* -roots, -hostRoots, -fileHandler
Specific:
* -to <coordinateSystem> -from <coordinateSystem>
- Display -help-compat when compatibility or ignored options are available
STYLE: capitalization of options text
- advanced options are not displayed with -help, but only with
-help-full, which helps retain a better overview of the standard
options. Replaces previous ad hoc suppression of -listSwitches,
-listRegisteredSwitches etc.
- relocate some standard functionality to TimePaths to allow a lighter
means of managing time directories without using the entire Time
mechanism.
- optional enableLibs for Time construction (default is on)
and a corresponding argList::noLibs() and "-no-libs" option
STYLE:
- mark Time::outputTime() as deprecated MAY-2016
- use pre-increment for runTime, although there is no difference in
behaviour or performance.
- add additional control via a Foam::infoDetailLevel flag, which is
supported by a 'DetailLevel' macro. Eg,
DetailLevel << "some information" << nl
- When infoDetailLevel is zero, the stdout for all Foam::system() calls
are also redirected to stderr to prevent child output from
appearing on the parent.
- close stdin before exec in system call.
-dry-run : perform check only and no writing
-dry-run-write: perform check and write to the next time step. This
also triggers function objects that fire on the call to write()
Improvements to existing functionality
--------------------------------------
- MPI is initialised without thread support if it is not needed e.g. uncollated
- Use native c++11 threading; avoids problem with static destruction order.
- etc/cellModels now only read if needed.
- etc/controlDict can now be read from the environment variable FOAM_CONTROLDICT
- Uniform files (e.g. '0/uniform/time') are now read only once on the master only
(with the masterUncollated or collated file handlers)
- collated format writes to 'processorsNNN' instead of 'processors'. The file
format is unchanged.
- Thread buffer and file buffer size are no longer limited to 2Gb.
The global controlDict file contains parameters for file handling. Under some
circumstances, e.g. running in parallel on a system without NFS, the user may
need to set some parameters, e.g. fileHandler, before the global controlDict
file is read from file. To support this, OpenFOAM now allows the global
controlDict to be read as a string set to the FOAM_CONTROLDICT environment
variable.
The FOAM_CONTROLDICT environment variable can be set to the content the global
controlDict file, e.g. from a sh/bash shell:
export FOAM_CONTROLDICT=$(foamDictionary $FOAM_ETC/controlDict)
FOAM_CONTROLDICT can then be passed to mpirun using the -x option, e.g.:
mpirun -np 2 -x FOAM_CONTROLDICT simpleFoam -parallel
Note that while this avoids the need for NFS to read the OpenFOAM configuration
the executable still needs to load shared libraries which must either be copied
locally or available via NFS or equivalent.
New: Multiple IO ranks
----------------------
The masterUncollated and collated fileHandlers can now use multiple ranks for
writing e.g.:
mpirun -np 6 simpleFoam -parallel -ioRanks '(0 3)'
In this example ranks 0 ('processor0') and 3 ('processor3') now handle all the
I/O. Rank 0 handles 0,1,2 and rank 3 handles 3,4,5. The set of IO ranks should always
include 0 as first element and be sorted in increasing order.
The collated fileHandler uses the directory naming processorsNNN_XXX-YYY where
NNN is the total number of processors and XXX and YYY are first and last
processor in the rank, e.g. in above example the directories would be
processors6_0-2
processors6_3-5
and each of the collated files in these contains data of the local ranks
only. The same naming also applies when e.g. running decomposePar:
decomposePar -fileHandler collated -ioRanks '(0 3)'
New: Distributed data
---------------------
The individual root directories can be placed on different hosts with different
paths if necessary. In the current framework it is necessary to specify the
root per slave process but this has been simplified with the option of specifying
the root per host with the -hostRoots command line option:
mpirun -np 6 simpleFoam -parallel -ioRanks '(0 3)' \
-hostRoots '("machineA" "/tmp/" "machineB" "/tmp")'
The hostRoots option is followed by a list of machine name + root directory, the
machine name can contain regular expressions.
New: hostCollated
-----------------
The new hostCollated fileHandler automatically sets the 'ioRanks' according to
the host name with the lowest rank e.g. to run simpleFoam on 6 processors with
ranks 0-2 on machineA and ranks 3-5 on machineB with the machines specified in
the hostfile:
mpirun -np 6 --hostfile hostfile simpleFoam -parallel -fileHandler hostCollated
This is equivalent to
mpirun -np 6 --hostfile hostfile simpleFoam -parallel -fileHandler collated -ioRanks '(0 3)'
This example will write directories:
processors6_0-2/
processors6_3-5/
A typical example would use distributed data e.g. no two nodes, machineA and
machineB, each with three processes:
decomposePar -fileHandler collated -case cavity
# Copy case (constant/*, system/*, processors6/) to master:
rsync -a cavity machineA:/tmp/
# Create root on slave:
ssh machineB mkdir -p /tmp/cavity
# Run
mpirun --hostfile hostfile icoFoam \
-case /tmp/cavity -parallel -fileHandler hostCollated \
-hostRoots '("machineA" "/tmp" "machineB" "/tmp")'
Contributed by Mattijs Janssens
- since PackedBoolList is now a compatibility typedef for bitSet,
it is useful to have an additional means of distinction.
STYLE: simplify internal version tests and compiler defines.
- the API version is now conveyed via the OPENFOAM define directly.
The older OPENFOAM_PLUS define is provided for existing code.
- this removes an OS-specific dependency (eg, drand48_r is not POSIX)
and allows easier use of other random number generators.
The Rand48 generator has identical behaviour and period as the
lrand48() library routine, but holds its own seed and state
(which makes it re-entrant) and can be combined with other
random distributions.
However, when using the modified form to obtain scalar values
they will not be identical to what drand48() yields.
This is because drand48() uses the raw 48-bit values to directly
set the mantissa of an IEEE double where as the newer distribution
normalizes based on the 32-bit value.
STYLE: simplify code in Random::shuffle and use Swap
- use succincter method names that more closely resemble dictionary
and HashTable method names. This improves method name consistency
between classes and also requires less typing effort:
args.found(optName) vs. args.optionFound(optName)
args.readIfPresent(..) vs. args.optionReadIfPresent(..)
...
args.opt<scalar>(optName) vs. args.optionRead<scalar>(optName)
args.read<scalar>(index) vs. args.argRead<scalar>(index)
- the older method names forms have been retained for code compatibility,
but are now deprecated
- allows (for example) splitting a user string on whitespace and
passing this to system as a list of arguments, thus bypassing any
implicit use of 'sh'.
- system() with optional background, for spawning processes.
- in rare cases we may wish to have command-line arguments that are
non-mandatory. This can now be reflected in the usage output, provided
that the argList::nonMandatoryArgs() has been used.
- added setRootCaseNonMandatoryArgs.H that applies the
argList::nonMandatoryArgs() settings and otherwise performs largely
as per setRootCase.H, except that the check for mandatory arguments
is deferred to later user code.
- consistent with C++ STL conventions, the reverse iterators should
use operator++ to transit the list from rbegin() to rend().
The previous implementation used raw pointers, which meant that they
had the opposite behaviour: operator-- to transit from rbegin() to
rend().
The updated version only has operator++ defined, thus the compiler
should catch any possible instances where people were using the old
(incorrect) versions.
- updated forAllReverseIters() and forAllConstReverseIters() macros to
be consistent with new implementation and with C++ STL conventions.
Original commit message:
------------------------
Parallel IO: New collated file format
When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor. Processor directories are named 'processorN',
where N is the processor number.
This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor. The files are stored in a single
directory named 'processors'.
The new format produces significantly fewer files - one per field, instead of N
per field. For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.
The file writing can be threaded allowing the simulation to continue running
while the data is being written to file. NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".
The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:
OptimisationSwitches
{
...
//- Parallel IO file handler
// uncollated (default), collated or masterUncollated
fileHandler uncollated;
//- collated: thread buffer size for queued file writes.
// If set to 0 or not sufficient for the file size threading is not used.
// Default: 2e9
maxThreadFileBufferSize 2e9;
//- masterUncollated: non-blocking buffer size.
// If the file exceeds this buffer size scheduled transfer is used.
// Default: 2e9
maxMasterFileBufferSize 2e9;
}
When using the collated file handling, memory is allocated for the data in the
thread. maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated. If the data exceeds this size, the write does not use threading.
When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer. If the
data exceeds this size, the system uses scheduled communication.
The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters. Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.
A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated
An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling
The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.
Initially the listSwitches functions depended directly on argList functionality
but this has now been factored out so that the listSwitches functions are more
general and require only debug functionality.
Provides better context for the available boundary conditions, fvOptions,
functionObjects etc. and thus returns only those available to and compatible
with the particular application.
e.g.
pimpleFoam -help
Usage: pimpleFoam [OPTIONS]
options:
-case <dir> specify alternate case directory, default is the cwd
-listFunctionObjects
List functionObjects
-listFvOptions List fvOptions
-listRegisteredSwitches
List switches registered for run-time modification
-listScalarBCs List scalar field boundary conditions (fvPatchField<scalar>)
-listSwitches List switches declared in libraries but not set in
etc/controlDict
-listTurbulenceModels
List turbulenceModels
-listUnsetSwitches
List switches declared in libraries but not set in
etc/controlDict
-listVectorBCs List vector field boundary conditions (fvPatchField<vector>)
-noFunctionObjects
do not execute functionObjects
-parallel run in parallel
-postProcess Execute functionObjects only
-roots <(dir1 .. dirN)>
slave root directories for distributed running
-srcDoc display source code in browser
-doc display application documentation in browser
-help print the usage
pimpleFoam listTurbulenceModels
pimpleFoam -listTurbulenceModels
/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: dev |
| \\ / A nd | Web: www.OpenFOAM.org |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : dev-39c46019e44f
Exec : pimpleFoam -listTurbulenceModels
Date : Jun 10 2017
Time : 21:37:49
Host : "dm"
PID : 675
Case : /home/dm2/henry/OpenFOAM/OpenFOAM-dev
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
SetNaN : Initialising allocated memory to NaN (FOAM_SETNAN).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Turbulence models
3
(
LES
RAS
laminar
)
RAS models
18
(
LRR
LamBremhorstKE
LaunderSharmaKE
LienCubicKE
LienLeschziner
RNGkEpsilon
SSG
ShihQuadraticKE
SpalartAllmaras
kEpsilon
kOmega
kOmegaSST
kOmegaSSTLM
kOmegaSSTSAS
kkLOmega
qZeta
realizableKE
v2f
)
LES models
10
(
DeardorffDiffStress
Smagorinsky
SpalartAllmarasDDES
SpalartAllmarasDES
SpalartAllmarasIDDES
WALE
dynamicKEqn
dynamicLagrangian
kEqn
kOmegaSSTDES
)
Further work will be needed to support the -listTurbulenceModels option in
multiphase solvers.
- some functionality similar to what the standary library <iterator>
provides.
* stdFoam::begin() and stdFoam::end() do type deduction,
which means that many cases it is possible to manage these types
of changes.
For example, when managing a number of indices:
Map<labelHashSet> lookup;
1) Longhand:
for
(
Map<labelHashSet>::const_iterator iter = lookup.begin();
iter != lookup.end();
++iter
)
{ .... }
1b) The same, but wrapped via a macro:
forAllConstIter(Map<labelHashSet>, lookup, iter)
{ .... }
2) Using stdFoam begin/end templates directly
for
(
auto iter = stdFoam::begin(lookup);
iter != stdFoam::end(lookup);
++iter
)
{ .... }
2b) The same, but wrapped via a macro:
forAllConstIters(lookup, iter)
{ .... }
Note that in many cases it is possible to simply use a range-based for.
Eg,
labelList myList;
for (auto val : myList)
{ ... }
for (const auto& val : myList)
{ ... }
These however will not work with any of the OpenFOAM hash-tables,
since the standard C++ concept of an iterator would return a key,value
pair when deferencing the *iter.
The deduction methods also exhibits some slightly odd behaviour with
some PtrLists (needs some more investigation).
- ensure proper and sensible handling of empty names.
Eg, isDir(""), isFile("") are no-ops, and avoid file-stat
- rmDir:
* optional 'silent' option to suppress messages.
* removes all possible sub-entries, instead of just giving up on
the first problem encountered.
- reduced code duplication in etcFiles
ENH: provide WM_USER_RESOURCE_DIRNAME define (in foamVersion.H)
- this is still a hard-coded value, but at least centrally available
- was previously only within string expansions, but cover dictionaries
as well for consistency
ENH: replace the never-used fileName::caseName() functionality
- stringOps::inplaceReplaceVar() is more general
stringOps::inplaceReplaceVar(myfile, "FOAM_CASE");
STYLE: relax parameter passing when calling some POSIX 'query' functions.
- A std::string is sufficient since the functions use a plain C-string.
Eg, getEnv("SOMETHING").
Retain more stringent Foam::word for things like setEnv, since this
could be useful.
- Cleanup/centralize handling of -decomposeParDict by relocating
common code into argList. Ensures that all processes receive
identical information about the -decomposeParDict opton.
- Only use alternative decomposeParDict for simpleFoam/motorBike
tutorial so that this will be included in the test loop for snappy.
- Added Mattijs' fix for surfaceRedistributePar.
This supports the abstraction of the set of fields from the field code
generation macros making it easier to change the set of fields supported
by OpenFOAM. This functionality is demonstrated in the updated
fvPatchFields macros and will be applied to the rest of the field code
generation macros in the future.
