- more dictionary-like methods, enforce keyType::LITERAL for all
lookups to avoid any spurious keyword matching.
- new readEntry, readIfPresent methods
- The get() method replaces the now deprecate lookup() method.
- Deprecate lookupOrFailsafe()
Failsafe behaviour is now an optional parameter for lookupOrDefault,
which makes it easier to tailor behaviour at runtime.
- output of the names is now always flatted without line-breaks.
Thus,
os << flatOutput(someEnumNames.names()) << nl;
os << someEnumNames << nl;
both generate the same output.
- Constructor now uses C-string (const char*) directly instead of
Foam::word in its initializer_list.
- Remove special enum + initializer_list constructor form since
it can create unbounded lookup indices.
- Removd old hasEnum, hasName forms that were provided during initial
transition from NamedEnum.
- Added static_assert on Enum contents to restrict to enum or
integral values. Should not likely be using this class to enumerate
other things since it internally uses an 'int' for its values.
Changed volumeType accordingly to enumerate on its type (enum),
not the class itself.
- with the 'cwd' optimization switch it is possible to select the
preferred behaviour for the cwd() function.
A value of 0 causes cwd() to return the physical directory,
which is what getcwd() and `pwd -P` return.
Until now, this was always the standard behaviour.
With a value of 1, cwd() instead returns the logical directory,
which what $PWD contains and `pwd -L` returns.
If any of the sanity checks fail (eg, PWD points to something other
than ".", etc), a warning is emitted and the physical cwd() is
returned instead.
Apart from the optical difference in the output, this additional
control helps workaround file systems with whitespace or other
characters in the directory that normally cause OpenFOAM to balk.
Using a cleaner symlink elsewhere should skirt this issue.
Eg,
cd $HOME
ln -s "/mounted volume/user/workdir" workdir
cd workdir
# start working with OpenFOAM
- The problem is not 100% reproducible, but be somewhat conservative
and now only close(stdin) for pure background (daemon-like) use of
Foam::system(), where it can be useful.
Using Foam::system() for launching background processes is currently
only used for the doc browser.
- add additional control via a Foam::infoDetailLevel flag, which is
supported by a 'DetailLevel' macro. Eg,
DetailLevel << "some information" << nl
- When infoDetailLevel is zero, the stdout for all Foam::system() calls
are also redirected to stderr to prevent child output from
appearing on the parent.
- close stdin before exec in system call.
- convenient way to temporarily suspend the SIGFPE handler.
Eg,
// Normal code with SIGFPE enabled ...
sigFpe::set();
{
// Ignore SIGFPE in this scope ...
sigFpe::ignore noFpeHandling;
...
}
- link CGAL (clang version) without reference to mpfr,gmp libraries
- use offset address in printStack for Darwin as well
- alternative handling of feexcept on Darwin
Improvements to existing functionality
--------------------------------------
- MPI is initialised without thread support if it is not needed e.g. uncollated
- Use native c++11 threading; avoids problem with static destruction order.
- etc/cellModels now only read if needed.
- etc/controlDict can now be read from the environment variable FOAM_CONTROLDICT
- Uniform files (e.g. '0/uniform/time') are now read only once on the master only
(with the masterUncollated or collated file handlers)
- collated format writes to 'processorsNNN' instead of 'processors'. The file
format is unchanged.
- Thread buffer and file buffer size are no longer limited to 2Gb.
The global controlDict file contains parameters for file handling. Under some
circumstances, e.g. running in parallel on a system without NFS, the user may
need to set some parameters, e.g. fileHandler, before the global controlDict
file is read from file. To support this, OpenFOAM now allows the global
controlDict to be read as a string set to the FOAM_CONTROLDICT environment
variable.
The FOAM_CONTROLDICT environment variable can be set to the content the global
controlDict file, e.g. from a sh/bash shell:
export FOAM_CONTROLDICT=$(foamDictionary $FOAM_ETC/controlDict)
FOAM_CONTROLDICT can then be passed to mpirun using the -x option, e.g.:
mpirun -np 2 -x FOAM_CONTROLDICT simpleFoam -parallel
Note that while this avoids the need for NFS to read the OpenFOAM configuration
the executable still needs to load shared libraries which must either be copied
locally or available via NFS or equivalent.
New: Multiple IO ranks
----------------------
The masterUncollated and collated fileHandlers can now use multiple ranks for
writing e.g.:
mpirun -np 6 simpleFoam -parallel -ioRanks '(0 3)'
In this example ranks 0 ('processor0') and 3 ('processor3') now handle all the
I/O. Rank 0 handles 0,1,2 and rank 3 handles 3,4,5. The set of IO ranks should always
include 0 as first element and be sorted in increasing order.
The collated fileHandler uses the directory naming processorsNNN_XXX-YYY where
NNN is the total number of processors and XXX and YYY are first and last
processor in the rank, e.g. in above example the directories would be
processors6_0-2
processors6_3-5
and each of the collated files in these contains data of the local ranks
only. The same naming also applies when e.g. running decomposePar:
decomposePar -fileHandler collated -ioRanks '(0 3)'
New: Distributed data
---------------------
The individual root directories can be placed on different hosts with different
paths if necessary. In the current framework it is necessary to specify the
root per slave process but this has been simplified with the option of specifying
the root per host with the -hostRoots command line option:
mpirun -np 6 simpleFoam -parallel -ioRanks '(0 3)' \
-hostRoots '("machineA" "/tmp/" "machineB" "/tmp")'
The hostRoots option is followed by a list of machine name + root directory, the
machine name can contain regular expressions.
New: hostCollated
-----------------
The new hostCollated fileHandler automatically sets the 'ioRanks' according to
the host name with the lowest rank e.g. to run simpleFoam on 6 processors with
ranks 0-2 on machineA and ranks 3-5 on machineB with the machines specified in
the hostfile:
mpirun -np 6 --hostfile hostfile simpleFoam -parallel -fileHandler hostCollated
This is equivalent to
mpirun -np 6 --hostfile hostfile simpleFoam -parallel -fileHandler collated -ioRanks '(0 3)'
This example will write directories:
processors6_0-2/
processors6_3-5/
A typical example would use distributed data e.g. no two nodes, machineA and
machineB, each with three processes:
decomposePar -fileHandler collated -case cavity
# Copy case (constant/*, system/*, processors6/) to master:
rsync -a cavity machineA:/tmp/
# Create root on slave:
ssh machineB mkdir -p /tmp/cavity
# Run
mpirun --hostfile hostfile icoFoam \
-case /tmp/cavity -parallel -fileHandler hostCollated \
-hostRoots '("machineA" "/tmp" "machineB" "/tmp")'
Contributed by Mattijs Janssens
- controlled by the the 'printExecutionFormat' InfoSwitch in
etc/controlDict
// Style for "ExecutionTime = " output
// - 0 = seconds (with trailing 's')
// - 1 = day-hh:mm:ss
ExecutionTime = 112135.2 s ClockTime = 113017 s
ExecutionTime = 1-07:08:55.20 ClockTime = 1-07:23:37
- Callable via the new Time::printExecutionTime() method,
which also helps to reduce clutter in the applications.
Eg,
runTime.printExecutionTime(Info);
vs
Info<< "ExecutionTime = " << runTime.elapsedCpuTime() << " s"
<< " ClockTime = " << runTime.elapsedClockTime() << " s"
<< nl << endl;
--
ENH: return elapsedClockTime() and clockTimeIncrement as double
- previously returned as time_t, which is less portable.
- generalize some of the library extensions (.so vs .dylib).
Provide as wmake 'sysFunctions'
- added note about unsupported/incomplete system support
- centralize detection of ThirdParty packages into wmake/ subdirectory
by providing a series of scripts in the spirit of GNU autoconfig.
For example,
have_boost, have_readline, have_scotch, ...
Each of the `have_<package>` scripts will generally provide the
following type of functions:
have_<package> # detection
no_<package> # reset
echo_<package> # echoing
and the following type of variables:
HAVE_<package> # unset or 'true'
<package>_ARCH_PATH # root for <package>
<package>_INC_DIR # include directory for <package>
<package>_LIB_DIR # library directory for <package>
This simplifies the calling scripts:
if have_metis
then
wmake metisDecomp
fi
As well as reducing clutter in the corresponding Make/options:
EXE_INC = \
-I$(METIS_INC_DIR) \
-I../decompositionMethods/lnInclude
LIB_LIBS = \
-L$(METIS_LIB_DIR) -lmetis
Any additional modifications (platform-specific or for an external build
system) can now be made centrally.
- a partial selection from https://github.com/mrklein/openfoam-os-x
with adjustments. The primary purpose is to reduce header-level
incompatibilities and to provide a common set of make rules to allow
easier patching (or re-integration).
- this removes an OS-specific dependency (eg, drand48_r is not POSIX)
and allows easier use of other random number generators.
The Rand48 generator has identical behaviour and period as the
lrand48() library routine, but holds its own seed and state
(which makes it re-entrant) and can be combined with other
random distributions.
However, when using the modified form to obtain scalar values
they will not be identical to what drand48() yields.
This is because drand48() uses the raw 48-bit values to directly
set the mantissa of an IEEE double where as the newer distribution
normalizes based on the 32-bit value.
STYLE: simplify code in Random::shuffle and use Swap
- eliminate iterators from PackedList since they were unused, had
lower performance than direct access and added unneeded complexity.
- eliminate auto-vivify for the PackedList '[] operator.
The set() method provides any required auto-vivification and
removing this ability from the '[]' operator allows for a lower
when accessing the values. Replaced the previous cascade of iterators
with simpler reference class.
PackedBoolList:
- (temporarily) eliminate logic and addition operators since
these contained partially unclear semantics.
- the new test() method tests the value of a single bit position and
returns a bool without any ambiguity caused by the return type
(like the get() method), nor the const/non-const access (like
operator[] has). The name corresponds to what std::bitset uses.
- more consistent use of PackedBoolList test(), set(), unset() methods
for fewer operation and clearer code. Eg,
if (list.test(index)) ... | if (list[index]) ...
if (!list.test(index)) ... | if (list[index] == 0u) ...
list.set(index); | list[index] = 1u;
list.unset(index); | list[index] = 0u;
- deleted the operator=(const labelUList&) and replaced with a setMany()
method for more clarity about the intended operation and to avoid any
potential inadvertent behaviour.
- clockValue class for managing the clock values only, with a null
constructor that does not query the system clock (can defer to later).
Can also be used directly for +/- operations.
- refactor clockTime, cpuTime, clock to reduce storage.
- define regExp::results_type using SubStrings container for handling
groups. This makes a later shift to std::smatch easier, but changes
the regExp API for matching with groups. Previously had list element
0 for regex group 1, now list element 0 is the entire match and list
element 1 is regex group 1.
Old:
List<std::string> mat;
if (re.match(text, mat)) Info<< "group 1: " << mat[0] << nl;
New:
regExp::results_type mat;
if (re.match(text, mat)) Info<< "group 1: " << mat.str(1) << nl;
- include amount of free system memory in profiling, which can give an
indication of when swapping is about to start
- profilingSummary utility to collect profiling from parallel
calculations. Collects profiling information from processor
directories and summarize the time spent and number of calls as (max
avg min) values.
- allows (for example) splitting a user string on whitespace and
passing this to system as a list of arguments, thus bypassing any
implicit use of 'sh'.
- system() with optional background, for spawning processes.
- similar to word::validate to allow stripping of invalid characters
without triggering a FatalError.
- use this validated fileName in Foam::readDir to avoid problems when
a directory contains files with invalid characters in their names
- adjust rmDir to handle filenames with invalid characters
- fileName::equals() static method to compare strings while ignoring
any differences that are solely due to duplicate slashes
- Any trailing whitespace when parsing from strings or character buffers
is ignored rather than being treated as an error. This is consistent
with behaviour when reading from an Istream and with leading whitespace
being ignored in the underlying atof/atod, strtof/strtod... functions.
- Allow parsing directly from a std::string instead of just from a 'char*'.
This reflects the C++11 addition of std::stod to complement the C
functions strtod. This also makes it easier to parse string directly
without using an IStringStream.
- Two-parameter parsing methods return success/failure.
Eg,
if (readInt32(str, &int32Val)) ...
- One-parameter parsing methods return the value on success or
emit a FatalIOError.
Eg,
const char* buf;
int32Val = readInt32(buf, &);
- Improved consistency when parsing unsigned ints.
Use strtoimax and strtoumax throughout.
- Rename readDoubleScalar -> readDouble, readFloatScalar -> readFloat.
Using the primitive name directly instead of the Foam typedef for
better consistency with readInt32 etc.
- Clean/improve parseNasScalar.
Handle normal numbers directly, reduce some operations.
Original commit message:
------------------------
Parallel IO: New collated file format
When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor. Processor directories are named 'processorN',
where N is the processor number.
This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor. The files are stored in a single
directory named 'processors'.
The new format produces significantly fewer files - one per field, instead of N
per field. For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.
The file writing can be threaded allowing the simulation to continue running
while the data is being written to file. NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".
The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:
OptimisationSwitches
{
...
//- Parallel IO file handler
// uncollated (default), collated or masterUncollated
fileHandler uncollated;
//- collated: thread buffer size for queued file writes.
// If set to 0 or not sufficient for the file size threading is not used.
// Default: 2e9
maxThreadFileBufferSize 2e9;
//- masterUncollated: non-blocking buffer size.
// If the file exceeds this buffer size scheduled transfer is used.
// Default: 2e9
maxMasterFileBufferSize 2e9;
}
When using the collated file handling, memory is allocated for the data in the
thread. maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated. If the data exceeds this size, the write does not use threading.
When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer. If the
data exceeds this size, the system uses scheduled communication.
The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters. Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.
A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated
An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling
The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.
- use allocator class to wrap the stream pointers instead of passing
them into ISstream, OSstream and using a dynamic cast to delete
then. This is especially important if we will have a bidirectional
stream (can't delete twice!).
STYLE:
- file stream constructors with std::string (C++11)
- for rewind, explicit about in|out direction. This is not currently
important, but avoids surprises with any future bidirectional access.
- combined string streams in StringStream.H header.
Similar to <sstream> include that has both input and output string
streams.
- allows configuration without an environment variable.
For compatibility still respect FOAM_SIGFPE and FOAM_SETNAN
env-variables
- The env-variables are now treated as true/false switch values.
Previously there was just a check for env exists or not, but this
can be fairly fragile for a user's environment.
- disable automatically upgrading copyrights in files since changes to
not automatically imply a change in copyright. Eg, fixing a typo in
comments, or changing a variable from 'loopI' to 'loopi' etc.
