- previously always called dlclose on opened libraries when destroying
the dlLibraryTable. However, by force closing the libraries the
situation can arise that the library is missing its own code that it
needs on unload (#1524). This is also sometimes evident when closing
VTK libraries for runTimePostProcessing (#354, #1585).
- The new default is to not forcibly dlclose any libraries, unless
the dlcloseOnTerminate OptimisationSwitch specifies otherwise.
- The dlLibraryTable::close() method can be used to explicitly close
all libraries and clear the list.
- The dlLibraryTable::clear() method now only clears the entries,
without a dlclose.
- This reflects the pre-existing coding situation where const_cast was
used throughout to effect the same.
STYLE: fix private/protected access
- CodedField, codedMixedFvPatchField
- libs() singleton method for global library handling
- explicit handling of empty filename for dlLibraryTable open/close.
Largely worked before, but now be more explicit about its behaviour.
- add (key, dict) constructor and open() methods.
More similarity to dimensionedType, Enum etc, and there is no
ambiguity with the templated open().
- construct or open from initializer_list of names
- optional verbosity when opening with auxiliary table,
avoid duplicate messages or spurious messages for these.
- basename and fullname methods (migrated from dynamicCode).
- centralise low-level load/unload hooks
- adjust close to also dlclose() aliased library names.
- this can be useful for preloading of libraries, or for utilities
that don't use system/controlDict.
The additional libraries can be specified individually:
myUtil -lib lib1 -lib lib2 -lib lib3
When specified like this, the options add up.
Or as a 'captured' list using OpenFOAM's standard arg list handling:
myUtil -lib '(' lib1 lib2 lib3 ')'
myUtil -lib \( lib1 lib2 lib3 \)
or as single argument list:
myUtil -lib '("lib1" "lib2" "lib3")'
When specified as a single argument, would normally take advantage
of the transparent handling of word vs fileName and omit the string
quotes:
myUtil -lib '(lib1 lib2 lib3)'
ENH: dlOpen error messages now propagated into dlLibraryTable
- this makes the context more relevant and also avoids the previous
annoyance of double warnings (one from the POSIX loader, and one
from dlLibraryTable)
STYLE: mark -noFunctionObjects and -withFunctionObjects as 'advanced'
- reduces clutter. Still visible with -help-full
- reduces some dictionary clutter and probably looks less confusing
than having an ending that may not correspond to the current OS.
Eg, "fvOptions" instead of "libfvOptions.so", "libfvOptions.dylib" ...
- convenience dlOpen method for multiple files
- for codedFunctionObject and CodedSource the main code snippets
were not included in the SHA1 calculation, which meant that many
changes would not be noticed and no new library would be compiled.
As a workaround, a dummy 'code' entry could be used solely for the
purposes of generating a SHA1, but this is easily forgotten.
We now allow tracking of the dynamicCodeContext for the coded
objects and append to the SHA1 hasher with specific entries.
This should solve the previous misbehaviour.
We additionally add information about the ordering of the code
sections. Suppose we have a coded function object (all code
segments are optional) with the following:
codeExecute "";
codeWrite #{ Info<< "Called\n"; #};
which we subsequently change to this:
codeExecute #{ Info<< "Called\n"; #};
codeWrite "";
If the code strings are simply concatenated together, the SHA1 hashes
will be identical. We thus 'salt' with their semantic locations,
choosing tags that are unlikely to occur within the code strings
themselves.
- simplify the coded templates with constexpr for the SHA1sum
information.
- Correct the CodedSource to use 'codeConstrain' instead of
'codeSetValue' for consistency with the underlying functions.
- this is identical to either of these solutions:
* getEnv("FOAM_CASE")
* stringOps::expand("<case>")
but with a closer resemblance to argList or Time globalPath(),
which makes the intent clearer.
Avoids using raw strings in the caller, which improves compile-time checks.
Used in situations where a class has no derivation path or other
access to a time registry or command args.
- Use the OPENFOAM define (eg, 1806, 1812), which normally corresponds
to a major release, to define an API level. This remains consistent
within a release cycle and means that it is possible to manage
several sub-versions and continue to have a consistent lookup.
The current API value is updated automatically during the build
and cached as meta data for later use, even when the wmake/ directory
is missing or OpenFOAM has not yet be initialized.
The version information reported on program start or with -help
usage adjusted to reflect this. The build tag from git now also
carries the date as being more meaningful to trace than a hash
value.
- Update etc/bashrc and etc/cshrc to obtain the project directory
directly instead of via its prefix directory. The value obtained
corresponds to an absolute path, from which the prefix directory
can be obtained.
The combination of these changes removes the reliance on any
particular directory naming convention.
For example,
With an 1812 version (API level):
WM_PROJECT_VERSION=myVersion
installed as /some/path/somewhere/openfoam-mySandbox
This makes the -prefix, -foamInstall, -projectVersion, -version
values of foamEtcFiles, and similar entries for foamConfigurePaths
superfluous.
WM_PROJECT_INST_DIR is no longer required or used
ENH: improve handling and discovery of ThirdParty
- improve the flexibility and reusability of ThirdParty packs to cover
various standard use cases:
1. Unpacking initial release tar files with two parallel directories
- OpenFOAM-v1812/
- ThirdParty-v1812/
2. With an adjusted OpenFOAM directory name, for whatever reason
- OpenFOAM-v1812-myCustom/
- openfoam-1812-other-info/
3. Operating with/without ThirdParty directory
To handle these use cases, the following discovery is used.
Note PROJECT = the OpenFOAM directory `$WM_PROJECT_DIR`
PREFIX = the parent directory
VERSION = `$WM_PROJECT_VERSION`
API = `$WM_PROJECT_API`, as per `foamEtcFiles -show-api`
0. PROJECT/ThirdParty
- for single-directory installations
1. PREFIX/ThirdParty-VERSION
- this corresponds to the traditional approach
2. PREFIX/ThirdParty-vAPI
- allows for an updated value of VERSION (eg, v1812-myCustom)
without requiring a renamed ThirdParty. The API value
would still be '1812' and the original ThirdParty-v1812/
would be found.
3. PREFIX/ThirdParty-API
- this is the same as the previous example, but using an unadorned
API value. This also makes sense if the chosen version name also
uses the unadorned API value in its naming
(eg, 1812-patch190131, 1812.19W03)
4. PREFIX/ThirdParty-common
- permits maximum reuse for various versions, but only for
experienced user who are aware of potential version
incompatibilities
Directory existence is checked as is the presence of an Allwmake file
or a platforms/ directory. This reduces the potential of false positive
matches and limits the selection to directories that are either
with sources (has the Allwmake file), or pre-compiled binaries (has
the platforms/ directory).
If none of the explored directories are found to be suitable,
it reverts to using a PROJECT/ThirdParty dummy location since
this is within the project source tree and can be trusted to
have no negative side-effects.
ENH: add csh support to foamConfigurePaths
- this removes the previously experienced inconsistence in config file
contents.
REMOVED: foamExec
- was previously used when switching versions and before the
bashrc/cshrc discovery logic was added. It is now obsolete.
- Within strings it is preferable to use the "<etc>" instead.
Most use cases for the old "~OpenFOAM" expansion have been obsoleted
by the #includeEtc directive.
- use keyType::option enum to consolidate searching options.
These enumeration names should be more intuitive to use
and improve code readability.
Eg, lookupEntry(key, keyType::REGEX);
vs lookupEntry(key, false, true);
or
Eg, lookupEntry(key, keyType::LITERAL_RECURSIVE);
vs lookupEntry(key, true, false);
- new findEntry(), findDict(), findScoped() methods with consolidated
search options for shorter naming and access names more closely
aligned with other components. Behave simliarly to the
methods lookupEntryPtr(), subDictPtr(), lookupScopedEntryPtr(),
respectively. Default search parameters consistent with lookupEntry().
Eg, const entry* e = dict.findEntry(key);
vs const entry* e = dict.lookupEntryPtr(key, false, true);
- added '*' and '->' dereference operators to dictionary searchers.
- add additional control via a Foam::infoDetailLevel flag, which is
supported by a 'DetailLevel' macro. Eg,
DetailLevel << "some information" << nl
- When infoDetailLevel is zero, the stdout for all Foam::system() calls
are also redirected to stderr to prevent child output from
appearing on the parent.
- close stdin before exec in system call.
- a partial selection from https://github.com/mrklein/openfoam-os-x
with adjustments. The primary purpose is to reduce header-level
incompatibilities and to provide a common set of make rules to allow
easier patching (or re-integration).
Original commit message:
------------------------
Parallel IO: New collated file format
When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor. Processor directories are named 'processorN',
where N is the processor number.
This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor. The files are stored in a single
directory named 'processors'.
The new format produces significantly fewer files - one per field, instead of N
per field. For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.
The file writing can be threaded allowing the simulation to continue running
while the data is being written to file. NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".
The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:
OptimisationSwitches
{
...
//- Parallel IO file handler
// uncollated (default), collated or masterUncollated
fileHandler uncollated;
//- collated: thread buffer size for queued file writes.
// If set to 0 or not sufficient for the file size threading is not used.
// Default: 2e9
maxThreadFileBufferSize 2e9;
//- masterUncollated: non-blocking buffer size.
// If the file exceeds this buffer size scheduled transfer is used.
// Default: 2e9
maxMasterFileBufferSize 2e9;
}
When using the collated file handling, memory is allocated for the data in the
thread. maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated. If the data exceeds this size, the write does not use threading.
When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer. If the
data exceeds this size, the system uses scheduled communication.
The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters. Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.
A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated
An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling
The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.
- the purpose is more explicit, without needing to check documentation
about what the bool parameter means.
STYLE: improve formatting of fileName documentation
Pstream: added maxCommsSize setting to do (unstructured) parallel transfers in blocks.
Tested:
- with maxCommsSize 0 produces exactly same result as plus.develop
- compiles with label64
- with maxCommsSize e.g. 3 produces exactly same result as plus.develop
- with maxCommsSize=0 exactly the same messages (with Pstream::debug = 1) as plus.develop
See merge request !85
This is controlled by the setting maxCommsSize in etc/controlDict which
specifies the max number of bytes per exchange. If set to <= 0 it
is ignored. This max size of messages is important when doing e.g.
load balancing which can send over whole meshes.
replace system() call with vfork/exec combination (issue #185)
Tested systemCall function object, dynamicCode, but should be rechecked with IB+openmpi
@Prashant
See merge request !55
The normal library system() command uses 'fork', which causes
problems on IB+OPENMPI.
STYLE: add Foam:: qualifier to system calls to make them easier to spot.
