- extend time searching mechanisms to support searching in the case
global path: times(), findClosestTime(), findInstancePath()
- extend IOobject path() and objectPath() methods with
IOobjectOption::Layout variants
The changes are needed for addressing a global file structure
in parallel (#3431)
- make memory streams header-only (simpler)
- add sub-views and direct seek for span streams
New IOobject convenience methods
- IOobject::instanceValue() : return the IOobject instance as a scalar
value (or 0). Effectively the same as instant(io.instance()).value()
but with far less typing.
- IOobject::fileModificationChecking_masterOnly() : combines checks for
time-stamp and inotify variants
STYLE: minor adjustments for Enum
- can be used in most places where checkType=false is used
ENH: add non-const get() method to HashPtrTable
- allows checking and modification (symmetric with PtrList methods)
STYLE: improve annotations in fileOperations headers
- can be used with this type of code:
ITstream* streamPtr = dict.findStream(name);
if (streamPtr)
{
auto& is = *streamPtr;
...
}
versus:
const entry* eptr = dict.findEntry(name);
if (eptr && eptr->isStream())
{
auto& is = eptr->stream();
...
}
ENH: add findStream(), streamPtr(), isStream() to dictionary search
- symmetric with findDict(), dictPtr(), isDict() methods
STYLE: use findDict() instead of found() + subDict() pairing
COMP: define is_globalIOobject trait at top of IOobject header
- more visibility, permits reuse for specializations etc.
- replace typeGlobal() global function with is_globalIOobject
traits for more consistent and easier overriding.
- relocate typeFilePath() global function as a member of IOobject
for consistency with typeHeaderOk.
BUG: faSchemes, fvSchemes not marked as global file types
- caused issues with collated
- extends the enumeration (NO_REGISTER, REGISTER, LEGACY_REGISTER).
Can be used to tweak registration preference where required and
potentially (TDB) to define a different default value in the future
- ensures that read failures can be properly detected
COMP: include refPtr.H instead of autoPtr.H in IOobject.H
- ensures inclusion of autoPtr/refPtr/tmp/stdFoam
ENH: add IOobject::resetHeader() method
- when re-using an IOobject for repeated read operations it enforces
resetting of headerClassName, scalar/label sizes etc prior to
reading. Permits convenient resetting of the name too (optional).
Example,
IOobject rio("none", ..., IOobject::LAZY_READ);
rio.resetHeader("U")
if (returnReduceOr(rio.typeHeaderOk<volVectorField>(false)))
...
io.resetHeader("p")
if (returnReduceOr(rio.typeHeaderOk<volScalarField>(false)))
...
- useful when regular contents are to be read via an IOobject and
returned.
Eg, dictionary propsDict(IOdictionary::readContents(dictIO));
vs. dictionary propsDict(static_cast<dictionary&&>(IOdictionary(dictIO)));
Commonly these would have simply been constructed directly as the
IO container:
eg, IOdictionary propsDict(dictIO);
However, that style may not ensure proper move semantics for return
types.
Now,
=====
labelList decomp(labelIOList::readContents(io));
... something
return decomp;
=====
Previously,
=====
labelIOList decomp(io);
// Hope for the best...
return decomp;
// Or be explicit and ensure elision occurs...
return labelList(std::move(static_cast<labelList&>(decomp)));
=====
Note:
labelList list(labelIOList(io));
looks like a good idea, but generally fails to compile
- encompasses isReadOptional or isReadRequired check
STYLE: allow LAZY_READ as a shorter synonym for READ_IF_PRESENT
- add helper for downgrading MUST_READ... to LAZY_READ
- returns UPtrList view (read-only or read/write) of the objects
- shorter names for IOobject checks: hasHeaderClass(), isHeaderClass()
- remove unused IOobject::isHeaderClassName(const word&) method.
The typed versions are preferable/recommended, but can still check
directly if needed:
(io.headerClassName() == "foo")
- local writeHeaderEntry helper was not marked as file-scope static.
- use do/while to simplify handling of padding spaces
ENH: IOobject - copy construct, resetting name and local component
- when copying with a new local component, this is simpler than
constructing from all of the components, which was previously the
only possibility for setting a new local component.
- adds handling of negative start times for masterUncollatedFileOperation
as well (#1112).
- handle failures *after* restoring non-parRun mode.
This ensures exit(FatalError) will exit MPI properly as well.
STYLE: replace "polyMesh" with polyMesh::meshSubDir
STYLE: adjust IOobject read/write enumerated values
- provision for possible bitwise handling
COMP: implicit cast scope name to C++-string in IOobject::scopedName
- handles 'const char*' and allows a check for an empty scope name
COMP: avoid potential name conflict in local function (Istream)
- reportedly some resolution issues (unconfirmed) with Fujitsu clang
- simplifies local toggling.
- centralize fileModification static variables into IOobject.
They were previously scattered between IOobject and regIOobject
- support selective enable/disable of the file banner.
ENH: improve code isolation for decomposedBlockData
- use readBlockEntry/writeBlockEntry to encapsulate the IO handling,
which ensures more consistency
- new decomposedBlockData::readHeader for chaining into the
block header information.
- remove unused constructors for decomposedBlockData
ENH: minor cleanup of collated fileOperations
- improves interface and data consistency.
Older signatures are still active (via the Foam_IOstream_extras
define).
- refine internals for IOstreamOption streamFormat, versionNumber
ENH: improve data alignment for IOstream and IOobject
- fit sizeof label/scalar into unsigned char
STYLE: remove dead code
- this was previously suppressed for ASCII format as being 'clutter',
but without it there is no context for interpreting the type of data
contained in ASCII files: potentially leading to integer overflows
when reading in ParaView etc.
This adds a 'geometry' scheme section to the system/fvSchemes:
geometry
{
type highAspectRatio;
}
These 'fvGeometryMethod's are used to calculate
- deltaCoeffs
- nonOrthoCoeffs
etc and can even modify the basic face/cellCentres calculation.
- make stream constructors explicit
- remove "using std::ifstream", "using std::iofstream" statements
for a cleaner namespace.
* copy/move assignments for ITstream
* IStringStream: default construct and construct from std::string
instead of Foam::string
- reduce some overhead in masterOFstream
- simplify Pstream handling of string variants (#1525)
- this is principally for cases where reduced indentation is desired,
such as when streaming to a memory location. If the indentation size
is zero or one, only a single space will be used to separate the
key/value.
This change does not affect the stream allocation size, since the
extra data falls within the padding.
ENH: relocate label/scalar sizes from Istream to IOstream.
- could allow future use for output streams as well?
Due to padding, reorganization has no effect on allocated size
of output streams.
STYLE: add read/write name qualifier to beginRaw, endRaw
- removes ambiguity for bi-directional streams
STYLE: fix inconsistent 'const' qualifier on std::streamsize
- base Ostream was without const, some derived streams with const
Integration of VOF MULES new interfaces. Update of VOF solvers and all instances
of MULES in the code.
Integration of reactingTwoPhaseEuler and reactingMultiphaseEuler solvers and sub-models
Updating reactingEuler tutorials accordingly (most of them tested)
New eRefConst thermo used in tutorials. Some modifications at thermo specie level
affecting mostly eThermo. hThermo mostly unaffected
New chtMultiRegionTwoPhaseEulerFoam solver for quenching and tutorial.
Phases sub-models for reactingTwoPhaseEuler and reactingMultiphaseEuler were moved
to src/phaseSystemModels/reactingEulerFoam in order to be used by BC for
chtMultiRegionTwoPhaseEulerFoam.
Update of interCondensatingEvaporatingFoam solver.
- extracts values from the arch "LSB;label=32;scalar=64" header entry
to provision for managing dissimilar primitive sizes.
Compensate for the additional IOobject members by narrowing the types
for the (objectState, readOption, writeOption) enumerations
- in parameter form, check if it headerClassName() corresponds to the
specified class name
- in templated form, check if headerClassName() corresponds to the
Type:typeName value.
Specialization for <void> always returns true (ie, no checks)
- centralizes IOobject handling and treatment of alternative locations.
If an alternative file location is specified, it will be used instead.
- provide decompositionMethod::canonicalName instead of using
"decomposeParDict" in various places.
- IOstreamOption class to encapsulate format, compression, version.
This is ordered to avoid internal padding in the structure, which
reduces several bytes of memory overhead for stream objects
and other things using this combination of data.
Byte-sizes:
old IOstream:48 PstreamBuffers:88 Time:928
new IOstream:24 PstreamBuffers:72 Time:904
====
STYLE: remove support for deprecated uncompressed/compressed selectors
In older versions, the system/controlDict used these types of
specifications:
writeCompression uncompressed;
writeCompression compressed;
As of DEC-2009, these were deprecated in favour of using normal switch
names:
writeCompression true;
writeCompression false;
writeCompression on;
writeCompression off;
Now removed these deprecated names and treat like any other unknown
input and issue a warning. Eg,
Unknown compression specifier 'compressed', assuming no compression
====
STYLE: provide Enum of stream format names (ascii, binary)
====
COMP: fixed incorrect IFstream construct in FIREMeshReader
- spurious bool argument (presumably meant as uncompressed) was being
implicitly converted to a versionNumber. Now caught by making
IOstreamOption::versionNumber constructor explicit.
- bad version specifier in changeDictionary
Original commit message:
------------------------
Parallel IO: New collated file format
When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor. Processor directories are named 'processorN',
where N is the processor number.
This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor. The files are stored in a single
directory named 'processors'.
The new format produces significantly fewer files - one per field, instead of N
per field. For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.
The file writing can be threaded allowing the simulation to continue running
while the data is being written to file. NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".
The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:
OptimisationSwitches
{
...
//- Parallel IO file handler
// uncollated (default), collated or masterUncollated
fileHandler uncollated;
//- collated: thread buffer size for queued file writes.
// If set to 0 or not sufficient for the file size threading is not used.
// Default: 2e9
maxThreadFileBufferSize 2e9;
//- masterUncollated: non-blocking buffer size.
// If the file exceeds this buffer size scheduled transfer is used.
// Default: 2e9
maxMasterFileBufferSize 2e9;
}
When using the collated file handling, memory is allocated for the data in the
thread. maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated. If the data exceeds this size, the write does not use threading.
When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer. If the
data exceeds this size, the system uses scheduled communication.
The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters. Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.
A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated
An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling
The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.
- Resolve ambiguity by using the following rules:
1) starts with '/' => absolute file-system path
2) starts with './' or '../' => file-system path relative to CWD
3) otherwise treat as relative to the case
STYLE: allow write access to headerClassName
- in specific cases it can be useful to suppress searching the instances.
For example, if one only wishes to check if a "points" is available at
the given time instance, without searching backwards through all
times.
