- in many cases the local communicator has the same communication
pattern for several timesteps so it can be reused instead of being
recreated. Avoids repeated MPI_Comm_free()/MPI_Comm_group() calls
- STYLE: eliminate bitSet intermediate when determining the sub-ranks
- can be used to pre-allocate space for internal+boundary field for
flat addressing (issue #3364).
For example:
DynamicField<Type> fld
(
std::pair<label,label>
(
GeoMesh::size(mesh),
GeoMesh::size(mesh)
+ (extra ? GeoMesh::boundary_size(mesh) : label(0))
)
);
ENH: base changes for GeometricField flags
- FieldBase 'localBoundaryConsistency_' and 'localBoundaryTolerance_'
for central handling of local boundary consistency checks.
- FieldBase 'unifiedGeometricField' for future extensions to GeometricField
- split/duplicate functionality
- rework inter-node/intra-node handling to allow selection of
splitting based on 'shared' or hostname (default).
- always creates node communicators at startup:
* commInterNode() - between nodes
* commLocalNode() - within a node
- world-comm is now always a duplicate of MPI_COMM_WORLD to provide
better separation from other processes.
NB:
the inter-node comm is a slight exception to other communicators
in that we always retain its list of (global) ranks, even if the
local process is not in that communicator.
This can help when constructing topology-aware patterns.
FIX: non-participating ranks still had knowledge of their potential siblings
- after create by group, the procIDs_ of non-participating ranks
should be empty (except for the inter-node exception)
- the maxCommsSize variable is used to 'chunk' large data transfers
(eg, with PstreamBuffers) into a multi-pass send/recv sequence.
The send/recv windows for chunk-wise transfers:
iter data window
---- -----------
0 [0, 1*chunk]
1 [1*chunk, 2*chunk]
2 [2*chunk, 3*chunk]
...
Since we mostly send/recv in bytes, the current internal limit
for MPI counts (INT_MAX) can be hit rather quickly.
The chunking limit should thus also be INT_MAX, but since it is
rather tedious to specify such large numbers, can instead use
maxCommsSize = -1
to specify (INT_MAX-1) as the limit.
The default value of maxCommsSize = 0 (ie, no chunking).
Note
~~~~
In previous versions, the number of chunks was determined by the
sender sizes. This required an additional MPI_Allreduce to establish
an overall consistent number of chunks to walk. This additional
overhead each time meant that maxCommsSize was rarely actually
enabled.
We can, however, instead rely on the send/recv buffers having been
consistently sized and simply walk through the local send/recvs until
no further chunks need to be exchanged. As an additional enhancement,
the message tags are connected to chunking iteration, which allows
the setup of all send/recvs without an intermediate Allwait.
ENH: extend UPstream::probeMessage to use int64 instead of int for sizes
- "buffered" corresponds to MPI_Bsend (buffered send),
whereas the old name "blocking" is misleading since the
regular MPI_Send also blocks until completion
(ie, buffer can be reused).
ENH: IPstream::read() returns std::streamsize instead of label (#3152)
- previously returned a 'label' but std::streamsize is consistent with
the input parameter and will help with later adjustments.
- use <label> instead of <int> for internal accounting of the message
size, for consistency with the underyling List<char> buffers used.
- improve handling for corner case of IPstream receive with
non-blocking, although this combination is not used anywhere
- was previously limited to 'char' whereas gatherv/scatterv
already supported various integer and float types
STYLE: rebundle allToAll declarations with macros
ENH: provide a version of allToAllConsensus returning the Map
- simplifies use and avoids ambiguities in the send/recv parameters
- the Map version will now also transmit zero value data if they exist
in the Map. Unlike the List version, zero values are not necessary to
signal connectivity with a Map.
COMP: forwarding template parameters for NBX routines
ENH: consolidate PstreamBuffers size exchange options
- had a variety of nearly identical backends for all-to-all,
gather/scatter. Now combined internally with a dispatch enumeration
which provides better control over which size exchange algorithm
is used.
DEFEATURE: remove experimental full-NBX PstreamBuffers variant
- no advantages seen compared to the hybrid NBX/PEX approach.
Removal reduces some code cruft.
DEFEATURE: remove experimental "double non-blocking" NBX version
- the idea was to avoid blocking receives for very large data transfers,
but that is usually better accomplished with a hybrid NBX/PEX approach
like PstreamBuffers allows
- previously had an additional stack for freedRequests_,
which were used to 'remember' locations into the list of
outstandingRequests_ that were handled by 'waitRequest()'.
This was principally done for sanity checks on shutdown,
but we now just test for any outstanding requests that
are *not* MPI_REQUEST_NULL instead (much simpler).
The framework with freedRequests_ also had a provision to 'recycle'
them by popping from that stack, but this is rather fragile since it
would only triggered by some collectives
(MPI_Iallreduce, MPI_Ialltoall, MPI_Igather, MPI_Iscatter)
with no guarantee that these will all be properly removed again.
There was also no pruning of extraneous indices.
ENH: consolidate internal reset/push of requests
- replace duplicate code with inline functions
reset_request(), push_request()
ENH: null out trailing requests
- extra safety (paranoia) for the UPstream::Request versions
of finishedRequests(), waitAnyRequest()
CONFIG: document nPollProcInterfaces in etc/controlDict
- still experimental, but at least make the keyword known
- simplifies code, consistent with other matrix transfer functions.
Use a setter method.
STYLE: AMIInterpolation::upToDate(bool) setter method
ENH: add guards to avoid float-compressed transfer of integral types
STYLE: drop unused debug member from abstract interface classes
- since ensight format is always float and also always written
component-wise, perform the double -> float narrowing when
extracting the components. This reduces the amount of data
transferred between processors.
ENH: avoid vtk/ensight parallel communication of empty messages
- since ensight writes by element type (eg, tet, hex, polyhedral) the
individual written field sections will tend to be relatively sparse.
Skip zero-size messages, which should help reduce some of the
synchronization bottlenecks.
ENH: use 'data chunking' when writing ensight files in parallel
- since ensight fields are written on a per-element basis, the
corresponding segment can become rather sparsely distributed. With
'data chunking', we attempt to get as many send/recv messages in
before flushing the buffer for writing. This should make the
sequential send/recv less affected by the IO time.
ENH: allow use of an external buffer when writing ensight components
STYLE: remove last vestiges of autoPtr<ensightFile> for output routines
- the very old 'writer' class was fully stateless and always templated
on an particular output type.
This is now replaced with a 'coordSetWriter' with similar concepts
as previously introduced for surface writers (#1206).
- writers change from being a generic state-less set of routines to
more properly conforming to the normal notion of a writer.
- Parallel data is done *outside* of the writers, since they are used
in a wide variety of contexts and the caller is currently still in
a better position for deciding how to combine parallel data.
ENH: update sampleSets to sample on per-field basis (#2347)
- sample/write a field in a single step.
- support for 'sampleOnExecute' to obtain values at execution
intervals without writing.
- support 'sets' input as a dictionary entry (as well as a list),
which is similar to the changes for sampled-surface and permits use
of changeDictionary to modify content.
- globalIndex for gather to reduce parallel communication, less code
- qualify the sampleSet results (properties) with the name of the set.
The sample results were previously without a qualifier, which meant
that only the last property value was actually saved (previous ones
overwritten).
For example,
```
sample1
{
scalar
{
average(line,T) 349.96521;
min(line,T) 349.9544281;
max(line,T) 350;
average(cells,T) 349.9854619;
min(cells,T) 349.6589286;
max(cells,T) 350.4967271;
average(line,epsilon) 0.04947733869;
min(line,epsilon) 0.04449639927;
max(line,epsilon) 0.06452856475;
}
label
{
size(line,T) 79;
size(cells,T) 1720;
size(line,epsilon) 79;
}
}
```
ENH: update particleTracks application
- use globalIndex to manage original parcel addressing and
for gathering. Simplify code by introducing a helper class,
storing intermediate fields in hash tables instead of
separate lists.
ADDITIONAL NOTES:
- the regionSizeDistribution largely retains separate writers since
the utility of placing sum/dev/count for all fields into a single file
is questionable.
- the streamline writing remains a "soft" upgrade, which means that
scalar and vector fields are still collected a priori and not
on-the-fly. This is due to how the streamline infrastructure is
currently handled (should be upgraded in the future).
- if the system/decomposeParDict is missing, skip check for matching
number of processor dirs. This can make job dispatch easier.
Does not apply if -decomposeParDict was explicitly specified.
STYLE: adjust naming of host/slaves in argList
- previously always called dlclose on opened libraries when destroying
the dlLibraryTable. However, by force closing the libraries the
situation can arise that the library is missing its own code that it
needs on unload (#1524). This is also sometimes evident when closing
VTK libraries for runTimePostProcessing (#354, #1585).
- The new default is to not forcibly dlclose any libraries, unless
the dlcloseOnTerminate OptimisationSwitch specifies otherwise.
- The dlLibraryTable::close() method can be used to explicitly close
all libraries and clear the list.
- The dlLibraryTable::clear() method now only clears the entries,
without a dlclose.
- timeVaryingUniformFixedValue -> uniformFixedValue
- allows a variety of functions (eg, coded, expressions, tables, ...)
- more similarity to finiteVolume patch type
STYLE: remove unused timeVarying... from etc/controlDict
- remove (unused) Istream constructors, prune some unused methods,
rationalize write() vs writeDict().
Deprecate inconsistent construction order.
- handle empty names for ".ftr" surface patches (for plain triSurface
format) with double-quoted strings for more reliable streaming.
Written on a single line.
This is _backward_ compatible, but if users have been parsing these
files manually, they will need to adjust their code.
Previously:
```
(
frt-fairing:001%1
empty
windshield:002%2
empty
...
)
```
Updated (with example handling of empty name):
```
(
frt-fairing:001%1 empty
windshield:002%2 ""
...
)
```
- now use debug 2 for scanner and debug 4 for parser.
Provided better feedback about what is being parsed (debug mode)
- relocate debug application to applications/tools/foamExprParserInfo
- this cannot be left as a configurable value (on windows), since it
needs to be enabled even prior to reading the etc/controlDict file,
in case the OpenFOAM installation path itself contains spaces.
- can be used to check the validity of input values.
Example:
dict.getCheck<label>("nIters", greaterOp1<label>(0));
dict.getCheck<scalar>("relax", scalarMinMax::zero_one());
- use 'get' prefix for more regular dictionary methods.
Eg, getOrDefault() as alternative to lookupOrDefault()
- additional ops for convenient construction of predicates
ENH: make dictionary writeOptionalEntries integer
- allow triggering of Fatal if default values are used
ENH: additional scalarRange static methods: ge0, gt0, zero_one
- use GREAT instead of VGREAT for internal placeholders
- additional MinMax static methods: gt, le
- having whitespace in fileName can be somewhat fragile since it means
that the fileName components do not necessarily correspond to a
'Foam::word'. But in many cases it will work provided that spaces
are not present in the final portion of the simulation directory
itself.
InfoSwitches
{
// Allow space character in fileName (use with caution)
allowSpaceInFileName 0;
}
- now use doClean=true as default for fileName::validate(). Was false.
Unlike fileName::clean() this requires no internal string rewrite
since the characters are being copied. Also handle any path
separator transformations (ie, backslash => forward slash) at the
same time. This makes it resemble the std::filesystem a bit more.
- Extended runTimePostProcessing to include access to "live"
simulation objects such a geometry patches and sampled surfaces
stored on the "functionObjectObjects" registry.
- Add 'live' runTimePostProcessing of cloud data.
Extracts position and fields from the cloud via its objectRegistry writer
- For the "live" simulation objects, there are two new volume filters
that work directly with the OpenFOAM volume fields:
* iso-surface
* cutting planes
Both use the VTK algorithms directly and support multiple values.
Eg, can make multiple iso-levels or multiple planes parallel to each
other.
- When VTK has been compiled with MPI-support, parallel rendering will
be used.
- Additional title text properties (shadow, italic etc)
- Simplified handling of scalar-bar and visibility switches
- Support multiple text positions. Eg, for adding watermark text.
- changed the sectorCoeffs keyword to 'point' from 'axisPt'
for more similarity with other dictionaries.
Continue to accept 'axisPt' for compatibility.
