- for interface polling previously required that both send and recv
requests were completed before evaluating (values or matrix update).
However, only the recv needs to be complete, which helps disentangle
the inter-rank waiting.
NB: this change is possible following (1f5cf3958b) that replaced
UPstream::resetRequests() call in favour of UPstream::waitRequests()
- UPstream exit with a non-zero return code is raised by things like
exit(FatalError) which means there is no reason to believe that
any/all of the buffered sends, requests etc have completed.
Thus avoid detaching buffers, freeing communicators etc in this
situation. This makes exit(1) behave much more like abort(), but
without any stack trace. Should presumably help with avoiding
deadlocks on exit.
ENH: support transfer from a wrapped MPI request to global list
- allows coding with a list UPstream::Request and subsequently either
retain that list or transfer into the global list.
- can use traits to distinguish label vs scalar types and
setComponents to properly index into single or multi-component
types without needing template specialisations for the task.
This avoids the need for a concrete translation unit and the
reported problem of multiply-defined specialisations when the header
is included in different places.
- the default (uninitialised) value for edge connections of -1
could be confused with a tagged finiteArea patch, which used
(-patchid-1) encoding. This would lead to messages about erroneous
processor-processor addressing, but is in fact an mismatched edge
connection.
Now tag the finiteArea patch as (-patchid-2) to avoid this ambiguity
and correctly generate an "Undefined connection:" message instead.
Properly flush the VTP writers before raising a FatalError
to ensure that they are not prematurely truncated.
Open Point:
The base problem of "Undefined connection:" is largely related to
multiply-connected face edges (ie, from the underlying volume mesh).
Not easily remedied in the finiteArea generation.
TUT: basic finiteArea setup on motorBike
- have read(nullptr, count) and readRaw(nullptr, count) act like a
forward seek instead of failing.
This lets it be used to advance through a file without needing to
allocate (and discard) storage space etc.
- construct from components, or use word::null to ensure
consistent avoid naming between IOobject vs dimensioned type.
- support construct with parameter ordering as per DimensionedField
ENH: instantiate a uniformDimensionedLabelField
- eg, for registering standalone integer counters
- directory discovery originally designed for a sub-dir location
(eg, etc/openfoam) but failed if called from within the sub-dir
itself.
Now simply assume it is located in the project directory or the etc/
sub-dir, so that it can also be relocated into the project directory
in the future (pending changes to RPM and debian packaging)
- for querying all outstanding requests:
if (UPstream::finishedRequests(startRequest)) ...
if (UPstream::finishedRequests(startRequest, -1)) ...
- for querying slice of outstanding requests:
if (UPstream::finishedRequests(startRequest, 10)) ...
- simplifies communication structuring with intra-host communication.
Can be used for IO only, or for specialised communication.
Demand-driven construction. Gathers the SHA1 of host names when
determining the connectivity. Internally uses an MPI_Gather of the
digests and a MPI_Bcast of the unique host indices.
NOTE:
does not use MPI_Comm_splt or MPI_Comm_splt_type since these
return MPI_COMM_NULL on non-participating process which does not
easily fit into the OpenFOAM framework.
Additionally, if using the caching version of
UPstream::commInterHost() and UPstream::commIntraHost()
the topology is determined simultaneously
(ie, equivalent or potentially lower communication).
- make sizing of commsStruct List demand-driven as well
for more robustness, fewer unneeded allocations.
- fix potential latent bug with allBelow/allNotBelow for proc 0
(linear communication).
ENH: remove unused/unusable UPstream::communicator optional parameter
- had constructor option to avoid constructing the MPI backend,
but this is not useful and inconsistent with what the reset or
destructor expect.
STYLE: local use of UPstream::communicator
- automatically frees communicator when it leaves scope
- these are primarily when encountering sparse (eg, inter-host)
communicators. Additional UPstream convenience methods:
is_rank(comm)
=> True if process corresponds to a rank in the communicators.
Can be a master rank or a sub-rank.
is_parallel(comm)
=> True if parallel algorithm or exchange is used on the process.
same as
(parRun() && (nProcs(comm) > 1) && is_rank(comm))
- for robustness with small edges (which can occur with snappy meshes),
the Le() and magLe() are limited to SMALL (commit a0f1e98d24).
Now use factor sqrt(1/3) in the components to maintain magnitude of 1.
ENH: add fvMesh::unitSf() and faMesh::unitLe() methods
- simple wrappers around Sf()/magSf() and Le()/magLe() but with
the potential for additional/alternative corrections.
STYLE: thisDb() in faMesh code to simplify future changes in storage
ENH: do not register finite-area geometric fields
- consistent with finite-volume treatment
- replace the "one-size-fits-all" approach of tensor field inv()
with individual 'failsafe' inverts.
The inv() field function historically just checked the first entry
to detect 2D cases and adjusted/readjusted *all* tensors accordingly
(to avoid singularity tensors and/or noisy inversions).
This seems to have worked reasonably well with 3D volume meshes, but
breaks down for 2D area meshes, which can be axis-aligned
differently on different sections of the mesh.
- with (nPollProcInterfaces < 0) it does the following:
- loop, waiting for some requests to finish
- for each out-of-date interface, check if its associated
requests have now finished (ie, the ready() check).
- if ready() -> call updateInterfaceMatrix()
In contrast to (nPollProcInterfaces > 0) which loops a specified
number of times with several calls to MPI_Test each time, the
(nPollProcInterfaces < 0) variant relies on internal MPI looping
within MPI_Waitsome to progress communication.
The actual dispatch still remains non-deterministic (ie, waiting for
some requests to finish does not mean that any particular interface
is eligible for update, or in any particular order). However, using
Waitsome places the tight looping into the MPI layer, which results
in few calls and eliminates behaviour dependent on the value of
nPollProcInterfaces.
TUT: add polling to windAroundBuildings case (for testing purposes)
