- the old Pstream::scatter routines (which were largely a misnomer)
have been superseded by various broadcast routines, but were left in
the code with #ifndef/#ifdef Foam_Pstream_scatter_nobroadcast
guards. Now noisily deprecate them, and remove the old manual tree
communication in favour of MPI broadcast and/or
serialize/de-serialize with wrapped Pstream::broadcast
- consolidate various gather methods to include the communication
structure directly. No functional change, but reduces the number of
methods.
ENH: add parallel guard to UPstream::whichCommunication() method
- returns List::null() as the schedule for non-parallel instead
of an inappropriate linear or tree schedule
ENH: Pstream::listGatherValues, Pstream::listScatterValues
- like the existing UPstream versions but supporting non-contiguous
- explicit use of UPstream::worldComm in globalIndex methods
for more clarity
- adjust method declaration ordering:
de-emphasize the processor-local convenience methods
- consistent use of leading tag dispatch,
remove unused enum-based dispatch tag
- add begin()/cbegin() with offset (as per List containers)
BUG: missing use of communicator in globalIndex gatherNonLocal
- does not affect any existing code (which all use worldComm anyhow)
- MPI_Gatherv requires contiguous data, but a byte-wise transfer can
quickly exceed the 'int' limits used for MPI sizes/offsets. Thus
gather label/scalar components when possible to increase the
effective size limit.
For non-contiguous types (or large contiguous data) now also
reverts to manual handling
ENH: handle contiguous data in GAMGAgglomeration gather values
- delegate to globalIndex::gatherValues static method (new)
- used low-level MPI gather, but the wrapping routine contains an
additional safety check for is_contiguous which is not defined for
various std::pair<..> combination.
So std::pair<label,vector> (which is actually contiguous, but not
declared as is_contiguous) would falsely trip the check.
Avoid by simply gathering unbundled values instead.
- UPstream::mpiGather (MPI_Gather) - used by Pstream::listGatherValues
- UPstream::mpiScatter (MPI_Scatter) - used by Pstream::listScatterValues
These are much simpler forms for gather/scatter of fixed-sized
contiguous types data types (eg, primitives, simple pairs etc).
In the gather form, creates a list of gathered values on the master
process. The subranks have a list size of zero.
Similarly, scatter will distribute a list of values to single values
on each process.
Instead of
labelList sendSizes(Pstream::nProcs());
sendSizes[Pstream::myProcNo()] = sendData.size();
Pstream::gatherList(sendSizes);
Can write
const labelList sendSizes
(
UPstream::listGatherValues<label>(sendData.size())
);
// Less code, lower overhead and list can be const.
For scattering an individual value only,
instead of
labelList someValues;
if (Pstream::master()) someValues = ...;
Pstream::gatherList(sendSizes);
const label localValue
(
someValues[Pstream::myProcNo()]
);
Can write
labelList someValues;
if (Pstream::master()) someValues = ...;
Pstream::gatherList(sendSizes);
const label localValue
(
UPstream::listScatterValues<label>(someValues)
);
Can of course also mix listGatherValues to assemble a list on master
and use Pstream::scatterList to distribute.
ENH: adjusted globalIndex gather methods
- added mpiGather() method [contiguous data only] using MPI_Gatherv
- respect localSize if gathering master data to ensure that a
request for 0 master elements is properly handled.
