- using the List containers, and not their low-level data_bytes(),
size_bytes() methods is more convenient and allows future
adjustments to be centralized
ENH: trivial intptr_t wrapper for MPI_Win
STYLE: minor adjustments to mpirunDebug
- 'if constexpr (...)'
* instead of std::enable_if
* terminate template recursion
* compile-time elimination of code
- use C++14 '_t', '_v' versions,
eg, std::is_integral_v<T> instead of std::is_integral<T>::value
- std::begin, std::end, std::void_t instead of prev stdFoam versions
- provide is_contiguous_v<..> as short form of is_contiguous<..>::value
with the additional benefit of removing any cv qualifiers.
ENH: include is_rotational_vectorspace trait
- tests for vector-space and nComponents > 1 (ie, not sphericalTensor)
ENH: improve robustness of pTraits_.. tests by removing cv qualifiers
- can use UList signature since the routines do not resize the list
or attempt to broadcast it: useful for SubList handling.
ENH: add IPstream/OPstream send/recv static methods
- provide a globalIndex::calcOffsets() taking an indirect list, which
enables convenient offsets calculation from a variety of inputs.
- new CompactListList unpack variant: copy_unpack()
The copy_unpack() works somewhat like std::copy() in that it writes
the generated sublists to iterator positions, which makes this
type of code possible:
CompactListList<label> compact = ...;
DynamicList<face> extracted;
compact.copy_unpack<face>
(
std::back_inserter(extracted),
labelRange(4, 10)
);
-and-
const label nOldFaces = allFaces.size();
allFaces.resize(allFaces + nNewFaces);
auto iter = allFaces.begin(nOldFaces);
iter = compact.copy_unpack<face>(iter, /* selection 1 */);
...
iter = compact.copy_unpack<face>(iter, /* selection 2 */);
ENH: globalIndex resize()
- can be used to shrink or grow the offsets table.
Any extension of the offsets table corresponds to 'slots'
with 0 local size.
- report location with previous good offset and the new count that
would cause overflow. Simpler to report and the (very long) list
of input sizes is not particularly useful for diagnostics either.
ENH: add globalIndex comparison operators
- for outputting lists of globalIndex
- was using UPstream::procIDs(), which returns the sub-ranks with
respect to the parent communicator. This is normally just an
identity list (single-world) but with multi-world the indexing
is incorrect. Use UPstream::allProcs() instead.
- consistent with sumOp
ENH: globalIndex with gatherNonLocal tag, and use leading dispatch tags
- useful for gather/write where the master data can be written
separately. Leading vs trailing dispatch tags for more similarity to
other C++ conventions.
- simplifies coding
* finishedRequest(), waitRequest(), waitRequests() with parRun guards
* nRequests() is noexcept
- more consistent use of UPstream::defaultCommsType in branching
- for indirect lists we use element-wise output streaming and read
back as a regular list. This approach cannot however work with
non-blocking mode - the receive buffers will simply not be filled
before attempting to read from them.
For contiguous data, the lowest overhead solution is to locally
flatten the indirect list and use the regular gather routines
for non-blocking mode. For non-contiguous data, can continue to
use the element-wise output, but cannot use non-blocking for it.
STYLE: use non-blocking consistently as default for globalIndex gather(s)
- most of the front-facing code was already using non-blocking,
but there were a few low-level routines defaulting to scheduled
(but never relied upon in the code).
- additional Pstream::broadcasts() method to serialize/deserialize
multiple items.
- revoke the broadcast specialisations for std::string and List(s) and
use a generic broadcasting template. In most cases, the previous
specialisations would have required two broadcasts:
(1) for the size
(2) for the contiguous content.
Now favour reduced communication over potential local (intermediate)
storage that would have only benefited a few select cases.
ENH: refine PstreamBuffers access methods
- replace 'bool hasRecvData(label)' with 'label recvDataCount(label)'
to recover the number of unconsumed receive bytes from specified
processor. Can use 'labelList recvDataCounts()' to recover the
number of unconsumed receive bytes from all processor.
- additional peekRecvData() method (for transcribing contiguous data)
ENH: globalIndex whichProcID - check for isLocal first
- reasonable to assume that local items are searched for more
frequently, so do preliminary check for isLocal before performing
a more costly binary search of globalIndex offsets
ENH: masterUncollatedFileOperation - bundled scatter of status
- MPI_Gatherv requires contiguous data, but a byte-wise transfer can
quickly exceed the 'int' limits used for MPI sizes/offsets. Thus
gather label/scalar components when possible to increase the
effective size limit.
For non-contiguous types (or large contiguous data) now also
reverts to manual handling
ENH: handle contiguous data in GAMGAgglomeration gather values
- delegate to globalIndex::gatherValues static method (new)
- eliminate redundant size_ accounting
- drop extra 'Container' template parameter and replace functionality
with more flexible pack/unpack methods.
There is also a pack() method that handles indirect lists of lists
that can be used, for example, to pack a patch slice of faces.
Drop the 'operator()' method in favour of unpack to expose and properly
document the conversion. Should revisit the corresponding code in
some places for optimization potential.
- align some method names with globalIndex:
totalSize(), maxSize() etc
- PstreamBuffers nProcs() and allProcs() methods to recover the rank
information consistent with the communicator used for construction
- allowClearRecv() methods for more control over buffer reuse
For example,
pBufs.allowClearRecv(false);
forAll(particles, particlei)
{
pBufs.clear();
fill...
read via IPstream(..., pBufs);
}
This preserves the receive buffers memory allocation between calls.
- finishedNeighbourSends() method as compact wrapper for
finishedSends() when send/recv ranks are identically
(eg, neighbours)
- hasSendData()/hasRecvData() methods for PstreamBuffers.
Can be useful for some situations to skip reading entirely.
For example,
pBufs.finishedNeighbourSends(neighProcs);
if (!returnReduce(pBufs.hasRecvData(), orOp<bool>()))
{
// Nothing to do
continue;
}
...
On an individual basis:
for (const int proci : pBufs.allProcs())
{
if (pBufs.hasRecvData(proci))
{
...
}
}
Also conceivable to do the following instead (nonBlocking only):
if (!returnReduce(pBufs.hasSendData(), orOp<bool>()))
{
// Nothing to do
pBufs.clear();
continue;
}
pBufs.finishedNeighbourSends(neighProcs);
...
- reduces later surprises and simplifies effort for the caller
- more flexible globalIndex scatter with auto-sized return field.
- Avoid communication for scattering into zero-sized fields.
GIT: relocate globalIndex (is independent of mesh)
STYLE: include label/scalar Fwd in contiguous.H
STYLE: unneed commSchedule include in GeometricField
