ENH: avoid prior communication when using mapDistribute

- in most cases can simply construct mapDistribute with the sendMap
  and have it take care of communication and addressing for the
  corresponding constructMap.

  This removes code duplication, which in some cases was also using
  much less efficient mechanisms (eg, combineReduce on list of
  lists, or an allGatherList on the send sizes etc) and also
  reduces the number of places where Pstream::exchange/exchangeSizes
  is being called.

ENH: reduce communication in turbulentDFSEMInlet

- was doing an allGatherList to populate a mapDistribute.
  Now simply use PstreamBuffers mechanisms directly.
This commit is contained in:
Mark Olesen
2023-02-14 11:24:22 +01:00
parent 1ce7a62209
commit fb69a54bc3
12 changed files with 105 additions and 534 deletions

View File

@ -146,7 +146,6 @@ Foam::parLagrangianDistributor::distributeLagrangianPositions
labelListList subMap;
// Allocate transfer buffers
PstreamBuffers pBufs(Pstream::commsTypes::nonBlocking);
@ -283,33 +282,24 @@ Foam::parLagrangianDistributor::distributeLagrangianPositions
lpi.rename(cloudName);
}
// Work the send indices (subMap) into a mapDistributeBase
labelListList sizes(Pstream::nProcs());
labelList& nsTransPs = sizes[Pstream::myProcNo()];
nsTransPs.setSize(Pstream::nProcs());
forAll(subMap, sendProcI)
{
nsTransPs[sendProcI] = subMap[sendProcI].size();
}
// Send sizes across. Note: blocks.
Pstream::combineReduce(sizes, Pstream::listEq());
// Until now (FEB-2023) we have always used processor ordering for the
// construct map (whereas mapDistribute has local transfers first),
// so we'll stick with that for now, but can likely just use the subMap
// directly with mapDistribute and have it determine the constructMap.
labelList recvSizes;
Pstream::exchangeSizes(subMap, recvSizes);
labelListList constructMap(Pstream::nProcs());
label constructSize = 0;
forAll(constructMap, procI)
labelListList constructMap(Pstream::nProcs());
forAll(constructMap, proci)
{
const label nRecv = sizes[procI][UPstream::myProcNo()];
labelList& map = constructMap[procI];
map.setSize(nRecv);
forAll(map, i)
{
map[i] = constructSize++;
}
const label len = recvSizes[proci];
constructMap[proci] = identity(len, constructSize);
constructSize += len;
}
return autoPtr<mapDistributeBase>::New
(
constructSize,