ENH: improved UPstream gather/scatter functionality

- added UPstream::allGatherValues() with a direct call to MPI_Allgather.
  This enables possible benefit from a variety of internal algorithms
  and simplifies the caller

    Old:
        labelList nPerProc
        (
            UPstream::listGatherValues<label>(patch_.size(), myComm)
        );
        Pstream::broadcast(nPerProc, myComm);

    New:

        const labelList nPerProc
        (
            UPstream::allGatherValues<label>(patch_.size(), myComm)
        );

- Pstream::allGatherList uses MPI_Allgather for contiguous values
  instead of the hand-rolled tree walking involved with
  gatherList/scatterList.

-
- simplified the calling parameters for mpiGather/mpiScatter.

  Since send/recv data types are identical, the send/recv count
  is also always identical. Eliminates the possibility of any
  discrepancies.

  Since this is a low-level call, it does not affect much code.
  Currently just Foam::profilingPstream and a UPstream internal.

BUG: call to MPI_Allgather had hard-coded MPI_BYTE (not the data type)

- a latent bug since it is currently only passed char data anyhow
This commit is contained in:
Mark Olesen
2023-05-17 14:26:05 +02:00
parent b687c4927c
commit 5eebc75845
15 changed files with 140 additions and 145 deletions

View File

@ -266,9 +266,8 @@ int main(int argc, char *argv[])
UPstream::mpiGather
(
myDigest.cdata_bytes(), // Send
SHA1Digest::max_size(), // Num send per proc
digests.data_bytes(), // Recv
SHA1Digest::max_size(), // Num recv per proc
SHA1Digest::max_size(), // Num send/recv per rank
UPstream::commGlobal()
);
}

View File

@ -62,10 +62,10 @@ void writeProcStats
)
{
// Determine surface bounding boxes, faces, points
List<treeBoundBox> surfBb(Pstream::nProcs());
surfBb[Pstream::myProcNo()] = treeBoundBox(s.points());
Pstream::gatherList(surfBb);
List<treeBoundBox> surfBb
(
UPstream::listGatherValues<treeBoundBox>(treeBoundBox(s.points()))
);
labelList nPoints(UPstream::listGatherValues<label>(s.points().size()));
labelList nFaces(UPstream::listGatherValues<label>(s.size()));