so the write thread does not have to do any parallel communication. This avoids
the bugs in the threading support in OpenMPI.
Patch contributed by Mattijs Janssens
Resolves bug-report https://bugs.openfoam.org/view.php?id=2669
- warn or fatal if Pstream::init or Pstream::exit are called multiple
times.
- additional Pstream::initNull method as failsafe to initialize MPI
when the underlying OpenFOAM process is not running in parallel but
the application still needs MPI.
- Pstream::exit() can now also be called without having used MPI::init(),
which means it can be used to cleanup serial process or for
applications that used the special purpose Pstream::initNull()
mechanism.
This is controlled by the setting maxCommsSize in etc/controlDict which
specifies the max number of bytes per exchange. If set to <= 0 it
is ignored. This max size of messages is important when doing e.g.
load balancing which can send over whole meshes.
//- Disallow default shallow-copy assignment
//
// Assignment of UList<T> may need to be either shallow (copy pointer)
// or deep (copy elements) depending on context or the particular type
// of list derived from UList and it is confusing and prone to error
// for the default assignment to be either. The solution is to
// disallow default assignment and provide separate 'shallowCopy' and
// 'deepCopy' member functions.
void operator=(const UList<T>&) = delete;
//- Copy the pointer held by the given UList.
inline void shallowCopy(const UList<T>&);
//- Copy elements of the given UList.
void deepCopy(const UList<T>&);
Contributed by Mattijs Janssens.
1. Any non-blocking data exchange needs to know in advance the sizes to
receive so it can size the buffer. For "halo" exchanges this is not
a problem since the sizes are known in advance but or all other data
exchanges these sizes need to be exchanged in advance.
This was previously done by having all processors send the sizes of data to
send to the master and send it back such that all processors
- had the same information
- all could work out who was sending what to where and hence what needed to
be received.
This is now changed such that we only send the size to the
destination processor (instead of to all as previously). This means
that
- the list of sizes to send is now of size nProcs v.s. nProcs*nProcs before
- we cut out the route to the master and back by using a native MPI
call
It causes a small change to the API of exchange and PstreamBuffers -
they now return the sizes of the local buffers only (a labelList) and
not the sizes of the buffers on all processors (labelListList)
2. Reversing the order of the way in which the sending is done when
scattering information from the master processor to the other
processors. This is done in a tree like fashion. Each processor has a
set of processors to receive from/ send to. When receiving it will
first receive from the processors with the least amount of
sub-processors (i.e. the ones which return first). When sending it
needs to do the opposite: start sending to the processor with the
most amount of sub-tree since this is the critical path.
So now we can read multiple times (using UIPstream) from PstreamBuffers
PstreamBuffers pBufs
UIPstream str1(procI, pBufs);
str1>> ..
UIPstream str2(procI, pBufs);
str1>> ..
