- move construct from components. Construct with optional IO control
- separate init() method (as per polyMesh) to delay evaluation of
globalData and base geometry.
- faMesh removeFiles method
ENH: faBoundaryMeshEntries for reading faBoundary files without a mesh
ENH: adjust debug output for {fa,fae,fv,fvs}patchField::New
- add alternative constraint type selection for faePatchField.
- unify handling of "patchType" reading.
Make less noisy when reporting dictionary defaults.
- allows reuse by finiteArea, for example.
- simplify edge looping with face thisLabel/nextLabel method
ENH: additional storage checks for mesh weights (faMesh + fvMesh)
- allow finite-area field decomposition without edge weights.
STYLE: use tmp New in various places. Simpler updateGeom check
- bundles frequently used 'gather/scatter' patterns more consistently.
- combineAllGather -> combineGather + broadcast
- listCombineAllGather -> listCombineGather + broadcast
- mapCombineAllGather -> mapCombineGather + broadcast
- allGatherList -> gatherList + scatterList
- reduce -> gather + broadcast (ie, allreduce)
- The allGatherList currently wraps gatherList/scatterList, but may be
replaced with a different algorithm in the future.
STYLE: PstreamCombineReduceOps.H is mostly unneeded now
STYLE: LduInterfaceFieldPtrsList as alias instead of a class
STYLE: define patch lists typedefs when defining the base patch
- eg, polyPatchList typedef within polyPatch.H
INT: relocate GeometricField::Boundary -> GeometricBoundaryField
- was internal to GeometricField but moving it outside simplifies
forward declarations etc. Code adapted from openfoam.org
Two problems:
- flipping inside snappyHexMesh is not done in a parallel
consistent way. So e.g. the octree-cached inside/outside information
has already been calculated. For now flipping of
distributedTriSurfaceMesh is disabled.
- octree-cached inside/outside information was using already
cached information and would only work for outwards pointing
volumes
- when writing surface formats (eg, vtk, ensight etc) the sampled
surfaces merge the faces/points originating from different
processors into a single surface (ie, patch gatherAndMerge).
Previous versions of mergePoints simply merged all points possible,
which proves to be rather slow for larger meshes. This has now been
modified to only consider boundary points, which reduces the number
of points to consider. As part of this change, the reference point
is now always equivalent to the min of the bounding box, which
reduces the number of search loops. The merged points retain their
original order.
- inplaceMergePoints version to simplify use and improve code
robustness and efficiency.
ENH: make PrimitivePatch::boundaryPoints() less costly
- if edge addressing does not already exist, it will now simply walk
the local face edges directly to define the boundary points.
This avoids a rather large overhead of the full faceFaces,
edgeFaces, faceEdges addressing.
This operation is now more important since it is used in the revised
patch gatherAndMerge.
ENH: topological merge for mesh-based surfaces in surfaceFieldValue
- gather/scatter types of operations can avoid AllToAll communication
and use simple MPI gather (or scatter) to establish the receive sizes.
New methods: finishedGathers() / finishedScatters()
- eliminate redundant size_ accounting
- drop extra 'Container' template parameter and replace functionality
with more flexible pack/unpack methods.
There is also a pack() method that handles indirect lists of lists
that can be used, for example, to pack a patch slice of faces.
Drop the 'operator()' method in favour of unpack to expose and properly
document the conversion. Should revisit the corresponding code in
some places for optimization potential.
- align some method names with globalIndex:
totalSize(), maxSize() etc
- less communication than gatherList/scatterList
ENH: refine send granularity in Pstream::exchange
STYLE: ensure PstreamBuffers and defaultCommsType agree
- simpler loops for lduSchedule
- for metis-like graphs there is no guarantee that a zero-sized graph
has an offsets list with size 1 or size 0, so always use
numCells = max(0, xadj.size()-1)
this was already done in most places, but missed in the
decomposeGeneral method
STYLE: use sumOp<label>() instead of plusOp<label>()
- PstreamBuffers nProcs() and allProcs() methods to recover the rank
information consistent with the communicator used for construction
- allowClearRecv() methods for more control over buffer reuse
For example,
pBufs.allowClearRecv(false);
forAll(particles, particlei)
{
pBufs.clear();
fill...
read via IPstream(..., pBufs);
}
This preserves the receive buffers memory allocation between calls.
- finishedNeighbourSends() method as compact wrapper for
finishedSends() when send/recv ranks are identically
(eg, neighbours)
- hasSendData()/hasRecvData() methods for PstreamBuffers.
Can be useful for some situations to skip reading entirely.
For example,
pBufs.finishedNeighbourSends(neighProcs);
if (!returnReduce(pBufs.hasRecvData(), orOp<bool>()))
{
// Nothing to do
continue;
}
...
On an individual basis:
for (const int proci : pBufs.allProcs())
{
if (pBufs.hasRecvData(proci))
{
...
}
}
Also conceivable to do the following instead (nonBlocking only):
if (!returnReduce(pBufs.hasSendData(), orOp<bool>()))
{
// Nothing to do
pBufs.clear();
continue;
}
pBufs.finishedNeighbourSends(neighProcs);
...
The logic was not maintaining consistent sets of constraints
on different processors. A single processor with a full
match (very easy with 0 local faces) would invalidate
adding the constraint.
- decomposePar: -no-fields to suppress decomposition of fields
- makeFaMesh: -no-decompose to suppress creation of *ProcAddressing
and fields, -no-fields to suppress decomposition of fields only
- the patch remapping in faFieldDecomposer calls weights
internalField() which can trigger parallel communication on the
complete mesh for some processors only (ie, blocks).
Force a priori creation of weights instead.
- ensure that the complete mesh (reconstruction helper)
is serial when adding patches.
- when creating a finite-area mesh in parallel, need to determine
the equivalent ProcAddressing for the faMesh.
In the faceProcAddressing the collected and sorted order was being
scattered directly back to the individual processors instead of only
the sections relevant to each particular processor.
This caused the observed jumbled order for reconstructed fields.
- allows reuse of an int64_t scotch library with label-size 32
and/or label-size 64.
COMP: prefer scotch/metis/kahip libraries with label-size qualifiers
- as noted in #2200, mpirun may insert mpi libraries higher in the
library loader which can cause masking of our ThirdParty libraries
of the same name. With scotch (for example), the operating system
may have an int32 version installed but we have an int64 version
compiled under ThirdParty. Runing in serial is fine, but in parallel
we resolve to the (incorrect) system version due to the adjustments
in mpirun.
- adjust the ThirdParty make scripts to also create corresponding
links (eg, 'ln -s libscotch.so libscotch-int64.so') and prefer
linkage with these qualified libraries.
Eg, -L$(SCOTCH_LIB_DIR) -lscotch$(SCOTCH_LIBNAME_SUFFIX)
this prevent accidental runtime linkage with the system versions.
STYLE: simplify scotch interface code by using local functions
- A bare-bones reconstructor for finiteArea meshes when processor
meshes are available (in parallel) but an equivalent serial faMesh
is needed for reconstruction or decomposition.
In these situations, a serial version of the faMesh is needed,
but preferably without reconstructing the entire volume mesh.
It uses the finiteVolume faceProcAddressing in addition to
the geometric information available from the underlying polyMesh.
The resulting equivalent faMesh can be used for basic operations,
but caution should be exercised before attempting large operations.
- can be used for block-like meshes that are not aligned with the global
coordinate directions. Alternatively, for general testing purposes.
Example,
method simple;
coeffs
{
n ( 2 2 2 );
transform
{
origin (-0.15 0.15 0);
e1 (1 1 0);
e3 (0 0 1);
}
}
- make regionName an optional constructor parameter, which eliminates
a separate set of constructors and construction tables. Adjust
internals to treat a missing/empty regionName as a no-op.
- pass in fallback dictionary content via new IOdictionary constructor
with a pointer
ENH: further relax check for matching number of processor dirs
- if the "numberOfSubdomains" entry is missing (or even zero)
ignore checks of processor dirs as meaningless.
- deprecated Feb-2018, but not marked as such.
The set() method originally enforce an additional run-time check
(Fatal if pointer was already set), but this was rarely used.
In fact, the set() method was invariably used in constructors
where the pointer by definition was unset.
Can now mark as deprecated to catch the last of these.
We prefer reset() for similarity with std::unique_ptr
Eg,
FOAM_EXTRA_CXXFLAGS="-DFoam_autoPtr_deprecate_setMethod" wmake
- prefix FOAM_MPI and library directories with 'sys-' for system
versions for uniform identication.
WM_MPLIB | libdir (FOAM_MPI) | old naming |
SYSTEMMPI | sys-mpi | mpi |
SYSTEMOPENMPI | sys-openmpi | openmpi-system |
- prefix preferences with 'prefs.' to make them more easily
identifiable, and update bin/tools/create-mpi-config accordingly
Old name: config.{csh,sh}/openmpi
New name: config.{csh,sh}/prefs.openmpi
- additional mpi preferences now available:
* prefs.intelmpi
* prefs.mpich
...
CONFIG: added hook for EASYBUILDMPI (eb-mpi), somewhat like USERMPI
- EasyBuild uses mpicc when compiling, so no explicit wmake rules are
used
ENH: support different major versions for system openmpi
- for example, with
WM_MPLIB=SYSTEMOPENMPI2
defines FOAM_MPI=sys-openmpi2 and thus creates lib/sys-openmpi2
ENH: centralize handling of mpi as 'mpi-rules'
Before:
sinclude $(GENERAL_RULES)/mplib$(WM_MPLIB)
sinclude $(DEFAULT_RULES)/mplib$(WM_MPLIB)
ifeq (,$(FOAM_MPI_LIBBIN))
FOAM_MPI_LIBBIN := $(FOAM_LIBBIN)/$(FOAM_MPI)
endif
After:
include $(GENERAL_RULES)/mpi-rules
- also allows variants such as SYSTEMOPENMPI2 to be handled separately
- returns a range of `int` values that can be iterated across.
For example,
for (const int proci : Pstream::subProcs()) { ... }
instead of
for
(
int proci = Pstream::firstSlave();
proci <= Pstream::lastSlave();
++proci
)
{
...
}
- returns a range of `int` values that can be iterated across.
For example,
for (const int proci : Pstream::allProcs()) { ... }
instead of
for (label proci = 0; proci < Pstream::nProcs(); ++proci) { ... }
- introduce WM_COMPILE_CONTROL variable to convey control information
into the build rules.
The convention (as per spack):
- '+' to select a feature
- '~' to deselect a feature
Eg, to select the gold linker, and disable openmp
(spaces are not required):
WM_COMPILE_CONTROL="+gold ~openmp"
CONFIG: accept FOAM_EXTRA_LDFLAGS for AMD, gold, Mingw linkers
CONFIG: generalize PROJECT_LIBS (-ldl used almost universally)
- When compiling additional modules or user code, we need more control
for the installation locations beyond the usual FOAM_USER_LIBBIN,
FOAM_SITE_LIBBIN, FOAM_LIBBIN, and wish to have these values be
modifiable without editing files.
- provide wmake rules for handling standard defaults:
* GENERAL_RULES/module-path-user
* GENERAL_RULES/module-path-group
* GENERAL_RULES/module-path-project
which are incorporated as follows:
Make/options:
include $(GENERAL_RULES)/module-path-user
Make/files:
LIB = $(FOAM_MODULE_LIBBIN)/libMyLibrary
By default these would compile into FOAM_USER_{APPBIN,LIBBIN} but
could be adjusted at compilation time. For example,
```
wmake -module-prefix=/path/my-install-location
```
Or
```
./Allwmake -module-prefix=/path/my-install-location
./Allwmake -prefix=/path/my-install-location
```
Or
```
FOAM_MODULE_PREFIX=/path/my-install-location ./Allwmake
```
ENH: add -no-recursion option for AllwmakeParseArguments
- more descriptive naming than the -fromWmake option (still supported)
- remove wmake/scripts/wmake.{cmake,wmake}-args since the -prefix
handling and -no-recursion is now directly handled by AllwmakeParseArguments
- previously introduced `getOrDefault` as a dictionary _get_ method,
now complete the transition and use it everywhere instead of
`lookupOrDefault`. This avoids mixed usage of the two methods that
are identical in behaviour, makes for shorter names, and promotes
the distinction between "lookup" access (ie, return a token stream,
locate and return an entry) and "get" access (ie, the above with
conversion to concrete types such as scalar, label etc).
- Provide Allwmake-mpi scripts for handling the MPI-only build segment.
- Adjust Make/options to support a FOAM_MPI_LIBBIN target location.
This will simply default to FOAM_LIBBIN/FOAM_MPI, but allows different
types of builds with out-of-tree targets.
Example,
Build OpenFOAM with default MPI settings (eg, system openmpi)
```
./Allwmake
```
Rebuild MPI-layers with different MPI.
For any given and known type
```
othermpi()
{
export WM_MPLIB=OPENMPI
export FOAM_MPI=openmpi-3.1.3
export OPAL_PREFIX=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/$FOAM_MPI
export MPI_ARCH_PATH="$OPAL_PREFIX"
}
```
May wish to build/rebuild ptscotch
```
(
othermpi
$WM_THIRD_PARTY_DIR/makeSCOTCH
)
```
Rebuild the mpi-dependent parts. Can also provide install location
```
for script in $(find src -name Allwmake-mpi)
do
(
other_mpi
export FOAM_MPI_LIBBIN="$HOME/tmp/install-prefix/lib/$FOAM_MPI"
)
done
```
- dependency handling relocated from cmakeFunctions to wmakeFunctions
and reused for mpi-versioned builds. This allows more checks for
configuration parameters and removes hard-code build path
information.
CONFIG: remove spurious mplibHPMPI entries
CONFIG: remove ADIOS1 rules (antiquated)
