used in conjunction with the new loadBalancing option in constant/chemistryProperties:
loadBalancing on;
which enables per-cell CPU time caching used by the loadBalancer to redistribute
the mesh. Currently this option is only provided for chemistry integration but
the implementation is general and in future options will be provided to balance
other local cell loads, in particular Lagrangian particles.
The loadBalancer in enabled by specifying a distributor entry in
constant/dynamicMeshDict, e.g.
distributor
{
type loadBalancer;
libs ("libfvMeshDistributors.so");
multiConstraint true;
// How often to redistribute
redistributionInterval 10;
// Maximum fractional cell distribution imbalance
// before rebalancing
maxImbalance 0.1;
}
with which the mesh is checked for more than 10% load-imbalance every 10
time-steps and redistributed using a multi-constraint method, i.e. separate CPU
load weights are provided for each of the loads, currently that is the chemistry
integration load and the CPU time taken for the rest of the simulation,
transport equations solution etc.
The fvMeshDistributors::loadBalancer uses the distributor specified in
system/decomposeParDict to redistribute the mesh based on the cell CPU loads,
e.g. to use the Zoltan RCB method specify:
distributor zoltan;
libs ("libzoltanDecomp.so");
zoltanCoeffs
{
lb_method rcb;
}
Unfortunately only a few available redistribution methods support
multi-constraints: Zoltan::RCB, MeTiS, parMeTiS and xtraPuLP, of these only
Zoltan::RCB is currently available in OpenFOAM. Load-balancing is possible
without using a multi-constraint method (i.e. using any of the other
decomposition methods provided with OpenFOAM and Zoltan) by summing the various
CPU loads which is selected by setting:
multiConstraint false;
but the load-balancing is likely to be a lot less effective with this option.
Due to the licencing issues with parMeTiS interfacing to xtraPuLP might be the
best option for further work on load-balancing in OpenFOAM, or MeTiS could be
used in parallel by first agglomerating the distribution graph on the master
processor and redistributing the result; this pseudo-parallel option is already
provided for the Scotch method.
This utility initialises the U, k and epsilon fields (if available) to
the output of the atmBoundaryLayer model. This is the same model as used
in the atmBoundaryLayerInlet.* boundary conditions and in the
waveAtmBoundaryLayer wave model.
The settings for the initialisation are read from a
system/setAtmBoundaryLayerDict file and are identical to the settings
required by the other use cases. An example of the settings required
within a system/setAtmBoundaryLayerDict file is shown below:
zDir (0 0 1); // Vertical direction
flowDir (1 0 0); // Direction of far-field flow
Zref 20; // Reference height
Uref 10; // Speed at reference height
z0 uniform 0.1; // Roughness height
zGround uniform 0; // Ground height
which provides better resolution of the CPU time than times function used
previously. This will be needed for load-balancing chemistry, Lagrangian
etc. for which the CPU time spent per cell is required.
Now that Cp and Cv are cached it is more convenient and consistent and slightly
more efficient to cache thermal conductivity kappa rather than thermal
diffusivity alpha which is not a fundamental property, the appropriate form
depending on the energy solved for. kappa is converted into the appropriate
thermal diffusivity for the energy form solved for by dividing by the
corresponding cached heat capacity when required, which is efficient.
Following the addition of the new moments functionObject, all related
functionality was removed from sizeDistribution.
In its revised version, sizeDistribution allows for different kinds of
weighted region averaging in case of field-dependent representative
particle properties.
A packaged function has also been added to allow for command line solver
post-processing.
For example, the following function object specification returns the
volume-based number density function:
numberDensity
{
type sizeDistribution;
libs ("libmultiphaseEulerFoamFunctionObjects.so");
writeControl writeTime;
populationBalance bubbles;
functionType numberDensity;
coordinateType volume;
setFormat raw;
}
The same can be achieved using a packaged function:
#includeFunc sizeDistribution
(
populationBalance=bubbles,
functionType=numberDensity,
coordinateType=volume,
funcName=numberDensity
)
Or on the command line:
multiphaseEulerFoam -postProcess -func "
sizeDistribution
(
populationBalance=bubbles,
functionType=numberDensity,
coordinateType=volume,
funcName=numberDensity
)"
Patch contributed by Institute of Fluid Dynamics,
Helmholtz-Zentrum Dresden - Rossendorf (HZDR)
This function calculates integral (integer moments) or mean properties
(mean, variance, standard deviation) of a size distribution computed with
multiphaseEulerFoam. It has to be run with multiphaseEulerFoam, either
at run-time or with -postProcess. It will not work with the postProcess
utility.
The following function object specification for example returns the first
moment of the volume-based number density function which is equivalent to
the phase fraction of the particulate phase:
moments
{
type moments;
libs ("libmultiphaseEulerFoamFunctionObjects.so");
executeControl timeStep;
writeControl writeTime;
populationBalance bubbles;
momentType integerMoment;
coordinateType volume;
order 1;
}
The same can be achieved using a packaged function:
#includeFunc moments
(
populationBalance=bubbles,
momentType=integerMoment,
coordinateType=volume,
order=1,
funcName=moments
)
Or on the command line:
multiphaseEulerFoam -postProcess -func "
moments
(
populationBalance=bubbles,
momentType=integerMoment,
coordinateType=volume,
order=1,
funcName=moments
)"
Patch contributed by Institute of Fluid Dynamics,
Helmholtz-Zentrum Dresden - Rossendorf (HZDR)
This correction applies to the handling of the implicit part of the matrix for
the segregates solution of the tensor components, e.g. when solving for the
Reynolds stress in cases with rotated cyclic patches.
Zoltan only work in parallel so zoltanDecomp can only be used for redistribution
but is much more flexible than ptscotch and provides a range of geometric, graph
and hypergraph methods which can operate in either "partition" or "repartition",
the latter being particularly useful for dynamic load-balancing by migrating
cells between processors rather than creating a completely different
decomposition, thus reducing communication.
Class
Foam::zoltanDecomp
Description
Zoltan redistribution in parallel
Note: Zoltan methods do not support serial operation.
Parameters
- lb_method : The load-balancing algorithm
- block : block partitioning
- random : random partitioning
- rcb : recursive coordinate bisection
- rib : ecursive inertial bisection
- hsfc : Hilbert space-filling curve partitioning
- reftree : refinement tree based partitioning
- graph : choose from collection of methods for graphs
- hypergraph : choose from a collection of methods for hypergraphs
- lb_approach The desired load balancing approach. Only lb_method =
hypergraph or graph uses the lb_approach parameter. Valid values are
- partition : Partition without reference to the current distribution,
recommended for static load balancing.
- repartition : Partition starting from the current data distribution
to keep data migration low, recommended for dynamic load balancing.
- refine : Quickly improve the current data distribution
Default values
- debug_level 0
- imbalance_tol 1.05
- lb_method graph
- lb_approach repartition
Usage
To select the Zoltan graph repartition method add the following entries to
decomposeParDict:
distributor zoltan;
libs ("libzoltanRenumber.so");
The Zoltan lb_method and lb_approach can be changed by adding the
corresponding entries to the optional zoltanCeoffs sub-dictionary, e.g.:
zoltanCoeffs
{
lb_method hypergraph;
lb_approach partition;
}
An example of using Zoltan for redistribution during snappyHexMesh is provided
commented out in
tutorials/incompressible/simpleFoam/motorBike/system/decomposeParDict
and fordynamic load-balancing in
tutorials/multiphase/interFoam/RAS/floatingObject/system/decomposeParDict.
Note that Zoltan must first be compiled in ThirdParty-dev by downloading from
the link in the README file and running Allwmake and then compiling zoltanDecomp
by running Allwmake in src/parallel/decompose.
The new DistributeableMeshObject MeshObject provides the interface for the
distribute(const mapDistributePolyMesh&) function used to distribute mesh
related objects following redistribution.
also adding optional "libs" entry to renumberMeshDict so that the
libzoltanRenumber.so can be loaded at run-time rather than having to recompile
and relink the renumberMesh utility to support it.
This switch controls whether or not the model clips a droplet's outlet
mass flow rate to zero or permits it to become negative and thereby
represent condensation. By default condensation is not permitted.
