ParMETIS is a parallel version of METIS and can be used as an alternative to
ptScotch or Zoltan, supporting multi-constraints and redistribution:
Description
ParMetis redistribution in parallel
Note: parMetis methods do not support serial operation.
Parameters
- Method of decomposition
- kWay: multilevel k-way
- geomKway: combined coordinate-based and multi-level k-way
- adaptiveRepart: balances the work load of a graph
- Options
- options[0]: The specified options are used if options[0] = 1
- options[1]: Specifies the level of information to be returned during
the execution of the algorithm. Timing information can be obtained by
setting this to 1. Additional options for this parameter can be obtained
by looking at parmetis.h. Default: 0.
- options[2]: Random number seed for the routine
- options[3]: Specifies whether the sub-domains and processors are coupled
or un-coupled. If the number of sub-domains desired (i.e., nparts) and
the number of processors that are being used is not the same, then these
must be un-coupled. However, if nparts equals the number of processors,
these can either be coupled or de-coupled. If sub-domains and processors
are coupled, then the initial partitioning will be obtained implicitly
from the graph distribution. However, if sub-domains are un-coupled from
processors, then the initial partitioning needs to be obtained from the
initial values assigned to the part array.
- itr: Parameter which describes the ratio of inter-processor communication
time compared to data redistribution time. Should be set between 0.000001
and 1000000.0. If set high, a repartitioning with a low edge-cut will be
computed. If it is set low, a repartitioning that requires little data
redistribution will be computed. Good values for this parameter can be
obtained by dividing inter-processor communication time by data
redistribution time. Otherwise, a value of 1000.0 is recommended.
Default: 1000.
The ParMETIS sources can be downloaded and compiled in ThirdParty-dev using the
link in the README file and the compilation commands in Allwmake.
Note the specific license under which ParMETIS is released:
Copyright & License Notice
--------------------------
The ParMETIS package is copyrighted by the Regents of the
University of Minnesota. It can be freely used for educational and
research purposes by non-profit institutions and US government
agencies only. Other organizations are allowed to use ParMETIS
only for evaluation purposes, and any further uses will require prior
approval. The software may not be sold or redistributed without prior
approval. One may make copies of the software for their use provided
that the copies, are not sold or distributed, are used under the same
terms and conditions.
As unestablished research software, this code is provided on an
``as is'' basis without warranty of any kind, either expressed or
implied. The downloading, or executing any part of this software
constitutes an implicit agreement to these terms. These terms and
conditions are subject to change at any time without prior notice.
This avoids error messages from all processors if the dictionary being parsed is
global, in which case the error message from all processors would be the same so
only the master need print it. This makes many common error messages the same
running in serial and parallel for user convenience.
to ensure the values on the patch are consistent with the boundary condition
specification and to avoid the need to specify a potentially inconsistent
"value" entry.
This boundary condition is applied to a patch which bounds a solid body,
wholly or partially. It represents the body as a lumped mass, i.e. by a
single temperature which is fixed across the patch. The body has a
volume which is either specified by the user, or is calculated when the
patch describes a closed volume (including in 2D meshes). Starting from
an initial temperature, the change in temperature is the calculated over
time according to a specified applied power source and the heat
transferred across the boundary, using a specified density and specific
heat capacity of the lumped mass.
Optional CPU load caching can be switched-on for Lagrangian cloud tracking
and/or chemistry integration using the new cpuLoad switch in the cloudProperties
or chemistryProperties dictionary files respectively and used for
multi-constraint load-balancing by the fvMeshDistributorsLoadBalancer specified
in the dynamicMeshDict file
distributor
{
type loadBalancer;
libs ("libfvMeshDistributors.so");
multiConstraint true;
redistributionInterval 10;
}
which used the distributor specified in the decomposeParDict file, e.g.
numberOfSubdomains 12;
decomposer simple;
distributor zoltan;
libs ("libzoltanDecomp.so");
simpleCoeffs
{
n (2 2 3);
}
zoltanCoeffs
{
lb_method rcb;
}
The incompressibleDenseParticleFluid/cyclone case has been updated to
demonstrate this new functionality and shows a speedup ~50% using the Zoltan RCB
multi-constraint distributor. The multicomponentFluid/counterFlowFlame2D_GRI
case has also been updated to use the new cpuLoad switch.
The majority of input parameters now support automatic unit conversion.
Units are specified within square brackets, either before or after the
value. Primitive parameters (e.g., scalars, vectors, tensors, ...),
dimensioned types, fields, Function1-s and Function2-s all support unit
conversion in this way.
Unit conversion occurs on input only. OpenFOAM writes out all fields and
parameters in standard units. It is recommended to use '.orig' files in
the 0 directory to preserve user-readable input if those files are being
modified by pre-processing applications (e.g., setFields).
For example, to specify a volumetric flow rate inlet boundary in litres
per second [l/s], rather than metres-cubed per second [m^3/s], in 0/U:
boundaryField
{
inlet
{
type flowRateInletVelocity;
volumetricFlowRate 0.1 [l/s];
value $internalField;
}
...
}
Or, to specify the pressure field in bar, in 0/p:
internalField uniform 1 [bar];
Or, to convert the parameters of an Arrhenius reaction rate from a
cm-mol-kcal unit system, in constant/chemistryProperties:
reactions
{
methaneReaction
{
type irreversibleArrhenius;
reaction "CH4^0.2 + 2O2^1.3 = CO2 + 2H2O";
A 6.7e12 [(mol/cm^3)^-0.5/s];
beta 0;
Ea 48.4 [kcal/mol];
}
}
Or, to define a time-varying outlet pressure using a CSV file in which
the pressure column is in mega-pascals [MPa], in 0/p:
boundaryField
{
outlet
{
type uniformFixedValue;
value
{
type table;
format csv;
nHeaderLine 1;
units ([s] [MPa]); // <-- new units entry
columns (0 1);
mergeSeparators no;
file "data/pressure.csv";
outOfBounds clamp;
interpolationScheme linear;
}
}
...
}
(Note also that a new 'columns' entry replaces the old 'refColumn' and
'componentColumns'. This is is considered to be more intuitive, and has
a consistent syntax with the new 'units' entry. 'columns' and
'componentColumns' have been retained for backwards compatibility and
will continue to work for the time being.)
Unit definitions can be added in the global or case controlDict files.
See UnitConversions in $WM_PROJECT_DIR/etc/controlDict for examples.
Currently available units include:
Standard: kg m s K kmol A Cd
Derived: Hz N Pa J W g um mm cm km l ml us ms min hr mol
rpm bar atm kPa MPa cal kcal cSt cP % rad rot deg
A user-time unit is also provided if user-time is in operation. This
allows it to be specified locally whether a parameter relates to
real-time or to user-time. For example, to define a mass source that
ramps up from a given engine-time (in crank-angle-degrees [CAD]) over a
duration in real-time, in constant/fvModels:
massSource1
{
type massSource;
points ((1 2 3));
massFlowRate
{
type scale;
scale linearRamp;
start 20 [CAD];
duration 50 [ms];
value 0.1 [g/s];
}
}
Specified units will be checked against the parameter's dimensions where
possible, and an error generated if they are not consistent. For the
dimensions to be available for this check, the code requires
modification, and work propagating this change across OpenFOAM is
ongoing. Unit conversions are still possible without these changes, but
the validity of such conversions will not be checked.
Units are no longer permitted in 'dimensions' entries in field files.
These 'dimensions' entries can now, instead, take the names of
dimensions. The names of the available dimensions are:
Standard: mass length time temperature
moles current luminousIntensity
Derived: area volume rate velocity momentum acceleration density
force energy power pressure kinematicPressure
compressibility gasConstant specificHeatCapacity
kinematicViscosity dynamicViscosity thermalConductivity
volumetricFlux massFlux
So, for example, a 0/epsilon file might specify the dimensions as
follows:
dimensions [energy/mass/time];
And a 0/alphat file might have:
dimensions [thermalConductivity/specificHeatCapacity];
*** Development Notes ***
A unit conversion can construct trivially from a dimension set,
resulting in a "standard" unit with a conversion factor of one. This
means the functions which perform unit conversion on read can be
provided dimension sets or unit conversion objects interchangeably.
A basic `dict.lookup<vector>("Umean")` call will do unit conversion, but
it does not know the parameter's dimensions, so it cannot check the
validity of the supplied units. A corresponding lookup function has been
added in which the dimensions or units can be provided; in this case the
corresponding call would be `dict.lookup<vector>("Umean", dimVelocity)`.
This function enables additional checking and should be used wherever
possible.
Function1-s and Function2-s have had their constructors and selectors
changed so that dimensions/units must be specified by calling code. In
the case of Function1, two unit arguments must be given; one for the
x-axis and one for the value-axis. For Function2-s, three must be
provided.
In some cases, it is desirable (or at least established practice), that
a given non-standard unit be used in the absence of specific
user-defined units. Commonly this includes reading angles in degrees
(rather than radians) and reading times in user-time (rather than
real-time). The primitive lookup functions and Function1 and Function2
selectors both support specifying a non-standard default unit. For
example, `theta_ = dict.lookup<scalar>("theta", unitDegrees)` will read
an angle in degrees by default. If this is done within a model which
also supports writing then the write call must be modified accordingly
so that the data is also written out in degrees. Overloads of writeEntry
have been created for this purpose. In this case, the angle theta should
be written out with `writeEntry(os, "theta", unitDegrees, theta_)`.
Function1-s and Function2-s behave similarly, but with greater numbers
of dimensions/units arguments as before.
The non-standard user-time unit can be accessed by a `userUnits()`
method that has been added to Time. Use of this user-time unit in the
construction of Function1-s should prevent the need for explicit
user-time conversion in boundary conditions and sub-models and similar.
Some models might contain non-typed stream-based lookups of the form
`dict.lookup("p0") >> p0_` (e.g., in a re-read method), or
`Umean_(dict.lookup("Umean"))` (e.g., in an initialiser list). These
calls cannot facilitate unit conversion and are therefore discouraged.
They should be replaced with
`p0_ = dict.lookup<scalar>("p0", dimPressure)` and
`Umean_(dict.lookup<vector>("Umean", dimVelocity))` and similar whenever
they are found.
The alpha equation in this model has been changed to a d2dt2 scheme,
rather than a hard-coded correction on a ddt scheme that relies on the
old-time value remaining unchanged from the previous timestep. This
modification makes the model compatible with some forms of mesh change.
The 'featureAngle' control defines the included angle between faces
above which layer extrusion is prevented. Its use within snappyHexMesh
was incorrect in that the cosine was not being taken before being
compared to dot products of face normals. This has now been corrected.
Existing 'featureAngle' settings may need changing to recover equivalent
behaviour. Any angle previously set to 58 degrees or above previously
resulted in no reduction of layer coverage. A large angle between 90 and
180 degrees is likely to be an appropriate replacement for cases such as
this. Angles previously set to 57 degrees and below can be equivalently
replaced by a value of (180/pi)*arccos(<angle>*(pi/180)).
Note that changing the feature angle also affects the slip feature
angle. The slip feature angle is taken to be half the feature angle if a
'slipFeatureAngle' is not specified.
lookupCompoundScoped is used by dictionary via stringOps to provide fast access
to compound tokens without the need to construct the container for every access,
e.g.
listU List<vector> ((1.1 2.1 1.1) (2.1 3.2 4.1) (4.3 5.3 0));
magU1 #calc "mag($listU[1])";
magU2 #calc "mag($listU[2])";
where the listU is declared as a List<vector> compound token which is then
accessed directly by $listU without the need to specify the type and the data is
accessed directly by reference which is more efficient, particularly if the list
is large.
The new token::typeName() function is used in stringOps to provide dictionary
with better support for automatic type inference for variables in #calc
expressions.
This change corrects some of the logic governing whether a geometry is
determined to result in a valid intersection or not. It has fixed some
issues with regards to intersections being erroneously duplicated or
omitted and has thereby increased the proportion of intersections that
result in unity or near-unity coverage.
Cloud functions are now not created at all during foamPostProcess. These
functions mostly usually compute state during tracking, so they cannot
produce anything useful when run as a post-process. In addition, in a
few cases, their initialisation can overwrites valid post-processing
files created at run-time.
A more "complete" system might distinguish between the few cloud
functions that can be evaluated instantaneously (e.g., sizeDistribution,
relativeVelocity, ...) and those those that cannot (e.g., fluxes,
patchCollisionDensity, ...) and only construct and execute the former
within foamPostProcess. This could be achieved with appropriate
support/funding.
In addition, volFieldValue and surfaceFieldValue have been modified so
that they do not write anything if no fields are available. This also
prevents the overwriting of valid run-time results in some cases.
