Non-Conformal Coupled (NCC): Conservative coupling of non-conforming patches

This major development provides coupling of patches which are
non-conformal, i.e. where the faces of one patch do not match the faces
of the other. The coupling is fully conservative and second order
accurate in space, unlike the Arbitrary Mesh Interface (AMI) and
associated ACMI and Repeat AMI methods which NCC replaces.

Description:

A non-conformal couple is a connection between a pair of boundary
patches formed by projecting one patch onto the other in a way that
fills the space between them. The intersection between the projected
surface and patch forms new faces that are incorporated into the finite
volume mesh. These new faces are created identically on both sides of
the couple, and therefore become equivalent to internal faces within the
mesh. The affected cells remain closed, meaning that the area vectors
sum to zero for all the faces of each cell. Consequently, the main
benefits of the finite volume method, i.e. conservation and accuracy,
are not undermined by the coupling.

A couple connects parts of mesh that are otherwise disconnected and can
be used in the following ways:

+ to simulate rotating geometries, e.g. a propeller or stirrer, in which
  a part of the mesh rotates with the geometry and connects to a
  surrounding mesh which is not moving;
+ to connect meshes that are generated separately, which do not conform
  at their boundaries;
+ to connect patches which only partially overlap, in which the
  non-overlapped section forms another boundary, e.g. a wall;
+ to simulate a case with a geometry which is periodically repeating by
  creating multiple couples with different transformations between
  patches.

The capability for simulating partial overlaps replaces the ACMI
functionality, currently provided by the 'cyclicACMI' patch type, and
which is unreliable unless the couple is perfectly flat. The capability
for simulating periodically repeating geometry replaces the Repeat AMI
functionality currently provided by the 'cyclicRepeatAMI' patch type.

Usage:

The process of meshing for NCC is very similar to existing processes for
meshing for AMI. Typically, a mesh is generated with an identifiable set
of internal faces which coincide with the surface through which the mesh
will be coupled. These faces are then duplicated by running the
'createBaffles' utility to create two boundary patches. The points are
then split using 'splitBaffles' in order to permit independent motion of
the patches.

In AMI, these patches are assigned the 'cyclicAMI' patch type, which
couples them using AMI interpolation methods.

With NCC, the patches remain non-coupled, e.g. a 'wall' type. Coupling
is instead achieved by running the new 'createNonConformalCouples'
utility, which creates additional coupled patches of type
'nonConformalCyclic'. These appear in the 'constant/polyMesh/boundary'
file with zero faces; they are populated with faces in the finite volume
mesh during the connection process in NCC.

For a single couple, such as that which separates the rotating and
stationary sections of a mesh, the utility can be called using the
non-coupled patch names as arguments, e.g.

    createNonConformalCouples -overwrite rotatingZoneInner rotatingZoneOuter

where 'rotatingZoneInner' and 'rotatingZoneOuter' are the names of the
patches.

For multiple couples, and/or couples with transformations,
'createNonConformalCouples' should be run without arguments. Settings
will then be read from a configuration file named
'system/createNonConformalCouplesDict'. See
'$FOAM_ETC/caseDicts/annotated/createNonConformalCouplesDict' for
examples.

Boundary conditions must be specified for the non-coupled patches. For a
couple where the patches fully overlap, boundary conditions
corresponding to a slip wall are typically applied to fields, i.e
'movingWallSlipVelocity' (or 'slip' if the mesh is stationary) for
velocity U, 'zeroGradient' or 'fixedFluxPressure' for pressure p, and
'zeroGradient' for other fields.  For a couple with
partially-overlapping patches, boundary conditions are applied which
physically represent the non-overlapped region, e.g. a no-slip wall.

Boundary conditions also need to be specified for the
'nonConformalCyclic' patches created by 'createNonConformalCouples'. It
is generally recommended that this is done by including the
'$FOAM_ETC/caseDicts/setConstraintTypes' file in the 'boundaryField'
section of each of the field files, e.g.

    boundaryField
    {
        #includeEtc "caseDicts/setConstraintTypes"

        inlet
        {
             ...
        }

        ...
    }

For moving mesh cases, it may be necessary to correct the mesh fluxes
that are changed as a result of the connection procedure. If the
connected patches do not conform perfectly to the mesh motion, then
failure to correct the fluxes can result in noise in the pressure
solution.

Correction for the mesh fluxes is enabled by the 'correctMeshPhi' switch
in the 'PIMPLE' (or equivalent) section of 'system/fvSolution'. When it
is enabled, solver settings are required for 'MeshPhi'. The solution
just needs to distribute the error enough to dissipate the noise. A
smooth solver with a loose tolerance is typically sufficient, e.g. the
settings in 'system/fvSolution' shown below:

    solvers
    {
        MeshPhi
        {
            solver          smoothSolver;
            smoother        symGaussSeidel;
            tolerance       1e-2;
            relTol          0;
        }
        ...
    }

    PIMPLE
    {
         correctMeshPhi      yes;
         ...
    }

The solution of 'MeshPhi' is an inexpensive computation since it is
applied only to a small subset of the mesh adjacent to the
couple. Conservation is maintained whether or not the mesh flux
correction is enabled, and regardless of the solution tolerance for
'MeshPhi'.

Advantages of NCC:

+ NCC maintains conservation which is required for many numerical
  schemes and algorithms to operate effectively, in particular those
  designed to maintain boundedness of a solution.

+ Closed-volume systems no longer suffer from accumulation or loss of
  mass, poor convergence of the pressure equation, and/or concentration
  of error in the reference cell.

+ Partially overlapped simulations are now possible on surfaces that are
  not perfectly flat. The projection fills space so no overlaps or
  spaces are generated inside contiguously overlapping sections, even if
  those sections have sharp angles.

+ The finite volume faces created by NCC have geometrically accurate
  centres. This makes the method second order accurate in space.

+ The polyhedral mesh no longer requires duplicate boundary faces to be
  generated in order to run a partially overlapped simulation.

+ Lagrangian elements can now transfer across non-conformal couplings in
  parallel.

+ Once the intersection has been computed and applied to the finite
  volume mesh, it can use standard cyclic or processor cyclic finite
  volume boundary conditions, with no need for additional patch types or
  matrix interfaces.

+ Parallel communication is done using the standard
  processor-patch-field system. This is more efficient than alternative
  systems since it has been carefully optimised for use within the
  linear solvers.

+ Coupled patches are disconnected prior to mesh motion and topology
  change and reconnected afterwards. This simplifies the boundary
  condition specification for mesh motion fields.

Resolved Bug Reports:

+ https://bugs.openfoam.org/view.php?id=663
+ https://bugs.openfoam.org/view.php?id=883
+ https://bugs.openfoam.org/view.php?id=887
+ https://bugs.openfoam.org/view.php?id=1337
+ https://bugs.openfoam.org/view.php?id=1388
+ https://bugs.openfoam.org/view.php?id=1422
+ https://bugs.openfoam.org/view.php?id=1829
+ https://bugs.openfoam.org/view.php?id=1841
+ https://bugs.openfoam.org/view.php?id=2274
+ https://bugs.openfoam.org/view.php?id=2561
+ https://bugs.openfoam.org/view.php?id=3817

Deprecation:

NCC replaces the functionality provided by AMI, ACMI and Repeat AMI.
ACMI and Repeat AMI are insufficiently reliable to warrant further
maintenance so are removed in an accompanying commit to OpenFOAM-dev.
AMI is more widely used so will be retained alongside NCC for the next
version release of OpenFOAM and then subsequently removed from
OpenFOAM-dev.
This commit is contained in:
Will Bainbridge
2022-05-09 14:35:11 +01:00
parent 94679fa88d
commit 569fa31d09
254 changed files with 18751 additions and 4327 deletions

View File

@ -1,5 +1,4 @@
decomposePar.C
domainDecomposition.C
dimFieldDecomposer.C
fvFieldDecomposer.C
pointFieldDecomposer.C

View File

@ -1,14 +1,16 @@
EXE_INC = \
-I$(LIB_SRC)/parallel/decompose/decompositionMethods/lnInclude \
-I$(LIB_SRC)/finiteVolume/lnInclude \
-I$(LIB_SRC)/dynamicMesh/lnInclude \
-I$(LIB_SRC)/lagrangian/basic/lnInclude \
-I$(LIB_SRC)/meshTools/lnInclude \
-I$(LIB_SRC)/parallel/parallel/lnInclude \
-I$(LIB_SRC)/parallel/decompose/decompositionMethods/lnInclude \
-I$(LIB_SRC)/lagrangian/basic/lnInclude \
-I$(LIB_SRC)/regionModels/regionModel/lnInclude
EXE_LIBS = \
-ldynamicMesh \
-lgenericPatchFields \
-ldecompositionMethods -L$(FOAM_LIBBIN)/dummy -lmetisDecomp -lscotchDecomp \
-lfiniteVolume \
-lmeshTools\
-lparallel \
-ldecompositionMethods \
-llagrangian \
-lregionModels
-lregionModels \
-lgenericPatchFields

View File

@ -79,6 +79,7 @@ Usage
\*---------------------------------------------------------------------------*/
#include "processorRunTimes.H"
#include "domainDecomposition.H"
#include "decompositionMethod.H"
#include "argList.H"
@ -114,14 +115,12 @@ namespace Foam
void decomposeUniform
(
const bool copyUniform,
const domainDecomposition& decomposition,
const Time& processorDb,
const bool distributeUniform,
const Time& runTime,
const Time& procRunTime,
const word& regionDir = word::null
)
{
const Time& runTime = decomposition.mesh().time();
// Any uniform data to copy/link?
const fileName uniformDir(regionDir/"uniform");
if (fileHandler().isDir(runTime.timePath()/uniformDir))
@ -131,9 +130,9 @@ void decomposeUniform
<< endl;
const fileName timePath =
fileHandler().filePath(processorDb.timePath());
fileHandler().filePath(procRunTime.timePath());
if (copyUniform || decomposition.distributed())
if (copyUniform || distributeUniform)
{
if (!fileHandler().exists(timePath/uniformDir))
{
@ -171,9 +170,9 @@ void decomposeUniform
}
void writeDecomposition(const domainDecomposition& decomposition)
void writeDecomposition(const domainDecomposition& meshes)
{
const labelList& procIds = decomposition.cellToProc();
const labelList& procIds = meshes.cellToProc();
// Write the decomposition as labelList for use with 'manual'
// decomposition method.
@ -182,8 +181,8 @@ void writeDecomposition(const domainDecomposition& decomposition)
IOobject
(
"cellDecomposition",
decomposition.mesh().facesInstance(),
decomposition.mesh(),
meshes.completeMesh().facesInstance(),
meshes.completeMesh(),
IOobject::NO_READ,
IOobject::NO_WRITE,
false
@ -203,12 +202,12 @@ void writeDecomposition(const domainDecomposition& decomposition)
IOobject
(
"cellDist",
decomposition.mesh().time().timeName(),
decomposition.mesh(),
meshes.completeMesh().time().timeName(),
meshes.completeMesh(),
IOobject::NO_READ,
IOobject::AUTO_WRITE
),
decomposition.mesh(),
meshes.completeMesh(),
dimless,
scalarField(scalarList(procIds))
);
@ -309,18 +308,20 @@ int main(int argc, char *argv[])
}
// Set time from database
#include "createTime.H"
Info<< "Create time\n" << endl;
processorRunTimes runTimes(Foam::Time::controlDictName, args);
// Allow override of time
instantList times = timeSelector::selectIfPresent(runTime, args);
const instantList times = runTimes.selectComplete(args);
// Get region names
const wordList regionNames(selectRegionNames(args, runTime));
const wordList regionNames =
selectRegionNames(args, runTimes.completeTime());
// Handle existing decomposition directories
{
// Determine the processor count from the directories
label nProcs = fileHandler().nProcs(runTime.path());
label nProcs = fileHandler().nProcs(runTimes.completeTime().path());
if (forceOverwrite)
{
@ -339,7 +340,7 @@ int main(int argc, char *argv[])
(
fileHandler().readDir
(
runTime.path(),
runTimes.completeTime().path(),
fileType::directory
)
);
@ -379,12 +380,17 @@ int main(int argc, char *argv[])
<< " domains, use the -force option or manually" << nl
<< "remove processor directories before decomposing. e.g.,"
<< nl
<< " rm -rf " << runTime.path().c_str() << "/processor*"
<< " rm -rf " << runTimes.completeTime().path().c_str()
<< "/processor*"
<< nl
<< exit(FatalError);
}
}
// Get the decomposition dictionary
const dictionary decomposeParDict =
decompositionMethod::decomposeParDict(runTimes.completeTime());
// Decompose all regions
forAll(regionNames, regioni)
{
@ -394,12 +400,12 @@ int main(int argc, char *argv[])
Info<< "\n\nDecomposing mesh " << regionName << nl << endl;
// Determine the existing processor count directly
label nProcs = fileHandler().nProcs(runTime.path(), regionDir);
const label nProcs =
fileHandler().nProcs(runTimes.completeTime().path(), regionDir);
// Get requested numberOfSubdomains
const label nDomains =
decompositionMethod::decomposeParDict(runTime)
.lookup<label>("numberOfSubdomains");
decomposeParDict.lookup<label>("numberOfSubdomains");
// Give file handler a chance to determine the output directory
const_cast<fileOperation&>(fileHandler()).setNProcs(nDomains);
@ -421,74 +427,70 @@ int main(int argc, char *argv[])
Info<< "Using existing processor directories" << nl;
}
Info<< "Create mesh" << endl;
fvMesh mesh
(
IOobject
(
regionName,
runTime.timeName(),
runTime,
IOobject::NO_READ,
IOobject::NO_WRITE,
false
),
false
);
// Get flag to determine whether or not to distribute uniform data
const label distributeUniform =
decomposeParDict.lookupOrDefault<bool>("distributed", false);
// Create decomposition
domainDecomposition decomposition(mesh);
// Create meshes
Info<< "Create mesh" << endl;
domainDecomposition meshes(runTimes, regionName);
meshes.readComplete();
// Read or generate a decomposition as necessary
if (decomposeFieldsOnly)
{
decomposition.read();
meshes.readProcs();
if (!copyZero)
{
meshes.readAddressing();
meshes.readUpdate();
}
}
else
{
decomposition.decompose();
decomposition.write(decomposeSets);
if (writeCellDist)
{
writeDecomposition(decomposition);
}
meshes.decompose();
meshes.writeProcs(decomposeSets);
if (writeCellDist) writeDecomposition(meshes);
fileHandler().flush();
}
// Field maps. These are preserved if decomposing multiple times.
PtrList<fvFieldDecomposer> fieldDecomposerList
(
decomposition.nProcs()
meshes.nProcs()
);
PtrList<dimFieldDecomposer> dimFieldDecomposerList
(
decomposition.nProcs()
meshes.nProcs()
);
PtrList<pointFieldDecomposer> pointFieldDecomposerList
(
decomposition.nProcs()
meshes.nProcs()
);
// Loop over all times
forAll(times, timeI)
{
// Set the time
runTime.setTime(times[timeI], timeI);
decomposition.setTime(times[timeI], timeI);
runTimes.setTime(times[timeI], timeI);
Info<< "Time = " << runTime.userTimeName() << endl;
Info<< "Time = " << runTimes.completeTime().userTimeName() << endl;
// Update the mesh
const fvMesh::readUpdateState state = mesh.readUpdate();
// Update the meshes, if necessary
fvMesh::readUpdateState state = fvMesh::UNCHANGED;
if (!decomposeFieldsOnly || !copyZero)
{
state = meshes.readUpdate();
}
// Update the decomposition
// Write the mesh out, if necessary
if (decomposeFieldsOnly)
{
decomposition.readUpdate();
// Nothing to do
}
else if (state == fvMesh::POINTS_MOVED)
{
decomposition.writePoints();
meshes.writeProcs(false);
}
else if
(
@ -496,12 +498,9 @@ int main(int argc, char *argv[])
|| state == fvMesh::TOPO_PATCH_CHANGE
)
{
decomposition.decompose();
decomposition.write(decomposeSets);
if (writeCellDist)
{
writeDecomposition(decomposition);
}
meshes.writeProcs(decomposeSets);
if (writeCellDist) writeDecomposition(meshes);
fileHandler().flush();
}
// Clear the field maps if there has been topology change
@ -511,7 +510,7 @@ int main(int argc, char *argv[])
|| state == fvMesh::TOPO_PATCH_CHANGE
)
{
for (label proci = 0; proci < decomposition.nProcs(); proci++)
for (label proci = 0; proci < meshes.nProcs(); proci++)
{
fieldDecomposerList.set(proci, nullptr);
dimFieldDecomposerList.set(proci, nullptr);
@ -528,43 +527,38 @@ int main(int argc, char *argv[])
{
// Copy the field files from the <time> directory to the
// processor*/<time> directories without altering them
const fileName completeTimePath =
runTimes.completeTime().timePath();
fileName prevTimePath;
for (label proci = 0; proci < decomposition.nProcs(); proci++)
fileName prevProcTimePath;
for (label proci = 0; proci < meshes.nProcs(); proci++)
{
Time processorDb
(
Time::controlDictName,
args.rootPath(),
args.caseName()
/fileName(word("processor") + name(proci))
);
processorDb.setTime(runTime);
const Time& procRunTime =
meshes.procMeshes()[proci].time();
if (fileHandler().isDir(runTime.timePath()))
if (fileHandler().isDir(completeTimePath))
{
const fileName timePath
const fileName procTimePath
(
fileHandler().objectPath
(
IOobject
(
"",
processorDb.timeName(),
processorDb
procRunTime.timeName(),
procRunTime
),
word::null
)
);
if (timePath != prevTimePath)
if (procTimePath != prevProcTimePath)
{
Info<< "Processor " << proci
<< ": copying " << runTime.timePath() << nl
<< " to " << timePath << endl;
fileHandler().cp(runTime.timePath(), timePath);
prevTimePath = timePath;
<< ": copying " << completeTimePath << nl
<< " to " << procTimePath << endl;
fileHandler().cp(completeTimePath, procTimePath);
prevProcTimePath = procTimePath;
}
}
}
@ -574,49 +568,73 @@ int main(int argc, char *argv[])
// Decompose the fields
// Search for list of objects for this time
IOobjectList objects(mesh, runTime.timeName());
IOobjectList objects
(
meshes.completeMesh(),
runTimes.completeTime().timeName()
);
// Construct the vol fields
PtrList<volScalarField> volScalarFields;
readFields(mesh, objects, volScalarFields);
readFields(meshes.completeMesh(), objects, volScalarFields);
PtrList<volVectorField> volVectorFields;
readFields(mesh, objects, volVectorFields);
readFields(meshes.completeMesh(), objects, volVectorFields);
PtrList<volSphericalTensorField> volSphericalTensorFields;
readFields(mesh, objects, volSphericalTensorFields);
readFields
(
meshes.completeMesh(),
objects,
volSphericalTensorFields
);
PtrList<volSymmTensorField> volSymmTensorFields;
readFields(mesh, objects, volSymmTensorFields);
readFields(meshes.completeMesh(), objects, volSymmTensorFields);
PtrList<volTensorField> volTensorFields;
readFields(mesh, objects, volTensorFields);
readFields(meshes.completeMesh(), objects, volTensorFields);
// Construct the dimensioned fields
PtrList<DimensionedField<scalar, volMesh>> dimScalarFields;
readFields(mesh, objects, dimScalarFields);
readFields(meshes.completeMesh(), objects, dimScalarFields);
PtrList<DimensionedField<vector, volMesh>> dimVectorFields;
readFields(mesh, objects, dimVectorFields);
readFields(meshes.completeMesh(), objects, dimVectorFields);
PtrList<DimensionedField<sphericalTensor, volMesh>>
dimSphericalTensorFields;
readFields(mesh, objects, dimSphericalTensorFields);
readFields
(
meshes.completeMesh(),
objects,
dimSphericalTensorFields
);
PtrList<DimensionedField<symmTensor, volMesh>>
dimSymmTensorFields;
readFields(mesh, objects, dimSymmTensorFields);
readFields(meshes.completeMesh(), objects, dimSymmTensorFields);
PtrList<DimensionedField<tensor, volMesh>> dimTensorFields;
readFields(mesh, objects, dimTensorFields);
readFields(meshes.completeMesh(), objects, dimTensorFields);
// Construct the surface fields
PtrList<surfaceScalarField> surfaceScalarFields;
readFields(mesh, objects, surfaceScalarFields);
readFields(meshes.completeMesh(), objects, surfaceScalarFields);
PtrList<surfaceVectorField> surfaceVectorFields;
readFields(mesh, objects, surfaceVectorFields);
readFields(meshes.completeMesh(), objects, surfaceVectorFields);
PtrList<surfaceSphericalTensorField>
surfaceSphericalTensorFields;
readFields(mesh, objects, surfaceSphericalTensorFields);
readFields
(
meshes.completeMesh(),
objects,
surfaceSphericalTensorFields
);
PtrList<surfaceSymmTensorField> surfaceSymmTensorFields;
readFields(mesh, objects, surfaceSymmTensorFields);
readFields
(
meshes.completeMesh(),
objects,
surfaceSymmTensorFields
);
PtrList<surfaceTensorField> surfaceTensorFields;
readFields(mesh, objects, surfaceTensorFields);
readFields(meshes.completeMesh(), objects, surfaceTensorFields);
// Construct the point fields
const pointMesh& pMesh = pointMesh::New(mesh);
const pointMesh& pMesh = pointMesh::New(meshes.completeMesh());
PtrList<pointScalarField> pointScalarFields;
readFields(pMesh, objects, pointScalarFields);
PtrList<pointVectorField> pointVectorFields;
@ -633,7 +651,7 @@ int main(int argc, char *argv[])
(
fileHandler().readDir
(
runTime.timePath()/cloud::prefix,
runTimes.completeTime().timePath()/cloud::prefix,
fileType::directory
)
);
@ -672,8 +690,8 @@ int main(int argc, char *argv[])
{
IOobjectList sprayObjs
(
mesh,
runTime.timeName(),
meshes.completeMesh(),
runTimes.completeTime().timeName(),
cloud::prefix/cloudDirs[i],
IOobject::MUST_READ,
IOobject::NO_WRITE,
@ -695,7 +713,7 @@ int main(int argc, char *argv[])
cloudI,
new Cloud<indexedParticle>
(
mesh,
meshes.completeMesh(),
cloudDirs[i],
false
)
@ -707,7 +725,7 @@ int main(int argc, char *argv[])
cloudI,
new List<SLList<indexedParticle*>*>
(
mesh.nCells(),
meshes.completeMesh().nCells(),
static_cast<SLList<indexedParticle*>*>(nullptr)
)
);
@ -726,7 +744,11 @@ int main(int argc, char *argv[])
label celli = iter().cell();
// Check
if (celli < 0 || celli >= mesh.nCells())
if
(
celli < 0
|| celli >= meshes.completeMesh().nCells()
)
{
FatalErrorInFunction
<< "Illegal cell number " << celli
@ -735,10 +757,11 @@ int main(int argc, char *argv[])
<< " at position "
<< iter().position() << nl
<< "Cell number should be between 0 and "
<< mesh.nCells()-1 << nl
<< meshes.completeMesh().nCells()-1 << nl
<< "On this mesh the particle should"
<< " be in cell "
<< mesh.findCell(iter().position())
<< meshes.completeMesh().findCell
(iter().position())
<< exit(FatalError);
}
@ -754,8 +777,8 @@ int main(int argc, char *argv[])
// Read fields
IOobjectList lagrangianObjects
(
mesh,
runTime.timeName(),
meshes.completeMesh(),
runTimes.completeTime().timeName(),
cloud::prefix/cloudDirs[cloudI],
IOobject::MUST_READ,
IOobject::NO_WRITE,
@ -856,7 +879,7 @@ int main(int argc, char *argv[])
Info<< endl;
// split the fields over processors
for (label proci = 0; proci < decomposition.nProcs(); proci++)
for (label proci = 0; proci < meshes.nProcs(); proci++)
{
Info<< "Processor " << proci << ": field transfer" << endl;
@ -869,10 +892,11 @@ int main(int argc, char *argv[])
proci,
new fvFieldDecomposer
(
mesh,
decomposition.procMeshes()[proci],
decomposition.procFaceAddressing()[proci],
decomposition.procCellAddressing()[proci]
meshes.completeMesh(),
meshes.procMeshes()[proci],
meshes.procFaceAddressing()[proci],
meshes.procCellAddressing()[proci],
meshes.procFaceAddressingBf()[proci]
)
);
}
@ -916,10 +940,10 @@ int main(int argc, char *argv[])
proci,
new dimFieldDecomposer
(
mesh,
decomposition.procMeshes()[proci],
decomposition.procFaceAddressing()[proci],
decomposition.procCellAddressing()[proci]
meshes.completeMesh(),
meshes.procMeshes()[proci],
meshes.procFaceAddressing()[proci],
meshes.procCellAddressing()[proci]
)
);
}
@ -949,7 +973,7 @@ int main(int argc, char *argv[])
)
{
const pointMesh& procPMesh =
pointMesh::New(decomposition.procMeshes()[proci]);
pointMesh::New(meshes.procMeshes()[proci]);
if (!pointFieldDecomposerList.set(proci))
{
@ -960,7 +984,7 @@ int main(int argc, char *argv[])
(
pMesh,
procPMesh,
decomposition.procPointAddressing()[proci]
meshes.procPointAddressing()[proci]
)
);
}
@ -989,10 +1013,10 @@ int main(int argc, char *argv[])
{
lagrangianFieldDecomposer fieldDecomposer
(
mesh,
decomposition.procMeshes()[proci],
decomposition.procFaceAddressing()[proci],
decomposition.procCellAddressing()[proci],
meshes.completeMesh(),
meshes.procMeshes()[proci],
meshes.procFaceAddressing()[proci],
meshes.procCellAddressing()[proci],
cloudDirs[cloudI],
lagrangianPositions[cloudI],
cellParticles[cloudI]
@ -1069,8 +1093,9 @@ int main(int argc, char *argv[])
decomposeUniform
(
copyUniform,
decomposition,
decomposition.procMeshes()[proci].time(),
distributeUniform,
runTimes.completeTime(),
meshes.procMeshes()[proci].time(),
regionDir
);
@ -1082,8 +1107,9 @@ int main(int argc, char *argv[])
decomposeUniform
(
copyUniform,
decomposition,
decomposition.procMeshes()[proci].time()
distributeUniform,
runTimes.completeTime(),
meshes.procMeshes()[proci].time()
);
}
}

View File

@ -1,241 +0,0 @@
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Copyright (C) 2011-2022 OpenFOAM Foundation
\\/ M anipulation |
-------------------------------------------------------------------------------
License
This file is part of OpenFOAM.
OpenFOAM is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with OpenFOAM. If not, see <http://www.gnu.org/licenses/>.
Class
Foam::domainDecomposition
Description
Automatic domain decomposition class for finite-volume meshes
SourceFiles
domainDecomposition.C
decomposeMesh.C
\*---------------------------------------------------------------------------*/
#ifndef domainDecomposition_H
#define domainDecomposition_H
#include "fvMesh.H"
#include "volFields.H"
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
namespace Foam
{
/*---------------------------------------------------------------------------*\
Class domainDecomposition Declaration
\*---------------------------------------------------------------------------*/
class domainDecomposition
{
// Private Data
//- Reference to the complete mesh
const fvMesh& mesh_;
//- Number of processors in decomposition
const label nProcs_;
//- Optional: points at the facesInstance
autoPtr<pointIOField> facesInstancePointsPtr_;
//- Is the decomposition data to be distributed for each processor
bool distributed_;
// Processor mesh to complete mesh addressing
//- Labels of points for each processor
labelListList procPointAddressing_;
//- Labels of faces for each processor
// Note: Face turning index is stored as the sign on addressing
// Only the processor boundary faces are affected: if the sign of
// the index is negative, the processor face is the reverse of the
// original face. In order to do this properly, all face
// indices will be incremented by 1 and the decremented as
// necessary to avoid the problem of face number zero having no
// sign.
List<DynamicList<label>> procFaceAddressing_;
//- Labels of cells for each processor
labelListList procCellAddressing_;
//- Processor times
PtrList<Time> procRunTimes_;
//- Processor meshes
PtrList<fvMesh> procMeshes_;
// Private Member Functions
//- Call the decomposition method and return the processor index that
// each cell is being distributed to
labelList distributeCells();
//- Mark all elements with value or -2 if occur twice
static void mark
(
const labelList& zoneElems,
const label zoneI,
labelList& elementToZone
);
//- Add face to inter-processor patch
void addInterProcFace
(
const label facei,
const label ownerProc,
const label nbrProc,
List<Map<label>>&,
List<DynamicList<DynamicList<label>>>&
) const;
//- Generate sub patch info for processor cyclics
template<class BinaryOp>
inline void processInterCyclics
(
const labelList& cellToProc,
const polyBoundaryMesh& patches,
List<DynamicList<DynamicList<label>>>& interPatchFaces,
List<Map<label>>& procNbrToInterPatch,
List<labelListList>& subPatchIDs,
List<labelListList>& subPatchStarts,
bool owner,
BinaryOp bop
) const;
//- Validate that the decomposition has been generated or read
void validate() const;
public:
//- Runtime type information
TypeName("domainDecomposition");
// Constructors
//- Construct from reference to fvMesh
domainDecomposition(const fvMesh& mesh);
//- Destructor
virtual ~domainDecomposition();
// Member Functions
//- Access the global mesh
const fvMesh& mesh() const
{
return mesh_;
}
//- Return the number of processors in the decomposition
label nProcs() const
{
return nProcs_;
}
//- Is the decomposition data to be distributed for each processor?
bool distributed() const
{
return distributed_;
}
//- Access the labels of points for each processor
const labelListList& procPointAddressing() const
{
validate();
return procPointAddressing_;
}
//- Access the labels of faces for each processor (see notes above)
const List<DynamicList<label>>& procFaceAddressing() const
{
validate();
return procFaceAddressing_;
}
//- Access the labels of cells for each processor
const labelListList& procCellAddressing() const
{
validate();
return procCellAddressing_;
}
//- Return the processor meshes
const PtrList<fvMesh>& procMeshes() const
{
validate();
return procMeshes_;
}
//- Generate the decomposition
void decompose();
//- Reset the time
void setTime(const instant&, const label newIndex);
//- Read additional (faceInstance) points, if necessary
void readPoints();
//- Read in the decomposition addressing
void readAddressing();
//- Read in the meshes and the decomposition
void read();
//- Read update the meshes and the decomposition
fvMesh::readUpdateState readUpdate();
//- Return the distribution as an FV field for writing
labelList cellToProc() const;
//- Read additional (faceInstance) points, if necessary
void writePoints() const;
//- Write the decomposition addressing
void writeAddressing() const;
//- Write the decomposed meshes and associated data
void write(const bool decomposeSets) const;
};
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
} // End namespace Foam
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
#endif
// ************************************************************************* //

View File

@ -25,95 +25,38 @@ License
#include "fvFieldDecomposer.H"
// * * * * * * * * * * * * * * * * Constructors * * * * * * * * * * * * * * //
// * * * * * * * * * * * * * Private Member Functions * * * * * * * * * * * //
Foam::labelList Foam::fvFieldDecomposer::patchFieldDecomposer::alignAddressing
Foam::label Foam::fvFieldDecomposer::completePatchID
(
const labelUList& addressingSlice,
const label addressingOffset
const label procPatchi
) const
{
labelList addressing(addressingSlice.size());
const fvPatch& procPatch = procMesh_.boundary()[procPatchi];
forAll(addressing, i)
if (procPatchi < completeMesh_.boundary().size())
{
// Subtract one to align addressing.
addressing[i] = addressingSlice[i] - (addressingOffset + 1);
return procPatchi;
}
else if (isA<processorCyclicFvPatch>(procPatch))
{
return refCast<const processorCyclicFvPatch>(procPatch).referPatchID();
}
else
{
return -1;
}
return addressing;
}
// * * * * * * * * * * * * * * * * Constructors * * * * * * * * * * * * * * //
Foam::fvFieldDecomposer::patchFieldDecomposer::patchFieldDecomposer
(
const labelUList& addressingSlice,
const label addressingOffset
const labelUList& addressing
)
:
labelList(alignAddressing(addressingSlice, addressingOffset)),
directFvPatchFieldMapper(static_cast<const labelList&>(*this))
{}
Foam::labelList Foam::fvFieldDecomposer::processorVolPatchFieldDecomposer::
alignAddressing
(
const fvMesh& mesh,
const labelUList& addressingSlice
) const
{
labelList addressing(addressingSlice.size());
const labelList& own = mesh.faceOwner();
const labelList& neighb = mesh.faceNeighbour();
forAll(addressing, i)
{
// Subtract one to align addressing.
label ai = mag(addressingSlice[i]) - 1;
if (ai < neighb.size())
{
// This is a regular face. it has been an internal face
// of the original mesh and now it has become a face
// on the parallel boundary.
// Give face the value of the neighbour.
if (addressingSlice[i] >= 0)
{
// I have the owner so use the neighbour value
addressing[i] = neighb[ai];
}
else
{
addressing[i] = own[ai];
}
}
else
{
// This is a face that used to be on a cyclic boundary
// but has now become a parallel patch face. I cannot
// do the interpolation properly (I would need to look
// up the different (face) list of data), so I will
// just grab the value from the owner cell
addressing[i] = own[ai];
}
}
return addressing;
}
Foam::fvFieldDecomposer::processorVolPatchFieldDecomposer::
processorVolPatchFieldDecomposer
(
const fvMesh& mesh,
const labelUList& addressingSlice
)
:
labelList(alignAddressing(mesh, addressingSlice)),
labelList(mag(addressing) - 1),
directFvPatchFieldMapper(static_cast<const labelList&>(*this))
{}
@ -123,34 +66,22 @@ Foam::fvFieldDecomposer::fvFieldDecomposer
const fvMesh& completeMesh,
const fvMesh& procMesh,
const labelList& faceAddressing,
const labelList& cellAddressing
const labelList& cellAddressing,
const surfaceLabelField::Boundary& faceAddressingBf
)
:
completeMesh_(completeMesh),
procMesh_(procMesh),
faceAddressing_(faceAddressing),
cellAddressing_(cellAddressing),
patchFieldDecomposers_(procMesh_.boundary().size()),
processorVolPatchFieldDecomposers_(procMesh_.boundary().size())
faceAddressingBf_(faceAddressingBf),
patchFieldDecomposers_(procMesh_.boundary().size())
{
forAll(procMesh_.boundary(), procPatchi)
{
const fvPatch& procPatch = procMesh.boundary()[procPatchi];
const label completePatchi = completePatchID(procPatchi);
// Determine the index of the corresponding complete patch
label completePatchi = -1;
if (procPatchi < completeMesh_.boundary().size())
{
completePatchi = procPatchi;
}
else if (isA<processorCyclicFvPatch>(procPatch))
{
completePatchi =
refCast<const processorCyclicPolyPatch>
(procPatch.patch()).referPatchID();
}
// If there is a corresponding complete patch, then create a mapper
// If there is a corresponding complete patch then create a patch mapper
if (completePatchi >= 0)
{
patchFieldDecomposers_.set
@ -158,22 +89,7 @@ Foam::fvFieldDecomposer::fvFieldDecomposer
procPatchi,
new patchFieldDecomposer
(
procPatch.patchSlice(faceAddressing_),
completeMesh_.boundaryMesh()[completePatchi].start()
)
);
}
// If this is a processor patch then create a processor mapper
if (procPatchi >= completeMesh_.boundary().size())
{
processorVolPatchFieldDecomposers_.set
(
procPatchi,
new processorVolPatchFieldDecomposer
(
completeMesh_,
procPatch.patchSlice(faceAddressing_)
faceAddressingBf[completePatchi]
)
);
}

View File

@ -63,52 +63,12 @@ public:
public labelList,
public directFvPatchFieldMapper
{
// Private Member Functions
labelList alignAddressing
(
const labelUList& addressingSlice,
const label addressingOffset
) const;
public:
// Constructors
//- Construct given addressing
patchFieldDecomposer
(
const labelUList& addressingSlice,
const label addressingOffset
);
};
//- Processor patch field decomposer class. Maps either owner or
// neighbour data (no interpolate anymore - processorFvPatchField
// holds neighbour data)
class processorVolPatchFieldDecomposer
:
public labelList,
public directFvPatchFieldMapper
{
// Private Member Functions
labelList alignAddressing
(
const fvMesh& mesh,
const labelUList& addressingSlice
) const;
public:
//- Construct given addressing
processorVolPatchFieldDecomposer
(
const fvMesh& mesh,
const labelUList& addressingSlice
);
patchFieldDecomposer(const labelUList& addressing);
};
@ -128,23 +88,35 @@ private:
//- Reference to cell addressing
const labelList& cellAddressing_;
//- Reference to face addressing boundary field
const surfaceLabelField::Boundary& faceAddressingBf_;
//- List of patch field decomposers
PtrList<patchFieldDecomposer> patchFieldDecomposers_;
//- ...
PtrList<processorVolPatchFieldDecomposer>
processorVolPatchFieldDecomposers_;
// Private Member Functions
//- Helper: map & optionally flip a (face) field
//- Convert a processor patch to the corresponding complete patch index
label completePatchID(const label procPatchi) const;
//- Map cell values to faces
template<class Type>
static tmp<Field<Type>> mapField
static tmp<Field<Type>> mapCellToFace
(
const labelUList& owner,
const labelUList& neighbour,
const Field<Type>& field,
const labelUList& addressing
);
//- Map face values to faces
template<class Type>
static tmp<Field<Type>> mapFaceToFace
(
const Field<Type>& field,
const labelUList& mapAndSign,
const bool applyFlip
const labelUList& addressing,
const bool isFlux
);
@ -158,7 +130,8 @@ public:
const fvMesh& completeMesh,
const fvMesh& procMesh,
const labelList& faceAddressing,
const labelList& cellAddressing
const labelList& cellAddressing,
const surfaceLabelField::Boundary& faceAddressingBf
);
//- Disallow default bitwise copy construction

View File

@ -33,35 +33,50 @@ License
// * * * * * * * * * * * * * * * Member Functions * * * * * * * * * * * * * //
template<class Type>
Foam::tmp<Foam::Field<Type>> Foam::fvFieldDecomposer::mapField
Foam::tmp<Foam::Field<Type>> Foam::fvFieldDecomposer::mapCellToFace
(
const labelUList& owner,
const labelUList& neighbour,
const Field<Type>& field,
const labelUList& mapAndSign,
const bool applyFlip
const labelUList& addressing
)
{
tmp<Field<Type>> tfld(new Field<Type>(mapAndSign.size()));
tmp<Field<Type>> tfld(new Field<Type>(addressing.size()));
Field<Type>& fld = tfld.ref();
if (applyFlip)
forAll(addressing, i)
{
forAll(mapAndSign, i)
{
if (mapAndSign[i] < 0)
{
fld[i] = -field[-mapAndSign[i] - 1];
}
else
{
fld[i] = field[mapAndSign[i] - 1];
}
}
fld[i] =
field
[
addressing[i] > 0
? neighbour[addressing[i] - 1]
: owner[- addressing[i] + 1]
];
}
else
return tfld;
}
template<class Type>
Foam::tmp<Foam::Field<Type>> Foam::fvFieldDecomposer::mapFaceToFace
(
const Field<Type>& field,
const labelUList& addressing,
const bool isFlux
)
{
tmp<Field<Type>> tfld(new Field<Type>(addressing.size()));
Field<Type>& fld = tfld.ref();
forAll(addressing, i)
{
// Ignore face flipping
fld.map(field, mag(mapAndSign) - 1);
fld[i] =
(isFlux && addressing[i] < 0 ? -1 : +1)
*field[mag(addressing[i]) - 1];
}
return tfld;
}
@ -74,9 +89,8 @@ Foam::fvFieldDecomposer::decomposeField
const bool allowUnknownPatchFields
) const
{
// 1. Create the complete field with dummy patch fields
// Create dummy patch fields
PtrList<fvPatchField<Type>> patchFields(procMesh_.boundary().size());
forAll(procMesh_.boundary(), procPatchi)
{
patchFields.set
@ -91,7 +105,7 @@ Foam::fvFieldDecomposer::decomposeField
);
}
// Create the field for the processor
// Create the processor field with the dummy patch fields
tmp<GeometricField<Type, fvPatchField, volMesh>> tresF
(
new GeometricField<Type, fvPatchField, volMesh>
@ -113,35 +127,17 @@ Foam::fvFieldDecomposer::decomposeField
);
GeometricField<Type, fvPatchField, volMesh>& resF = tresF.ref();
// 2. Change the fvPatchFields to the correct type using a mapper
// constructor (with reference to the now correct internal field)
// Change the patch fields to the correct type using a mapper constructor
// (with reference to the now correct internal field)
typename GeometricField<Type, fvPatchField, volMesh>::
Boundary& bf = resF.boundaryFieldRef();
forAll(bf, procPatchi)
{
const fvPatch& procPatch = procMesh_.boundary()[procPatchi];
// Determine the index of the corresponding complete patch
label completePatchi = -1;
if (procPatchi < completeMesh_.boundary().size())
{
completePatchi = procPatchi;
}
else if (isA<processorCyclicFvPatch>(procPatch))
{
const label referPatchi =
refCast<const processorCyclicPolyPatch>
(procPatch.patch()).referPatchID();
if (field.boundaryField()[referPatchi].overridesConstraint())
{
completePatchi = referPatchi;
}
}
const label completePatchi = completePatchID(procPatchi);
if (completePatchi != -1)
if (completePatchi == procPatchi)
{
bf.set
(
@ -157,6 +153,10 @@ Foam::fvFieldDecomposer::decomposeField
}
else if (isA<processorCyclicFvPatch>(procPatch))
{
const label nbrCompletePatchi =
refCast<const processorCyclicFvPatch>(procPatch)
.referPatch().nbrPatchID();
bf.set
(
procPatchi,
@ -164,9 +164,12 @@ Foam::fvFieldDecomposer::decomposeField
(
procPatch,
resF(),
processorVolPatchFieldDecomposers_[procPatchi]
mapCellToFace
(
field.primitiveField()
labelUList(),
completeMesh_.lduAddr().patchAddr(nbrCompletePatchi),
field.primitiveField(),
faceAddressingBf_[procPatchi]
)
)
);
@ -180,9 +183,12 @@ Foam::fvFieldDecomposer::decomposeField
(
procPatch,
resF(),
processorVolPatchFieldDecomposers_[procPatchi]
mapCellToFace
(
field.primitiveField()
completeMesh_.owner(),
completeMesh_.neighbour(),
field.primitiveField(),
faceAddressingBf_[procPatchi]
)
)
);
@ -206,7 +212,6 @@ Foam::fvFieldDecomposer::decomposeField
}
}
// Create the field for the processor
return tresF;
}
@ -218,34 +223,14 @@ Foam::fvFieldDecomposer::decomposeField
const GeometricField<Type, fvsPatchField, surfaceMesh>& field
) const
{
// Problem with addressing when a processor patch picks up both internal
// faces and faces from cyclic boundaries. This is a bit of a hack, but
// I cannot find a better solution without making the internal storage
// mechanism for surfaceFields correspond to the one of faces in polyMesh
// (i.e. using slices)
Field<Type> allFaceField(field.mesh().nFaces());
const SubList<label> faceAddressingIf
(
faceAddressing_,
procMesh_.nInternalFaces()
);
forAll(field.primitiveField(), i)
{
allFaceField[i] = field.primitiveField()[i];
}
forAll(field.boundaryField(), patchi)
{
const Field<Type> & p = field.boundaryField()[patchi];
const label patchStart = field.mesh().boundaryMesh()[patchi].start();
forAll(p, i)
{
allFaceField[patchStart + i] = p[i];
}
}
// 1. Create the complete field with dummy patch fields
// Create dummy patch fields
PtrList<fvsPatchField<Type>> patchFields(procMesh_.boundary().size());
forAll(procMesh_.boundary(), procPatchi)
{
patchFields.set
@ -260,6 +245,7 @@ Foam::fvFieldDecomposer::decomposeField
);
}
// Create the processor field with the dummy patch fields
tmp<GeometricField<Type, fvsPatchField, surfaceMesh>> tresF
(
new GeometricField<Type, fvsPatchField, surfaceMesh>
@ -275,14 +261,10 @@ Foam::fvFieldDecomposer::decomposeField
),
procMesh_,
field.dimensions(),
mapField
mapFaceToFace
(
field,
labelList::subList
(
faceAddressing_,
procMesh_.nInternalFaces()
),
faceAddressingIf,
isFlux(field)
),
patchFields
@ -290,18 +272,17 @@ Foam::fvFieldDecomposer::decomposeField
);
GeometricField<Type, fvsPatchField, surfaceMesh>& resF = tresF.ref();
// 2. Change the fvsPatchFields to the correct type using a mapper
// constructor (with reference to the now correct internal field)
// Change the patch fields to the correct type using a mapper constructor
// (with reference to the now correct internal field)
typename GeometricField<Type, fvsPatchField, surfaceMesh>::
Boundary& bf = resF.boundaryFieldRef();
forAll(procMesh_.boundary(), procPatchi)
{
const fvPatch& procPatch = procMesh_.boundary()[procPatchi];
if (procPatchi < completeMesh_.boundary().size())
const label completePatchi = completePatchID(procPatchi);
if (completePatchi == procPatchi)
{
bf.set
(
@ -317,7 +298,6 @@ Foam::fvFieldDecomposer::decomposeField
}
else if (isA<processorCyclicFvPatch>(procPatch))
{
// Do our own mapping. Avoids a lot of mapping complexity.
bf.set
(
procPatchi,
@ -325,10 +305,10 @@ Foam::fvFieldDecomposer::decomposeField
(
procPatch,
resF(),
mapField
mapFaceToFace
(
allFaceField,
procPatch.patchSlice(faceAddressing_),
field.boundaryField()[completePatchi],
faceAddressingBf_[procPatchi],
isFlux(field)
)
)
@ -336,7 +316,6 @@ Foam::fvFieldDecomposer::decomposeField
}
else if (isA<processorFvPatch>(procPatch))
{
// Do our own mapping. Avoids a lot of mapping complexity.
bf.set
(
procPatchi,
@ -344,10 +323,10 @@ Foam::fvFieldDecomposer::decomposeField
(
procPatch,
resF(),
mapField
mapFaceToFace
(
allFaceField,
procPatch.patchSlice(faceAddressing_),
field.primitiveField(),
faceAddressingBf_[procPatchi],
isFlux(field)
)
)
@ -360,7 +339,6 @@ Foam::fvFieldDecomposer::decomposeField
}
}
// Create the field for the processor
return tresF;
}