diff --git a/doc/src/comm_modify.rst b/doc/src/comm_modify.rst index a3f7dba4eb..489ab407ee 100644 --- a/doc/src/comm_modify.rst +++ b/doc/src/comm_modify.rst @@ -15,12 +15,15 @@ Syntax .. parsed-literal:: - *mode* value = *single* or *multi* = communicate atoms within a single or multiple distances + *mode* value = *single*, *multi*, or *multi/old* = communicate atoms within a single or multiple distances *cutoff* value = Rcut (distance units) = communicate atoms from this far away - *cutoff/multi* type value + *cutoff/multi* collection value + collection = atom collection or collection range (supports asterisk notation) + value = Rcut (distance units) = communicate atoms for selected types from this far away + *reduce/multi* arg = none = reduce number of communicated ghost atoms for multi style + *cutoff/multi/old* type value type = atom type or type range (supports asterisk notation) value = Rcut (distance units) = communicate atoms for selected types from this far away - *multi/reduce* arg = none = reduce number of communicated ghost atoms for multi style *group* value = group-ID = only communicate atoms in the group *vel* value = *yes* or *no* = do or do not communicate velocity info with ghost atoms @@ -29,7 +32,7 @@ Examples .. code-block:: LAMMPS - comm_modify mode multi + comm_modify mode multi reduce/multi comm_modify mode multi group solvent comm_modift mode multi cutoff/multi 1 10.0 cutoff/multi 2*4 15.0 comm_modify vel yes @@ -63,13 +66,18 @@ sub-domain. The distance is by default the maximum of the neighbor cutoff across all atom type pairs. For many systems this is an efficient algorithm, but for systems with -widely varying cutoffs for different type pairs, the *multi* mode can -be faster. In this case, each atom type is assigned its own distance +widely varying cutoffs for different type pairs, the *multi* or *multi/old* mode can +be faster. In *multi*, each atom is assigned to a collection which should +correspond to a set of atoms with similar interaction cutoffs. +In this case, each atom collection is assigned its own distance cutoff for communication purposes, and fewer atoms will be -communicated. See the :doc:`neighbor multi ` command for a -neighbor list construction option that may also be beneficial for -simulations of this kind. The *multi* mode is compatable with both the -*multi* and *multi/old* neighbor styles. +communicated. in *multi/old*, a similar technique is used but atoms +are grouped by atom type. See the :doc:`neighbor multi ` and +:doc:`neighbor multi/old ` commands for +neighbor list construction options that may also be beneficial for +simulations of this kind. The *multi* communiction mode is only compatable +with the *multi* neighbor style. The *multi/old* communication mode is compatble +with both the *multi* and *multi/old* neighbor styles. The *cutoff* keyword allows you to extend the ghost cutoff distance for communication mode *single*\ , which is the distance from the borders @@ -89,18 +97,24 @@ warning is printed, if this bond based estimate is larger than the communication cutoff used. The *cutoff/multi* option is equivalent to *cutoff*\ , but applies to -communication mode *multi* instead. Since in this case the communication -cutoffs are determined per atom type, a type specifier is needed and -cutoff for one or multiple types can be extended. Also ranges of types -using the usual asterisk notation can be given. For granular pair styles, -the default cutoff is set to the sum of the current maximum atomic radii -for each type. +communication mode *multi* instead. Since the communication +cutoffs are determined per atom collections, a collection specifier is needed and +cutoff for one or multiple collections can be extended. Also ranges of collections +using the usual asterisk notation can be given. +Note that the arguments for *cutoff/multi* are parsed right before each +simulation to account for potential changes in the number of collections. +Custom cutoffs are preserved between runs but if collections are redefined, +one may want to respecify communication cutoffs. +For granular pair styles,the default cutoff is set to the sum of the +current maximum atomic radii for each collection. +The *cutoff/multi/old* option is similar to *cutoff/multi* except it +operates on atom types as opposed to collections. -The *multi/reduce* option applies to *multi* and sets communication -cutoffs for different sized particles based on the largest interaction distance -between two particles in the same multi grouping. This reduces the number of +The *reduce/multi* option applies to *multi* and sets the communication +cutoff for a particle equal to the maximum interaction distance between particles +in the same collection. This reduces the number of ghost atoms that need to be communicated. This method is only compatible with the -*multi* neighbor style and requires only half neighbor lists and Newton on. +*multi* neighbor style and requires a half neighbor list and Newton on. See the :doc:`neighbor multi ` command for more information. These are simulation scenarios in which it may be useful or even diff --git a/doc/src/neigh_modify.rst b/doc/src/neigh_modify.rst index c6f573ef08..6ee21d7419 100644 --- a/doc/src/neigh_modify.rst +++ b/doc/src/neigh_modify.rst @@ -14,7 +14,7 @@ Syntax .. parsed-literal:: - keyword = *delay* or *every* or *check* or *once* or *cluster* or *include* or *exclude* or *page* or *one* or *binsize* or *multi/custom* + keyword = *delay* or *every* or *check* or *once* or *cluster* or *include* or *exclude* or *page* or *one* or *binsize* or *collection/type* or *collection/interval* *delay* value = N N = delay building until this many steps since last build *every* value = M @@ -47,9 +47,12 @@ Syntax N = max number of neighbors of one atom *binsize* value = size size = bin size for neighbor list construction (distance units) - *multi/custom* values = N arg1 ... argN - N = number of custom groups - arg = N separate types or ranges of types (see below) + *collection/type* values = N arg1 ... argN + N = number of custom collections + arg = N separate lists of types (see below) + *collection/interval* values = N arg1 ... argN + N = number of custom collections + arg = N separate cutoffs for intervals (see below) Examples """""""" @@ -61,7 +64,8 @@ Examples neigh_modify exclude group frozen frozen check no neigh_modify exclude group residue1 chain3 neigh_modify exclude molecule/intra rigid - neigh_modify multi/custom 2 1*2 3*4 + neigh_modify collection/type 2 1*2,5 3*4 + neigh_modify collection/interval 2 1.0 10.0 Description """"""""""" @@ -192,8 +196,9 @@ atom can have. The *binsize* option allows you to specify what size of bins will be used in neighbor list construction to sort and find neighboring atoms. By default, for :doc:`neighbor style bin `, LAMMPS uses bins -that are 1/2 the size of the maximum pair cutoff. For :doc:`neighbor style multi `, the bins are 1/2 the size of the minimum pair -cutoff. Typically these are good values for minimizing the time for +that are 1/2 the size of the maximum pair cutoff. For :doc:`neighbor style multi `, +the bins are 1/2 the size of the collection interaction cutoff. +Typically these are good values for minimizing the time for neighbor list construction. This setting overrides the default. If you make it too big, there is little overhead due to looping over bins, but more atoms are checked. If you make it too @@ -201,19 +206,28 @@ small, the optimal number of atoms is checked, but bin overhead goes up. If you set the binsize to 0.0, LAMMPS will use the default binsize of 1/2 the cutoff. -The *multi/custom* option allows you to define custom groups of atom +The *collection/type* option allows you to define custom collections of atom types for the *multi* neighbor mode. By grouping atom types with similar cutoffs, one may be able to improve performance by reducing -overhead. You must first specify the number of custom groups N to be -defined followed by N ranges of types. The range can be specified as a +overhead. You must first specify the number of custom collections N to be +defined followed by N lists of types. Each list consists of a series of type +ranges separated by commas. The range can be specified as a single numeric value, or a wildcard asterisk can be used to specify a range of values. This takes the form "\*" or "\*n" or "n\*" or "m\*n". For example, if M = the number of atom types, then an asterisk with no numeric values means all types from 1 to M. A leading asterisk means all types from 1 to n (inclusive). A trailing asterisk means all types from n to M (inclusive). A middle asterisk means all types from m to n (inclusive). -Note that any atom types not included in a custom group will be automatically -placed within a separate group. +Note that all atom types must be included in a custom collection. + +The *collection/interval* option provides a similar capability. +This command allows a user to define custom collections by specifying a +series of cutoff intervals. LAMMPS will automatically sort atoms into these intervals +based on their type-dependent cutoffs or their finite size. +You must first specify the number of custom collections N to be +defined followed by N values representing the upper cutoff of each interval. +This command is particularly useful for granular pairstyles where the interaction distance +of particles depends on their radius and may not depend on their atom type. Restrictions """""""""""" diff --git a/doc/src/neighbor.rst b/doc/src/neighbor.rst index 52d59dae75..bd58f7045d 100644 --- a/doc/src/neighbor.rst +++ b/doc/src/neighbor.rst @@ -56,13 +56,13 @@ the largest cutoff distance between any pair of atom types and a single set of bins is defined to search over for all atom types. This can be inefficient if one pair of types has a very long cutoff, but other type pairs have a much shorter cutoff. The *multi* style uses -different sized bins for groups of different sized particles. Different +different sized bins for collections of different sized particles. Different sets of bins are then used to construct the neighbor lists as as further described by Shire, Hanley, and Stratford :ref:`(Shire) `. This imposes some extra setup overhead, but the searches themselves -may be much faster. By default, separate groups of particles are defined +may be much faster. By default, separate collections of particles are defined for each atom type. For systems with two or more types with similar -cutoffs, one can reduce the extra overhead by defining custom groupings +cutoffs, one can reduce the extra overhead by defining custom collections using the :doc:`neigh_modify ` command. See the :doc:`comm_modify mode bytype ` command for compatible communication options that may be beneficial for simulations of this kind. diff --git a/examples/multi/in.colloid b/examples/multi/in.colloid index b332560777..835032f2e2 100644 --- a/examples/multi/in.colloid +++ b/examples/multi/in.colloid @@ -30,8 +30,8 @@ velocity all create 1.44 87287 loop geom # multi neighbor and comm for efficiency neighbor 1 multi #multi/old -neigh_modify delay 0 multi/custom 2 1*4 5 -comm_modify mode multi multi/reduce +neigh_modify delay 0 collection/type 2 1*4 5 +comm_modify mode multi reduce/multi # colloid potential diff --git a/examples/multi/in.granular b/examples/multi/in.granular index 8f1743f90d..f43e9505e4 100644 --- a/examples/multi/in.granular +++ b/examples/multi/in.granular @@ -29,7 +29,7 @@ velocity all create 1.44 87287 loop geom neighbor 1 multi #multi/old neigh_modify delay 0 -comm_modify mode multi vel yes multi/reduce +comm_modify mode multi vel yes reduce/multi # colloid potential diff --git a/src/comm.cpp b/src/comm.cpp index 3f2d4ecde6..c84f0249ef 100644 --- a/src/comm.cpp +++ b/src/comm.cpp @@ -35,7 +35,7 @@ #include "update.h" #include - +#include #ifdef _OPENMP #include #endif @@ -59,6 +59,7 @@ Comm::Comm(LAMMPS *lmp) : Pointers(lmp) cutghostuser = 0.0; cutusermulti = nullptr; cutusermultiold = nullptr; + ncollections_prior = 0; ghost_velocity = 0; user_procgrid[0] = user_procgrid[1] = user_procgrid[2] = 0; @@ -339,24 +340,17 @@ void Comm::modify_params(int narg, char **arg) error->all(FLERR,"Use cutoff keyword to set cutoff in single mode"); if (mode == Comm::MULTIOLD) error->all(FLERR,"Use cutoff/multi/old keyword to set cutoff in multi/old mode"); - if (domain->box_exist == 0) + if (domain->box_exist == 0) error->all(FLERR, "Cannot set cutoff/multi before simulation box is defined"); - ncollections = neighbor->ncollections; - if (iarg+3 > narg) - error->all(FLERR,"Illegal comm_modify command"); - if (cutusermulti == nullptr) { - memory->create(cutusermulti,ncollections,"comm:cutusermulti"); - for (i=0; i < ncollections; ++i) - cutusermulti[i] = -1.0; - } - utils::bounds(FLERR,arg[iarg+1],1,ncollections,nlo,nhi,error); + + // save arguments so they can be parsed in comm->setup() + // ncollections can be changed by neigh_modify commands cut = utils::numeric(FLERR,arg[iarg+2],false,lmp); cutghostuser = MAX(cutghostuser,cut); if (cut < 0.0) error->all(FLERR,"Invalid cutoff in comm_modify command"); - for (i=nlo; i<=nhi; ++i) - cutusermulti[i] = cut; + usermultiargs.emplace_back(arg[iarg+1], cut); iarg += 3; } else if (strcmp(arg[iarg],"cutoff/multi/old") == 0) { int i,nlo,nhi; diff --git a/src/comm.h b/src/comm.h index 877941523c..2afdfd8765 100644 --- a/src/comm.h +++ b/src/comm.h @@ -33,6 +33,8 @@ class Comm : protected Pointers { double cutghost[3]; // cutoffs used for acquiring ghost atoms double cutghostuser; // user-specified ghost cutoff (mode == 0) double *cutusermulti; // per collection user ghost cutoff (mode == 1) + std::vector> usermultiargs; + // collection args for custom ghost cutoffs double *cutusermultiold; // per type user ghost cutoff (mode == 2) int recv_from_partition; // recv proc layout from this partition int send_to_partition; // send my proc layout to this partition diff --git a/src/comm_brick.cpp b/src/comm_brick.cpp index ca0fea8757..3cb841d20f 100644 --- a/src/comm_brick.cpp +++ b/src/comm_brick.cpp @@ -30,6 +30,7 @@ #include #include +#include using namespace LAMMPS_NS; @@ -194,12 +195,9 @@ void CommBrick::setup() if (mode == Comm::MULTI) { // build initial collection array - neighbor->build_collection(0); - - if(cutusermulti and ncollections != neighbor->ncollections) - error->all(FLERR, "Cannot change number of collections after defining comm_modify multi/cutoff"); - else ncollections = neighbor->ncollections; - + neighbor->build_collection(0); + ncollections = neighbor->ncollections; + // reallocate memory for multi-style communication at setup if ncollections change if(ncollections_prior != ncollections){ if(multilo) free_multi(); @@ -207,8 +205,24 @@ void CommBrick::setup() allocate_multi(maxswap); memory->create(cutghostmulti,ncollections,3,"comm:cutghostmulti"); + memory->grow(cutusermulti,ncollections,"comm:cutusermulti"); + for(i = ncollections_prior; i < ncollections; i++) + cutusermulti[i] = -1.0; + ncollections_prior = ncollections; } + + // parse any cutoff/multi commands + int nhi, nlo; + for(auto it = usermultiargs.begin(); it != usermultiargs.end(); it ++) { + utils::bounds(FLERR,it->first,1,ncollections,nlo,nhi,error); + if(nhi >= ncollections) + error->all(FLERR, "Unused collection id in comm_modify cutoff/multi command"); + for (j=nlo; j<=nhi; ++j) + cutusermulti[j] = it->second; + } + usermultiargs.clear(); + double **cutcollectionsq = neighbor->cutcollectionsq; // If using multi/reduce, communicate particles a distance equal diff --git a/src/comm_tiled.cpp b/src/comm_tiled.cpp index accdbf7b04..c06ad52d6c 100644 --- a/src/comm_tiled.cpp +++ b/src/comm_tiled.cpp @@ -31,6 +31,7 @@ #include #include +#include using namespace LAMMPS_NS; @@ -182,12 +183,8 @@ void CommTiled::setup() if (mode == Comm::MULTI) { // build collection from scratch as it is needed for atom exchange neighbor->build_collection(0); - - if(cutusermulti and ncollections != neighbor->ncollections) - error->all(FLERR, "Cannot change number of collections after defining comm_modify multi/cutoff"); - ncollections = neighbor->ncollections; - + // allocate memory for multi-style communication at setup as ncollections can change if(ncollections_prior != ncollections){ memory->destroy(cutghostmulti); @@ -197,8 +194,24 @@ void CommTiled::setup() for(i = 0; i < maxswap; i ++) grow_swap_send_multi(i,DELTA_PROCS); + memory->grow(cutusermulti,ncollections,"comm:cutusermulti"); + for(i = ncollections_prior; i < ncollections; i++) + cutusermulti[i] = -1.0; + ncollections_prior = ncollections; + } + + // parse any cutoff/multi commands + int nhi, nlo; + for(auto it = usermultiargs.begin(); it != usermultiargs.end(); it ++) { + utils::bounds(FLERR,it->first,1,ncollections,nlo,nhi,error); + if(nhi >= ncollections) + error->all(FLERR, "Unused collection id in comm_modify cutoff/multi command"); + + for (j=nlo; j<=nhi; ++j) + cutusermulti[j] = it->second; } + usermultiargs.clear(); double **cutcollectionsq = neighbor->cutcollectionsq; // If using multi/reduce, communicate particles a distance equal