diff --git a/doc/src/Howto_output.rst b/doc/src/Howto_output.rst index 792c8f8398..bf04b39781 100644 --- a/doc/src/Howto_output.rst +++ b/doc/src/Howto_output.rst @@ -56,10 +56,10 @@ of each atom. Local datums are calculated by each processor based on the atoms it owns, but there may be zero or more per atom, e.g. a list of bond distances. -A per-grid datum is one or more values per grid point, for a grid -which overlays the simulation domain. The grid points and the data -they store are distributed across processors; each processor owns the -grid points which fall within its sub-domain. +A per-grid datum is one or more values per grid cell, for a grid which +overlays the simulation domain. The grid cells and the data they +store are distributed across processors; each processor owns the grid +cells whose center point falls within its sub-domain. .. _scalar: @@ -97,7 +97,7 @@ Per-grid data ------------------------ Per-grid data can come in two kinds: a vector of values (one per grid -point), or a 2d array of values (multiple values per grid point). The +cekk), or a 2d array of values (multiple values per grid ckk). The doc page for a "compute" or "fix" that generates data will specify names for both the grid(s) and datum(s) it produces, e.g. per-grid vectors or arrays, which can be referenced by other commands. @@ -213,7 +213,7 @@ global and can also be used as input to other output commands. Note that the :doc:`fix ave/grid ` command can also average the same per-atom quantities within spatial bins, but it does -this for a distributed grid whose grid points are owned by different +this for a distributed grid whose grid cells are owned by different processors. It outputs per-grid data, not global data, so it is more efficient for large numbers of averaging bins. @@ -269,7 +269,7 @@ These are produced as output values which can be used as input to other output commands. The :doc:`compute property/grid ` command takes -a list of one or more pre-defined per-grid attributes (id, grid point +a list of one or more pre-defined per-grid attributes (id, grid cell coords, etc) and stores the values in a per-grid vector or array. These are produced as output values which can be used as input to the :doc:`dump grid ` command. @@ -321,7 +321,7 @@ The chief difference between the :doc:`fix ave/grid ` and :doc:`fix ave/chunk ` commands when used in this context is that the former uses a distributed grid, while the latter uses a global grid. Distributed means that each processor owns the -subset of grid points within its sub-domain. Global means that each +subset of grid cells within its sub-domain. Global means that each processor owns a copy of the entire grid. The :doc:`fix ave/grid ` command is thus more efficient for large grids. diff --git a/doc/src/compute_property_grid.rst b/doc/src/compute_property_grid.rst index b2edd18dde..90b48d3a7c 100644 --- a/doc/src/compute_property_grid.rst +++ b/doc/src/compute_property_grid.rst @@ -18,7 +18,7 @@ Syntax .. parsed-literal:: attributes = id, ix, iy, iz, x, y, z, xs, ys, zs, xc, yc, zc, xsc, ysc, zsc - id = ID of grid point, x fastest, y next, z slowest + id = ID of grid cell, x fastest, y next, z slowest ix,iy,iz = grid indices in each dimension (1 to N inclusive) x,y,z = coords of lower left corner of grid cell xs,ys,zs = scaled coords of lower left corner of grid cell (0.0 to 1.0) @@ -39,9 +39,9 @@ Description Define a computation that stores the specified attributes of a distributed grid. In LAMMPS, distributed grids are regular 2d or 3d grids which overlay a 2d or 3d simulation domain. Each processor owns -the grid points within its sub-domain. See the :doc:`Howto grid -` doc page for details of how distributed grids can be -defined by various commands and referenced. +the grid cells whose center points lie within its sub-domain. See the +:doc:`Howto grid ` doc page for details of how distributed +grids can be defined by various commands and referenced. This compute stores the specified attributes of grids as per-grid data so they can be accessed by other :doc:`output commands ` @@ -53,14 +53,14 @@ to output per-grid values from other computes of fixes, the grid size specified for this command must be consistent with the grid sizes used by the other commands. -The *id* attribute stores the grid ID for each grid point. For a +The *id* attribute stores the grid ID for each grid cell. For a global grid of size Nx by Ny by Nz (in 3d simulations) the grid IDs range from 1 to Nx*Ny*Nz. They are ordered with the X index of the 3d grid varying fastest, then Y, then Z slowest. For 2d grids (in 2d simulations), the grid IDs range from 1 to Nx*Ny, with X varying fastest and Y slowest. -The *ix*, *iy*, *iz* attributes are the indices of a grid point in +The *ix*, *iy*, *iz* attributes are the indices of a grid cell in each dimension. They range from 1 to Nx inclusive in the X dimension, and similar for Y and Z. @@ -91,8 +91,8 @@ Output info This compute calculates a per-grid vector or array depending on the number of input values. The length of the vector or number of array rows (distributed across all processors) is Nx * Ny * Nz. For access -by other commands, the name of the grid produced by this command is -"grid". The name of its data is "data". +by other commands, the name of the single grid produced by this +command is "grid". The name of its per-grid data is "data". The (x,y,z) and (xc,yc,zc) coordinates are in distance :doc:`units `. diff --git a/doc/src/fix_ave_grid.rst b/doc/src/fix_ave_grid.rst index 7453a2ea8a..e32560193a 100644 --- a/doc/src/fix_ave_grid.rst +++ b/doc/src/fix_ave_grid.rst @@ -183,8 +183,8 @@ i.e. total-mass/volume. The output values are in units of 1/volume or density (mass/volume). See the :doc:`units ` command page for the definition of density for each choice of units, e.g. gram/cm\^3. -The *temp* value means the temperature is computed for each grid cell, -by the formula +The *temp* value computes the temperature for each grid cell, by the +formula .. math:: @@ -444,24 +444,24 @@ No information about this fix is written to :doc:`binary restart files `. None of the :doc:`fix_modify ` options are relevant to this fix. -This fix computes a global array of values which can be accessed by -various :doc:`output commands `. The values can only be -accessed on timesteps that are multiples of *Nfreq* since that is when -averaging is performed. The global array has # of rows = the number -of grids *grid* as calculated by the specified :doc:`compute -property/grid ` command. The # of columns = -M+1+Nvalues, where M = 1 to 4, depending on whether the optional -columns for OrigID and CoordN are used, as explained above. Following -the optional columns, the next column contains the count of atoms in -the grid, and the remaining columns are the Nvalue quantities. When -the array is accessed with a row I that exceeds the current number of -grids, than a 0.0 is returned by the fix instead of an error, since -the number of grids can vary as a simulation runs depending on how -that value is computed by the compute grid/atom command. +This fix calculates a per-grid array which has one column for each of +the specified input values. The units for each column with be in the +:doc:`units ` for the per-atom or per-grid quantity for the +corresponding input value. If the fix is used in per-atom mode, it +also calculates a per-grid vector with the count of atoms in each grid +cell. The number of rows in the per-grid array and number of values +in the per-grid vector (distributed across all processors) is Nx * +Ny * Nz. -The array values calculated by this fix are treated as "intensive", -since they are typically already normalized by the count of atoms in -each grid. +For access by other commands, the name of the single grid produced by +this fix is "grid". The names of its two per-grid datums are "data" +for the per-grid array and "count" for the per-grid vector. Both +datums can be accessed by various :doc:`output commands +`. + +In per-atom mode, the per-grid array values calculated by this fix are +treated as "intensive", since they are typically already normalized by +the count of atoms in each grid cell. No parameter of this fix can be used with the *start/stop* keywords of the :doc:`run ` command. This fix is not invoked during diff --git a/doc/src/fix_ttm.rst b/doc/src/fix_ttm.rst index 73bd70f76f..2cbb0c7799 100644 --- a/doc/src/fix_ttm.rst +++ b/doc/src/fix_ttm.rst @@ -386,13 +386,13 @@ electronic subsystem energies reported at the end of the timestep. The vector values calculated are "extensive". -Th fix ttm/grid command also calculates a per-grid vector which store -the electron temperature for each grid cell in temperature :doc:`units -`. which can be accessed by various :doc:`output commands -`. The length of the vector (distributed across all -processors) is Nx * Ny * Nz. For access by other commands, the name -of the grid produced by fix ttm/grid is "grid". The name of its data -is "data". +The fix ttm/grid command also calculates a per-grid vector which +stores the electron temperature for each grid cell in temperature +:doc:`units `. which can be accessed by various :doc:`output +commands `. The length of the vector (distributed +across all processors) is Nx * Ny * Nz. For access by other commands, +the name of the single grid produced by fix ttm/grid is "grid". The +name of its per-grid data is "data". No parameter of the fixes can be used with the *start/stop* keywords of the :doc:`run ` command. The fixes are not invoked during diff --git a/src/EXTRA-FIX/fix_ttm_grid.cpp b/src/EXTRA-FIX/fix_ttm_grid.cpp index 08f1e43c32..59726ae63b 100644 --- a/src/EXTRA-FIX/fix_ttm_grid.cpp +++ b/src/EXTRA-FIX/fix_ttm_grid.cpp @@ -496,9 +496,6 @@ void FixTTMGrid::reset_grid() return; } else delete gridnew; - // DEBUG - if (comm->me == 0) printf("Remapping grid on step %ld\n",update->ntimestep); - // delete grid data which doesn't need to persist from previous to new decomp memory->destroy(grid_buf1); @@ -601,7 +598,7 @@ void FixTTMGrid::unpack_reverse_grid(int /*flag*/, void *vbuf, int nlist, int *l } /* ---------------------------------------------------------------------- - pack old grid values to buf to send to another proc + pack old grid values to buf to send to another proc ------------------------------------------------------------------------- */ void FixTTMGrid::pack_remap_grid(void *vbuf, int nlist, int *list) diff --git a/src/fix_ave_grid.cpp b/src/fix_ave_grid.cpp index e68591c7e3..dcc1016729 100644 --- a/src/fix_ave_grid.cpp +++ b/src/fix_ave_grid.cpp @@ -29,6 +29,9 @@ #include "update.h" #include "variable.h" +// DEBUG +#include "comm.h" + #include using namespace LAMMPS_NS; @@ -390,7 +393,6 @@ FixAveGrid::FixAveGrid(LAMMPS *lmp, int narg, char **arg) : // instantiate the Grid class and allocate per-grid memory - double maxdist; if (modeatom) maxdist = 0.5 * neighbor->skin; else if (modegrid) maxdist = 0.0; @@ -1889,17 +1891,122 @@ void FixAveGrid::unpack_reverse_grid(int /*flag*/, void *vbuf, int nlist, int *l } } +/* ---------------------------------------------------------------------- + pack old grid values to buf to send to another proc +------------------------------------------------------------------------- */ + +void FixAveGrid::pack_remap_grid(void *vbuf, int nlist, int *list) +{ + auto buf = (double *) vbuf; + double *src; + //double *src = + // &T_electron_previous[nzlo_out_previous][nylo_out_previous][nxlo_out_previous]; + + for (int i = 0; i < nlist; i++) buf[i] = src[list[i]]; +} + +/* ---------------------------------------------------------------------- + unpack another proc's own values from buf and set own ghost values +------------------------------------------------------------------------- */ + +void FixAveGrid::unpack_remap_grid(void *vbuf, int nlist, int *list) +{ + auto buf = (double *) vbuf; + double *dest; + //double *dest = &T_electron[nzlo_out][nylo_out][nxlo_out]; + + for (int i = 0; i < nlist; i++) dest[list[i]] = buf[i]; +} + /* ---------------------------------------------------------------------- subset of grid assigned to each proc may have changed called by load balancer when proc subdomains are adjusted - not supported for now, b/c requires per-grid values to persist, i.e. a remap() + persist per-grid data by performing a grid remap ------------------------------------------------------------------------- */ void FixAveGrid::reset_grid() { - error->all(FLERR,"Fix ave/grid does not support load balancing (yet)"); -} + // check if new grid partitioning is different on any proc + // if not, just return + if (dimension == 2) { + int tmp[8]; + Grid2d *gridnew = new Grid2d(lmp, world, nxgrid, nygrid); + gridnew->set_distance(maxdist); + gridnew->setup_grid(tmp[0], tmp[1], tmp[2], tmp[3], + tmp[4], tmp[5], tmp[6], tmp[7]); + + if (grid2d->identical(gridnew)) { + delete gridnew; + return; + } else delete gridnew; + + } else { + + int tmp[12]; + Grid3d *gridnew = new Grid3d(lmp, world, nxgrid, nygrid, nzgrid); + gridnew->set_distance(maxdist); + gridnew->setup_grid(tmp[0], tmp[1], tmp[2], tmp[3], tmp[4], tmp[5], + tmp[6], tmp[7], tmp[8], tmp[9], tmp[10], tmp[11]); + + if (grid3d->identical(gridnew)) { + delete gridnew; + return; + } else delete gridnew; + } + + // DEBUG + if (comm->me == 0) printf("Remapping grid on step %ld\n",update->ntimestep); + + /* + // delete grid data which doesn't need to persist from previous to new decomp + + memory->destroy(grid_buf1); + memory->destroy(grid_buf2); + memory->destroy3d_offset(T_electron_old, nzlo_out, nylo_out, nxlo_out); + memory->destroy3d_offset(net_energy_transfer, nzlo_out, nylo_out, nxlo_out); + + // make copy of ptrs to grid data which does need to persist + + grid_previous = grid; + T_electron_previous = T_electron; + nxlo_out_previous = nxlo_out; + nylo_out_previous = nylo_out; + nzlo_out_previous = nzlo_out; + + // allocate new per-grid data for new decomposition + + allocate_grid(); + + // perform remap from previous decomp to new decomp + + int nremap_buf1,nremap_buf2; + grid->setup_remap(grid_previous,nremap_buf1,nremap_buf2); + + double *remap_buf1,*remap_buf2; + memory->create(remap_buf1, nremap_buf1, "ave/grid:remap_buf1"); + memory->create(remap_buf2, nremap_buf2, "ave/grid:remap_buf2"); + + grid->remap(Grid3d::FIX,this,1,sizeof(double),remap_buf1,remap_buf2,MPI_DOUBLE); + + memory->destroy(remap_buf1); + memory->destroy(remap_buf2); + + // delete grid data and grid for previous decomposition + + memory->destroy3d_offset(T_electron_previous, + nzlo_out_previous, nylo_out_previous, + nxlo_out_previous); + delete grid_previous; + + // zero new net_energy_transfer + // in case compute_vector accesses it on timestep 0 + + outflag = 0; + memset(&net_energy_transfer[nzlo_out][nylo_out][nxlo_out],0, + ngridout*sizeof(double)); + */ +} /* ---------------------------------------------------------------------- return index of grid associated with name diff --git a/src/fix_ave_grid.h b/src/fix_ave_grid.h index 0503092233..71f1a34d8a 100644 --- a/src/fix_ave_grid.h +++ b/src/fix_ave_grid.h @@ -35,6 +35,8 @@ class FixAveGrid : public Fix { void pack_reverse_grid(int, void *, int, int *) override; void unpack_reverse_grid(int, void *, int, int *) override; + void pack_remap_grid(void *, int, int *) override; + void unpack_remap_grid(void *, int, int *) override; void reset_grid() override; @@ -53,6 +55,7 @@ class FixAveGrid : public Fix { int modeatom, modegrid; int discardflag, normflag, aveflag, nwindow; + double maxdist; int running_count; int window_count,window_oldest,window_newest;