doc page updates, start adding remap support to fix ave/grid

doc/src/Howto_output.rst

@@ -56,10 +56,10 @@ of each atom.  Local datums are calculated by each processor based on
 the atoms it owns, but there may be zero or more per atom, e.g. a list
 of bond distances.
 
-A per-grid datum is one or more values per grid point, for a grid
+A per-grid datum is one or more values per grid cell, for a grid which
-which overlays the simulation domain.  The grid points and the data
+overlays the simulation domain.  The grid cells and the data they
-they store are distributed across processors; each processor owns the
+store are distributed across processors; each processor owns the grid
-grid points which fall within its sub-domain.
+cells whose center point falls within its sub-domain.
 
 .. _scalar:
 
@@ -97,7 +97,7 @@ Per-grid data
 ------------------------
 
 Per-grid data can come in two kinds: a vector of values (one per grid
-point), or a 2d array of values (multiple values per grid point).  The
+cekk), or a 2d array of values (multiple values per grid ckk).  The
 doc page for a "compute" or "fix" that generates data will specify
 names for both the grid(s) and datum(s) it produces, e.g. per-grid
 vectors or arrays, which can be referenced by other commands.
@@ -213,7 +213,7 @@ global and can also be used as input to other output commands.
 
 Note that the :doc:`fix ave/grid <fix_ave_grid>` command can also
 average the same per-atom quantities within spatial bins, but it does
-this for a distributed grid whose grid points are owned by different
+this for a distributed grid whose grid cells are owned by different
 processors.  It outputs per-grid data, not global data, so it is more
 efficient for large numbers of averaging bins.
 
@@ -269,7 +269,7 @@ These are produced as output values which can be used as input to
 other output commands.
 
 The :doc:`compute property/grid <compute_property_grid>` command takes
-a list of one or more pre-defined per-grid attributes (id, grid point
+a list of one or more pre-defined per-grid attributes (id, grid cell
 coords, etc) and stores the values in a per-grid vector or array.
 These are produced as output values which can be used as input to the
 :doc:`dump grid <dump>` command.
@@ -321,7 +321,7 @@ The chief difference between the :doc:`fix ave/grid <fix_ave_grid>`
 and :doc:`fix ave/chunk <fix_ave_chunk>` commands when used in this
 context is that the former uses a distributed grid, while the latter
 uses a global grid.  Distributed means that each processor owns the
-subset of grid points within its sub-domain.  Global means that each
+subset of grid cells within its sub-domain.  Global means that each
 processor owns a copy of the entire grid.  The :doc:`fix ave/grid
 <fix_ave_grid>` command is thus more efficient for large grids.
 

doc/src/compute_property_grid.rst

@@ -18,7 +18,7 @@ Syntax
   .. parsed-literal::
 
        attributes = id, ix, iy, iz, x, y, z, xs, ys, zs, xc, yc, zc, xsc, ysc, zsc
-         id = ID of grid point, x fastest, y next, z slowest
+         id = ID of grid cell, x fastest, y next, z slowest
          ix,iy,iz = grid indices in each dimension (1 to N inclusive)
          x,y,z = coords of lower left corner of grid cell
          xs,ys,zs = scaled coords of lower left corner of grid cell (0.0 to 1.0)
@@ -39,9 +39,9 @@ Description
 Define a computation that stores the specified attributes of a
 distributed grid.  In LAMMPS, distributed grids are regular 2d or 3d
 grids which overlay a 2d or 3d simulation domain.  Each processor owns
-the grid points within its sub-domain.  See the :doc:`Howto grid
+the grid cells whose center points lie within its sub-domain.  See the
-<Howto_grid>` doc page for details of how distributed grids can be
+:doc:`Howto grid <Howto_grid>` doc page for details of how distributed
-defined by various commands and referenced.
+grids can be defined by various commands and referenced.
 
 This compute stores the specified attributes of grids as per-grid data
 so they can be accessed by other :doc:`output commands <Howto_output>`
@@ -53,14 +53,14 @@ to output per-grid values from other computes of fixes, the grid size
 specified for this command must be consistent with the grid sizes
 used by the other commands.
 
-The *id* attribute stores the grid ID for each grid point.  For a
+The *id* attribute stores the grid ID for each grid cell.  For a
 global grid of size Nx by Ny by Nz (in 3d simulations) the grid IDs
 range from 1 to Nx*Ny*Nz.  They are ordered with the X index of the 3d
 grid varying fastest, then Y, then Z slowest.  For 2d grids (in 2d
 simulations), the grid IDs range from 1 to Nx*Ny, with X varying
 fastest and Y slowest.
 
-The *ix*, *iy*, *iz* attributes are the indices of a grid point in
+The *ix*, *iy*, *iz* attributes are the indices of a grid cell in
 each dimension.  They range from 1 to Nx inclusive in the X dimension,
 and similar for Y and Z.
 
@@ -91,8 +91,8 @@ Output info
 This compute calculates a per-grid vector or array depending on the
 number of input values.  The length of the vector or number of array
 rows (distributed across all processors) is Nx * Ny * Nz.  For access
-by other commands, the name of the grid produced by this command is
+by other commands, the name of the single grid produced by this
-"grid".  The name of its data is "data".
+command is "grid".  The name of its per-grid data is "data".
 
 The (x,y,z) and (xc,yc,zc) coordinates are in distance :doc:`units
 <units>`.

doc/src/fix_ave_grid.rst

@@ -183,8 +183,8 @@ i.e. total-mass/volume.  The output values are in units of 1/volume or
 density (mass/volume).  See the :doc:`units <units>` command page for
 the definition of density for each choice of units, e.g. gram/cm\^3.
 
-The *temp* value means the temperature is computed for each grid cell,
+The *temp* value computes the temperature for each grid cell, by the
-by the formula
+formula
 
 .. math::
 
@@ -444,24 +444,24 @@ No information about this fix is written to :doc:`binary restart files
 <restart>`.  None of the :doc:`fix_modify <fix_modify>` options are
 relevant to this fix.
 
-This fix computes a global array of values which can be accessed by
+This fix calculates a per-grid array which has one column for each of
-various :doc:`output commands <Howto_output>`.  The values can only be
+the specified input values.  The units for each column with be in the
-accessed on timesteps that are multiples of *Nfreq* since that is when
+:doc:`units <units>` for the per-atom or per-grid quantity for the
-averaging is performed.  The global array has # of rows = the number
+corresponding input value.  If the fix is used in per-atom mode, it
-of grids *grid* as calculated by the specified :doc:`compute
+also calculates a per-grid vector with the count of atoms in each grid
-property/grid <compute_property_grid>` command.  The # of columns =
+cell.  The number of rows in the per-grid array and number of values
-M+1+Nvalues, where M = 1 to 4, depending on whether the optional
+in the per-grid vector (distributed across all processors) is Nx *
-columns for OrigID and CoordN are used, as explained above.  Following
+Ny * Nz.
-the optional columns, the next column contains the count of atoms in
-the grid, and the remaining columns are the Nvalue quantities.  When
-the array is accessed with a row I that exceeds the current number of
-grids, than a 0.0 is returned by the fix instead of an error, since
-the number of grids can vary as a simulation runs depending on how
-that value is computed by the compute grid/atom command.
 
-The array values calculated by this fix are treated as "intensive",
+For access by other commands, the name of the single grid produced by
-since they are typically already normalized by the count of atoms in
+this fix is "grid".  The names of its two per-grid datums are "data"
-each grid.
+for the per-grid array and "count" for the per-grid vector.  Both
+datums can be accessed by various :doc:`output commands
+<Howto_output>`.
+
+In per-atom mode, the per-grid array values calculated by this fix are
+treated as "intensive", since they are typically already normalized by
+the count of atoms in each grid cell.
 
 No parameter of this fix can be used with the *start/stop* keywords of
 the :doc:`run <run>` command.  This fix is not invoked during

doc/src/fix_ttm.rst

@@ -386,13 +386,13 @@ electronic subsystem energies reported at the end of the timestep.
 
 The vector values calculated are "extensive".
 
-Th fix ttm/grid command also calculates a per-grid vector which store
+The fix ttm/grid command also calculates a per-grid vector which
-the electron temperature for each grid cell in temperature :doc:`units
+stores the electron temperature for each grid cell in temperature
-<units>`. which can be accessed by various :doc:`output commands
+:doc:`units <units>`. which can be accessed by various :doc:`output
-<Howto_output>`.  The length of the vector (distributed across all
+commands <Howto_output>`.  The length of the vector (distributed
-processors) is Nx * Ny * Nz.  For access by other commands, the name
+across all processors) is Nx * Ny * Nz.  For access by other commands,
-of the grid produced by fix ttm/grid is "grid".  The name of its data
+the name of the single grid produced by fix ttm/grid is "grid".  The
-is "data".
+name of its per-grid data is "data".
 
 No parameter of the fixes can be used with the *start/stop* keywords
 of the :doc:`run <run>` command.  The fixes are not invoked during

src/EXTRA-FIX/fix_ttm_grid.cpp

@@ -496,9 +496,6 @@ void FixTTMGrid::reset_grid()
     return;
   } else delete gridnew;
 
-  // DEBUG
-  if (comm->me == 0) printf("Remapping grid on step %ld\n",update->ntimestep);
-  
   // delete grid data which doesn't need to persist from previous to new decomp
 
   memory->destroy(grid_buf1);
@@ -601,7 +598,7 @@ void FixTTMGrid::unpack_reverse_grid(int /*flag*/, void *vbuf, int nlist, int *l
 }
 
 /* ----------------------------------------------------------------------
-   pack old grid  values to buf to send to another proc
+   pack old grid values to buf to send to another proc
 ------------------------------------------------------------------------- */
 
 void FixTTMGrid::pack_remap_grid(void *vbuf, int nlist, int *list)

src/fix_ave_grid.cpp

@@ -29,6 +29,9 @@
 #include "update.h"
 #include "variable.h"
 
+// DEBUG
+#include "comm.h"
+
 #include <cstring>
 
 using namespace LAMMPS_NS;
@@ -390,7 +393,6 @@ FixAveGrid::FixAveGrid(LAMMPS *lmp, int narg, char **arg) :
 
   // instantiate the Grid class and allocate per-grid memory
 
-  double maxdist;
   if (modeatom) maxdist = 0.5 * neighbor->skin;
   else if (modegrid) maxdist = 0.0;
 
@@ -1889,17 +1891,122 @@ void FixAveGrid::unpack_reverse_grid(int /*flag*/, void *vbuf, int nlist, int *l
   }
 }
 
+/* ----------------------------------------------------------------------
+   pack old grid values to buf to send to another proc
+------------------------------------------------------------------------- */
+
+void FixAveGrid::pack_remap_grid(void *vbuf, int nlist, int *list)
+{
+  auto buf = (double *) vbuf;
+  double *src;
+  //double *src =
+  //  &T_electron_previous[nzlo_out_previous][nylo_out_previous][nxlo_out_previous];
+
+  for (int i = 0; i < nlist; i++) buf[i] = src[list[i]];
+}
+
+/* ----------------------------------------------------------------------
+   unpack another proc's own values from buf and set own ghost values
+------------------------------------------------------------------------- */
+
+void FixAveGrid::unpack_remap_grid(void *vbuf, int nlist, int *list)
+{
+  auto buf = (double *) vbuf;
+  double *dest;
+  //double *dest = &T_electron[nzlo_out][nylo_out][nxlo_out];
+
+  for (int i = 0; i < nlist; i++) dest[list[i]] = buf[i];
+}
+
 /* ----------------------------------------------------------------------
    subset of grid assigned to each proc may have changed
    called by load balancer when proc subdomains are adjusted
-   not supported for now, b/c requires per-grid values to persist, i.e. a remap()
+   persist per-grid data by performing a grid remap
 ------------------------------------------------------------------------- */
 
 void FixAveGrid::reset_grid()
 {
-  error->all(FLERR,"Fix ave/grid does not support load balancing (yet)");
+  // check if new grid partitioning is different on any proc
-}
+  // if not, just return
 
+  if (dimension == 2) {
+    int tmp[8];
+    Grid2d *gridnew = new Grid2d(lmp, world, nxgrid, nygrid);
+    gridnew->set_distance(maxdist);
+    gridnew->setup_grid(tmp[0], tmp[1], tmp[2], tmp[3],
+                        tmp[4], tmp[5], tmp[6], tmp[7]);
+
+    if (grid2d->identical(gridnew)) {
+      delete gridnew;
+      return;
+    } else delete gridnew;
+
+  } else {
+
+    int tmp[12];
+    Grid3d *gridnew = new Grid3d(lmp, world, nxgrid, nygrid, nzgrid);
+    gridnew->set_distance(maxdist);
+    gridnew->setup_grid(tmp[0], tmp[1], tmp[2], tmp[3], tmp[4], tmp[5],
+                        tmp[6], tmp[7], tmp[8], tmp[9], tmp[10], tmp[11]);
+  
+    if (grid3d->identical(gridnew)) {
+      delete gridnew;
+      return;
+    } else delete gridnew;
+  }
+
+  // DEBUG
+  if (comm->me == 0) printf("Remapping grid on step %ld\n",update->ntimestep);
+
+  /*
+  // delete grid data which doesn't need to persist from previous to new decomp
+
+  memory->destroy(grid_buf1);
+  memory->destroy(grid_buf2);
+  memory->destroy3d_offset(T_electron_old, nzlo_out, nylo_out, nxlo_out);
+  memory->destroy3d_offset(net_energy_transfer, nzlo_out, nylo_out, nxlo_out);
+
+  // make copy of ptrs to grid data which does need to persist
+
+  grid_previous = grid;
+  T_electron_previous = T_electron;
+  nxlo_out_previous = nxlo_out;
+  nylo_out_previous = nylo_out;
+  nzlo_out_previous = nzlo_out;
+
+  // allocate new per-grid data for new decomposition
+
+  allocate_grid();
+
+  // perform remap from previous decomp to new decomp
+
+  int nremap_buf1,nremap_buf2;
+  grid->setup_remap(grid_previous,nremap_buf1,nremap_buf2);
+
+  double *remap_buf1,*remap_buf2;
+  memory->create(remap_buf1, nremap_buf1, "ave/grid:remap_buf1");
+  memory->create(remap_buf2, nremap_buf2, "ave/grid:remap_buf2");
+
+  grid->remap(Grid3d::FIX,this,1,sizeof(double),remap_buf1,remap_buf2,MPI_DOUBLE);
+
+  memory->destroy(remap_buf1);
+  memory->destroy(remap_buf2);
+
+  // delete grid data and grid for previous decomposition
+
+  memory->destroy3d_offset(T_electron_previous,
+                           nzlo_out_previous, nylo_out_previous,
+                           nxlo_out_previous);
+  delete grid_previous;
+
+  // zero new net_energy_transfer
+  // in case compute_vector accesses it on timestep 0
+
+  outflag = 0;
+  memset(&net_energy_transfer[nzlo_out][nylo_out][nxlo_out],0,
+         ngridout*sizeof(double));
+  */
+}
 
 /* ----------------------------------------------------------------------
    return index of grid associated with name

src/fix_ave_grid.h

@@ -35,6 +35,8 @@ class FixAveGrid : public Fix {
 
   void pack_reverse_grid(int, void *, int, int *) override;
   void unpack_reverse_grid(int, void *, int, int *) override;
+  void pack_remap_grid(void *, int, int *) override;
+  void unpack_remap_grid(void *, int, int *) override;
 
   void reset_grid() override;
 
@@ -53,6 +55,7 @@ class FixAveGrid : public Fix {
   int modeatom, modegrid;
   int discardflag, normflag, aveflag, nwindow;
 
+  double maxdist;
   int running_count;
   int window_count,window_oldest,window_newest;