change proc count to color for MPI_Comm_split

This commit is contained in:
Steven J. Plimpton
2018-08-02 14:47:20 -06:00
parent b2c75cc0b9
commit b2d1332d46
3 changed files with 32 additions and 34 deletions

View File

@ -1352,24 +1352,27 @@ specified file is "none", then no log files are created. Using a
"log"_log.html command in the input script will override this setting. "log"_log.html command in the input script will override this setting.
Option -plog will override the name of the partition log files file.N. Option -plog will override the name of the partition log files file.N.
-mpi P :pre -mpi color :pre
If used, this must be the first command-line argument after the LAMMPS If used, this must be the first command-line argument after the LAMMPS
executable name. It is only used when running LAMMPS in client/server executable name. It is only used when LAMMPS is launched by an mpirun
mode with the "mpi/one" mode of messaging provided by the command which also launches another executable(s) (the other
"message"_message.html command and the CSlib library LAMMPS links with executable could be LAMMPS as well) at the same time. The color is an
from the lib/message directory. See the "message"_message.html integer value which should be different for each executable (another
command for more details application may set this value in a different way). LAMMPS and the
other executable(s) perform an MPI_Comm_split() with their own colors
In the mpi/one mode of messaging, both executables (the client and the to shrink the MPI_COMM_WORLD communication to be the subset of
server) are launched by one mpirun command. P should be specified as
the number of processors (MPI tasks) the first executable is running
on (could be the client or the server code).
This information is required so that both codes can shrink the
MPI_COMM_WORLD communicator they are part of to the subset of
processors they are actually running on. processors they are actually running on.
Currently, this is only used in LAMMPS to perform client/server
messaging with another application. LAMMPS can act as either a client
or server (or both).
Specifically, this refers to the "mpi/one" mode of messaging provided
by the "message"_message.html command and the CSlib library LAMMPS
links with from the lib/message directory. See the
"message"_message.html command for more details.
-nocite :pre -nocite :pre
Disable writing the log.cite file which is normally written to list Disable writing the log.cite file which is normally written to list

View File

@ -91,12 +91,11 @@ Mpi/one mode of messaging:
Launch LAMMPS twice in a single mpirun command: Launch LAMMPS twice in a single mpirun command:
mpirun -np 2 lmp_mpi -mpi 2 -in in.message.client -v mode mpione -log log.client : -np 4 lmp_mpi -mpi 2 -in in.message.server -v mode mpione -log log.server mpirun -np 2 lmp_mpi -mpi 0 -in in.message.client -v mode mpione -log log.client : -np 4 lmp_mpi -mpi 1 -in in.message.server -v mode mpione -log log.server
The two -np values determine how many procs the client and the server The two -np values determine how many procs the client and the server
run on. run on.
A LAMMPS executable run in this manner must use the -mpi P A LAMMPS executable run in this manner must use the -mpi color
command-line option as their first option, where P is the number of command-line option as their first option, where color is set to
processors the first code in the mpirun command (client or server) is one integer value for the 1st app, and another value for the 2nd app.
running on.

View File

@ -86,17 +86,17 @@ LAMMPS::LAMMPS(int narg, char **arg, MPI_Comm communicator)
// check if -mpi is first arg // check if -mpi is first arg
// if so, then 2 apps were launched with one mpirun command // if so, then 2 apps were launched with one mpirun command
// this means passed communicator (e.g. MPI_COMM_WORLD) is bigger than LAMMPS // this means passed communicator (e.g. MPI_COMM_WORLD) is bigger than LAMMPS
// e.g. for client/server coupling with another code // e.g. for client/server coupling with another code
// communicator needs to shrink to be just LAMMPS // in the future LAMMPS might leverage this in other ways
// syntax: -mpi P1 means P1 procs for one app, P-P1 procs for second app // universe communicator needs to shrink to be just LAMMPS
// LAMMPS could be either app, based on proc IDs // syntax: -mpi color
// color = integer for this app, different than other app(s)
// do the following: // do the following:
// perform an MPI_Comm_split() to create a new LAMMPS-only subcomm // perform an MPI_Comm_split() to create a new LAMMPS-only subcomm
// NOTE: assuming other app is doing the same thing, else will hang! // NOTE: this assumes other app(s) does same thing, else will hang!
// re-create universe with subcomm // re-create universe with subcomm
// store full two-app comm in cscomm // store full multi-app comm in cscomm
// cscomm is used by CSLIB package to exchange messages w/ other app // cscomm is used by CSLIB package to exchange messages w/ other app
// eventually should extend to N > 2 apps launched with one mpirun command
int iarg = 1; int iarg = 1;
if (narg-iarg >= 2 && (strcmp(arg[iarg],"-mpi") == 0 || if (narg-iarg >= 2 && (strcmp(arg[iarg],"-mpi") == 0 ||
@ -104,13 +104,9 @@ LAMMPS::LAMMPS(int narg, char **arg, MPI_Comm communicator)
int me,nprocs; int me,nprocs;
MPI_Comm_rank(communicator,&me); MPI_Comm_rank(communicator,&me);
MPI_Comm_size(communicator,&nprocs); MPI_Comm_size(communicator,&nprocs);
int p1 = atoi(arg[iarg+1]); int color = atoi(arg[iarg+1]);
if (p1 <= 0 || p1 >= nprocs)
error->universe_all(FLERR,"Invalid command-line argument");
int which = 0;
if (me >= p1) which = 1;
MPI_Comm subcomm; MPI_Comm subcomm;
MPI_Comm_split(communicator,which,me,&subcomm); MPI_Comm_split(communicator,color,me,&subcomm);
cscomm = communicator; cscomm = communicator;
communicator = subcomm; communicator = subcomm;
delete universe; delete universe;
@ -656,8 +652,8 @@ LAMMPS::~LAMMPS()
delete [] suffix; delete [] suffix;
delete [] suffix2; delete [] suffix2;
// free the MPI comm created by -mpi command-line arg // free the MPI comm created by -mpi command-line arg processed in constructor
// it was passed to universe as if origianl universe world // it was passed to universe as if original universe world
// may have been split later by partitions, universe will free the splits // may have been split later by partitions, universe will free the splits
// free a copy of uorig here, so check in universe destructor will still work // free a copy of uorig here, so check in universe destructor will still work