change platform::mkdir() to create entire path like "mkdir -p"

update the documentation of the shell command to current state
and specifically explain the difference between built-in and
external commands and correct where built in commands were described
as calling external commands.

doc/src/shell.rst

@@ -20,7 +20,7 @@ Syntax
          dir1,dir2 = one or more directories to create
        *mv* args = old new
          old = old filename
-         new = new filename
+         new = new filename or destination folder
        *rm* args = file1 file2 ...
          file1,file2 = one or more filenames to delete
        *rmdir* args = dir1 dir2 ...
@@ -36,8 +36,8 @@ Examples
 
    shell cd sub1
    shell cd ..
-   shell mkdir tmp1 tmp2 tmp3
+   shell mkdir tmp1 tmp2/tmp3
-   shell rmdir tmp1
+   shell rmdir tmp1 tmp2
    shell mv log.lammps hold/log.1
    shell rm TMP/file1 TMP/file2
    shell putenv LAMMPS_POTENTIALS=../../potentials
@@ -50,34 +50,39 @@ Description
 Execute a shell command.  A few simple file-based shell commands are
 supported directly, in Unix-style syntax.  Any command not listed
 above is passed as-is to the C-library system() call, which invokes
-the command in a shell.
+the command in a shell.  To use the external executable instead of
+the built-in version one needs to use a full path, for example
+*/bin/rm* instead of *rm*.  The built-in commands will also work
+on operating systems, that do not - by default - provide the
+corresponding external executables (like *mkdir* on Windows).
 
-This is means to invoke other commands from your input script.  For
+This command provides a ways to invoke custom commands or executables
-example, you can move files around in preparation for the next section
+from your input script.  For example, you can move files around in
-of the input script.  Or you can run a program that pre-processes data
+preparation for the next section of the input script.  Or you can run a
-for input into LAMMPS.  Or you can run a program that post-processes
+program that pre-processes data for input into LAMMPS.  Or you can run a
-LAMMPS output data.
+program that post-processes LAMMPS output data.
 
 With the exception of *cd*, all commands, including ones invoked via a
 system() call, are executed by only a single processor, so that
-files/directories are not being manipulated by multiple processors.
+files/directories are not being manipulated by multiple processors
+concurrently which may result in unexpected errors or corrupted files.
 
-The *cd* command executes the Unix "cd" command to change the working
+The *cd* command changes the current working directory similar to
-directory.  All subsequent LAMMPS commands that read/write files will
+the ``cd`` command.  All subsequent LAMMPS commands that read/write files
-use the new directory.  All processors execute this command.
+will use the new directory.  All processors execute this command.
 
-The *mkdir* command executes the Unix "mkdir" command to create one or
+The *mkdir* command creates directories similar to the Unix ``mkdir -p``
-more directories.
+command.  That is, it will attempt to create the entire path of
+sub-directories if they do not exist yet.
 
-The *mv* command executes the Unix "mv" command to rename a file and/or
+The *mv* command renames a file and/or moves it to a new directory.
-move it to a new directory.
+It cannot rename files across filesystem boundaries or between drives.
 
-The *rm* command executes the Unix "rm" command to remove one or more
+The *rm* command deletes file similar to the Unix ``rm`` command.
-files.
 
-The *rmdir* command executes the Unix "rmdir" command to remove one or
+The *rmdir* command deletes directories similar to Unix ``rmdir`` command.
-more directories.  A directory must be empty to be successfully
+If a directory is not empty, its contents are also removed recursively
-removed.
+similar to the Unix ``rm -r`` command.
 
 The *putenv* command defines or updates an environment variable directly.
 Since this command does not pass through the shell, no shell variable
@@ -107,9 +112,9 @@ with 3 arguments: file1 10 file2.
 Restrictions
 """"""""""""
 
-LAMMPS does not detect errors or print warnings when any of these
+LAMMPS will do a best effort to detect errors and print suitable
-commands execute.  E.g. if the specified directory does not exist,
+warnings, but due to the nature of delegating commands to the C-library
-executing the *cd* command will silently do nothing.
+system() call, this is not always reliable.
 
 Related commands
 """"""""""""""""

src/input.cpp

@@ -1194,7 +1194,10 @@ void Input::shell()
     if (narg < 2) error->all(FLERR,"Illegal shell mkdir command");
     if (me == 0) {
       for (int i = 1; i < narg; i++) {
-        if (platform::mkdir(arg[i]) < 0)
+        rv = (platform::mkdir(arg[i]) < 0) ? errno : 0;
+        MPI_Reduce(&rv,&err,1,MPI_INT,MPI_MAX,0,world);
+        errno = err;
+        if (err != 0)
           error->warning(FLERR, "Shell command 'mkdir {}' failed with error '{}'",
                          arg[i],utils::getsyserror());
       }
@@ -1202,9 +1205,14 @@ void Input::shell()
   } else if (strcmp(arg[0],"mv") == 0) {
     if (narg != 3) error->all(FLERR,"Illegal shell mv command");
     if (me == 0) {
-      if (rename(arg[1],arg[2]) < 0) {
+      if (platform::path_is_directory(arg[2])) {
-        error->warning(FLERR, "Shell command 'mv {} {}' failed with error '{}'",
+        if (system(fmt::format("mv {} {}", arg[1], arg[2]).c_str()))
-                       arg[1],arg[2],utils::getsyserror());
+          error->warning(FLERR,"Shell command 'mv {} {}' returned with non-zero status", arg[1], arg[2]);
+      } else {
+        if (rename(arg[1],arg[2]) < 0) {
+          error->warning(FLERR, "Shell command 'mv {} {}' failed with error '{}'",
+                         arg[1],arg[2],utils::getsyserror());
+        }
       }
     }
   } else if (strcmp(arg[0],"rm") == 0) {
@@ -1237,23 +1245,20 @@ void Input::shell()
         error->warning(FLERR, "Shell command 'putenv {}' failed with error '{}'",
                        arg[i], utils::getsyserror());
     }
-  // use work string to concat args back into one string separated by spaces
+
-  // invoke string in shell via system()
+  // concat arguments and invoke string in shell via system()
 
   } else {
-    int n = 0;
+    if (me == 0) {
-    for (int i = 0; i < narg; i++) n += strlen(arg[i]) + 1;
+      std::string cmd = arg[0];
-    if (n > maxwork) reallocate(work,maxwork,n);
+      for (int i = 1; i < narg; i++) {
+        cmd += " ";
+        cmd += arg[i];
+      }
 
-    strcpy(work,arg[0]);
+      if (system(cmd.c_str()) != 0)
-    for (int i = 1; i < narg; i++) {
+        error->warning(FLERR,"Shell command {} returned with non-zero status", cmd);
-      strcat(work," ");
-      strcat(work,arg[i]);
     }
-
-    if (me == 0)
-      if (system(work) != 0)
-        error->warning(FLERR,"Shell command returned with non-zero status");
   }
 }
 

src/platform.cpp

@@ -21,6 +21,7 @@
 
 #include <mpi.h>
 #include <exception>
+#include <deque>
 
 ////////////////////////////////////////////////////////////////////////
 // include system headers and tweak system settings
@@ -752,16 +753,31 @@ int platform::chdir(const std::string &path)
 }
 
 /* ----------------------------------------------------------------------
-   Create a directory
+   Create a directory. Create entire path if necessary.
 ------------------------------------------------------------------------- */
 
 int platform::mkdir(const std::string &path)
 {
+  std::deque<std::string> dirlist = { path };
+  std::string dirname = path_dirname(path);
+
+  while ((dirname != ".") && (dirname != "")) {
+    dirlist.push_front(dirname);
+    dirname = path_dirname(dirname);
+  }
+
+  int rv;
+  for (const auto &dir : dirlist) {
+    if (!path_is_directory(dir)) {
 #if defined(_WIN32)
-  return ::_mkdir(path.c_str());
+      rv = ::_mkdir(dir.c_str());
 #else
-  return ::mkdir(path.c_str(), S_IRWXU | S_IRGRP | S_IXGRP);
+      rv = ::mkdir(dir.c_str(), S_IRWXU | S_IRGRP | S_IXGRP);
 #endif
+      if (rv != 0) return rv;
+    }
+  }
+  return 0;
 }
 
 /* ----------------------------------------------------------------------

src/platform.h

@@ -271,6 +271,11 @@ namespace platform {
   int chdir(const std::string &path);
 
   /*! Create a directory
+   *
+   * Unlike the the ``mkdir()`` or ``_mkdir()`` functions of the
+   * C library, this function will also try to create non-existing sub-directories
+   * in case they don't exist, and thus behave like the ``mkdir -p`` command rather
+   * than plain ``mkdir`` or ``md`.
    *
    * \param  path  directory path
    * \return -1 if unsuccessful, otherwise >= 0  */
@@ -279,7 +284,7 @@ namespace platform {
 
   /*! Delete a directory
    *
-   * Unlike the the ``rmdir()`` or ``_rmdir()`` function of the
+   * Unlike the the ``rmdir()`` or ``_rmdir()`` functions of the
    * C library, this function will check for the contents of the
    * folder and recurse into any sub-folders, if necessary and
    * delete all contained folders and their contents before

unittest/utils/test_platform.cpp

@@ -323,6 +323,15 @@ TEST(Platform, path_and_directory)
     ASSERT_EQ(dirs.size(), 3);
     platform::rmdir("path_is_directory");
     ASSERT_FALSE(platform::path_is_directory("path_is_directory"));
+
+#if defined(_WIN32)
+    ASSERT_EQ(platform::mkdir("path_is_directory\\path_is_directory"),0);
+    ASSERT_TRUE(platform::path_is_directory("path_is_directory\\path_is_directory"));
+#else
+    ASSERT_EQ(platform::mkdir("path_is_directory/path_is_directory"),0);
+    ASSERT_TRUE(platform::path_is_directory("path_is_directory/path_is_directory"));
+#endif
+    platform::rmdir("path_is_directory");
 }
 
 TEST(Platform, get_change_directory)