diff --git a/doc/src/Build_development.rst b/doc/src/Build_development.rst index c674b2c258..4d8bf0d07f 100644 --- a/doc/src/Build_development.rst +++ b/doc/src/Build_development.rst @@ -122,32 +122,39 @@ Code Coverage and Unit Testing (CMake only) ------------------------------------------- The LAMMPS code is subject to multiple levels of automated testing -during development: integration testing (i.e. whether the code compiles -on various platforms and with a variety of settings), unit testing -(i.e. whether certain individual parts of the code produce the expected -results for given inputs), run testing (whether selected complete input -decks run without crashing for multiple configurations), and regression -testing (i.e. whether selected input examples reproduce the same -results over a given number of steps and operations within a given -error margin). The status of this automated testing can be viewed on -`https://ci.lammps.org `_. +during development: + +- Integration testing (i.e. whether the code compiles + on various platforms and with a variety of compilers and settings), +- Unit testing (i.e. whether certain functions or classes of the code + produce the expected results for given inputs), +- Run testing (i.e. whether selected input decks can run to completion + without crashing for multiple configurations), +- Regression testing (i.e. whether selected input examples reproduce the + same results over a given number of steps and operations within a + given error margin). + +The status of this automated testing can be viewed on `https://ci.lammps.org +`_. The scripts and inputs for integration, run, and regression testing are maintained in a `separate repository `_ -of the LAMMPS project on GitHub. +of the LAMMPS project on GitHub. A few tests are also run as GitHub +Actions and their configuration files are in the ``.github/workflows/`` +folder of the LAMMPS git tree. -The unit testing facility is integrated into the CMake build process -of the LAMMPS source code distribution itself. It can be enabled by +The unit testing facility is integrated into the CMake build process of +the LAMMPS source code distribution itself. It can be enabled by setting ``-D ENABLE_TESTING=on`` during the CMake configuration step. -It requires the `YAML `_ library and development -headers (if those are not found locally a recent version will be -downloaded and compiled along with LAMMPS and the test program) to -compile and will download and compile a specific recent version of the -`Googletest `_ C++ test framework -for implementing the tests. +It requires the `YAML `_ library and matching +development headers to compile (if those are not found locally a recent +version of that library will be downloaded and compiled along with +LAMMPS and the test programs) and will download and compile a specific +version of the `GoogleTest `_ C++ +test framework that is used to implement the tests. -.. admonition:: Software version requirements for testing +.. admonition:: Software version and LAMMPS configuration requirements :class: note The compiler and library version requirements for the testing @@ -155,7 +162,7 @@ for implementing the tests. example the default GNU C++ and Fortran compilers of RHEL/CentOS 7.x (version 4.8.x) are not sufficient. The CMake configuration will try to detect incompatible versions and either skip incompatible tests or - stop with an error. Also the number of tests will depend on + stop with an error. Also the number of available tests will depend on installed LAMMPS packages, development environment, operating system, and configuration settings. @@ -234,12 +241,31 @@ will be skipped if prerequisite features are not available in LAMMPS. time. Preference is given to parts of the code base that are easy to test or commonly used. -Tests for styles of the same kind of style (e.g. pair styles or bond -styles) are performed with the same test executable using different -input files in YAML format. So to add a test for another style of the -same kind it may be sufficient to add a suitable YAML file. -:doc:`Detailed instructions for adding tests ` are -provided in the Programmer Guide part of the manual. +Tests as shown by the ``ctest`` program are command lines defined in the +``CMakeLists.txt`` files in the ``unittest`` directory tree. A few +tests simply execute LAMMPS with specific command line flags and check +the output to the screen for expected content. A large number of unit +tests are special tests programs using the `GoogleTest framework +`_ and linked to the LAMMPS +library that test individual functions or create a LAMMPS class +instance, execute one or more commands and check data inside the LAMMPS +class hierarchy. There are also tests for the C-library, Fortran, and +Python module interfaces to LAMMPS. The Python tests use the Python +"unittest" module in a similar fashion than the others use `GoogleTest`. +These special test programs are structured to perform multiple +individual tests internally and each of those contains several checks +(aka assertions) for internal data being changed as expected. + +Tests for force computing or modifying styles (e.g. styles for non-bonded +and bonded interactions and selected fixes) are run by using a more generic +test program that reads its input from files in YAML format. The YAML file +provides the information on how to customized the test program to test +a specific style and - if needed - with specific settings. +To add a test for another, similar style (e.g. a new pair style) it is +usually sufficient to add a suitable YAML file. :doc:`Detailed +instructions for adding tests ` are provided in the +Programmer Guide part of the manual. A description of what happens +during the tests is given below. Unit tests for force styles ^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/doc/utils/sphinx-config/false_positives.txt b/doc/utils/sphinx-config/false_positives.txt index c74e2a79f3..4f5fe6fdaf 100644 --- a/doc/utils/sphinx-config/false_positives.txt +++ b/doc/utils/sphinx-config/false_positives.txt @@ -3797,6 +3797,7 @@ unimodal uninstall unitarg unitless +unittest Universite unix unmaintained diff --git a/unittest/force-styles/test_angle_style.cpp b/unittest/force-styles/test_angle_style.cpp index 3476ae8dde..010fabd6e2 100644 --- a/unittest/force-styles/test_angle_style.cpp +++ b/unittest/force-styles/test_angle_style.cpp @@ -27,6 +27,7 @@ #include "atom.h" #include "compute.h" #include "exceptions.h" +#include "fix.h" #include "fmt/format.h" #include "force.h" #include "info.h" @@ -528,6 +529,59 @@ TEST(AngleStyle, omp) if (!verbose) ::testing::internal::GetCapturedStdout(); }; +TEST(AngleStyle, numdiff) +{ + if (!LAMMPS::is_installed_pkg("EXTRA-FIX")) GTEST_SKIP(); + if (test_config.skip_tests.count(test_info_->name())) GTEST_SKIP(); + + LAMMPS::argv args = {"AngleStyle", "-log", "none", "-echo", "screen", "-nocite"}; + + ::testing::internal::CaptureStdout(); + LAMMPS *lmp = init_lammps(args, test_config, true); + + std::string output = ::testing::internal::GetCapturedStdout(); + if (verbose) std::cout << output; + + if (!lmp) { + std::cerr << "One or more prerequisite styles are not available " + "in this LAMMPS configuration:\n"; + for (auto &prerequisite : test_config.prerequisites) { + std::cerr << prerequisite.first << "_style " << prerequisite.second << "\n"; + } + GTEST_SKIP(); + } + + EXPECT_THAT(output, StartsWith("LAMMPS (")); + EXPECT_THAT(output, HasSubstr("Loop time")); + + // abort if running in parallel and not all atoms are local + const int nlocal = lmp->atom->nlocal; + ASSERT_EQ(lmp->atom->natoms, nlocal); + + if (!verbose) ::testing::internal::CaptureStdout(); + lmp->input->one("fix diff all numdiff 2 6.05504e-6"); + lmp->input->one("run 2 post no"); + if (!verbose) ::testing::internal::GetCapturedStdout(); + Fix *ifix = lmp->modify->get_fix_by_id("diff"); + if (ifix) { + double epsilon = test_config.epsilon * 5.0e8; + ErrorStats stats; + double **f1 = lmp->atom->f; + double **f2 = ifix->array_atom; + SCOPED_TRACE("EXPECT FORCES: numdiff"); + for (int i = 0; i < nlocal; ++i) { + EXPECT_FP_LE_WITH_EPS(f1[i][0], f2[i][0], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][1], f2[i][1], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][2], f2[i][2], epsilon); + } + if (print_stats) + std::cerr << "numdiff stats: " << stats << " epsilon: " << epsilon << std::endl; + } + if (!verbose) ::testing::internal::CaptureStdout(); + cleanup_lammps(lmp, test_config); + if (!verbose) ::testing::internal::GetCapturedStdout(); +} + TEST(AngleStyle, single) { if (test_config.skip_tests.count(test_info_->name())) GTEST_SKIP(); diff --git a/unittest/force-styles/test_bond_style.cpp b/unittest/force-styles/test_bond_style.cpp index f7ecd835b0..185d28089e 100644 --- a/unittest/force-styles/test_bond_style.cpp +++ b/unittest/force-styles/test_bond_style.cpp @@ -27,6 +27,7 @@ #include "bond.h" #include "compute.h" #include "exceptions.h" +#include "fix.h" #include "fmt/format.h" #include "force.h" #include "info.h" @@ -530,6 +531,60 @@ TEST(BondStyle, omp) if (!verbose) ::testing::internal::GetCapturedStdout(); }; + +TEST(BondStyle, numdiff) +{ + if (!LAMMPS::is_installed_pkg("EXTRA-FIX")) GTEST_SKIP(); + if (test_config.skip_tests.count(test_info_->name())) GTEST_SKIP(); + + LAMMPS::argv args = {"BondStyle", "-log", "none", "-echo", "screen", "-nocite"}; + + ::testing::internal::CaptureStdout(); + LAMMPS *lmp = init_lammps(args, test_config, true); + + std::string output = ::testing::internal::GetCapturedStdout(); + if (verbose) std::cout << output; + + if (!lmp) { + std::cerr << "One or more prerequisite styles are not available " + "in this LAMMPS configuration:\n"; + for (auto &prerequisite : test_config.prerequisites) { + std::cerr << prerequisite.first << "_style " << prerequisite.second << "\n"; + } + GTEST_SKIP(); + } + + EXPECT_THAT(output, StartsWith("LAMMPS (")); + EXPECT_THAT(output, HasSubstr("Loop time")); + + // abort if running in parallel and not all atoms are local + const int nlocal = lmp->atom->nlocal; + ASSERT_EQ(lmp->atom->natoms, nlocal); + + if (!verbose) ::testing::internal::CaptureStdout(); + lmp->input->one("fix diff all numdiff 2 6.05504e-6"); + lmp->input->one("run 2 post no"); + if (!verbose) ::testing::internal::GetCapturedStdout(); + Fix *ifix = lmp->modify->get_fix_by_id("diff"); + if (ifix) { + double epsilon = test_config.epsilon * 5.0e8; + ErrorStats stats; + double **f1 = lmp->atom->f; + double **f2 = ifix->array_atom; + SCOPED_TRACE("EXPECT FORCES: numdiff"); + for (int i = 0; i < nlocal; ++i) { + EXPECT_FP_LE_WITH_EPS(f1[i][0], f2[i][0], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][1], f2[i][1], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][2], f2[i][2], epsilon); + } + if (print_stats) + std::cerr << "numdiff stats: " << stats << " epsilon: " << epsilon << std::endl; + } + if (!verbose) ::testing::internal::CaptureStdout(); + cleanup_lammps(lmp, test_config); + if (!verbose) ::testing::internal::GetCapturedStdout(); +} + TEST(BondStyle, single) { if (test_config.skip_tests.count(test_info_->name())) GTEST_SKIP(); diff --git a/unittest/force-styles/test_dihedral_style.cpp b/unittest/force-styles/test_dihedral_style.cpp index 662d63909d..efc37b9e03 100644 --- a/unittest/force-styles/test_dihedral_style.cpp +++ b/unittest/force-styles/test_dihedral_style.cpp @@ -27,6 +27,7 @@ #include "compute.h" #include "dihedral.h" #include "exceptions.h" +#include "fix.h" #include "fmt/format.h" #include "force.h" #include "info.h" @@ -531,3 +532,57 @@ TEST(DihedralStyle, omp) cleanup_lammps(lmp, test_config); if (!verbose) ::testing::internal::GetCapturedStdout(); }; + + +TEST(DihedralStyle, numdiff) +{ + if (!LAMMPS::is_installed_pkg("EXTRA-FIX")) GTEST_SKIP(); + if (test_config.skip_tests.count(test_info_->name())) GTEST_SKIP(); + + LAMMPS::argv args = {"DihedralStyle", "-log", "none", "-echo", "screen", "-nocite"}; + + ::testing::internal::CaptureStdout(); + LAMMPS *lmp = init_lammps(args, test_config, true); + + std::string output = ::testing::internal::GetCapturedStdout(); + if (verbose) std::cout << output; + + if (!lmp) { + std::cerr << "One or more prerequisite styles are not available " + "in this LAMMPS configuration:\n"; + for (auto &prerequisite : test_config.prerequisites) { + std::cerr << prerequisite.first << "_style " << prerequisite.second << "\n"; + } + GTEST_SKIP(); + } + + EXPECT_THAT(output, StartsWith("LAMMPS (")); + EXPECT_THAT(output, HasSubstr("Loop time")); + + // abort if running in parallel and not all atoms are local + const int nlocal = lmp->atom->nlocal; + ASSERT_EQ(lmp->atom->natoms, nlocal); + + if (!verbose) ::testing::internal::CaptureStdout(); + lmp->input->one("fix diff all numdiff 2 6.05504e-6"); + lmp->input->one("run 2 post no"); + if (!verbose) ::testing::internal::GetCapturedStdout(); + Fix *ifix = lmp->modify->get_fix_by_id("diff"); + if (ifix) { + double epsilon = test_config.epsilon * 5.0e8; + ErrorStats stats; + double **f1 = lmp->atom->f; + double **f2 = ifix->array_atom; + SCOPED_TRACE("EXPECT FORCES: numdiff"); + for (int i = 0; i < nlocal; ++i) { + EXPECT_FP_LE_WITH_EPS(f1[i][0], f2[i][0], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][1], f2[i][1], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][2], f2[i][2], epsilon); + } + if (print_stats) + std::cerr << "numdiff stats: " << stats << " epsilon: " << epsilon << std::endl; + } + if (!verbose) ::testing::internal::CaptureStdout(); + cleanup_lammps(lmp, test_config); + if (!verbose) ::testing::internal::GetCapturedStdout(); +} diff --git a/unittest/force-styles/test_improper_style.cpp b/unittest/force-styles/test_improper_style.cpp index dc1b846b5a..ba3618d3dc 100644 --- a/unittest/force-styles/test_improper_style.cpp +++ b/unittest/force-styles/test_improper_style.cpp @@ -26,6 +26,7 @@ #include "atom.h" #include "compute.h" #include "exceptions.h" +#include "fix.h" #include "fmt/format.h" #include "force.h" #include "improper.h" @@ -524,3 +525,56 @@ TEST(ImproperStyle, omp) cleanup_lammps(lmp, test_config); if (!verbose) ::testing::internal::GetCapturedStdout(); }; + +TEST(ImproperStyle, numdiff) +{ + if (!LAMMPS::is_installed_pkg("EXTRA-FIX")) GTEST_SKIP(); + if (test_config.skip_tests.count(test_info_->name())) GTEST_SKIP(); + + LAMMPS::argv args = {"ImproperStyle", "-log", "none", "-echo", "screen", "-nocite"}; + + ::testing::internal::CaptureStdout(); + LAMMPS *lmp = init_lammps(args, test_config, true); + + std::string output = ::testing::internal::GetCapturedStdout(); + if (verbose) std::cout << output; + + if (!lmp) { + std::cerr << "One or more prerequisite styles are not available " + "in this LAMMPS configuration:\n"; + for (auto &prerequisite : test_config.prerequisites) { + std::cerr << prerequisite.first << "_style " << prerequisite.second << "\n"; + } + GTEST_SKIP(); + } + + EXPECT_THAT(output, StartsWith("LAMMPS (")); + EXPECT_THAT(output, HasSubstr("Loop time")); + + // abort if running in parallel and not all atoms are local + const int nlocal = lmp->atom->nlocal; + ASSERT_EQ(lmp->atom->natoms, nlocal); + + if (!verbose) ::testing::internal::CaptureStdout(); + lmp->input->one("fix diff all numdiff 2 6.05504e-6"); + lmp->input->one("run 2 post no"); + if (!verbose) ::testing::internal::GetCapturedStdout(); + Fix *ifix = lmp->modify->get_fix_by_id("diff"); + if (ifix) { + double epsilon = test_config.epsilon * 5.0e8; + ErrorStats stats; + double **f1 = lmp->atom->f; + double **f2 = ifix->array_atom; + SCOPED_TRACE("EXPECT FORCES: numdiff"); + for (int i = 0; i < nlocal; ++i) { + EXPECT_FP_LE_WITH_EPS(f1[i][0], f2[i][0], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][1], f2[i][1], epsilon); + EXPECT_FP_LE_WITH_EPS(f1[i][2], f2[i][2], epsilon); + } + if (print_stats) + std::cerr << "numdiff stats: " << stats << " epsilon: " << epsilon << std::endl; + } + if (!verbose) ::testing::internal::CaptureStdout(); + cleanup_lammps(lmp, test_config); + if (!verbose) ::testing::internal::GetCapturedStdout(); +} diff --git a/unittest/force-styles/test_main.cpp b/unittest/force-styles/test_main.cpp index 80f1ca4e30..1cf428eee4 100644 --- a/unittest/force-styles/test_main.cpp +++ b/unittest/force-styles/test_main.cpp @@ -47,7 +47,7 @@ void EXPECT_STRESS(const std::string &name, double *stress, const stress_t &expe EXPECT_FP_LE_WITH_EPS(stress[3], expected_stress.xy, epsilon); EXPECT_FP_LE_WITH_EPS(stress[4], expected_stress.xz, epsilon); EXPECT_FP_LE_WITH_EPS(stress[5], expected_stress.yz, epsilon); - if (print_stats) std::cerr << name << " stats" << stats << std::endl; + if (print_stats) std::cerr << name << " stats: " << stats << std::endl; } void EXPECT_FORCES(const std::string &name, Atom *atom, const std::vector &f_ref, @@ -64,7 +64,7 @@ void EXPECT_FORCES(const std::string &name, Atom *atom, const std::vector &x_ref, @@ -81,7 +81,7 @@ void EXPECT_POSITIONS(const std::string &name, Atom *atom, const std::vector &v_ref, @@ -98,7 +98,7 @@ void EXPECT_VELOCITIES(const std::string &name, Atom *atom, const std::vector