more documentation for adding unittest
This commit is contained in:
Axel Kohlmeyer
committed by
Richard Berger
Richard Berger
parent
2a8cc331e7
commit
bbe50ab5c1
@ -21,16 +21,202 @@ tests can be added by simply adding more of those input files. Those
|
||||
tests should be seen more as a hybrid between unit and regression tests.
|
||||
|
||||
When adding tests it is recommended to also :ref:`enable support for
|
||||
code coverage reporting <testing>`, so that it is possible to monitor
|
||||
how large a fraction of the code of a given file is executed during
|
||||
the tests.
|
||||
code coverage reporting <testing>`, and study the coverage reports
|
||||
so that it is possible to monitor which parts of the code of a given
|
||||
file are executed during the tests and which tests would need to be
|
||||
added to increase the coverage.
|
||||
|
||||
The tests are grouped into categories and corresponding folders.
|
||||
The following sections describe how the tests are implemented and
|
||||
executed in those categories with increasing complexity of tests
|
||||
and implementation.
|
||||
|
||||
|
||||
Adding tests for styles computing forces
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Tests for utility functions
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
These tests are driven by programs in the ``unittest/utils`` folder
|
||||
and most closely resemble conventional unit tests. There is one test
|
||||
program for each namespace or group of classes or file. The naming
|
||||
convention for the sources and executables is that they start with
|
||||
with ``test_``. The following sources and groups of tests are currently
|
||||
available:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: auto
|
||||
:align: left
|
||||
|
||||
* - File name:
|
||||
- Test name:
|
||||
- Description:
|
||||
* - ``test_fmtlib.cpp``
|
||||
- FmtLib
|
||||
- Tests for ``fmtlib::`` functions used by LAMMPS
|
||||
* - ``test_math_eigen_impl.cpp``
|
||||
- MathEigen
|
||||
- Tests for ``MathEigen::`` classes and functions
|
||||
* - ``test_mempool.cpp``
|
||||
- MemPool
|
||||
- Tests for :cpp:class:`MyPage <LAMMPS_NS::MyPage>` and :cpp:class:`MyPoolChunk <LAMMPS_NS::MyPoolChunk>`
|
||||
* - ``test_tokenizer.cpp``
|
||||
- Tokenizer
|
||||
- Tests for :cpp:class:`Tokenizer <LAMMPS_NS::Tokenizer>` and :cpp:class:`ValueTokenizer <LAMMPS_NS::ValueTokenizer>`
|
||||
* - ``test_utils.cpp``
|
||||
- Utils
|
||||
- Tests for ``utils::`` :doc:`functions <Developer_utils>`
|
||||
|
||||
To add tests either an existing source file needs to be modified or a
|
||||
new source file needs to be added to the distribution and enabled for
|
||||
testing. To add a new file suitable CMake script code needs to be added
|
||||
to the ``CMakeLists.txt`` file in the ``unittest/utils`` folder. Example:
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
add_executable(test_tokenizer test_tokenizer.cpp)
|
||||
target_link_libraries(test_tokenizer PRIVATE lammps GTest::GMockMain GTest::GMock GTest::GTest)
|
||||
add_test(Tokenizer test_tokenizer)
|
||||
|
||||
This adds instructions to build the ``test_tokenizer`` executable from
|
||||
``test_tokenizer.cpp`` and links it with the googletest libraries and the
|
||||
LAMMPS library as well as it uses the ``main()`` function from the
|
||||
GMock library of googletest. The third line registers the executable
|
||||
as a test program to be run from ``ctest`` under the name ``Tokenizer``.
|
||||
|
||||
The test executable itself will execute multiple individual tests
|
||||
through the googletest framework. In this case each test consists of
|
||||
creating a tokenizer class instance with a given string and explicit or
|
||||
default separator choice, and then executing member functions of the
|
||||
class and comparing their results with expected values. A few examples:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
TEST(Tokenizer, empty_string)
|
||||
{
|
||||
Tokenizer t("", " ");
|
||||
ASSERT_EQ(t.count(), 0);
|
||||
}
|
||||
|
||||
TEST(Tokenizer, two_words)
|
||||
{
|
||||
Tokenizer t("test word", " ");
|
||||
ASSERT_EQ(t.count(), 2);
|
||||
}
|
||||
|
||||
TEST(Tokenizer, default_separators)
|
||||
{
|
||||
Tokenizer t(" \r\n test \t word \f");
|
||||
ASSERT_THAT(t.next(), Eq("test"));
|
||||
ASSERT_THAT(t.next(), Eq("word"));
|
||||
ASSERT_EQ(t.count(), 2);
|
||||
}
|
||||
|
||||
Each of these TEST functions will become an individual
|
||||
test run by the test program. When using the ``ctest``
|
||||
command as a front end to run the tests, their output
|
||||
will be suppressed and only a summary printed, but adding
|
||||
the '-V' option will then produce output from the tests
|
||||
above like the following:
|
||||
|
||||
.. code-block::
|
||||
|
||||
[...]
|
||||
1: [ RUN ] Tokenizer.empty_string
|
||||
1: [ OK ] Tokenizer.empty_string (0 ms)
|
||||
1: [ RUN ] Tokenizer.two_words
|
||||
1: [ OK ] Tokenizer.two_words (0 ms)
|
||||
1: [ RUN ] Tokenizer.default_separators
|
||||
1: [ OK ] Tokenizer.default_separators (0 ms)
|
||||
[...]
|
||||
|
||||
The MathEigen test collection has been adapted from a standalone test
|
||||
and does not use the googletest framework and thus not representative.
|
||||
The other test sources, however, can serve as guiding examples for
|
||||
additional tests.
|
||||
|
||||
Tests for individual LAMMPS commands
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
These tests are a bit more complex as they require to first create a
|
||||
:cpp:class:`LAMMPS <LAMMPS_NS::LAMMPS>` class instance and then use the
|
||||
:doc:`C++ API <Cplusplus>` to pass individual commands to that LAMMPS
|
||||
instance. For that reason these tests use a googletest "test fixture",
|
||||
i.e. a class derived from ``testing::Test`` that will create (and
|
||||
delete) the required LAMMPS class instance for each set of tests in a
|
||||
TEST_F() function. Please see the individual source files for different
|
||||
examples of setting up suitable test fixtures. Here is an example for
|
||||
implementing a test using a fixture by first checking the default
|
||||
value and then issuing LAMMPS commands and checking whether they
|
||||
have the desired effect:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
TEST_F(SimpleCommandsTest, ResetTimestep)
|
||||
{
|
||||
ASSERT_EQ(lmp->update->ntimestep, 0);
|
||||
|
||||
if (!verbose) ::testing::internal::CaptureStdout();
|
||||
lmp->input->one("reset_timestep 10");
|
||||
if (!verbose) ::testing::internal::GetCapturedStdout();
|
||||
ASSERT_EQ(lmp->update->ntimestep, 10);
|
||||
|
||||
if (!verbose) ::testing::internal::CaptureStdout();
|
||||
lmp->input->one("reset_timestep 0");
|
||||
if (!verbose) ::testing::internal::GetCapturedStdout();
|
||||
ASSERT_EQ(lmp->update->ntimestep, 0);
|
||||
}
|
||||
|
||||
Please note the use of the (global) verbose variable to control whether
|
||||
the LAMMPS command will be silent by capturing the output or not. In
|
||||
the default case, verbose == false, the test output will be compact and
|
||||
not mixed with LAMMPS output. However setting the verbose flag (via
|
||||
setting the TEST_ARGS environment variable, ``TEST_ARGS=-v``) can be
|
||||
helpful to understand why tests fail unexpectedly.
|
||||
|
||||
Another complexity of these tests stems from the need to capture
|
||||
situations where LAMMPS will stop with an error, i.e. handle so-called
|
||||
"death tests". Here the LAMMPS code will operate differently depending
|
||||
on whether it was configured to throw C++ exceptions on errors or call
|
||||
either ``exit()`` or ``MPI_Abort()``. In the latter case, the test code
|
||||
also needs to detect whether LAMMPS was compiled with the OpenMPI
|
||||
library, as OpenMPI is **only** compatible the death test options of the
|
||||
googletest library when C++ exceptions are enabled; otherwise those
|
||||
"death tests" must be skipped to avoid reporting bogus failures. The
|
||||
specifics of this step are implemented in the TEST_FAILURE()
|
||||
macro. These tests operate by capturing the screen output when executing
|
||||
the failing command and then comparing that with a provided regular
|
||||
expression string pattern. Example:
|
||||
|
||||
.. code-block:: C++
|
||||
|
||||
TEST_F(SimpleCommandsTest, UnknownCommand)
|
||||
{
|
||||
TEST_FAILURE(".*ERROR: Unknown command.*", lmp->input->one("XXX one two"););
|
||||
}
|
||||
|
||||
The following test programs are currently available:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:widths: auto
|
||||
:align: left
|
||||
|
||||
* - File name:
|
||||
- Test name:
|
||||
- Description:
|
||||
* - ``test_simple_commands.cpp``
|
||||
- SimpleCommands
|
||||
- Tests for LAMMPS commands that do not require a box
|
||||
* - ``test_lattice_region.cpp``
|
||||
- LatticeRegion
|
||||
- Tests to validate the :doc:`lattice <lattice>` and :doc:`region <region>` commands
|
||||
* - ``test_kim_commands.cpp``
|
||||
- KimCommands
|
||||
- Tests for several commands from the :ref:`KIM package <PKG-KIM>`
|
||||
* - ``test_reset_ids.cpp``
|
||||
- ResetIDs
|
||||
- Tests to validate the :doc:`reset_atom_ids <reset_atom_ids>` and :doc:`reset_mol_ids <reset_mol_ids>` commands
|
||||
|
||||
Adding tests for individual LAMMPS commands
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Adding tests for the C-style library interface
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
@ -44,8 +230,45 @@ Adding tests for the Fortran interface
|
||||
Adding tests for the C++-style library interface
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Adding tests for utility functions
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Adding tests for styles computing or modifying forces
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
These are tests common configurations for pair styles, bond styles,
|
||||
angle styles, kspace styles and certain fix styles. Those are tests
|
||||
driven by some test executables build from sources in the
|
||||
``unittest/force-styles`` folder and use LAMMPS input template and data
|
||||
files as well as input files in YAML format from the
|
||||
``unittest/force-styles/tests`` folder. The YAML file names have to
|
||||
follow some naming conventions so they get associated with the test
|
||||
programs and categorized and listed with canonical names in the list
|
||||
of tests as displayed by ``ctest -N``. If you add a new YAML file,
|
||||
you need to re-run CMake to update the corresponding list of tests.
|
||||
|
||||
A minimal YAML file for a (molecular) pair style test will looks
|
||||
something like the following (see ``mol-pair-zero.yaml``):
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
lammps_version: 24 Aug 2020
|
||||
date_generated: Tue Sep 15 09:44:21 202
|
||||
epsilon: 1e-14
|
||||
prerequisites: ! |
|
||||
atom full
|
||||
pair zero
|
||||
pre_commands: ! ""
|
||||
post_commands: ! ""
|
||||
input_file: in.fourmol
|
||||
pair_style: zero 8.0
|
||||
pair_coeff: ! |
|
||||
* *
|
||||
extract: ! ""
|
||||
natoms: 29
|
||||
init_vdwl: 0
|
||||
init_coul: 0
|
||||
|
||||
[...]
|
||||
|
||||
|
||||
Adding tests for programs in the tools folder
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
@ -1119,11 +1119,13 @@ gmail
|
||||
gmake
|
||||
gmask
|
||||
Gmask
|
||||
GMock
|
||||
gneb
|
||||
GNEB
|
||||
Goldfarb
|
||||
Gonzalez-Melchor
|
||||
googlemail
|
||||
googletest
|
||||
Gordan
|
||||
GPa
|
||||
gpu
|
||||
|
||||
Reference in New Issue
Block a user