124 lines
5.1 KiB
Plaintext
124 lines
5.1 KiB
Plaintext
The script `run_tests.py` in this folder is used to perform regression tests
|
|
using in-place example scripts.
|
|
|
|
What this single script does is to launch the selected LAMMPS binary
|
|
using a testing configuration defined in a `.yaml` file (e.g., `config.yaml`)
|
|
for the set of input scripts inside the given `examples/` subfolders,
|
|
and compare the output thermo with that in the existing log file with the same number of procs.
|
|
If there are multiple log files with the same input script (e.g., `log.melt.*.g++.1` and `log.melt.*.g++.4`),
|
|
the one with the highest number of procs is chosen.
|
|
|
|
The output includes the number of passed and failed tests and
|
|
an `output.xml` file in the JUnit XML format for downstream reporting.
|
|
The output and error of any crashed runs are logged.
|
|
|
|
A test with an input script is considered passed when the given LAMMPS binary produces
|
|
thermo output quantities consistent with those in the reference log file
|
|
within the specified tolerances in the test configuration `config.yaml` file.
|
|
|
|
With the current features, users can:
|
|
|
|
+ specify which LAMMPS binary version to test (e.g., the version from a commit, or those from `lammps-testing`)
|
|
+ specify the examples subfolders (thus the reference log files) seperately (e.g. from other LAMMPS versions or commits)
|
|
+ specify tolerances for individual quantities for any input script to override the global values
|
|
+ launch tests with `mpirun` with all supported command line features (multiple procs, multiple paritions, and suffices)
|
|
+ skip certain input files if not interested, or no reference log file exists
|
|
+ simplify the main LAMMPS builds, as long as a LAMMPS binary is available
|
|
|
|
Limitations:
|
|
|
|
- input scripts use thermo style multi (e.g., examples/peptide) do not work with the expected thermo output format
|
|
- input scripts that require partition runs (e.g. examples/neb) need a separate config file, e.g. "args: --partition 2x1"
|
|
- testing accelerator packages (GPU, INTEL, KOKKOS, OPENMP) need separate config files, "args: -sf omp -pk omp 4"
|
|
|
|
TODO:
|
|
|
|
+ keep track of the testing progress to resume the testing from the last checkpoint
|
|
+ distribute the input list across multiple processes via multiprocessing, or
|
|
split the list of input scripts into separate runs (there are 800+ input script under the top-level examples)
|
|
+ be able to be invoked from run_tests in the lammps-testing infrastruture
|
|
|
|
|
|
The following Python packages need to be installed into an activated environment:
|
|
|
|
python3 -m venv testing-env
|
|
source testing-env/bin/activate
|
|
pip install numpy pyyaml junit_xml
|
|
|
|
|
|
Example uses:
|
|
|
|
1) Simple use (using the provided tools/regression-tests/config.yaml and the examples/ folder at the top level)
|
|
python3 run_tests.py --lmp-bin=/path/to/lmp_binary
|
|
|
|
2) Use a custom testing configuration
|
|
python3 run_tests.py --lmp-bin=/path/to/lmp_binary --config-file=/path/to/config/file/config.yaml
|
|
|
|
3) Specify a list of example folders
|
|
python3 run_tests.py --lmp-bin=/path/to/lmp_binary --config-file=/path/to/config/file/config.yaml \
|
|
--example-folders="/path/to/examples/folder1;/path/to/examples/folder2"
|
|
|
|
The example folders can also be loaded from a text file list_subfolders1.txt:
|
|
python3 run_tests.py --lmp-bin=/path/to/lmp_binary --config-file=/path/to/config/file/config.yaml \
|
|
--list-input=list_subfolders1.txt --output-file=output1.txt --progress-file=progress1.yaml \
|
|
--log-file=run1.log
|
|
|
|
4) Test a LAMMPS binary with the whole top-level /examples folder in a LAMMPS source tree
|
|
python3 run_tests.py --lmp-bin=/path/to/lmp_binary --examples-top-level=/path/to/lammps/examples
|
|
|
|
5) Analyze (dry run) the LAMMPS binary annd whole top-level /examples folder in a LAMMPS source tree
|
|
and generate separate input lists for 8 workers:
|
|
python3 run_tests.py --lmp-bin=/path/to/lmp_binary --examples-top-level=/path/to/lammps/examples \
|
|
--dry-run --num-workers=8
|
|
|
|
This is used for splitting the subfolders into separate input lists and launching different instances
|
|
of run_tests.py simultaneously.
|
|
|
|
An example of the test configuration `config.yaml` is given as below.
|
|
|
|
---
|
|
lmp_binary: ""
|
|
nprocs: "4"
|
|
args: "-cite none"
|
|
mpiexec: "mpirun"
|
|
mpiexec_numproc_flag: "-np"
|
|
tolerance:
|
|
PotEng:
|
|
abs: 1e-4
|
|
rel: 1e-7
|
|
TotEng:
|
|
abs: 1e-4
|
|
rel: 1e-7
|
|
Press:
|
|
abs: 1e-4
|
|
rel: 1e-7
|
|
Temp:
|
|
abs: 1e-4
|
|
rel: 1e-7
|
|
E_vdwl:
|
|
abs: 1e-3
|
|
rel: 1e-7
|
|
overrides:
|
|
in.rigid.tnr:
|
|
Temp:
|
|
abs: 1e-3
|
|
rel: 1e-5
|
|
Press:
|
|
abs: 1e-2
|
|
rel: 1e-4
|
|
skip:
|
|
[ in.rigid.poems3,
|
|
in.rigid.poems4,
|
|
in.rigid.poems5,
|
|
]
|
|
|
|
nugget: 1.0
|
|
epsilon: 1e-16
|
|
|
|
An example of the list of input scripts in a text file `list_subfolders1.txt`
|
|
|
|
/home/codes/lammps/examples/melt
|
|
/home/codes/lammps/examples/body
|
|
/home/codes/lammps/examples/PACKAGES/dielectric
|
|
/home/codes/lammps/examples/PACKAGES/tally
|