ICODE-ADC/lammps
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix MPI issue in PySCF wrapper

Browse Source
This commit is contained in:
Steve Plimpton
2023-02-15 12:14:38 -07:00
parent 4bf9024c17
commit 76c00fe2ed
5 changed files with 323 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/QUANTUM/NWChem/in.water.qmmm.plugin
View File

@ -59,5 +59,5 @@ thermo 1
variable p equal extract_setting(world_size)
mdi plugin nwchem_mdi mdi "-role ENGINE -name NWChem -method LINK" &
extra "template.water.nw water.dimer.nw log.water.pwdft.qmmm.plugin.$p" &
extra "template.water.nw water.nw log.water.pwdft.qmmm.plugin.$p" &
command "run 1"

2
examples/QUANTUM/NWChem/in.zeolite.qmmm.plugin
View File

@ -31,7 +31,7 @@ pair_coeff 1 2 buck 27029.419922 0.19385082 148.099091
pair_coeff 1 3 lj/cut 0.09087 3.613
pair_coeff 1 4 lj/cut 0.0344258 3.238
pair_coeff 2 3 lj/cut 0.1419429 3.1
pair_coeff 2 4 lj/cut 0.035857762359063315 1.932779 # same as water dimer
pair_coeff 2 4 lj/cut 0.035857762359063315 1.932779 # same as water
pair_coeff 3 3 lj/cut 0.0 3.4
pair_coeff 4 4 lj/cut 0.0 2.65

154
examples/QUANTUM/PySCF/log.8Feb23.mixture.qmmm.mpi.1
View File

@ -1 +1,155 @@
LAMMPS (22 Dec 2022)
# QMMM with PySCF - mixture example
units real
atom_style full
pair_style lj/cut/coul/long 12
pair_modify mix arithmetic
bond_style harmonic
angle_style harmonic
dihedral_style none
improper_style none
kspace_style pppm 1e-5
read_data data.mixture
Reading data file ...
orthogonal box = (0 0 0) to (31.351877 31.351877 31.351877)
1 by 1 by 1 MPI processor grid
reading atoms ...
3081 atoms
reading velocities ...
3081 velocities
scanning bonds ...
4 = max bonds/atom
scanning angles ...
6 = max angles/atom
reading bonds ...
2055 bonds
reading angles ...
1033 angles
Finding 1-2 1-3 1-4 neighbors ...
special bond factors lj: 0 0 0
special bond factors coul: 0 0 0
4 = max # of 1-2 neighbors
3 = max # of 1-3 neighbors
3 = max # of 1-4 neighbors
4 = max # of special neighbors
special bonds CPU = 0.000 seconds
read_data CPU = 0.011 seconds
# QM atoms are IDs 1 to 9
# MM atoms are remaining atoms
group qm id 1:9
9 atoms in group qm
group mm subtract all qm
3072 atoms in group mm
# remove bonds/angles between QM atoms
# set charges to zero on QM atoms
delete_bonds qm multi remove special
System init for delete_bonds ...
PPPM initialization ...
using 12-bit tables for long-range coulomb (../kspace.cpp:342)
G vector (1/distance) = 0.25751777
grid = 24 24 24
stencil order = 5
estimated absolute RMS force accuracy = 0.0020473396
estimated relative force accuracy = 6.1655023e-06
using double precision KISS FFT
3d grid and FFT values/proc = 29791 13824
Generated 21 of 21 mixed pair_coeff terms from arithmetic mixing rule
Neighbor list info ...
update: every = 1 steps, delay = 0 steps, check = yes
max neighbors/atom: 2000, page size: 100000
master list distance cutoff = 14
ghost atom cutoff = 14
binsize = 7, bins = 5 5 5
1 neighbor lists, perpetual/occasional/extra = 1 0 0
(1) pair lj/cut/coul/long, perpetual
attributes: half, newton on
pair build: half/bin/newton
stencil: half/bin/3d
bin: standard
Deleting bonds ...
2048 total bonds, 2048 turned on, 0 turned off
1024 total angles, 1024 turned on, 0 turned off
0 total dihedrals, 0 turned on, 0 turned off
0 total impropers, 0 turned on, 0 turned off
Finding 1-2 1-3 1-4 neighbors ...
special bond factors lj: 0 0 0
special bond factors coul: 0 0 0
2 = max # of 1-2 neighbors
1 = max # of 1-3 neighbors
1 = max # of 1-4 neighbors
4 = max # of special neighbors
special bonds CPU = 0.000 seconds
set group qm charge 0.0
Setting atom values ...
9 settings made for charge
neighbor 2.0 bin
neigh_modify delay 0 every 1 check yes
# QMMM dynamics
timestep 0.01
fix 1 all nve
fix 2 qm mdi/qmmm direct elements N H C H Cl O H
fix_modify 2 energy yes
thermo_style custom step cpu temp ke evdwl ecoul epair emol elong f_2 pe etotal press
thermo 1
run 2
PPPM initialization ...
using 12-bit tables for long-range coulomb (../kspace.cpp:342)
G vector (1/distance) = 0.25751777
grid = 24 24 24
stencil order = 5
estimated absolute RMS force accuracy = 0.0020473396
estimated relative force accuracy = 6.1655023e-06
using double precision KISS FFT
3d grid and FFT values/proc = 29791 13824
Generated 21 of 21 mixed pair_coeff terms from arithmetic mixing rule
Per MPI rank memory allocation (min/avg/max) = 26.88 | 26.88 | 26.88 Mbytes
Step CPU Temp KinEng E_vdwl E_coul E_pair E_mol E_long f_2 PotEng TotEng Press
0 0 0 0 103310.44 35284.412 88790.313 863.33521 -49804.536 -349295.05 -259641.4 -259641.4 897319.08
1 16.658724 631.94375 5801.8092 97293.123 35284.413 82773 863.33444 -49804.536 -349294.57 -265658.24 -259856.43 852336.04
2 34.242441 2211.4634 20303.213 82388.368 35284.413 67868.246 863.33213 -49804.536 -349292.7 -280561.12 -260257.91 741101.53
Loop time of 34.2425 on 1 procs for 2 steps with 3081 atoms
Performance: 0.000 ns/day, 475589.761 hours/ns, 0.058 timesteps/s, 179.952 atom-step/s
100.0% CPU use with 1 MPI tasks x no OpenMP threads
MPI task timing breakdown:
Section | min time | avg time | max time |%varavg| %total
---------------------------------------------------------------
Pair | 0.048944 | 0.048944 | 0.048944 | 0.0 | 0.14
Bond | 0.00014683 | 0.00014683 | 0.00014683 | 0.0 | 0.00
Kspace | 0.0043212 | 0.0043212 | 0.0043212 | 0.0 | 0.01
Neigh | 0 | 0 | 0 | 0.0 | 0.00
Comm | 0.00016665 | 0.00016665 | 0.00016665 | 0.0 | 0.00
Output | 9.4962e-05 | 9.4962e-05 | 9.4962e-05 | 0.0 | 0.00
Modify | 34.189 | 34.189 | 34.189 | 0.0 | 99.84
Other | | 8.974e-05 | | | 0.00
Nlocal: 3081 ave 3081 max 3081 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Nghost: 17741 ave 17741 max 17741 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Neighs: 1.77144e+06 ave 1.77144e+06 max 1.77144e+06 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Total # of neighbors = 1771437
Ave neighs/atom = 574.95521
Ave special neighs/atom = 1.9941577
Neighbor list builds = 0
Dangerous builds = 0
Total wall time: 0:00:53

152
examples/QUANTUM/PySCF/log.8Feb23.water.qmmm.mpi.1
View File

@ -1 +1,153 @@
LAMMPS (22 Dec 2022)
# QMMM with PySCF - two water example
units real
atom_style full
pair_style lj/cut/coul/long 12
pair_modify mix arithmetic
bond_style harmonic
angle_style harmonic
dihedral_style none
improper_style none
kspace_style pppm 1e-5
read_data data.water
Reading data file ...
orthogonal box = (0 0 0) to (31.351877 31.351877 31.351877)
1 by 1 by 1 MPI processor grid
reading atoms ...
6 atoms
scanning bonds ...
2 = max bonds/atom
scanning angles ...
1 = max angles/atom
reading bonds ...
4 bonds
reading angles ...
2 angles
Finding 1-2 1-3 1-4 neighbors ...
special bond factors lj: 0 0 0
special bond factors coul: 0 0 0
2 = max # of 1-2 neighbors
1 = max # of 1-3 neighbors
1 = max # of 1-4 neighbors
2 = max # of special neighbors
special bonds CPU = 0.000 seconds
read_data CPU = 0.002 seconds
# QM atoms are 1st water
# MM atoms are 2nd water
group qm molecule 1
3 atoms in group qm
group mm molecule 2
3 atoms in group mm
# remove bonds/angles between QM atoms
# set charges to zero on QM atoms
delete_bonds qm multi remove special
System init for delete_bonds ...
PPPM initialization ...
using 12-bit tables for long-range coulomb (../kspace.cpp:342)
G vector (1/distance) = 0.21131887
grid = 12 12 12
stencil order = 5
estimated absolute RMS force accuracy = 0.0020500128
estimated relative force accuracy = 6.1735526e-06
using double precision KISS FFT
3d grid and FFT values/proc = 4913 1728
Generated 1 of 1 mixed pair_coeff terms from arithmetic mixing rule
Neighbor list info ...
update: every = 1 steps, delay = 0 steps, check = yes
max neighbors/atom: 2000, page size: 100000
master list distance cutoff = 14
ghost atom cutoff = 14
binsize = 7, bins = 5 5 5
1 neighbor lists, perpetual/occasional/extra = 1 0 0
(1) pair lj/cut/coul/long, perpetual
attributes: half, newton on
pair build: half/bin/newton
stencil: half/bin/3d
bin: standard
Deleting bonds ...
2 total bonds, 2 turned on, 0 turned off
1 total angles, 1 turned on, 0 turned off
0 total dihedrals, 0 turned on, 0 turned off
0 total impropers, 0 turned on, 0 turned off
Finding 1-2 1-3 1-4 neighbors ...
special bond factors lj: 0 0 0
special bond factors coul: 0 0 0
2 = max # of 1-2 neighbors
1 = max # of 1-3 neighbors
1 = max # of 1-4 neighbors
2 = max # of special neighbors
special bonds CPU = 0.000 seconds
set group qm charge 0.0
Setting atom values ...
3 settings made for charge
neighbor 2.0 bin
neigh_modify delay 0 every 1 check yes
# QMMM dynamics
timestep 2.0
fix 1 all nve
fix 2 qm mdi/qmmm direct elements O H
fix_modify 2 energy yes
thermo_style custom step cpu temp ke evdwl ecoul epair emol elong f_2 pe etotal press
thermo 1
run 2
PPPM initialization ...
using 12-bit tables for long-range coulomb (../kspace.cpp:342)
G vector (1/distance) = 0.19767375
grid = 10 10 10
stencil order = 5
estimated absolute RMS force accuracy = 0.0022851662
estimated relative force accuracy = 6.8817102e-06
using double precision KISS FFT
3d grid and FFT values/proc = 3375 1000
Generated 1 of 1 mixed pair_coeff terms from arithmetic mixing rule
Per MPI rank memory allocation (min/avg/max) = 7.165 | 7.165 | 7.165 Mbytes
Step CPU Temp KinEng E_vdwl E_coul E_pair E_mol E_long f_2 PotEng TotEng Press
0 0 0 0 -0.001256507 37.096223 -0.0091688448 0.95028479 -37.104135 -47941.423 -47940.482 -47940.482 -11.65711
1 2.1383388 274.73153 4.0946125 -0.001258094 37.106014 -0.0088915927 0.62503382 -37.113647 -47947.815 -47947.199 -47943.104 5.9530005
2 4.182352 121.44569 1.8100327 -0.0012589509 37.125978 -0.0083996825 0.32331257 -37.133119 -47938.851 -47938.536 -47936.726 14.80352
Loop time of 4.18244 on 1 procs for 2 steps with 6 atoms
Performance: 0.083 ns/day, 290.447 hours/ns, 0.478 timesteps/s, 2.869 atom-step/s
100.0% CPU use with 1 MPI tasks x no OpenMP threads
MPI task timing breakdown:
Section | min time | avg time | max time |%varavg| %total
---------------------------------------------------------------
Pair | 1.3909e-05 | 1.3909e-05 | 1.3909e-05 | 0.0 | 0.00
Bond | 9.667e-06 | 9.667e-06 | 9.667e-06 | 0.0 | 0.00
Kspace | 0.00031926 | 0.00031926 | 0.00031926 | 0.0 | 0.01
Neigh | 0 | 0 | 0 | 0.0 | 0.00
Comm | 6.235e-06 | 6.235e-06 | 6.235e-06 | 0.0 | 0.00
Output | 0.00011152 | 0.00011152 | 0.00011152 | 0.0 | 0.00
Modify | 4.182 | 4.182 | 4.182 | 0.0 | 99.99
Other | | 2.225e-05 | | | 0.00
Nlocal: 6 ave 6 max 6 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Nghost: 42 ave 42 max 42 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Neighs: 15 ave 15 max 15 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Total # of neighbors = 15
Ave neighs/atom = 2.5
Ave special neighs/atom = 1
Neighbor list builds = 0
Dangerous builds = 0
Total wall time: 0:00:06

22
examples/QUANTUM/PySCF/pyscf_mdi.py
View File

@ -107,8 +107,9 @@ dm_previous = None
# --------------------------------------------
def error(txt):
if me == 0: print("ERROR:",txt)
world.Abort()
if me == 0: print("ERROR:",txt,mpiexists=1)
if mpiexists: world.abort()
sys.exit()
# --------------------------------------------
# process non-MDI options to PySCF
@ -616,10 +617,12 @@ def MDI_Plugin_init_pyscf_mdi(plugin_state):
# --------------------------------------------
if __name__== "__main__":
# mdi_index = index in sys.argv of -mdi
# mdi_option = single arg in quotes that follows -mdi
# other_options = all non-MDI args
mdi_index = -1
mdi_option = ""
other_options = []
@ -630,19 +633,24 @@ if __name__== "__main__":
while iarg < narg:
arg = args[iarg]
if arg == "-mdi" or arg == "--mdi":
mdi_index = iarg
if narg > iarg+1: mdi_option = sys.argv[iarg+1]
else: error("PySCF -mdi argument not provided")
else: error("PySCF -mdi argument not provided",0)
iarg += 1
else: other_options.append(arg)
iarg += 1
if not mdi_option: error("PySCF -mdi option not provided")
if not mdi_option: error("PySCF -mdi option not provided",0)
# remove -mdi and its string from sys.argv
# so that PySCF does not try to process it
sys.argv.pop(mdi_index)
sys.argv.pop(mdi_index)
# disable this mode of MDI coupling for now
# until issue on PySCF side is fixed
error("PySCF does not currently support running as independent executable")
# call MDI_Init with just -mdi option
mdi.MDI_Init(mdi_option)