enhance custom easyblock for LAMMPS: add `sanity_check_test_inputs` custom easyconfig parameter, add mapping for zen4, also support building of non-stable LAMMPS versions by laraPPr · Pull Request #3336 · easybuilders/easybuild-easyblocks

laraPPr · 2024-05-23T11:39:32Z

Add the option to change the examples that will be run during the sanity-check in the EasyConfig

… sanity-check via the easyconfig

boegel · 2024-07-31T14:37:33Z

            'kokkos': [True, "Enable kokkos build.", CUSTOM],
            'kokkos_arch': [None, "Set kokkos processor arch manually, if auto-detection doesn't work.", CUSTOM],
            'user_packages': [None, "List user packages (without prefix PKG_ or USER-PKG_).", CUSTOM],
+            'check_files': [None, "List of tests for sanity-check.", CUSTOM],


Why not just rely on sanity_check_files if you need to override the default sanity check performed by this easyblock?

That did not work and than on slack they suggested I do this. sanity_check_files is just ignored if added to the easyconfig.

This is the output of sanity check if I add this to the easyconfig

sanity_check_files = [ 'balance', 'crack', 'friction', 'indent', 'melt', 'min', 'nemd', 'obstacle', ]

output of sanity-check you can see that it runs all the tests set in check_files in the easyblock

>> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/atm/in.atm"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/atm/in.atm"); l.finalize()'': FAILED >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/balance/in.balance"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/balance/in.balance"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/colloid/in.colloid"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/colloid/in.colloid"); l.finalize()'': FAILED >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/crack/in.crack"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/crack/in.crack"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/dipole/in.dipole"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/dipole/in.dipole"); l.finalize()'': FAILED >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/friction/in.friction"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/friction/in.friction"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/hugoniostat/in.hugoniostat"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/hugoniostat/in.hugoniostat"); l.finalize()'': FAILED >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/indent/in.indent"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/indent/in.indent"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/melt/in.melt"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/melt/in.melt"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/min/in.min"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/min/in.min"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/msst/in.msst"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/msst/in.msst"); l.finalize()'': FAILED >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/nemd/in.nemd"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/nemd/in.nemd"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/obstacle/in.obstacle"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/obstacle/in.obstacle"); l.finalize()'': OK >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/pour/in.pour"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/pour/in.pour"); l.finalize()'': FAILED >> running command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/voronoi/in.voronoi"); l.finalize()'' ... >> result for command 'cd /tmp/eb-nod3jf99/tmpx040qyst && LD_LIBRARY_PATH=$LIBRARY_PATH:$LD_LIBRARY_PATH mpirun -n 1 python -c 'from lammps import lammps; l=lammps(); l.file("/data/gent/vo/000/gvo00002/vsc46128/easybuild/RHEL8/cascadelake-ampere-ib/software/LAMMPS/2Aug2023_update2-foss-2023a-kokkos-dev_OBMD/exam ples/voronoi/in.voronoi"); l.finalize()'': FAILED

Ah, my bad check_files is used to determine which sanity check commands to run, and is orthogonal to the standard sanity_check_paths we have in EasyBuild which is used for checking whether specific files/directories exist or not.

To avoid confusion, maybe it's better to rename this custom check_files easyconfig parameter to something else to avoid confusion, for example sanity_check_test_inputs?

sounds good. Done.

boegel · 2024-07-31T14:37:53Z

            mkdir(site_packages, parents=True)

-            self.lammpsdir = os.path.join(self.builddir, '%s-stable_%s' % (self.name.lower(), self.version))
+            self.lammpsdir = os.path.join(self.builddir, '%s-*_%s' % (self.name.lower(), self.version))


This seems to be the only real necessary change, the PR title should reflect that?

boegel · 2024-08-02T15:51:12Z

            'kokkos': [True, "Enable kokkos build.", CUSTOM],
            'kokkos_arch': [None, "Set kokkos processor arch manually, if auto-detection doesn't work.", CUSTOM],
            'user_packages': [None, "List user packages (without prefix PKG_ or USER-PKG_).", CUSTOM],
+            'check_files': [None, "List of tests for sanity-check.", CUSTOM],


Ah, my bad check_files is used to determine which sanity check commands to run, and is orthogonal to the standard sanity_check_paths we have in EasyBuild which is used for checking whether specific files/directories exist or not.

To avoid confusion, maybe it's better to rename this custom check_files easyconfig parameter to something else to avoid confusion, for example sanity_check_test_inputs?

boegel

lgtm

boegel · 2024-08-14T20:34:20Z

@boegelbot please test @ jsc-zen3
EB_ARGS="LAMMPS-2Aug2023_update2-foss-2023a-kokkos.eb"

boegelbot · 2024-08-14T20:53:07Z

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3336 EB_ARGS="LAMMPS-2Aug2023_update2-foss-2023a-kokkos.eb" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3336 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 4687

Test results coming soon (I hope)...

Details

- notification for comment with ID 2289826172 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-08-14T21:06:24Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS LAMMPS-2Aug2023_update2-foss-2023a-kokkos.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/041341e9f6c9aed6b95f589792615d52 for a full test report.

LAMMPS: add option to change the examples that will be run during the…

53cfad5

… sanity-check via the easyconfig

laraPPr mentioned this pull request Jun 21, 2024

first dev of OBDM packages laraPPr/lammps#1

Merged

vsc46128 vscuser and others added 2 commits July 30, 2024 13:57

update python command to also work for patch releases

94e2836

Merge branch 'develop' into LAMMPS_checks

dcd3087

laraPPr changed the title ~~update LAMMPS check_files~~ update LAMMPS check_files and update python command for 2Aug2023 and older versions Jul 30, 2024

boegel added the enhancement label Jul 31, 2024

boegel reviewed Jul 31, 2024

View reviewed changes

boegel requested changes Aug 2, 2024

View reviewed changes

vsc46128 vscuser and others added 2 commits August 4, 2024 13:52

replace check_files with sanity_check_test_inputs

4794dbb

add mapping for zen4

c390e73

laraPPr changed the title ~~update LAMMPS check_files and update python command for 2Aug2023 and older versions~~ Add sanity_check_test_inputs, add mapping for zen4 and update python command for 2Aug2023 and older versions Aug 7, 2024

boegel added this to the release after 4.9.2 milestone Aug 14, 2024

boegel changed the title ~~Add sanity_check_test_inputs, add mapping for zen4 and update python command for 2Aug2023 and older versions~~ Add sanity_check_test_inputs, add mapping for zen4 and update python command for LAMMPS 2Aug2023 and older versions Aug 14, 2024

boegel changed the title ~~Add sanity_check_test_inputs, add mapping for zen4 and update python command for LAMMPS 2Aug2023 and older versions~~ Add sanity_check_test_inputs, add mapping for zen4, and update python command for LAMMPS 2Aug2023 and older versions Aug 14, 2024

boegel approved these changes Aug 14, 2024

View reviewed changes

boegel merged commit 99f7254 into easybuilders:develop Aug 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enhance custom easyblock for LAMMPS: add `sanity_check_test_inputs` custom easyconfig parameter, add mapping for zen4, also support building of non-stable LAMMPS versions#3336

enhance custom easyblock for LAMMPS: add `sanity_check_test_inputs` custom easyconfig parameter, add mapping for zen4, also support building of non-stable LAMMPS versions#3336
boegel merged 5 commits intoeasybuilders:developfrom
laraPPr:LAMMPS_checks

laraPPr commented May 23, 2024

Uh oh!

boegel Jul 31, 2024

Uh oh!

laraPPr Jul 31, 2024 •

edited

Loading

Uh oh!

laraPPr Jul 31, 2024

Uh oh!

boegel Aug 2, 2024

Uh oh!

laraPPr Aug 4, 2024

Uh oh!

boegel Jul 31, 2024

Uh oh!

boegel Aug 2, 2024

Uh oh!

boegel left a comment

Uh oh!

boegel commented Aug 14, 2024

Uh oh!

boegelbot commented Aug 14, 2024

Uh oh!

boegelbot commented Aug 14, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

laraPPr commented May 23, 2024

Uh oh!

boegel Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

laraPPr Jul 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laraPPr Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

boegel Aug 2, 2024

Choose a reason for hiding this comment

Uh oh!

laraPPr Aug 4, 2024

Choose a reason for hiding this comment

Uh oh!

boegel Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

boegel Aug 2, 2024

Choose a reason for hiding this comment

Uh oh!

boegel left a comment

Choose a reason for hiding this comment

Uh oh!

boegel commented Aug 14, 2024

Uh oh!

boegelbot commented Aug 14, 2024

Uh oh!

boegelbot commented Aug 14, 2024

Overview of tested easyconfigs (in order)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

laraPPr Jul 31, 2024 •

edited

Loading