Skip to content

limit parallelism of OpenBLAS tests with parallel property#3936

Merged
SebastianAchilles merged 1 commit intoeasybuilders:developfrom
Thyre:20250915163057_new_pr_openblas
Sep 15, 2025
Merged

limit parallelism of OpenBLAS tests with parallel property#3936
SebastianAchilles merged 1 commit intoeasybuilders:developfrom
Thyre:20250915163057_new_pr_openblas

Conversation

@Thyre
Copy link
Copy Markdown
Collaborator

@Thyre Thyre commented Sep 15, 2025 •
edited by boegel
Loading

(created using eb --new-pr)

Avoids heavily excessive CPU utilization on many-core machines, which also increases test times significantly.
Both OMP_NUM_THREADS and OPENBLAS_NUM_THREADS are checked in the source, so set both based on if OpenMP and / or threads are used.

Results in e.g:

== testing...
  >> running shell command:
        OMP_NUM_THREADS=8 OPENBLAS_NUM_THREADS=8   make tests  BINARY='64'  CC='gcc'  FC='gfortran'  INTERFACE64='1'  LIBPREFIX='libopenblas64'  MAKE_NB_JOBS='-1'  USE_OPENMP='1'  USE_THREAD='1'
        [started at: 2025-09-15 16:31:58]
        [working dir: /data/EasyBuild-develop/build/OpenBLAS/0.3.30/GCC-14.3.0/OpenBLAS-0.3.30]
        [output and state saved to /tmp/eb-7ptaw8j_/run-shell-cmd-output/OMP_NUM_THREADS8-7vewft3l]

Closes #3441

edit: this reduced installation time for OpenBLAS from ~14h to ~10min on JUWELS @ JSC

@Thyre
Copy link
Copy Markdown
Collaborator Author

Thyre commented Sep 15, 2025

@boegelbot please test @ jsc-zen3
EB_ARGS="OpenBLAS-0.3.30-GCC-14.3.0.eb OpenBLAS-0.3.29-GCC-14.2.0.eb OpenBLAS-0.3.27-GCC-13.3.0.eb OpenBLAS-0.3.24-GCC-13.2.0.eb --installpath=/tmp/$USER/ebpr-3936"

@Thyre Thyre added this to the 5.x milestone Sep 15, 2025
@Thyre Thyre added the change label Sep 15, 2025
@boegelbot
Copy link
Copy Markdown

boegelbot commented Sep 15, 2025

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3936 EB_ARGS="OpenBLAS-0.3.30-GCC-14.3.0.eb OpenBLAS-0.3.29-GCC-14.2.0.eb OpenBLAS-0.3.27-GCC-13.3.0.eb OpenBLAS-0.3.24-GCC-13.2.0.eb --installpath=/tmp/$USER/ebpr-3936" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3936 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 7959

Test results coming soon (I hope)...

Details

- notification for comment with ID 3292503330 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@Thyre
Copy link
Copy Markdown
Collaborator Author

Thyre commented Sep 15, 2025 •
edited
Loading

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS OpenBLAS-0.3.30-GCC-14.3.0.eb
  • SUCCESS OpenBLAS-0.3.29-GCC-14.2.0.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
Linux - Linux Arch Linux UNKNOWN, x86_64, AMD Ryzen 7 7800X3D 8-Core Processor (zen4), 1 x AMD Navi 48 [Radeon RX 9070/9070 XT/9070 GRE] (device id: 0x7550, gfx: gfx1201, driver: 6.16.7-arch1-1), 1 x AMD Raphael (device id: 0x164e, gfx: gfx1036, driver: 6.16.7-arch1-1), Python 3.13.7
See https://gist.github.com/Thyre/c25a2af9e47f196e3f4fca34bc32edbb for a full test report.

SebastianAchilles
SebastianAchilles approved these changes Sep 15, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@SebastianAchilles SebastianAchilles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegelbot
Copy link
Copy Markdown

boegelbot commented Sep 15, 2025

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS OpenBLAS-0.3.30-GCC-14.3.0.eb
  • SUCCESS OpenBLAS-0.3.29-GCC-14.2.0.eb
  • SUCCESS OpenBLAS-0.3.27-GCC-13.3.0.eb
  • SUCCESS OpenBLAS-0.3.24-GCC-13.2.0.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/39fa369d83b01b3e0434a0158a8098e9 for a full test report.

@SebastianAchilles
Copy link
Copy Markdown
Member

SebastianAchilles commented Sep 15, 2025

Going in, thanks @Thyre!

@SebastianAchilles SebastianAchilles merged commit 4889a40 into easybuilders:develop Sep 15, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SebastianAchilles SebastianAchilles SebastianAchilles approved these changes

Assignees

No one assigned

Labels

change

Projects

None yet

Milestone

5.1.2

Development

Successfully merging this pull request may close these issues.

LAPACK tests for OpenBLAS not controlled by maxparallel

3 participants

@Thyre @boegelbot @SebastianAchilles