Don't show command string again when showing in which folder it runs by Flamefire · Pull Request #5003 · easybuilders/easybuild-framework

Flamefire · 2025-09-24T16:01:54Z

Follow up to:

Make command environment of shell commands more discoverable in the log #4999

This should change

== 2025-09-15 09:27:24,887 run.py:457 INFO run_shell_cmd: command environment of
"export PYTHONPATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/lib/python3.12/site-packages:$PYTHONPATH && export PATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/bin:$PATH && PYTORCH_BUILD_VERSION=2.7.1 PYTORCH_BUILD_NUMBER=1 VERBOSE=0 MAX_JOBS=16 BLAS=FlexiBLAS WITH_BLAS=flexi USE_GFLAGS=0 USE_GLOG=0 USE_CUDSS=0 USE_CUSPARSELT=1 USE_UCC=0 USE_SYSTEM_UCC=0 BUILD_CUSTOM_PROTOBUF=0 USE_SYSTEM_EIGEN_INSTALL=0 USE_SYSTEM_PYBIND11=1 USE_IBVERBS=1 USE_CUDA=1 CUDNN_LIB_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/lib64 CUDNN_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/include USE_SYSTEM_NCCL=1 NCCL_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/NCCL/2.26.2-GCCcore-13.3.0-CUDA-12.6.0/include USE_ROCM=0 CMAKE_BUILD_TYPE=Release /software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/bin/python -m pip install --prefix=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on --verbose --no-deps --ignore-installed --no-index --no-build-isolation ."
will be saved to /tmp/easybuild-tmp/eb-ujvxdd0l/run-shell-cmd-output/export-g0peat6s/cmd.sh
Output will be logged to /tmp/easybuild-tmp/eb-ujvxdd0l/run-shell-cmd-output/export-g0peat6s/out.txt
== 2025-09-15 09:27:24,926 run.py:494 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-09-15 09:27:24,926 run.py:508 INFO Running shell command 'export PYTHONPATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/lib/python3.12/site-packages:$PYTHONPATH && export PATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/bin:$PATH && PYTORCH_BUILD_VERSION=2.7.1 PYTORCH_BUILD_NUMBER=1 VERBOSE=0 MAX_JOBS=16 BLAS=FlexiBLAS WITH_BLAS=flexi USE_GFLAGS=0 USE_GLOG=0 USE_CUDSS=0 USE_CUSPARSELT=1 USE_UCC=0 USE_SYSTEM_UCC=0 BUILD_CUSTOM_PROTOBUF=0 USE_SYSTEM_EIGEN_INSTALL=0 USE_SYSTEM_PYBIND11=1 USE_IBVERBS=1 USE_CUDA=1 CUDNN_LIB_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/lib64 CUDNN_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/include USE_SYSTEM_NCCL=1 NCCL_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/NCCL/2.26.2-GCCcore-13.3.0-CUDA-12.6.0/include USE_ROCM=0 CMAKE_BUILD_TYPE=Release /software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/bin/python -m pip install --prefix=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on --verbose --no-deps --ignore-installed --no-index --no-build-isolation .' in /dev/shm/s3248973-EasyBuild/PyTorch/2.7.1/foss-2024a-CUDA-12.6.0/pytorch-v2.7.1
== 2025-09-26 13:56:48,683 run.py:628 INFO Shell command completed successfully: export PYTHONPATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/lib/python3.12/site-packages:$PYTHONPATH && export PATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/bin:$PATH && PYTORCH_BUILD_VERSION=2.7.1 PYTORCH_BUILD_NUMBER=1 VERBOSE=0 MAX_JOBS=16 BLAS=FlexiBLAS WITH_BLAS=flexi USE_GFLAGS=0 USE_GLOG=0 USE_CUDSS=0 USE_CUSPARSELT=1 USE_UCC=0 USE_SYSTEM_UCC=0 BUILD_CUSTOM_PROTOBUF=0 USE_SYSTEM_EIGEN_INSTALL=0 USE_SYSTEM_PYBIND11=1 USE_IBVERBS=1 USE_CUDA=1 CUDNN_LIB_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/lib64 CUDNN_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/include USE_SYSTEM_NCCL=1 NCCL_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/NCCL/2.26.2-GCCcore-13.3.0-CUDA-12.6.0/include USE_ROCM=0 CMAKE_BUILD_TYPE=Release /software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/bin/python -m pip install --prefix=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on --verbose --no-deps --ignore-installed --no-index --no-build-isolation .
== 2025-09-26 13:56:48,684 run.py:630 INFO Output of 'export ...' shell command (stdout + stderr):

To

== 2025-09-15 09:27:24,887 run.py:457 INFO run_shell_cmd: command environment of
"export PYTHONPATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/lib/python3.12/site-packages:$PYTHONPATH && export PATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/bin:$PATH && PYTORCH_BUILD_VERSION=2.7.1 PYTORCH_BUILD_NUMBER=1 VERBOSE=0 MAX_JOBS=16 BLAS=FlexiBLAS WITH_BLAS=flexi USE_GFLAGS=0 USE_GLOG=0 USE_CUDSS=0 USE_CUSPARSELT=1 USE_UCC=0 USE_SYSTEM_UCC=0 BUILD_CUSTOM_PROTOBUF=0 USE_SYSTEM_EIGEN_INSTALL=0 USE_SYSTEM_PYBIND11=1 USE_IBVERBS=1 USE_CUDA=1 CUDNN_LIB_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/lib64 CUDNN_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/include USE_SYSTEM_NCCL=1 NCCL_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/NCCL/2.26.2-GCCcore-13.3.0-CUDA-12.6.0/include USE_ROCM=0 CMAKE_BUILD_TYPE=Release /software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/bin/python -m pip install --prefix=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on --verbose --no-deps --ignore-installed --no-index --no-build-isolation ."
will be saved to /tmp/easybuild-tmp/eb-ujvxdd0l/run-shell-cmd-output/export-g0peat6s/cmd.sh
Output will be logged to /tmp/easybuild-tmp/eb-ujvxdd0l/run-shell-cmd-output/export-g0peat6s/out.txt
== 2025-09-15 09:27:24,926 run.py:494 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-09-15 09:27:24,926 run.py:508 INFO Running 'export ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.7.1/foss-2024a-CUDA-12.6.0/pytorch-v2.7.1
== 2025-09-26 13:56:48,683 run.py:628 INFO Shell command completed successfully: export PYTHONPATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/lib/python3.12/site-packages:$PYTHONPATH && export PATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/bin:$PATH && PYTORCH_BUILD_VERSION=2.7.1 PYTORCH_BUILD_NUMBER=1 VERBOSE=0 MAX_JOBS=16 BLAS=FlexiBLAS WITH_BLAS=flexi USE_GFLAGS=0 USE_GLOG=0 USE_CUDSS=0 USE_CUSPARSELT=1 USE_UCC=0 USE_SYSTEM_UCC=0 BUILD_CUSTOM_PROTOBUF=0 USE_SYSTEM_EIGEN_INSTALL=0 USE_SYSTEM_PYBIND11=1 USE_IBVERBS=1 USE_CUDA=1 CUDNN_LIB_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/lib64 CUDNN_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/include USE_SYSTEM_NCCL=1 NCCL_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/NCCL/2.26.2-GCCcore-13.3.0-CUDA-12.6.0/include USE_ROCM=0 CMAKE_BUILD_TYPE=Release /software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/bin/python -m pip install --prefix=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on --verbose --no-deps --ignore-installed --no-index --no-build-isolation .
== 2025-09-26 13:56:48,684 run.py:630 INFO Output of 'export ...' shell command (stdout + stderr):

Shall we remove the duplication in the completed/failed message too?

Then I'd go for:

== 2025-09-15 09:27:24,887 run.py:457 INFO run_shell_cmd: command environment of
'export ...' shell command
will be saved to /tmp/easybuild-tmp/eb-ujvxdd0l/run-shell-cmd-output/export-g0peat6s/cmd.sh
Output will be logged to /tmp/easybuild-tmp/eb-ujvxdd0l/run-shell-cmd-output/export-g0peat6s/out.txt
== 2025-09-15 09:27:24,926 run.py:494 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-09-15 09:27:24,926 run.py:508 INFO Running 'export PYTHONPATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/lib/python3.12/site-packages:$PYTHONPATH && export PATH=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on/bin:$PATH && PYTORCH_BUILD_VERSION=2.7.1 PYTORCH_BUILD_NUMBER=1 VERBOSE=0 MAX_JOBS=16 BLAS=FlexiBLAS WITH_BLAS=flexi USE_GFLAGS=0 USE_GLOG=0 USE_CUDSS=0 USE_CUSPARSELT=1 USE_UCC=0 USE_SYSTEM_UCC=0 BUILD_CUSTOM_PROTOBUF=0 USE_SYSTEM_EIGEN_INSTALL=0 USE_SYSTEM_PYBIND11=1 USE_IBVERBS=1 USE_CUDA=1 CUDNN_LIB_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/lib64 CUDNN_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/cuDNN/9.5.1.17-CUDA-12.6.0/include USE_SYSTEM_NCCL=1 NCCL_INCLUDE_DIR=/data/horse/ws/s3248973-EasyBuild/easybuild-genoa/software/NCCL/2.26.2-GCCcore-13.3.0-CUDA-12.6.0/include USE_ROCM=0 CMAKE_BUILD_TYPE=Release /software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/bin/python -m pip install --prefix=/tmp/easybuild-tmp/eb-ujvxdd0l/tmp_8ohi9on --verbose --no-deps --ignore-installed --no-index --no-build-isolation .' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.7.1/foss-2024a-CUDA-12.6.0/pytorch-v2.7.1
== 2025-09-26 13:56:48,683 run.py:628 INFO Shell command completed successfully
== 2025-09-26 13:56:48,684 run.py:630 INFO Output of 'export ...' shell command (stdout + stderr):

I.e. show the full command only when the actual run starts and abbreviate otherwise. The part with "command environment of .. will be saved to" will not always be present as it depends on an argument

Use "shell command" or just "command" or "interactive [shell] command" in all places where it is logged. Additionally log the shortened command in more places like: "Running 'gcc ...' shell command in /tmp/fpp" "Output of 'gcc ...' shell command (stdout only):"

Flamefire · 2025-12-11T08:16:27Z

@boegel Can we get this in for the release? Preferably with the suggestion of having the full command only once.

…en it's actually being run)

boegel · 2025-12-16T08:57:55Z

I've tweaked this a bit.

I think it makes sense to only log the shell command being run once, and doing so when it's being run.

With the tweaks in 6277a67, here's how it looks now:

For very short commands:

== 2025-12-16 09:54:28,730 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'make ...' shell command will be saved to /tmp/eb-l0udl58t/run-shell-cmd-output/make-we16pztz/cmd.sh, output will be logged to /tmp/eb-l0udl58t/run-shell-cmd-output/make-we16pztz/out.txt
== 2025-12-16 09:54:28,735 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-16 09:54:28,735 run.py:518 INFO Running 'make ...' shell command in /tmp/vsc40023/easybuild_build/zlib/1.3.1/GCCcore-14.3.0/zlib-1.3.1:
        make  -j 16
== 2025-12-16 09:54:29,881 run.py:632 INFO 'make ...' shell command completed successfully
== 2025-12-16 09:54:29,881 run.py:634 INFO Output of 'make ...' shell command (stdout + stderr):
...

For longer commands:

== 2025-12-16 09:51:41,956 run.py:478 INFO run_shell_cmd: Script to start debug shell for '/user/gent/400/vsc40023/eb_arcaninescratch/RHEL9/zen2-ib/software/Python/3.13.5-GCCcore-14.3.0/bin/python ...' shell command will be saved to /tmp/eb-4osijv04/run-shell-cmd-output/python-6p43fn7u/cmd.sh, output will be logged to /tmp/eb-4osijv04/run-shell-cmd-output/python-6p43fn7u/out.txt
== 2025-12-16 09:51:41,959 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-16 09:51:41,959 run.py:518 INFO Running '/user/gent/400/vsc40023/eb_arcaninescratch/RHEL9/zen2-ib/software/Python/3.13.5-GCCcore-14.3.0/bin/python ...' shell command in /tmp/vsc40023/easybuild_build/matplotlib/3.10.5/gfbf-2025b/cycler/cycler-0.12.1:
        /user/gent/400/vsc40023/eb_arcaninescratch/RHEL9/zen2-ib/software/Python/3.13.5-GCCcore-14.3.0/bin/python -m pip install --prefix=/user/gent/400/vsc40023/eb_arcaninescratch/RHEL9/zen2-ib/software/matplotlib/3.10.5-gfbf-2025b  --verbose --no-deps --ignore-installed --no-build-isolation .
== 2025-12-16 09:51:44,821 run.py:632 INFO '/user/gent/400/vsc40023/eb_arcaninescratch/RHEL9/zen2-ib/software/Python/3.13.5-GCCcore-14.3.0/bin/python ...' shell command completed successfully
== 2025-12-16 09:51:44,821 run.py:634 INFO Output of '/user/gent/400/vsc40023/eb_arcaninescratch/RHEL9/zen2-ib/software/Python/3.13.5-GCCcore-14.3.0/bin/python ...' shell command (stdout + stderr):
...

or

== 2025-12-16 09:49:38,017 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'cmake ...' shell command will be saved to /tmp/eb-3xyftuj6/run-shell-cmd-output/cmake-g44x043b/cmd.sh, output will be logged to /tmp/eb-3xyftuj6/run-shell-cmd-output/cmake-g44x043b/out.txt
== 2025-12-16 09:49:38,020 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-16 09:49:38,020 run.py:518 INFO Running 'cmake ...' shell command in /tmp/vsc40023/easybuild_build/zlibng/2.2.5/GCCcore-14.3.0/easybuild_obj:
        cmake  -DCMAKE_INSTALL_PREFIX=/user/gent/400/vsc40023/eb_arcaninescratch/RHEL9/zen2-ib/software/zlib-ng/2.2.5-GCCcore-14.3.0 -DCMAKE_MESSAGE_LOG_LEVEL=STATUS -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_LIBDIR:PATH=lib -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DZLIB_ENABLE_TESTS=ON /tmp/vsc40023/easybuild_build/zlibng/2.2.5/GCCcore-14.3.0/zlib-ng-2.2.5
== 2025-12-16 09:50:02,478 run.py:632 INFO 'cmake ...' shell command completed successfully
== 2025-12-16 09:50:02,478 run.py:634 INFO Output of 'cmake ...' shell command (stdout + stderr):
...

I've put the shell command itself on a separate line, indented with a tab (so it stands out a bit, and is easy to copy-paste).
It's only logged in full right before it's being run, and it's easy to find shell commands being run by looking for Running.*shell command.

I feel that makes the log a whole lot more readable...

boegel

lgtm

Flamefire · 2025-12-16T12:32:36Z

Great thanks!

Flamefire added 3 commits September 24, 2025 18:00

Don't show command string again when showing in which folder it runs.

80c82eb

Fix test

67e19b1

boegel added the change label Oct 8, 2025

boegel added this to the next release (5.2.0?) milestone Oct 8, 2025

Merge branch 'develop' into run-shell-cmd-verbosity

616f453

boegel added 2 commits December 16, 2025 08:46

Merge branch 'develop' into run-shell-cmd-verbosity

3da01de

tweak logging of shell commands, only log full shell command once (wh…

6277a67

…en it's actually being run)

boegel added the enhancement label Dec 16, 2025

boegel approved these changes Dec 16, 2025

View reviewed changes

boegel enabled auto-merge December 16, 2025 08:59

boegel merged commit 3a6821f into easybuilders:develop Dec 16, 2025
40 checks passed

Flamefire deleted the run-shell-cmd-verbosity branch December 16, 2025 12:32

Flamefire mentioned this pull request Dec 18, 2025

Add line breaks in log for run-shell-cmd environment #5078

Merged

boegel changed the title ~~Don't show command string again when showing in which folder it runs.~~ Don't show command string again when showing in which folder it runs Jan 14, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't show command string again when showing in which folder it runs#5003

Don't show command string again when showing in which folder it runs#5003
boegel merged 6 commits intoeasybuilders:developfrom
Flamefire:run-shell-cmd-verbosity

Flamefire commented Sep 24, 2025 •

edited by boegel

Loading

Uh oh!

Flamefire commented Dec 11, 2025

Uh oh!

boegel commented Dec 16, 2025

Uh oh!

boegel left a comment

Uh oh!

Uh oh!

Flamefire commented Dec 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Flamefire commented Sep 24, 2025 • edited by boegel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Flamefire commented Dec 11, 2025

Uh oh!

boegel commented Dec 16, 2025

Uh oh!

boegel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Flamefire commented Dec 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Flamefire commented Sep 24, 2025 •

edited by boegel

Loading