Skip to content

fix Python shebangs for scripts in Python and Conda-like installations#3717

Merged
boegel merged 1 commit intoeasybuilders:developfrom
Flamefire:pythonshebangs
May 23, 2025
Merged

fix Python shebangs for scripts in Python and Conda-like installations#3717
boegel merged 1 commit intoeasybuilders:developfrom
Flamefire:pythonshebangs

Conversation

@Flamefire
Copy link
Copy Markdown
Contributor

@Flamefire Flamefire commented May 13, 2025

The core Python modules have scripts too that use python. However their shebangs use absolute paths to the installed python binary.
This causes serious problems with virtualenvs as e.g pip install would still use the core Python binary instead of the virtualenv binary leading to failures if $PIP_REQUIRE_VIRTUALENV is set.

Copy over the default shebang-fix from PythonPackage.

@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented May 13, 2025 •
edited
Loading

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS Python-3.10.4-GCCcore-11.3.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
julia.hpc.tu-dresden.de - Linux Rocky Linux 8.7 (Green Obsidian), x86_64, Intel(R) Xeon(R) Platinum 8276M CPU @ 2.20GHz (cascadelake), Python 3.8.13
See https://gist.github.com/Flamefire/9582c830d4e566d7acfa433abd1f4c44 for a full test report.

@boegel boegel added this to the release after 5.1.0 milestone May 21, 2025
@Flamefire
Fix Python shebangs for Python and Conda-like modules
5378bf3 
The core Python modules have scripts too that use `python`.
However their shebangs use absolute paths to the installed python
binary.
This causes serious problems with virtualenvs as e.g `pip install` would
still use the core Python binary instead of the virtualenv binary
leading to failures if `$PIP_REQUIRE_VIRTUALENV` is set.

Copy over the default shebang-fix from PythonPackage.
@boegel
Copy link
Copy Markdown
Member

boegel commented May 23, 2025

@boegelbot please test @ jsc-zen3
EB_ARGS="Python-2.7.18-GCCcore-12.3.0.eb Python-3.11.3-GCCcore-12.3.0.eb Python-3.13.1-GCCcore-14.2.0.eb Anaconda3-2024.06-1.eb Miniconda3-24.7.1-0.eb --installpath /tmp/$USER/pr3717"

@boegelbot
Copy link
Copy Markdown

boegelbot commented May 23, 2025

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3717 EB_ARGS="Python-2.7.18-GCCcore-12.3.0.eb Python-3.11.3-GCCcore-12.3.0.eb Python-3.13.1-GCCcore-14.2.0.eb Anaconda3-2024.06-1.eb Miniconda3-24.7.1-0.eb --installpath /tmp/$USER/pr3717" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3717 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 6446

Test results coming soon (I hope)...

Details

- notification for comment with ID 2903808331 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Copy Markdown

boegelbot commented May 23, 2025

Test report by @boegelbot

Overview of tested easyconfigs (in order)

Build succeeded for 4 out of 5 (5 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/6d9bd5efd146633a33dc25a56da69f1e for a full test report.

@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented May 23, 2025

== 2025-05-23 10:07:35,967 build_log.py:226 ERROR EasyBuild encountered an error (at easybuild/easybuild-framework/easybuild/base/exceptions.py:126 in init): Couldn't find file Anaconda3-2024.06-1-Linux-x86_64.sh anywhere, and downloading it didn't work either...

boegel
boegel approved these changes May 23, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel
Copy link
Copy Markdown
Member

boegel commented May 23, 2025

@boegelbot please test @ jsc-zen3
EB_ARGS="Anaconda3-2023.09-0.eb --installpath /tmp/$USER/pr3717"

@boegelbot
Copy link
Copy Markdown

boegelbot commented May 23, 2025

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3717 EB_ARGS="Anaconda3-2023.09-0.eb --installpath /tmp/$USER/pr3717" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3717 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 6449

Test results coming soon (I hope)...

Details

- notification for comment with ID 2904001712 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Copy Markdown

boegelbot commented May 23, 2025

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS Anaconda3-2023.09-0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/5e9f3b4f94479740ff6215698315cb6c for a full test report.

@boegel boegel merged commit 83a76c8 into easybuilders:develop May 23, 2025
17 checks passed
@Flamefire Flamefire deleted the pythonshebangs branch May 23, 2025 11:16
@boegel boegel changed the title Fix Python shebangs for Python and Conda-like modules fix Python shebangs for scripts in Python and Conda-like installations May 26, 2025
@branfosj branfosj mentioned this pull request Jun 5, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

bug fix

Projects

None yet

Milestone

5.1.0

Development

Successfully merging this pull request may close these issues.

3 participants

@Flamefire @boegel @boegelbot