make sure to raise an error if `pick_python_cmd` returns `False` for Python bundles/packages by casparvl · Pull Request #3430 · easybuilders/easybuild-easyblocks

casparvl · 2024-09-03T19:15:45Z

This was already done for PythonPackage, but not for PythonBundle. I ran into this here easybuilders/easybuild-easyconfigs#21307 (comment)

casparvl · 2024-09-03T19:25:03Z

When used with ReFrame-4.9.2.eb from easybuilders/easybuild-easyconfigs#21307 , with this PR, a proper error is raised:

== 2024-09-03 21:20:47,564 run.py:247 INFO running cmd: /usr/bin/python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'
== 2024-09-03 21:20:47,591 run.py:689 DEBUG cmd "/usr/bin/python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'" exited with exit code 0 and output:
3.6.8

== 2024-09-03 21:20:47,591 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-09-03 21:20:47,591 pythonpackage.py:130 DEBUG Minimal requirement for minor Python version not satisfied: 3.6.8 vs 3.7
== 2024-09-03 21:20:47,591 pythonpackage.py:159 DEBUG Python command '/usr/bin/python3' does not satisfy version requirements (maj: 3, min: 7), moving on
== 2024-09-03 21:20:47,611 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:126 in __init__): Failed to pick Python command to use (at easybuild/easyblocks/generic/pythonbundle.py:115 in prepare_step)
== 2024-09-03 21:20:47,612 build_log.py:267 INFO ... (took < 1 sec)
== 2024-09-03 21:20:47,615 config.py:700 DEBUG software install path as specified by 'installpath' and 'subdir_software': /home/it4i-casparl/.local/easybuild/software
== 2024-09-03 21:20:47,615 filetools.py:2013 INFO Removing lock /home/it4i-casparl/.local/easybuild/software/.locks/_home_it4i-casparl_.local_easybuild_software_ReFrame_4.6.2.lock...
== 2024-09-03 21:20:47,616 filetools.py:383 INFO Path /home/it4i-casparl/.local/easybuild/software/.locks/_home_it4i-casparl_.
local_easybuild_software_ReFrame_4.6.2.lock successfully removed.
== 2024-09-03 21:20:47,616 filetools.py:2017 INFO Lock removed: /home/it4i-casparl/.local/easybuild/software/.locks/_home_it4i-casparl_.local_easybuild_software_ReFrame_4.6.2.lock
== 2024-09-03 21:20:47,616 easyblock.py:4285 WARNING build failed (first 300 chars): Failed to pick Python command to use

Although it is still cryptic (it doesn't specifically complain about the version unless you run with --debug), at least behavior is now the same as for PythonPackage. And: at least EasyBuild doesn't just try to run a python command that doesn't exist on the system.

…mismatch with the requirements

casparvl · 2024-09-03T19:59:06Z

Test build in easybuilders/easybuild-easyconfigs#21307 (comment) (ended up in the other PR... but still shows it in the gist all right: https://gist.github.com/casparvl/07e98bf3cb8c43ee1a6f34a38b1eed6d . It now

errors out also for pythonbundle and
the error is more clear on the fact that there is a requirement for the python versions in the EasyConfigs (and that that might be why no valid python command is found)

…les#r1745639292

…None)

… is used when using system python

…ver and req_py_minver are guaranteed to be defined

bedroge · 2024-09-06T07:25:27Z

Test report by @bedroge

Overview of tested easyconfigs (in order)

FAIL (build issue) ReFrame-4.6.2.eb (partial log available at https://gist.github.com/bedroge/d99ddfdd92ae3a5beac6b7d85835ef41)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
interactive1 - Linux Rocky Linux 8.9, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.6.8
See https://gist.github.com/bedroge/658cf22fd1c22abb32ca8337a5027899 for a full test report.

This is an expected failure to demonstrate that the added functionality works: I tried building the ReFrame version from easybuilders/easybuild-easyconfigs#21307 using Python 3.6, while it needs at least 3.7.

== FAILED: Installation ended unsuccessfully (build directory: /dev/shm/p251204/ReFrame/4.6.2/system-system): build failed (first 300 chars): Failed to pick 
python command that satisfies requirements in the EasyConfigs (req_py_majver = 3, req_py_minver = 7) (took 0 secs)

bedroge · 2024-09-06T07:38:06Z

Test report by @bedroge

Overview of tested easyconfigs (in order)

SUCCESS pyBigWig-0.3.22-gfbf-2023a.eb
SUCCESS PyYAML-6.0-GCCcore-12.2.0.eb
SUCCESS sktime-0.25.0-gfbf-2023a.eb
SUCCESS NiBabel-4.0.2-foss-2022a.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/390dbacfe8775eb02397ca838813083c for a full test report.

bedroge · 2024-09-06T07:50:03Z

Test report by @bedroge

Overview of tested easyconfigs (in order)

SUCCESS ReFrame-4.6.2.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/ddf841dd1329ad5b5c6f7ac561446ff9 for a full test report.

This is the other easyconfig from easybuilders/easybuild-easyconfigs#21307, which requires at least Python 3.7, and 3.12 is being used. So, it also works as expected.

bedroge

Lgtm

Make sure to raise an error if pick_python_cmd returns False

67a3fa2

casparvl mentioned this pull request Sep 3, 2024

{devel}[SYSTEM] Reframe v4.6.2 easybuilders/easybuild-easyconfigs#21307

Merged

Caspar van Leeuwen added 4 commits September 3, 2024 21:25

python_cmd doesn't come from self for pythonbundle

32564cc

Raise more clear error message if the potential problem is a version …

4f6bd0d

…mismatch with the requirements

Fix typos

fa64b34

More clear error message for pythonbundle too

9cbe0a9

casparvl mentioned this pull request Sep 4, 2024

allow Python bundles and packages to specify a maximum Python version for the system toolchain #3431

Merged

bedroge reviewed Sep 5, 2024

View reviewed changes

Comment thread easybuild/easyblocks/generic/pythonbundle.py Outdated

Caspar van Leeuwen added 7 commits September 5, 2024 16:33

Fix https://github.com/easybuilders/easybuild-easyblocks/pull/3430/fi…

8b47e7b

…les#r1745639292

Make sure req_py_majver and req_py_minver are always initialized (to …

72c280c

…None)

Fix where the error is raised, it should only be in the codepath that…

5c4d3b6

… is used when using system python

No need to initialize, now that the error is raised in the if-else block

3c48b89

Fix indent

54beed9

Pull complicated error inside the 'if python' block, where req_py_maj…

0e6f795

…ver and req_py_minver are guaranteed to be defined

Remove stray whiteline

d9358c4

bedroge added 2 commits September 6, 2024 09:30

Capitalize Python everywhere, to keep things consistent

71c4adc

Capitalize Python

df1148c

bedroge changed the title ~~Make sure to raise an error if pick_python_cmd returns False~~ make sure to raise an error if pick_python_cmd returns False for Python bundles/packages Sep 6, 2024

bedroge approved these changes Sep 6, 2024

View reviewed changes

bedroge added this to the release after 4.9.2 milestone Sep 6, 2024

bedroge added the bug fix label Sep 6, 2024

bedroge merged commit 020ca79 into easybuilders:develop Sep 6, 2024

boegel changed the title ~~make sure to raise an error if pick_python_cmd returns False for Python bundles/packages~~ make sure to raise an error if pick_python_cmd returns False for Python bundles/packages Sep 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make sure to raise an error if `pick_python_cmd` returns `False` for Python bundles/packages#3430

make sure to raise an error if `pick_python_cmd` returns `False` for Python bundles/packages#3430
bedroge merged 14 commits intoeasybuilders:developfrom
casparvl:fix_python_req_pythonbundle

casparvl commented Sep 3, 2024 •

edited

Loading

Uh oh!

casparvl commented Sep 3, 2024

Uh oh!

casparvl commented Sep 3, 2024

Uh oh!

Uh oh!

bedroge commented Sep 6, 2024 •

edited

Loading

Uh oh!

bedroge commented Sep 6, 2024

Uh oh!

bedroge commented Sep 6, 2024 •

edited

Loading

Uh oh!

bedroge left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

casparvl commented Sep 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Sep 3, 2024

Uh oh!

casparvl commented Sep 3, 2024

Uh oh!

Uh oh!

bedroge commented Sep 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview of tested easyconfigs (in order)

Uh oh!

bedroge commented Sep 6, 2024

Overview of tested easyconfigs (in order)

Uh oh!

bedroge commented Sep 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview of tested easyconfigs (in order)

Uh oh!

bedroge left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

casparvl commented Sep 3, 2024 •

edited

Loading

bedroge commented Sep 6, 2024 •

edited

Loading

bedroge commented Sep 6, 2024 •

edited

Loading