Skip to content

tweak 'eb' wrapper script to correctly handle errors when python command being considered fails to run#4019

Merged
boegel merged 3 commits intoeasybuilders:developfrom
Poshi:develop
Jul 6, 2022
Merged

tweak 'eb' wrapper script to correctly handle errors when python command being considered fails to run#4019
boegel merged 3 commits intoeasybuilders:developfrom
Poshi:develop

Conversation

@Poshi
Copy link
Copy Markdown
Contributor

@Poshi Poshi commented Jun 6, 2022

There are setups (like ours), where the python command may not load due to missing dynamic libraries in the path. In those cases, the run is like:

$ /home/group/user/venv/bin/python -V
/home/group/user/venv/bin/python: error while loading shared libraries: libpython3.9.so.1.0: cannot open shared object file: No such file or directory

This causes an issue in the lines 73-75 of the eb script:

pyver=`$python_cmd -V 2>&1 | cut -f2 -d' '`

This execution returns the shown message thru stderr, stderr is redirected to stdout and it is cut to take the second space-separated field: the "error" word. So the $pyver is "error"!
This ends up in a $pyver_maj == "error" and a $pyver_min == "".
The next line, the one that compares the versions with the minimum ones, fails with:

/software/crgadm/software/EasyBuild/4.5.4/bin/eb: line 73: [: error: integer expression expected
/software/crgadm/software/EasyBuild/4.5.4/bin/eb: line 76: [: error: integer expression expected

because "error" is not an integer.

The solution goes to test if the command can be run, not just if the command exists. So I added a quick execution to test for its validity.
BTW, I think we can get rid of the original test command -v, but I let it, just in case.
This new version checks if the command exist and if the command can be run and return a version number. If the command does not exist, both tests will fail, if it works, both tests will work, and if the command exist but it does not work, only second test will fail. So I would only leave the second test. Up to you in a further commit :-)

@jfgrimm jfgrimm added this to the 4.x milestone Jun 6, 2022
jfgrimm
jfgrimm previously approved these changes Jun 7, 2022
Copy link
Copy Markdown
Member

@jfgrimm jfgrimm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@boegel shall we squeeze this into the 4.5.5 release?

@boegel boegel modified the milestones: 4.x, next release (4.5.6?) Jun 8, 2022
Comment thread eb Outdated
@boegel
Copy link
Copy Markdown
Member

boegel commented Jun 8, 2022

LGTM @boegel shall we squeeze this into the 4.5.5 release?

That ship has just sailed (EasyBuild v4.5.5 has just been released), and I've learned the hard way to not include potentially disruptive changes at the last minute...

The changes here look fine, but I'd rather have this battle tested in develop for a while, since shipping a broken eb command would be quite embarrassing :)

@boegel boegel changed the title Fix: errors when python command fails to run tweak eb wrapper script to correctly handle errors when python command fails to run Jun 8, 2022
Comment thread eb
# (using 'command -v', since 'which' implies an extra dependency)
command -v $python_cmd &> /dev/null
if [ $? -eq 0 ]; then
if { command -v "${python_cmd}" && "${python_cmd}" -V; } &> /dev/null; then
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your analysis that just the "${python_cmd}" -V should be sufficient, but on the other hand I don't think there's much to gain by dropping the command -v in terms of speed anyway

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, there's no speed gain (just some microsecond). But the command could be simplified from:
if { command -v "${python_cmd}" && "${python_cmd}" -V; } &> /dev/null; then
to
if "${python_cmd}" -V &> /dev/null; then
which looks cleaner to me. But up to you. This is neither a performance issue nor a latent error.

@jfgrimm
Copy link
Copy Markdown
Member

jfgrimm commented Jun 8, 2022

LGTM @boegel shall we squeeze this into the 4.5.5 release?

That ship has just sailed (EasyBuild v4.5.5 has just been released), and I've learned the hard way to not include potentially disruptive changes at the last minute...

The changes here look fine, but I'd rather have this battle tested in develop for a while, since shipping a broken eb command would be quite embarrassing :)

fair point

Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've battle tested this a bit in various scenarios, and didn't run into any trouble, so looks good to me.
Thanks a lot @Poshi!

@boegel boegel changed the title tweak eb wrapper script to correctly handle errors when python command fails to run tweak eb wrapper script to correctly handle errors when python command being considered fails to run Jul 6, 2022
@boegel boegel merged commit c1e5196 into easybuilders:develop Jul 6, 2022
@boegel boegel changed the title tweak eb wrapper script to correctly handle errors when python command being considered fails to run tweak 'eb' wrapper script to correctly handle errors when python command being considered fails to run Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants