don't pass down specific environment variables into submitted jobs#2643
Merged
akesandgren merged 1 commit intoeasybuilders:developfrom Nov 1, 2018
Merged
don't pass down specific environment variables into submitted jobs#2643akesandgren merged 1 commit intoeasybuilders:developfrom
akesandgren merged 1 commit intoeasybuilders:developfrom
Conversation
akesandgren
reviewed
Nov 1, 2018
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Strictly speaking, this change is not backward compatible, but based on the feedback in #2632 this is actually more of a bugfix rather than just a change in behavior...
The behavior with this change included basically corresponds EasyBuild versions prior to 3.7.0 (which were not compatible yet with GC3Pie 2.5.0), since with GC3Pie versions prior to 2.5.0 the environment variables that were specified were basically totally ignored (at least when submitting to SLURM). Am I getting that right @riccardomurri?
Currently, EasyBuild passes down all
$EASYBUILD_*environment variables to ensure that the EasyBuild configuration in the submitted job is correct, but that's actually useless since the entire configuration is also passed down via command line options in the submitted job (which override the$EASYBUILD_*environment variables)...Passing down of
$PYTHONPATH&$MODULEPATHis an attempt to make sure that EasyBuild & loaded modules are passed down into the submitted job as well. That's a pretty broken attempt though... For EasyBuild itself, at least$PATHmust be passed down too (cfr. #2632), and for loaded modules several other environment variables may be relevant as well.So, it's better to simply not pass down anything at all, since this provides (in some sense) more control to the user of
eb --jobw.r.t. setting up the job environment (which can also be done via a.bashrclogin script, for example).cc @akesandgren