list build dependencies *before* runtime dependencies, so dependencies listed via multi_deps are loaded first in toolchain environment#2839
Merged
bartoldeman merged 2 commits intoeasybuilders:developfrom Apr 24, 2019
Conversation
…s listed via multi_deps are loaded first...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While working on an easyconfig file that required
SciPy-bundleas a dependency and where multiplePythonversions were specified viamulti_deps, I noticed a problem we have overlooked in #2828.When
Pythonis included inmulti_deps, the installation procedure is performed iteratively, once for every listedPythonversion. Internally, this is done viabuilddependencies(to ensure no (strict) runtime dependency on a particularPythonversion is introduced in the generated module file).If this is combined with a dependency like
SciPy-bundleby which aPythonversion gets loaded by default if none is loaded yet (sincemulti_deps_load_defaultwas enabled when installingSciPy-bundle), a version conflict arises when theSciPy-bundlemodule (which in turns loads the module forPython3.7.2(for example)) was loaded first, and a(nother)Pythonversion was loaded afterwards...For example, with:
the problem occurs when the
Python/2.7.15-GCCcore-8.2.0module is being loaded for the 2nd iteration of the installation, since at that point theSciPy-bundle/2019.03-foss-2019amodule was already loaded, which in turns resulted in loadingPython/3.7.2-GCCcore-8.2.0by default since noPythonmodule was loaded yet at this point...The problem is easily fixed by simply switching the order of build dependencies vs (runtime) dependencies to ensure that build dependencies are loaded first (after the toolchain is loaded)...
In practice this most likely rarely matters in contexts where
multi_depsis not involved, so this change should be safe to make.We ran into a similar issue with the sanity check step while implementing the support for
multi_deps_load_defaultin #2828, see https://github.com/easybuilders/easybuild-framework/pull/2828/files#diff-a1bcbbc1aab6b59688b219f298fddb56R1344