enforce that hiddendependencies is a subset of dependencies#1078
enforce that hiddendependencies is a subset of dependencies#1078boegel merged 3 commits intoeasybuilders:developfrom
Conversation
|
issue for resolving deps via an existing hidden @ #1079 |
|
Test FAILed. |
There was a problem hiding this comment.
clarify what is allowed in hiddendependencies: the module name, software name, or the whole dependency tuple.
There was a problem hiding this comment.
For now the entries of hiddendependencies are passes through _parse_dependency, so they have to be name/version tuples (and can include suffix/toolchain optionally), like other dependencies in dependencies or builddependencies.
There was a problem hiding this comment.
add this to the docstring (and is there an issue for a more human-friendly version with just the package names?)
|
Test PASSed. |
|
@boegel: Your motivation for this change makes sense to me. Though, frankly speaking, I was never really a friend of these hidden dependencies... They were "invented" as a workaround for the Intel/Clang STL problem in the context of hierarchical modules. And I still believe this should be handled differently in the long run (by "collapsing" the GCC STL into the Intel/Clang install directories -- note: only the STL, not the full compiler). |
|
@geimer: true, support for hidden modules was added in the context of pushing GCC under icc/ifort in the Collapsing GCC into icc/ifort (or Clang) is probably better, agreed, but still needs to be looked into. Afterwards implementing support for hidden modules, with some input by @rtmclay, we realised this wasn't the best approach, but supporting hidden modules is still useful, e.g., as an easy way to hide an installation from users before declaring it production-quality. |
|
@stdweird: remarks fixed, pls rereview? |
|
Test PASSed. |
There was a problem hiding this comment.
it's a bit brute force. do you expect multiple entries to be removed here?
There was a problem hiding this comment.
wel, dependencies should be unique, but this is not enforced anywhere imho
|
@boegel minor remarks |
enforce that hiddendependencies is a subset of dependencies
by request of @stdweird
@ocaisa, @geimer: heads up on this one, since this makes things stricter compared to EasyBuild v1.15.x
We're enforcing here that all entries of
hiddendependenciesare also listed in thedependencieslist.Before EasyBuild resolving dependencies, it will filter out the dependencies that are listed as hidden from the
dependencieslist, so semantics w.r.t. installing missing modules hidden is retained.We want to avoid that contributed easyconfig files contain site-specific decision w.r.t. whether modules should be installed hidden or not.
By ensuring that
hiddendependenciesis a subset ofdependencies, it becomes trivial to make easyconfig files usinghiddendependenciessite-agnostic: simply remove thehiddendependenciesspecification.The
hiddendependencieseasyconfig parameter still has its merit (to indicate to--robotwhich modules must be installed hidden), and will not disappear.Sometime in the future, we will also add support for resolving entries in the
dependencieslist with an existing hidden module, as a fallback in case the module is not available non-hidden. When neither are available, the module will be installed non-hidden via the robot, unless it is included in thehiddendependencieslist (in which case it has to be available as a hidden module).@ocaisa, @geimer: thoughs on this?