take into account custom configuration options specified in easystack file + drop support for easystack files using 'software' top-level key#4057
Merged
boegel merged 57 commits intoeasybuilders:developfrom Nov 23, 2022
Conversation
…file, each time reconfiguring EasyBuild with EasyConfig-specific options that might be listed in the EasyStack file
…fic arguments from the EasyStack file. This commit should contain a functional implementation of the option support for EasyStack files. Now, we only need to clean it up...
… Probably need to pass more args there...
casparvl
commented
Sep 19, 2022
…nfiguration when reconfigure=True
casparvl
commented
Sep 19, 2022
casparvl
commented
Sep 19, 2022
boegel
reviewed
Sep 24, 2022
boegel
requested changes
Oct 12, 2022
…at way we can limit to only one 'if options.easystack'
boegel
reviewed
Oct 12, 2022
boegel
requested changes
Oct 12, 2022
…e we _used_ to have a clean_exit() call. Right after the call to rest_of_main we call clean_exit anyway.
boegel
requested changes
Oct 12, 2022
boegel
requested changes
Oct 12, 2022
… printing of the warning
boegel
requested changes
Oct 12, 2022
boegel
requested changes
Oct 12, 2022
boegel
requested changes
Oct 12, 2022
boegel
requested changes
Oct 12, 2022
… keyword alltogether
… keyword alltogether
boegel
reviewed
Oct 21, 2022
| run_hook(START, hooks) | ||
|
|
||
| if modtool is None: | ||
| # TODO: insert fast loop that validates if all command line options are valid. If there are errors in options, |
Member
There was a problem hiding this comment.
I opened an issue to follow up on this aspect: #4103
…syconfig caches in process_easystack
10bbb3c to
292498f
Compare
Contributor
Author
|
Todo, update documentation: https://github.com/easybuilders/easybuild/blob/develop/docs/Easystack-files.rst edit (by @boegel): see easybuilders/easybuild#832 |
casparvl
added a commit
to casparvl/easybuild
that referenced
this pull request
Oct 21, 2022
… + remove duplicate blank line in easystack test suite
Member
|
Merging #3749 caused a merge conflict here, which I've fixed just now. I'll propose some tweaks to the docs update in easybuilders/easybuild#832, so we can get this merged... |
…omly changing order of keys in older Python versions
boegel
approved these changes
Nov 23, 2022
This was referenced Nov 23, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #4052
After discussion with @boegel we've decided that the the approach in #4052 is not maintainable. By pushing the loop over the entries in an EasyStack file to
main.py, and by basically re-executing part ofmain.pyfor each item in the EasyStack file, we can properly reinitialize the configuration options for each item (if there are EasyConfig-specific options defined). This has the advantage that we can reuse the existing machinery of things likeset_up_configuration.It also means that EasyBuild builds a dependency tree per item in the EasyStack file. On the one hand, that is desirable since two items might have different
--from-prarguments, and thus the same dependencyA.ebmight in theory resolve to two different PRs depending on the options passed for that item in the EasyStack file. The downside is that since we don't have a single dependency graph, we cannot e.g. submit a single set of jobs with--job: we'll need to make sure EasyBuild becomes aware of the order in the EasyStack file, and make sure that jobs for later items wait for all jobs of earlier items. That's something we'll implement later though...Note that this is really a work in progress, a lot more functions need to be put in place before this properly works...