Skip to content

adding env var to force EasyBuild version during bootstrap#2382

Merged
akesandgren merged 1 commit intoeasybuilders:developfrom
FredHutch:bootstrap_force_version
Sep 17, 2018
Merged

adding env var to force EasyBuild version during bootstrap#2382
akesandgren merged 1 commit intoeasybuilders:developfrom
FredHutch:bootstrap_force_version

Conversation

@bmcgough
Copy link
Copy Markdown
Contributor

@bmcgough bmcgough commented Jan 17, 2018

We are automating EasyBuild within Docker and need to get a predictable EasyBuild version when bootstrapping.

@boegelbot
Copy link
Copy Markdown

boegelbot commented Jan 17, 2018

Travis test report: 10/10 runs failed - see https://travis-ci.org/easybuilders/easybuild-framework/builds/330108659

Only showing partial log for 1st failed test suite run 1625.1;
full log at https://travis-ci.org/easybuilders/easybuild-framework/jobs/330108660

...
WARNING: xmlrunner module not available, falling back to using unittest...

......Deprecated functionality, will no longer work in v10000000000000: almost kaput; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information
Deprecated functionality, will no longer work in v10000000000000: almost kaput; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information
Deprecated functionality, will no longer work in v10000000000000: almost kaput; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information
Deprecated functionality, will no longer work in v10000000000000: almost kaput; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information
Deprecated functionality, will no longer work in v10000000000000: almost kaput; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information
..........Skipping test_check_style, since pep8 is not available
...........Skipping test_empty_pr, no GitHub token available?
.......Skipping test_from_pr, no GitHub token available?
.Skipping test_from_pr, no GitHub token available?
.Skipping test_from_pr_x, no GitHub token available?
..................Skipping test_merge_pr, no GitHub token available?
...Skipping test_new_pr_delete, no GitHub token available?
.Skipping test_new_pr_dependencies, no GitHub token available?
.Skipping test_new_update_pr, no GitHub token available?
.....Skipping test_preview_pr, no GitHub token available?
....Skipping test_review_pr, no GitHub token available?
.........== backup of existing module file stored at /tmp/eb-Nc5em_/eb-7O8rVO/eb-aPslWR/tmpqW5fY_/modules/all/toy/0.0.lua.bak_20180117230235
== comparing module file with backup /tmp/eb-Nc5em_/eb-7O8rVO/eb-aPslWR/tmpqW5fY_/modules/all/toy/0.0.lua.bak_20180117230235; no differences found
................(skipping SvnRepository test)
..................................................Deprecated functionality, will no longer work in v4.0: Named argument 'default_fallback' for get_easyblock_class is deprecated, use 'error_on_missing_easyblock' instead; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information
......................................................................................................Failed to download https://pypi.python.org/simple/nosuchpackageonpypiever to determine available PyPI URLs for nosuchpackageonpypiever
...Skipping timeout test in test_download_file (working offline)
...............................................Skipping test_from_pr, no GitHub token available?
................== installing extension ext1  (1/1)...
..................== installing extension ext1  (1/1)...
....Skipping test_download_repo, no GitHub token available?
.Skipping test_fetch_easyconfigs_from_pr, no GitHub token available?
.Skipping test_fetch_latest_commit_sha, no GitHub token available?
.Skipping test_find_easybuild_easyconfig, no GitHub token available?
..Skipping test_install_github_token, no GitHub token available?
.Skipping test_read, no GitHub token available?
.Skipping test_read_api, no GitHub token available?
.Skipping test_validate_github_token, no GitHub token available?
.Skipping test_walk, no GitHub token available?
..........................................
WARNING: Found one or more non-allowed loaded (EasyBuild-generated) modules in current environment:
* toy/0.0-external-deps

This is not recommended since it may affect the installation procedure(s) performed by EasyBuild.

To make EasyBuild allow particular loaded modules, use the --allow-loaded-modules configuration option.
To specify action to take when loaded modules are detected, use --detect-loaded-modules={error,ignore,purge,unload,warn}.

See http://easybuild.readthedocs.io/en/latest/Detecting_loaded_modules.html for more information.


WARNING: Found one or more non-allowed loaded (EasyBuild-generated) modules in current environment:
* toy/0.0-external-deps

This is not recommended since it may affect the installation procedure(s) performed by EasyBuild.

To make EasyBuild allow particular loaded modules, use the --allow-loaded-modules configuration option.
To specify action to take when loaded modules are detected, use --detect-loaded-modules={error,ignore,purge,unload,warn}.

See http://easybuild.readthedocs.io/en/latest/Detecting_loaded_modules.html for more information.


WARNING: Found one or more non-allowed loaded (EasyBuild-generated) modules in current environment:
* toy/0.0-external-deps

This is not recommended since it may affect the installation procedure(s) performed by EasyBuild.

To make EasyBuild allow particular loaded modules, use the --allow-loaded-modules configuration option.
To specify action to take when loaded modules are detected, use --detect-loaded-modules={error,ignore,purge,unload,warn}.

See http://easybuild.readthedocs.io/en/latest/Detecting_loaded_modules.html for more information.

.....== backup of existing module file stored at /tmp/eb-Nc5em_/eb-7O8rVO/eb-c8wR2X/tmpbwcdan/modules/all/toy/0.0.lua.bak_20180117230713
== comparing module file with backup /tmp/eb-Nc5em_/eb-7O8rVO/eb-c8wR2X/tmpbwcdan/modules/all/toy/0.0.lua.bak_20180117230713; no differences found
................== backup of existing module file stored at /tmp/eb-Nc5em_/eb-7O8rVO/eb-MxWheO/tmpWvI9U5/modules/all/toy/0.0.lua.bak_20180117230806
== comparing module file with backup /tmp/eb-Nc5em_/eb-7O8rVO/eb-MxWheO/tmpWvI9U5/modules/all/toy/0.0.lua.bak_20180117230806; no differences found
..............................................................................................................................
----------------------------------------------------------------------
Ran 531 tests in 511.635s

OK

travis_time:end:1a3c3af0:start=1516230098670664160,finish=1516230611333477894,duration=512662813734
�[0K
�[32;1mThe command "python -O -m test.framework.suite" exited with 0.�[0m
travis_time:start:034b422e
�[0K$ EB_BOOTSTRAP_VERSION=$(grep '^EB_BOOTSTRAP_VERSION' $TRAVIS_BUILD_DIR/easybuild/scripts/bootstrap_eb.py | sed 's/[^0-9.]//g')

travis_time:end:034b422e:start=1516230611341647436,finish=1516230611352231579,duration=10584143
�[0K
�[32;1mThe command "EB_BOOTSTRAP_VERSION=$(grep '^EB_BOOTSTRAP_VERSION' $TRAVIS_BUILD_DIR/easybuild/scripts/bootstrap_eb.py | sed 's/[^0-9.]//g')" exited with 0.�[0m
travis_time:start:1cc11f36
�[0K$ EB_BOOTSTRAP_SHA256SUM=$(sha256sum $TRAVIS_BUILD_DIR/easybuild/scripts/bootstrap_eb.py | cut -f1 -d' ')

travis_time:end:1cc11f36:start=1516230611359412977,finish=1516230611370804022,duration=11391045
�[0K
�[32;1mThe command "EB_BOOTSTRAP_SHA256SUM=$(sha256sum $TRAVIS_BUILD_DIR/easybuild/scripts/bootstrap_eb.py | cut -f1 -d' ')" exited with 0.�[0m
travis_time:start:12ae1b72
�[0K$ EB_BOOTSTRAP_FOUND="$EB_BOOTSTRAP_VERSION $EB_BOOTSTRAP_SHA256SUM"

travis_time:end:12ae1b72:start=1516230611378525295,finish=1516230611384341105,duration=5815810
�[0K
�[32;1mThe command "EB_BOOTSTRAP_FOUND="$EB_BOOTSTRAP_VERSION $EB_BOOTSTRAP_SHA256SUM"" exited with 0.�[0m
travis_time:start:29d3a522
�[0K$ EB_BOOTSTRAP_EXPECTED="20170808.01 7a22faf5ee9d249807fd883e435008fb84c11129425ec09d3cc76918b36ef3d5"

travis_time:end:29d3a522:start=1516230611391815483,finish=1516230611397155988,duration=5340505
�[0K
�[32;1mThe command "EB_BOOTSTRAP_EXPECTED="20170808.01 7a22faf5ee9d249807fd883e435008fb84c11129425ec09d3cc76918b36ef3d5"" exited with 0.�[0m
travis_time:start:01996080
�[0K$ test "$EB_BOOTSTRAP_FOUND" = "$EB_BOOTSTRAP_EXPECTED" || (echo "Version check on bootstrap script failed $EB_BOOTSTRAP_FOUND" && exit 1)
Version check on bootstrap script failed 20170808.01 eee0d0c75f063c2fca25cdbe56dac1c77e286deea092dec580f52e858e8afd44

(bleep, bloop, I'm just a bot, please talk to my owner @boegel if you notice you me acting stupid)

akesandgren
akesandgren requested changes Aug 27, 2018
View reviewed changes
Comment thread easybuild/scripts/bootstrap_eb.py
else:
# install meta-package easybuild from PyPI
cmd.append('easybuild')
if forcedversion is not None:
Copy link
Copy Markdown
Contributor

@akesandgren akesandgren Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if forcedversion: should be enough here and is easier to read.

Comment thread easybuild/scripts/bootstrap_eb.py

forcedversion = EASYBUILD_BOOTSTRAP_FORCE_VERSION
if EASYBUILD_BOOTSTRAP_FORCE_VERSION is not None:
info("Forcing specified version %s..." % forcedversion)
Copy link
Copy Markdown
Contributor

@akesandgren akesandgren Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If forcedversion isn't use further down, then drop it and use EASYBUILD_BOOTSTRAP_FORCE_VERSION in the info(...).
If it is used then changing the if EASYBUILD.... to
if forcedversion:
makes it easier to read.

Copy link
Copy Markdown
Member

@boegel boegel Aug 27, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akesandgren Well, it is used further down, since it's being passed to stage1.

So, your second suggestion is right on the spot: @bmcgough please use if forcedversion: above

@boegel
Copy link
Copy Markdown
Member

boegel commented Aug 27, 2018

@bmcgough You'll need to update the version of the bootstrap script in bootstrap_eb.py and then also in .travis.yml where you'll also need to update the SHA256 checksum of bootstrap_eb.py to make Travis happy (we do this to ensure the version of the bootstrap script is bumped whenever any changes are made to it).

@boegel boegel added this to the 3.7.0 milestone Aug 27, 2018
@boegel
Copy link
Copy Markdown
Member

boegel commented Aug 27, 2018

Wow, just noticed how long this has been open...

I'm very sorry for not getting back to you on this @bmcgough, since it's a useful contribution, it somehow slipped through the cracks... :(

@boegel boegel mentioned this pull request Sep 16, 2018
Merged
@akesandgren akesandgren merged commit 4a23cfc into easybuilders:develop Sep 17, 2018
@migueldiascosta migueldiascosta mentioned this pull request Sep 24, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel left review comments

@akesandgren akesandgren akesandgren requested changes

Assignees

No one assigned

Labels

enhancement

Projects

None yet

Milestone

3.7.0

Development

Successfully merging this pull request may close these issues.

4 participants

@bmcgough @boegelbot @boegel @akesandgren