enhance bootstrap to auto-skip stage 0 in case a suitable setuptools is already available by boegel · Pull Request #1946 · easybuilders/easybuild-framework

boegel · 2016-10-06T12:09:37Z

No description provided.

…t enough

boegel · 2016-10-07T17:00:44Z

@riccardomurri up for testing this (w.r.t. #1887)?

riccardomurri · 2016-10-14T08:54:14Z

Installation failed on a (almost) vanilla Ubuntu 16.04::

ubuntu@eb-test:~$ wget https://raw.githubusercontent.com/boegel/easybuild-framework/auto_skip_stage0_bootstrap/easybuild/scripts/bootstrap_eb.py
...
2016-10-14 08:48:40 (1.40 MB/s) - 'bootstrap_eb.py' saved [41578/41578]

ubuntu@eb-test:~$ sudo apt install lmod
...
Setting up lmod (5.8-1) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...

ubuntu@eb-test:~$ source /usr/share/lmod/lmod/init/bash

ubuntu@eb-test:~$ python bootstrap_eb.py $HOME/easybuild
[[INFO]] EasyBuild bootstrap script (version 20161006.01, MD5: c91bf277b86f693463f05119f068e4de)
[[INFO]] Found Python 2.7.12 (default, Jul  1 2016, 15:12:24) ; [GCC 5.4.0 20160609]

[[INFO]] Installation prefix /home/ubuntu/easybuild
[[INFO]] Found module command '/usr/share/lmod/lmod/libexec/lmod' via $LMOD_CMD (Lmod), so using it.
[[INFO]] No suitable setuptools installation found, proceeding with stage 0...


[[INFO]] +++ STAGE 0: installing distribute via included (patched) distribute_setup.py...

Downloading http://hpcugent.github.io/easybuild/files/distribute-0.6.49-patched1.tar.gz
Extracting in /tmp/tmpTHecFs
Now working in /tmp/tmpTHecFs/distribute-0.6.49
Installing Distribute
[[INFO]] Installed setuptools version 0.6 (/tmp/tmpMEOFQY/lib/python2.7/site-packages/distribute-0.6.49-py2.7.egg/setuptools/__init__.pyc)


[[INFO]] +++ STAGE 1: installing EasyBuild in temporary dir with easy_install...

[[INFO]] installing EasyBuild with 'easy_install --quiet --upgrade --prefix=/tmp/tmpMEOFQY/eb_stage1 easybuild'
[[INFO]] running post install command 'easy_install --upgrade --prefix=/tmp/tmpMEOFQY/eb_stage1 vsc-base'
[[ERROR]] Failed to find right 'easy_install' command!

The starting VM image is a Ubuntu 16.04 with the python package installed
(which is not in the default base image; only python3 is).

boegel · 2016-10-14T09:03:08Z

@riccardomurri can you upload to debug output (EASYBUILD_BOOTSTRAP_DEBUG=1 python bootstrap_eb.py ...) to a gist?

riccardomurri · 2016-10-14T09:03:37Z

Same results on Ubuntu 14.04, with the addition that git is required (see #1962)

riccardomurri · 2016-10-14T09:37:06Z

@boegel here's the DEBUG log of the Ubuntu 14.04 failure:
https://gist.github.com/riccardomurri/a5c803f820cba43a8300e1c19e629faa

riccardomurri · 2016-10-24T14:16:15Z

@boegel here's the DEBUG log of the Ubuntu 14.04 failure:
https://gist.github.com/riccardomurri/a5c803f820cba43a8300e1c19e629faa

…TH when checking whether available easy_install is the right one

boegel · 2016-10-25T08:53:59Z

@riccardomurri Urgh, I indeed missing something... Please try again?

… version check for easy_install, bump version

boegel · 2016-11-03T23:29:08Z

@riccardomurri ping on retesting this?

…provided vsc-install tarball in stage2 if available

boegel · 2016-11-09T09:48:59Z

OK, I think this is good to go, it has been tested extensively in various contexts:

my laptop (OS X)
the OpenHPC build service where it does an offline install, cfr. https://build.openhpc.community/package/show/home:boegel/easybuild
an Arch Linux system that has Python3 as default Python (requires python2 bootstrap_.eb.py $PREFIX)

I would like to also verify that the offline install mode doesn't get broken in Travis, but I'll do that in a follow-up PR.

Thanks for all the feedback on this @riccardomurri!

boegel added 15 commits October 5, 2016 17:59

automagically script stage0 if available setuptools seems to be recen…

e2694ba

…t enough

make sure correct 'easy_install' command is used in stage 2 of bootstrap

aa1254b

bump version

bfa536a

reinstate testing of bootstrap script in Travis config

8a26334

also check vsc-base location

9aa6b44

renable debug mode for bootstrap in Travis config

9396545

moar debug info when checking setuptools

c69d390

moar debug, MOAR

43b5f0f

prepend location of setuptools installation to sys.path

2bd5785

replace easy_install.main by run_easy_install

b3df30c

disable debug mode for bootstrap in Travis config

f4d2ed7

bump version of bootstrap script

85d7370

fix remark

fcab541

Merge branch 'develop' into auto_skip_stage0_bootstrap

34ceff3

clean up check_easy_install_cmd

36d0b4b

boegel added this to the v3.0 milestone Oct 6, 2016

This was referenced Oct 6, 2016

bootstrap script fails on Ubuntu 16.04 #1887

Closed

bootstrap fails on Travis with Python 2.7.12 due to setuptools version requirement #1869

Closed

boegel mentioned this pull request Oct 11, 2016

easybuild (2.9.0) openhpc/ohpc#289

Closed

make sure path to easy_install installed in stage0 is included in $PA…

78e2ddd

…TH when checking whether available easy_install is the right one

boegel added 2 commits October 25, 2016 11:08

remove stray print command

b8d8095

move updating of $PATH after stage1, do local import of StringIO, fix…

8ffdf7a

… version check for easy_install, bump version

boegel added 2 commits November 4, 2016 00:29

Merge branch 'develop' into auto_skip_stage0_bootstrap

57c51f0

Merge branch 'develop' into auto_skip_stage0_bootstrap

0ecd01c

crbaird added a commit to openhpc/ohpc that referenced this pull request Nov 4, 2016

merge changes from easybuilders/easybuild-framework#1946 (#289)

8135414

boegel added 3 commits November 8, 2016 22:01

don't install latest setuptools in stage2 if stage0 was skipped, use …

663a6f2

…provided vsc-install tarball in stage2 if available

deal with sourcepath being potentially empty

b1b7681

call 'easy_install' using correct Python command

e7fed02

boegel merged commit 81f50b8 into easybuilders:develop Nov 9, 2016

boegel deleted the auto_skip_stage0_bootstrap branch November 9, 2016 09:49

boegel added a commit to boegel/easybuild-framework that referenced this pull request Nov 9, 2016

fix bugs in bootstrap script introduced in easybuilders#1946

53a5cdc

This was referenced Nov 9, 2016

fix issues with setuptools provided in step 0 of bootstrap script #1984

Merged

TypeError exception while using bootstrap_eb.py #1983

Closed

boegel mentioned this pull request Dec 10, 2016

offline install fails with 'ImportError: No module named vsc.install.shared_setup' #1926

Closed

boegel mentioned this pull request Nov 5, 2017

Bootstrap script fails on Ubuntu 16.04 #1798

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enhance bootstrap to auto-skip stage 0 in case a suitable setuptools is already available#1946

enhance bootstrap to auto-skip stage 0 in case a suitable setuptools is already available#1946
boegel merged 23 commits intoeasybuilders:developfrom
boegel:auto_skip_stage0_bootstrap

boegel commented Oct 6, 2016

Uh oh!

boegel commented Oct 7, 2016

Uh oh!

riccardomurri commented Oct 14, 2016

Uh oh!

boegel commented Oct 14, 2016

Uh oh!

riccardomurri commented Oct 14, 2016

Uh oh!

riccardomurri commented Oct 14, 2016

Uh oh!

riccardomurri commented Oct 24, 2016

Uh oh!

boegel commented Oct 25, 2016

Uh oh!

boegel commented Nov 3, 2016

Uh oh!

boegel commented Nov 9, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

boegel commented Oct 6, 2016

Uh oh!

boegel commented Oct 7, 2016

Uh oh!

riccardomurri commented Oct 14, 2016

Uh oh!

boegel commented Oct 14, 2016

Uh oh!

riccardomurri commented Oct 14, 2016

Uh oh!

riccardomurri commented Oct 14, 2016

Uh oh!

riccardomurri commented Oct 24, 2016

Uh oh!

boegel commented Oct 25, 2016

Uh oh!

boegel commented Nov 3, 2016

Uh oh!

boegel commented Nov 9, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants