added script for bootstrapping EasyBuild by boegel · Pull Request #531 · easybuilders/easybuild-framework

boegel · 2013-03-11T22:40:41Z

We should set up a dedicated Jenkins test for this, so we can rely on it to keep on working in the future...

…ldMeta en Easy_install easyblocks

… in Python; one issue remain with final EasyBuild installation (see easy-install.pth file in pylibdir)

…work into bootstrap

Bootstrap

…th easy_install, lower verbosity

…uild with EasyBuild

…l/easybuild

…ome cases (e.g. with a Tcl (non-C) environment modules installation)

…d to lib as a Python lib dir

stdweird · 2013-03-12T07:21:16Z

is this the same distutils.sysconfig stuff?

No, it's more complex than that actually... distutils.sysconfig only gives one or the other (lib or lib64), but apparently in some cases lib is used even though distutils.sysconfig returns a lib64 path. Hardcoding somewhere probably, so we really need to consider both.
This popped up during the EasyBuild hackathon in Cyprus when testing eb on JUROPA.

boegel · 2013-03-21T22:36:49Z

Fixed remarks, please reconsider.

stdweird · 2013-03-22T05:34:04Z

hardcoded modulecmd, not all sites have this. need script (also in EB) to look for propoer command. if you're not fixing it, open an issue.

Won't fix that now, since I need to figure out a good way to handle this.

Added a note w.r.t. the bootstrap script in #532.

stdweird · 2013-03-22T05:40:58Z

@boegel some small remarks, nothing really blocking though.

boegel · 2013-03-22T08:26:04Z

Adressed remarks, please reconsider. The v1.3 and up codepaths can't be tested now, since v1.3 is not released yet.

We'll need to retest the bootstrap after the v1.3 release, I made a note of that in https://github.com/hpcugent/easybuild/wiki/Release-process.

boegel · 2013-03-22T08:26:27Z

Argh, wait, testing failed, grr. Will fix that first.

… up more recent locally installed EasyBuild packages

boegel · 2013-03-22T10:13:02Z

Fixed the issue I was swearing about.
Please recheck, and test on your end, before I merge it in please.

stdweird · 2013-03-22T10:19:43Z

@boegel bootstrap script needs a version of some sort

boegel · 2013-03-22T10:27:31Z

@stdweird: You mean, a constant like EB_BOOTSTRAP_VERSION that we can easily forget to update? ;)

stdweird · 2013-03-22T10:28:59Z

as opposed to 'hey i tried the bootstrap that someone mailed but it doesn't work' and then you try to figure out what version it was since the other side is sure he dowmloaded it from the correct location?

stdweird · 2013-03-22T10:30:21Z

insert the check if this is run as root that is in main.py

… running as root (refs easybuilders#530)

stdweird · 2013-03-22T10:48:03Z

root check works

new bug in sudo env

== temporary log file in case of crash /tmp/easybuild-PiVCzK.log
Traceback (most recent call last):
  File "bootstrap_eb.py", line 533, in <module>
    main()
  File "bootstrap_eb.py", line 364, in main
    stage2(tmpdir, versions, install_path)
  File "bootstrap_eb.py", line 306, in stage2
    easybuild_main()
  File "/tmp/tmpOp4gPP/eb_stage1/lib/python2.6/site-packages/easybuild_framework-1.2.0-py2.6.egg/easybuild/main.py", line 256, in main
    os.chdir(os.environ['PWD'])
  File "/usr/lib64/python2.6/UserDict.py", line 22, in __getitem__
    raise KeyError(key)
KeyError: 'PWD'

use os.getcwd() instead os.environ['PWD'] ? or save os.getcwd() in some variable first

boegel · 2013-03-22T10:51:54Z

added check to disallow root, obtain global eb version in a more strict way, such that local easybuild-framework installations are ignored (i.e., use python -S)

Tested on 3 or 4 totally different systems, I'd say it's good to go now...

stdweird · 2013-03-22T10:52:42Z

@boegel try as this

[root@freeipa-sl64-2 tmp]# sudo -u stdweird python bootstrap_eb.py /tmp

boegel · 2013-03-22T10:56:55Z

@stdweird: I consider this a minor issue, because this also warrants the check in EasyBuild itself to be enhanced... Please open an issue for this, I won't fix it now.

added script for bootstrapping EasyBuild

stdweird · 2013-03-22T11:04:11Z

see #551 for the sudo issue

boegel and others added 20 commits February 11, 2013 17:32

add draft of bootstrap script, which still includes hacks for EasyBui…

041c791

…ldMeta en Easy_install easyblocks

Merge branch 'develop' into bootstrap

6d93a44

clean up bootstrap script, get rid of most os.system calls, do it all…

a1f18c2

… in Python; one issue remain with final EasyBuild installation (see easy-install.pth file in pylibdir)

fix shebang

d894281

Merge branch 'bootstrap' of https://github.com/boegel/easybuild-frame…

60777f8

…work into bootstrap

commit remarks

4cf8728

Merge pull request #4 from stdweird/bootstrap

f37505a

Bootstrap

fix problems/remarks, seems like it's working like it should now...

f5c1659

remove unneeded code, force specifying install path, use --upgrade wi…

6ab491f

…th easy_install, lower verbosity

change to tmp dir earlier to avoid leaving stuff behind

6497189

stop relying on which to figure out if modulecmd is available

15f6d73

make sure we're using the distribute setuptools when installing EasyB…

e4a0a33

…uild with EasyBuild

add copyright header

5213a29

use more compatible output redirection

937c512

set EASYBUILDPREFIX, don't leave stuff behind in /Users/kehoste/.loca…

3cb245a

…l/easybuild

clear PYTHONPATH again before stage2

2dd8a99

add more info messages w.r.t. install paths and EasyBuild configuration

12ed0d8

make sure we get an absolute path for the install path

19fcb89

use 'help' instead of '-H' because the latter seems to be broken in s…

a72f41c

…ome cases (e.g. with a Tcl (non-C) environment modules installation)

fix bootstrap script for systems that (sometimes) use lib64 as oppose…

9009f86

…d to lib as a Python lib dir

boegel mentioned this pull request Mar 11, 2013

Fix for bootstrap_eb.py easybuilders/easybuild#4

Closed

stdweird reviewed Mar 12, 2013
View reviewed changes

fixed remarks

f3b3970

stdweird reviewed Mar 22, 2013
View reviewed changes

thoroughly clean sys.path before step 0, we really don't want to pick…

d54e352

… up more recent locally installed EasyBuild packages

stdweird reviewed Mar 22, 2013
View reviewed changes

Comment thread easybuild/scripts/bootstrap_eb.py

Copy link
Copy Markdown

Contributor

stdweird Mar 22, 2013

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

insert the check if this is run as root that is in main.py

determine version using main.py directly (stricter control), disallow…

0d1f17e

… running as root (refs easybuilders#530)

boegel mentioned this pull request Mar 22, 2013

bootstrap script should fail early when run as root #530

Closed

python -m doesn't work with Python 2.4, so specify full path of main.py

27ea45a

boegel added a commit that referenced this pull request Mar 22, 2013

Merge pull request #531 from boegel/bootstrap

8c9e499

added script for bootstrapping EasyBuild

boegel merged commit 8c9e499 into easybuilders:develop Mar 22, 2013

Conversation

boegel commented Mar 11, 2013

Uh oh!

stdweird Mar 12, 2013

Choose a reason for hiding this comment

Uh oh!

boegel Mar 21, 2013

Choose a reason for hiding this comment

Uh oh!

boegel commented Mar 21, 2013

Uh oh!

stdweird Mar 22, 2013

Choose a reason for hiding this comment

Uh oh!

boegel Mar 22, 2013

Choose a reason for hiding this comment

Uh oh!

stdweird commented Mar 22, 2013

Uh oh!

boegel commented Mar 22, 2013

Uh oh!

boegel commented Mar 22, 2013

Uh oh!

boegel commented Mar 22, 2013

Uh oh!

stdweird commented Mar 22, 2013

Uh oh!

boegel commented Mar 22, 2013

Uh oh!

stdweird commented Mar 22, 2013

Uh oh!

stdweird Mar 22, 2013

Choose a reason for hiding this comment

Uh oh!

stdweird commented Mar 22, 2013

Uh oh!

boegel commented Mar 22, 2013

Uh oh!

stdweird commented Mar 22, 2013

Uh oh!

boegel commented Mar 22, 2013

Uh oh!

stdweird commented Mar 22, 2013

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants