Feature: make dry run output readable (and also of eb --search) by fgeorgatos · Pull Request #781 · easybuilders/easybuild-framework

fgeorgatos · 2013-12-06T23:33:49Z

Here is the why of this:
(sry, but I couldn't really read the other kind of output)

bash-3.2$ export CFGS=/Users/fotis/Desktop/arena/uni.lu/easybuild-easyconfigs/easybuild/easyconfigs
bash-3.2$ eb e/EMBOSS/EMBOSS-6.5.7-goolf-1.4.10.eb --dry-run -r
== temporary log file in case of crash /var/folders/3L/3LkVBznPF7WgIo752F4GjE+++TI/-Tmp-/easybuild-BkOeUk.log
== Dry run: printing build status of easyconfigs and dependencies
 * [ ] $CFGS/g/GCC/GCC-4.7.2.eb    # module: GCC/4.7.2
 * [ ] $CFGS/h/hwloc/hwloc-1.6.2-GCC-4.7.2.eb    # module: hwloc/1.6.2-GCC-4.7.2
 * [ ] $CFGS/o/OpenMPI/OpenMPI-1.6.4-GCC-4.7.2.eb    # module: OpenMPI/1.6.4-GCC-4.7.2
 * [ ] $CFGS/g/gompi/gompi-1.4.10.eb    # module: gompi/1.4.10
 * [ ] $CFGS/o/OpenBLAS/OpenBLAS-0.2.6-gompi-1.4.10-LAPACK-3.4.2.eb    # module: OpenBLAS/0.2.6-gompi-1.4.10-LAPACK-3.4.2
 * [ ] $CFGS/f/FFTW/FFTW-3.3.3-gompi-1.4.10.eb    # module: FFTW/3.3.3-gompi-1.4.10
 * [ ] $CFGS/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-1.4.10-OpenBLAS-0.2.6-LAPACK-3.4.2.eb    # module: ScaLAPACK/2.0.2-gompi-1.4.10-OpenBLAS-0.2.6-LAPACK-3.4.2
 * [ ] $CFGS/g/goolf/goolf-1.4.10.eb    # module: goolf/1.4.10
 * [ ] $CFGS/z/zlib/zlib-1.2.7-goolf-1.4.10.eb    # module: zlib/1.2.7-goolf-1.4.10
 * [ ] $CFGS/n/ncurses/ncurses-5.9-goolf-1.4.10.eb    # module: ncurses/5.9-goolf-1.4.10
 * [ ] $CFGS/l/libpng/libpng-1.5.13-goolf-1.4.10.eb    # module: libpng/1.5.13-goolf-1.4.10
 * [ ] $CFGS/c/CMake/CMake-2.8.4-goolf-1.4.10.eb    # module: CMake/2.8.4-goolf-1.4.10
 * [ ] $CFGS/l/libharu/libharu-2.2.0-goolf-1.4.10.eb    # module: libharu/2.2.0-goolf-1.4.10
 * [ ] $CFGS/e/EMBOSS/EMBOSS-6.5.7-goolf-1.4.10.eb    # module: EMBOSS/6.5.7-goolf-1.4.10
bash-3.2$ eb --search sympy
== temporary log file in case of crash /var/folders/3L/3LkVBznPF7WgIo752F4GjE+++TI/-Tmp-/easybuild-JOQqYB.log
== Searching for sympy in /Users/fotis/Desktop/arena/uni.lu/easybuild-easyconfigs/easybuild/easyconfigs
== * $CFGS/s/sympy/sympy-0.7.2-goolf-1.4.10-Python-2.7.3.eb
== * $CFGS/s/sympy/sympy-0.7.2-ictce-4.1.13-Python-2.7.3.eb
== temporary log file /var/folders/3L/3LkVBznPF7WgIo752F4GjE+++TI/-Tmp-/easybuild-JOQqYB.log has been removed.

Signed-off-by: Fotis Georgatos <[email protected]>

… etc Signed-off-by: Fotis Georgatos <[email protected]>

Signed-off-by: Fotis Georgatos <[email protected]>

hpcugentbot · 2013-12-06T23:33:51Z

Automatic reply from Jenkins: Can I test this?

…o feature_make_dry-run_output_readable

boegel · 2013-12-09T16:05:12Z

Jenkins: ok to test

boegel · 2013-12-09T16:06:55Z

You should pick up the robot path instead of grabbing the value of some random environment variable $CFGS.

Try and take into account somehow that the robot path might contain multiple entries, separated by os.pathsep (:).

Also, what happens if $CFGS is not set here?

OK, first, thanks for looking into this early on, because we'll need to time to settle on the preferred design.

The whole point of this, is to make output of --dry-run & eb --search more readable for the newcomers.
I see no way to avoid ambiguity if I rely on robotpath exclusively, because trimming down the output on the basis of that, creates confusion when an easyconfig exists on more than one paths. What's your stance on that? Any ideas to offer?

It's important to settle at this first, because that affects all the rest of the changes.
(and surely agree, that it should behave reasonably if CFGS is not defined ;-)

OK, I hadn't realized that $CFGS is now considered "random" variable, because in early versions of the tool (I think including v1.0) it was defined as the default robotpath, no? Admittedly, this code will not handle gracefully the situation of a multicomponent robotpath.

fgeorgatos · 2013-12-11T20:56:18Z

how does this get affected by #786 ?

Unless you come up with some other idea, I think we will be forced to settle to some convention (eg. match against variables finishing in easybuild/easyconfigs or something of that type). Any better offers?

…o feature_make_dry-run_output_readable

Signed-off-by: Fotis Georgatos <[email protected]>

This reverts commit e2dac32.

Signed-off-by: Fotis Georgatos <[email protected]>

JensTimmerman · 2013-12-12T15:11:01Z

What's with the CFGs variable? Why do you need this here? Can't you set EASYBUILD_ROBOT ?

I see no way this is getting merged in with the current CFGSVAR = 'CFGS'

configuration should be don in the config file,
Don't start using constants in the middle of helper functions all of a sudden.

add the CFG to the 'path' in the search_file, and allow it to take a template or something, and then set the template in the config file.

This way when other people want another output format they can easily change it, without having to change the python code again.

boegel · 2013-12-12T17:23:32Z

@JensTimmerman: No worries, I'm on it. :-)

…readable Conflicts: easybuild/main.py

…=' prefix in printed messages

…rch pick up robot paths, align output of search/dry-run, use options.search_short and options.dry_run_short

add --dry-short-short/-D and --search-short-S options, clean up implementation, add unit tests

fgeorgatos · 2013-12-13T01:19:24Z

one further improvement, could be to have the output of --search actually report also in markdown/github style, * [x],
if particular easyconfigs have been built; since this could come at computational expense though, I am not that sure I am a strong supporter of the idea; just give it a thought.

boegel · 2013-12-13T07:30:59Z

@fgeorgatos: I feel that goes beyond the intention of --search. Currently, it's using * <path> as output, which is also markdown style (just no checkboxes).

So, this should be ready to go, you think? Have you played with it?

boegel · 2013-12-13T21:43:34Z

I'm happy with this, the unit tests are happy, so merging it in. Thanks for initiating this @fgeorgatos!

…eadable Feature: make dry run output readable (and also of eb --search)

fgeorgatos · 2013-12-13T23:02:39Z

cool! sorry for being slow to reply... i am still on the road for today ;-)
nice to have it!
On Dec 13, 2013 10:43 PM, "Kenneth Hoste" [email protected] wrote:

Merged #781 #781.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/781
.

fgeorgatos added 5 commits December 7, 2013 00:03

trim down output of dry-run, when $CFGS var is defined

8b07547

Signed-off-by: Fotis Georgatos <[email protected]>

make output of dry-run appear in markdown style, ready for github use…

594535e

… etc Signed-off-by: Fotis Georgatos <[email protected]>

re-adjust how to print module

e2dac32

Signed-off-by: Fotis Georgatos <[email protected]>

fix the classic off-by-one bug

58b1d57

Signed-off-by: Fotis Georgatos <[email protected]>

also, compact output of eb --search, via $CFGS contraction

191e8d7

Signed-off-by: Fotis Georgatos <[email protected]>

Merge branch 'develop' of github.com:hpcugent/easybuild-framework int…

07475af

…o feature_make_dry-run_output_readable

fgeorgatos mentioned this pull request Dec 7, 2013

Shorten eb --search output ; default path for $CFGS variable easybuilders/easybuild-easyblocks#260

Closed

boegel reviewed Dec 9, 2013
View reviewed changes

fgeorgatos added 4 commits December 12, 2013 00:53

Merge branch 'develop' of github.com:hpcugent/easybuild-framework int…

6c98561

…o feature_make_dry-run_output_readable

add perpetrator, restore previous print style

c8a9d1a

Signed-off-by: Fotis Georgatos <[email protected]>

Revert "re-adjust how to print module"

c2773e8

This reverts commit e2dac32.

template CFGSVAR, avoid referencing -1 if cfgs==''

9a147ca

Signed-off-by: Fotis Georgatos <[email protected]>

boegel mentioned this pull request Dec 12, 2013

add support for specifying multiple robot paths #786

Merged

boegel added 5 commits December 12, 2013 18:26

Merge branch 'multiple_robot_paths' into feature_make_dry-run_output_…

cff22ae

…readable Conflicts: easybuild/main.py

add det_common_path_prefix function + unit tests

bb3209e

add optional prefix argument to print_msg, to disable inclusion of '=…

d18ce22

…=' prefix in printed messages

add --search-short/-S and --dry-run-short/-D command line options

9b4dc98

print shorter paths for search/dry-run based on robot paths, make sea…

4ec116f

…rch pick up robot paths, align output of search/dry-run, use options.search_short and options.dry_run_short

add checks for -D/--dry-run-short

da63766

boegel mentioned this pull request Dec 12, 2013

add --dry-short-short/-D and --search-short-S options, clean up implementation, add unit tests fgeorgatos/easybuild-framework#5

Merged

boegel and others added 2 commits December 13, 2013 00:02

add import for all in filetools.py

f3d1c25

Merge pull request #5 from boegel/feature_make_dry-run_output_readable

f171393

add --dry-short-short/-D and --search-short-S options, clean up implementation, add unit tests

boegel added a commit that referenced this pull request Dec 13, 2013

Merge pull request #781 from fgeorgatos/feature_make_dry-run_output_r…

de74527

…eadable Feature: make dry run output readable (and also of eb --search)

boegel merged commit de74527 into easybuilders:develop Dec 13, 2013

boegel mentioned this pull request Dec 14, 2013

pep8 cleanup in main.py #789

Merged

fgeorgatos deleted the feature_make_dry-run_output_readable branch December 15, 2013 00:38

Conversation

fgeorgatos commented Dec 6, 2013

Uh oh!

hpcugentbot commented Dec 6, 2013

Uh oh!

boegel commented Dec 9, 2013

Uh oh!

boegel Dec 9, 2013

Choose a reason for hiding this comment

Uh oh!

boegel Dec 9, 2013

Choose a reason for hiding this comment

Uh oh!

fgeorgatos Dec 9, 2013

Choose a reason for hiding this comment

Uh oh!

fgeorgatos Dec 12, 2013

Choose a reason for hiding this comment

Uh oh!

fgeorgatos commented Dec 11, 2013

Uh oh!

JensTimmerman commented Dec 12, 2013

Uh oh!

boegel commented Dec 12, 2013

Uh oh!

fgeorgatos commented Dec 13, 2013

Uh oh!

boegel commented Dec 13, 2013

Uh oh!

boegel commented Dec 13, 2013

Uh oh!

fgeorgatos commented Dec 13, 2013

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants