Updated custom_paths for picard-1.124 and above by nathanhaigh · Pull Request #796 · easybuilders/easybuild-easyblocks

nathanhaigh · 2016-01-05T02:35:55Z

A restructure of picard at v1.124 means only a single picard.jar file is created

hpcugentbot · 2016-01-05T02:35:56Z

Automatic reply from Jenkins: Can I test this?

boegel · 2016-01-05T09:11:10Z

rather than copy-pasting this whole block, it would be better to change the logic that defines the jar_files list?

How familiar are you with Python @nathanhaigh? Let us know if you're up for this.

I'm not familiar with Python - I'm a Perl guy! :)

I'll have a bash at it though!

Let me know if you need help.

boegel · 2016-01-05T09:11:19Z

Jenkins: ok to test

hpcugentbot · 2016-01-05T09:18:17Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1556/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-01-12T06:22:13Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1578/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

nathanhaigh · 2016-01-12T06:23:20Z

@boegel This now implements a comprehensive sanity_check_step for all versions of picard from 1.100 to the current 2.0.1. Versions prior to 1.100 may also work, but I haven't tested them.

boegel · 2016-01-17T13:51:01Z

hmm, do we want this one per line? I'd collapse this by listing multiple items on a single line, but keeping it down to max 120 chars line length?

boegel · 2016-01-17T13:54:24Z

Looks good, but let's make this less verbose/long (vertically)?

…ectively.

hpcugentbot · 2016-02-22T05:31:11Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1690/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

nathanhaigh · 2016-02-22T05:32:31Z

Do you really want multiple elements in jar_files to be placed on a single line? I was thinking that for the sake of some vertical real-estate it makes things more readable.

boegel · 2016-02-22T10:58:43Z

@nathanhaigh: well, it's a matter of preference, sure... There's no other easyblock I know of that has a list this long with one entry per line. I agree it makes things more readable, but how often is that really relevant in this case?
This makes me scroll a lot more when looking at other bits of the easyblock, which I find annoying.

Anyway, it's up to you in this case, I don't feel strongly enough about it to make this a blocker.

nathanhaigh · 2016-02-22T23:01:10Z

Ok, I'll leave it as it is :) I'm finished with this PR, so please merge when you're happy.

boegel · 2016-02-23T09:34:38Z

-        jar_files = ['picard']
-        if LooseVersion(self.version) < LooseVersion('1.115'):
-            jar_files.append('sam')
+        """All versions prior to 1.124 have these jar files"""


this shouldn't be a docstring, but a comment:

# all versions prior to 1.124 have these jar files

boegel · 2016-02-23T09:40:23Z

@nathanhaigh: sanity check fails for existing picard-1.39.eb with this...

Most likely because comparing 1.39 < 1.124 yields False...

We need a way around that, maybe by counting the digits in the minor version number?

hpcugentbot · 2016-02-23T11:41:27Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1693/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-03-02T20:16:02Z

@nathanhaigh: ping on the issue with picard-1.39.eb?

boegel · 2016-03-02T20:16:53Z

btw, apparently easybuilders/easybuild-easyconfigs#2572 was merged even though it requires this PR to be merged...

seems like I was not very careful with testing easybuilders/easybuild-easyconfigs#2572, and it got tested on top of your modified easyblock... :(

nathanhaigh · 2016-03-07T01:21:59Z

Actually, @boegel the version comparison works as expected:

>>> from distutils.version import LooseVersion
>>> LooseVersion('1.39') < LooseVersion('1.124')
True

I think the fail on picard 1.39 is because the following jars are not actually present:

AddOrReplaceReadGroups.jar
CheckIlluminaDirectory.jar
CollectRnaSeqMetrics.jar

I'll update the easyblock.

Some jar files were incorrectly assumed to be in picard versions < 1.124 but weren't. This broke installation of existing easyconfigs for the really old picard 1.39.

hpcugentbot · 2016-03-07T01:26:50Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1759/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-03-07T18:01:59Z

@nathanhaigh: I'm afraid picard 1.39 is still failing, see below...

I have a feeling we're over-specifying this...

The largest part of the easyblock now consists over making exceptions on which .jar files to check for, while we've actually just copied them...

Is that really worth the trouble? Shouldn't we just check for a couple of key .jar files instead?
Like picard* (and maybe sam*) for the older versions of picard, and the three current .jar for recent versions?

== 2016-03-07 18:55:59,137 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('CollectTargetedPcrMetrics.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,138 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('DownsampleSam.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,139 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('ExtractIlluminaBarcodes.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,139 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('FilterSamReads.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,143 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('IlluminaBasecallsToFastq.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,143 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('IlluminaBasecallsToSam.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,144 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('IntervalListTools.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,144 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('MakeSitesOnlyVcf.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,145 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('MarkIlluminaAdapters.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,146 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('MergeVcfs.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,146 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('ReorderSam.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,148 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('SplitVcfs.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39
== 2016-03-07 18:55:59,148 not available in optimized mode.EB_picard WARNING Sanity check: no file of ('VcfFormatConverter.jar',) in /user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/picard/1.39

nathanhaigh · 2016-03-07T19:59:57Z

yep - I agree :)

… different versions

hpcugentbot · 2016-03-07T20:12:04Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1768/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-03-08T09:07:47Z

retested with existing picard easyconfigs and easybuilders/easybuild-easyconfigs#2252, all is well, so going in

Thanks @nathanhaigh!

Updated custom_paths for picard-1.124 and above

Updated custom_paths for picard-1.124 and above

eb8b4f7

nathanhaigh mentioned this pull request Jan 5, 2016

{bio}[dummy/dummy] picard 1.120, picard 2.0.1 (REVIEW) easybuilders/easybuild-easyconfigs#2252

Merged

boegel reviewed Jan 5, 2016
View reviewed changes

boegel added this to the v2.6.0 milestone Jan 5, 2016

Comprehensive sanity_check_step for versions 1.100 to current 2.0.1

4eb1088

boegel reviewed Jan 17, 2016
View reviewed changes

boegel modified the milestones: v2.7.0, v2.6.0 Jan 23, 2016

nathanhaigh added 2 commits February 22, 2016 15:58

Use .append() and .extend() for adding single and multiple items resp…

91e7a03

…ectively.

No need for brackets

6af65d5

boegel reviewed Feb 23, 2016
View reviewed changes

docstrings to comments

0b77073

This was referenced Mar 2, 2016

Update picard to 2.1.0 including move to GitHub easybuilders/easybuild-easyconfigs#2572

Merged

Update files that have dummy as toolchain, are not a toolchain themse… easybuilders/easybuild-easyconfigs#2612

Merged

Moved some jar file sanity checks.

7fd82bc

Some jar files were incorrectly assumed to be in picard versions < 1.124 but weren't. This broke installation of existing easyconfigs for the really old picard 1.39.

Check only for key jar files to remove complicated checking logic for…

c3b59c1

… different versions

boegel added a commit that referenced this pull request Mar 8, 2016

Merge pull request #796 from nathanhaigh/picard-1.124

5280668

Updated custom_paths for picard-1.124 and above

boegel merged commit 5280668 into easybuilders:develop Mar 8, 2016

Conversation

nathanhaigh commented Jan 5, 2016

Uh oh!

hpcugentbot commented Jan 5, 2016

Uh oh!

boegel Jan 5, 2016

Choose a reason for hiding this comment

Uh oh!

nathanhaigh Jan 5, 2016

Choose a reason for hiding this comment

Uh oh!

boegel Jan 9, 2016

Choose a reason for hiding this comment

Uh oh!

boegel commented Jan 5, 2016

Uh oh!

hpcugentbot commented Jan 5, 2016

Uh oh!

hpcugentbot commented Jan 12, 2016

Uh oh!

nathanhaigh commented Jan 12, 2016

Uh oh!

boegel Jan 17, 2016

Choose a reason for hiding this comment

Uh oh!

boegel commented Jan 17, 2016

Uh oh!

hpcugentbot commented Feb 22, 2016

Uh oh!

nathanhaigh commented Feb 22, 2016

Uh oh!

boegel commented Feb 22, 2016

Uh oh!

nathanhaigh commented Feb 22, 2016

Uh oh!

boegel Feb 23, 2016

Choose a reason for hiding this comment

Uh oh!

boegel commented Feb 23, 2016

Uh oh!

hpcugentbot commented Feb 23, 2016

Uh oh!

boegel commented Mar 2, 2016

Uh oh!

boegel commented Mar 2, 2016

Uh oh!

nathanhaigh commented Mar 7, 2016

Uh oh!

hpcugentbot commented Mar 7, 2016

Uh oh!

boegel commented Mar 7, 2016

Uh oh!

nathanhaigh commented Mar 7, 2016

Uh oh!

hpcugentbot commented Mar 7, 2016

Uh oh!

boegel commented Mar 8, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants