Skip to content

{data,math}[foss/2018b] rootpy v1.0.1, root_numpy v4.8.0 w/ Python 3.6.6#10424

Merged
lexming merged 15 commits intoeasybuilders:developfrom
kelseymh:20200416_new_ROOT-based_Python_packages
Apr 24, 2020
Merged

{data,math}[foss/2018b] rootpy v1.0.1, root_numpy v4.8.0 w/ Python 3.6.6#10424
lexming merged 15 commits intoeasybuilders:developfrom
kelseymh:20200416_new_ROOT-based_Python_packages

Conversation

@kelseymh
Copy link
Copy Markdown
Contributor

rootpy-1.0.1-foss-2018b-Python-3.6.6.eb is an update for Python 3.6.6 support, which ROOT hasn't had up to now.

root_numpy-4.8.0-foss-2018b-Python-3.6.6.eb is an entirely new package for EasyBuild, which provides integrated ROOT and Numpy array support for TTrees.

@kelseymh
Copy link
Copy Markdown
Contributor Author

This PR (specifically the new root_numpy module) depends on #10377, which has passed its checks but hasn't been reviewed or merged. Should I merge that PR's two entries onto this branch, or wait for that PR to be merged and retrigger the checks here?

Copy link
Copy Markdown
Contributor

@lexming lexming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kelseymh if this easyconfig needs numpy v1.18.2 it will not be possible to merge it into our repository. We have a rule to limit the number of dependencies in a toolchain to a single version of each package. Therefore in foss/2018b it is only possible to use numpy v1.15.0 provided by Python-3.6.6-foss-2018b.eb. The reason being that this repository would be very bloated and difficult to manage if all combinations of versions were allowed for every toolchain.

If this new version of ROOT really needs numpy v1.18.2, you will be able to submit it for merging into foss/2020a once it is available (it will have numpy v1.18). Instead, if it can be used with numpy v1.15.0, we can accept it into foss/2018b. It just needs to depend on Python-3.6.6-foss-2018b.eb.

@kelseymh
Copy link
Copy Markdown
Contributor Author

Thank you, @lexming. I don't see why we can't use numpy v1.15.0; I'll have to do the build and make sure there are no hiccups. I chose the "latest and greatest" just because...

The dependence is actually that root_numpy uses both ROOT and numpy, combining the two into a useful Python analysis tool. It will depend on both ROOT/6.14.06-foss-2018b-Python-3.6.6.eb and Python-3.6.6-foss-2018b.eb. If I can use numpy-1.15.0-blah-blah, then I will update this PR.

@kelseymh kelseymh marked this pull request as draft April 23, 2020 02:41
kelseymh added a commit to kelseymh/easybuild-easyconfigs that referenced this pull request Apr 23, 2020
@kelseymh
Copy link
Copy Markdown
Contributor Author

I'm not sure why those extra files, especially the numpy-1.15.0 and Cython-0.29.16, got pulled in here. I used eb --update-pr=10424 and listed only the one file I changed (root_numpy). I apologize for any confusion.

@kelseymh kelseymh marked this pull request as ready for review April 23, 2020 05:15
I don't know why this file was pulled in by `eb --update-pr`, but it's not the correct version.
@lexming
Copy link
Copy Markdown
Contributor

lexming commented Apr 23, 2020

@kelseymh you can remove any file from this PR by prepending a colon to the name of the easyconfig

eb --update-pr 10424 --pr-commit-msg "some commit message" :numpy-1.15.0-foss-2018b-Python-3.6.6.eb

numpy and Cython are already provided by Python-3.6.6-foss-2018b.eb so those can be safely removed from this PR.

update: and GSL can also be removed as there is GSL-2.5-GCC-7.3.0-2.30.eb already.

…gs into 20200415_new_ROOT-based_Python_packages
@kelseymh kelseymh marked this pull request as draft April 23, 2020 14:58
@easybuilders easybuilders deleted a comment from boegelbot Apr 23, 2020
@easybuilders easybuilders deleted a comment from boegelbot Apr 23, 2020
@kelseymh
Copy link
Copy Markdown
Contributor Author

Question: in the previous round of failed checks, it complained about "http://" in the homepage entry for root_numpy. But it didn't complain about that in rootpy, and I see the same thing in both rootpy EasyConfigs (0.8.0 and my new 1.0.1).

Rootpy's Web site is only http://www.rootpy.org/. It doesn't have an https://www.rootpy.org/ server.

Does the validation test check whether https:// works, before complaining about it?

@kelseymh kelseymh marked this pull request as ready for review April 23, 2020 21:45
@lexming
Copy link
Copy Markdown
Contributor

lexming commented Apr 24, 2020

Does the validation test check whether https:// works, before complaining about it?

Yes

Thanks for updating this PR, I'll start testing it.

@lexming
Copy link
Copy Markdown
Contributor

lexming commented Apr 24, 2020

Test report by @lexming
FAILED
Build succeeded for 2 out of 3 (2 easyconfigs in this PR)
node150.hydra.os - Linux centos linux 7.7.1908, x86_64, Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz, Python 2.7.5
See https://gist.github.com/21ecdf82c27dc10e897b04561a59c6da for a full test report.

Comment thread easybuild/easyconfigs/r/rootpy/rootpy-1.0.1-foss-2018b-Python-3.6.6.eb Outdated
@kelseymh
Copy link
Copy Markdown
Contributor Author

@lexming Naive question: Should checksums change on a distribution like this? Why didn't my own builds of this .eb file (on TAMU HPRC) fail if the checksum was wrong?

@lexming
Copy link
Copy Markdown
Contributor

lexming commented Apr 24, 2020

@kelseymh you should check why that happened, is your first tarball corrupted? does its contents differ from the tarball that can be downloaded today?

@kelseymh
Copy link
Copy Markdown
Contributor Author

To be quite honest. I don't know where the previous checksum arose. The "7d85fc8f" checksum was in the file I initially committed in my local EC repository. I tried grepping for it in my other .eb files (i.e., a misplaced copy-paste), but no matches.

In the original version of rootpy-1.0.1, which I used to actually build the module for my group, the .eb file didn't have a checksums entry at all (the existing rootpy-0.8.0, which I copied, does not have one either).

The rootpy file which was downloaded as part of that (successful) installation, without a checksum. I ran shasum -a 256 on the saved sources/.../1.0.1.tar.gz file, and it came back with "your" (correct) checksum, not the one I committed.

@easybuilders easybuilders deleted a comment from boegelbot Apr 24, 2020
@lexming
Copy link
Copy Markdown
Contributor

lexming commented Apr 24, 2020

Test report by @lexming
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in this PR)
node150.hydra.os - Linux centos linux 7.7.1908, x86_64, Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz, Python 2.7.5
See https://gist.github.com/853bbd72bb76b083e7dda7f7844168d6 for a full test report.

Copy link
Copy Markdown
Contributor

@lexming lexming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build works well. We need one last change for rootpy adding all executables in bin to sanity_check_paths. I also suggest a couple changes to use a formatting more in line with the other easyconfigs.

Comment thread easybuild/easyconfigs/r/root_numpy/root_numpy-4.8.0-foss-2018b-Python-3.6.6.eb Outdated
Comment thread easybuild/easyconfigs/r/rootpy/rootpy-1.0.1-foss-2018b-Python-3.6.6.eb Outdated
Comment thread easybuild/easyconfigs/r/rootpy/rootpy-1.0.1-foss-2018b-Python-3.6.6.eb Outdated
@kelseymh
Copy link
Copy Markdown
Contributor Author

Thank you, @lexming . I'm glad those were all stylistic issues, and not functional. I'll learn all this stuff eventually :-)

@lexming
Copy link
Copy Markdown
Contributor

lexming commented Apr 24, 2020

Test report by @lexming
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in this PR)
node154.hydra.os - Linux centos linux 7.7.1908, x86_64, Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz, Python 2.7.5
See https://gist.github.com/26b0e91f46f42389cbf2e3306304640d for a full test report.

Copy link
Copy Markdown
Contributor

@lexming lexming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lexming
Copy link
Copy Markdown
Contributor

lexming commented Apr 24, 2020

Going in, thanks for following up on this PR @kelseymh !

@lexming lexming merged commit 9bfc639 into easybuilders:develop Apr 24, 2020
@kelseymh kelseymh deleted the 20200416_new_ROOT-based_Python_packages branch April 25, 2020 01:38
@boegel boegel changed the title Two new ROOT-based Python numeric modules, for use with HEP analysis code {data,math}[foss/2018b] rootpy v1.0.1, root_numpy v4.8.0 w/ Python 3.6.6 Apr 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants