Skip to content

avoid downloading old numpy version during install for h5py#13428

Merged
smoors merged 14 commits intoeasybuilders:developfrom
LHurst-JM:13424_h5py-install-fails
Jul 16, 2021
Merged

avoid downloading old numpy version during install for h5py#13428
smoors merged 14 commits intoeasybuilders:developfrom
LHurst-JM:13424_h5py-install-fails

Conversation

@LHurst-JM
Copy link
Copy Markdown
Contributor

By default h5py's setup.py has a setup_requires for exactly the minimum supported version of numpy (older then the one in SciPy-bundle, in some cases) based on the python interpretor version, resulting in pip attempting to fetch it during the EasyBuild install phase.

As the dependency is already managed through easy_build, and h5py's install_requires is for any version newer than the minimum version, this change suppresses the setup_requires (using the mechanism provided by h2py for "downstream packagers - e.g. Linux distros").

Resolves #13424

By default h5py's setup.py has a setup_requires for the minimum supported version of numpy (older then the one in SciPy-bundle, in some cases) based on the python interpretor version, resulting in pip attempting to fetch it during the install phase.

As the dependency is already managed through easy_build, and h5py's install_requires is for any version newer than the minimum version, this change suppresses the setup_requires (using the mechanism provided by h2py for "downstream packgers - e.g. Linux distros").
By default h5py's setup.py has a setup_requires for the minimum supported version of numpy (older then the one in SciPy-bundle, in some cases) based on the python interpretor version, resulting in pip attempting to fetch it during the install phase.

As the dependency is already managed through easy_build, and h5py's install_requires is for any version newer than the minimum version, this change suppresses the setup_requires (using the mechanism provided by h2py for "downstream packgers - e.g. Linux distros").
By default h5py's setup.py has a setup_requires for the minimum supported version of numpy (older then the one in SciPy-bundle, in some cases) based on the python interpretor version, resulting in pip attempting to fetch it during the install phase.

As the dependency is already managed through easy_build, and h5py's install_requires is for any version newer than the minimum version, this change suppresses the setup_requires (using the mechanism provided by h2py for "downstream packgers - e.g. Linux distros").
By default h5py's setup.py has a setup_requires for the minimum supported version of numpy (older then the one in SciPy-bundle, in some cases) based on the python interpretor version, resulting in pip attempting to fetch it during the install phase.

As the dependency is already managed through easy_build, and h5py's install_requires is for any version newer than the minimum version, this change suppresses the setup_requires (using the mechanism provided by h2py for "downstream packgers - e.g. Linux distros").
@Micket Micket added the bug fix label Jul 13, 2021
@Micket Micket added this to the next release (4.4.2?) milestone Jul 13, 2021
Copy link
Copy Markdown
Contributor

@Micket Micket left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a short comment that explains what it does?

Also, are only the 2020b versions affected?

@LHurst-JM
Copy link
Copy Markdown
Contributor Author

Maybe add a short comment that explains what it does?

Sure, I initially put one in then removed it as I didn't think it was the done thing to comment in .eb files 😉 I'll put one back.

Also, are only the 2020b versions affected?

No but the environment variable to selectively disable the numpy== setup_requires for packagers was only added in 3.0.0 (and >3.0.0 only has 2020b and 2021a easyconfigs at the moment), so the only workaround for older versions would be to patch the setup.py. I will add this to the 2021a .eb too, thanks for pointing that out to me.

LHurst-JM added 10 commits July 14, 2021 09:34
By default h5py's setup.py has a setup_requires for the minimum supported version of numpy (older then the one in SciPy-bundle, in some cases) based on the python interpretor version, resulting in pip attempting to fetch it during the install phase.

As the dependency is already managed through easy_build, and h5py's install_requires is for any version newer than the minimum version, this change suppresses the setup_requires (using the mechanism provided by h2py for "downstream packgers - e.g. Linux distros").
@LHurst-JM LHurst-JM requested a review from Micket July 14, 2021 08:41
Copy link
Copy Markdown
Contributor

@Micket Micket left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@smoors
Copy link
Copy Markdown
Contributor

smoors commented Jul 15, 2021

@boegelbot: please test @ generoso

@boegelbot
Copy link
Copy Markdown
Collaborator

@smoors: Request for testing this PR well received on generoso

PR test command 'EB_PR=13428 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_13428 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 17796

Test results coming soon (I hope)...

Details

- notification for comment with ID 880956725 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@smoors smoors changed the title Suppress h5py's dependency management avoid downloading old numpy version during install for h5py Jul 15, 2021
@boegelbot
Copy link
Copy Markdown
Collaborator

Test report by @boegelbot
FAILED
Build succeeded for 3 out of 5 (5 easyconfigs in total)
generoso-c1-s-2 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/ab0c24c0bd9509f1f516f9ccd427eb46 for a full test report.

@smoors
Copy link
Copy Markdown
Contributor

smoors commented Jul 15, 2021

Test report by @smoors
FAILED
Build succeeded for 14 out of 15 (5 easyconfigs in total)
node371.hydra.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 2.7.5
See https://gist.github.com/13a6555aea90c4d2f264200affccaef5 for a full test report.

@Micket
Copy link
Copy Markdown
Contributor

Micket commented Jul 15, 2021

Test report by @Micket
SUCCESS
Build succeeded for 21 out of 21 (5 easyconfigs in total)
alvis-c1 - Linux centos linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake), Python 3.6.8
See https://gist.github.com/e93b408f003cf07d7f539850cdde19bc for a full test report.

@smoors
Copy link
Copy Markdown
Contributor

smoors commented Jul 16, 2021

Test report by @smoors
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node251.hydra.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (broadwell), Python 2.7.5
See https://gist.github.com/5fc588dbf1ec48d90b493585a998c702 for a full test report.

Copy link
Copy Markdown
Contributor

@smoors smoors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@smoors
Copy link
Copy Markdown
Contributor

smoors commented Jul 16, 2021

Going in, thanks @LHurst-JM!

@smoors smoors merged commit 3496b2e into easybuilders:develop Jul 16, 2021
@LHurst-JM LHurst-JM deleted the 13424_h5py-install-fails branch August 2, 2021 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

h5py-3.1.0-foss-2020b fails

4 participants