Skip to content

use stand-alone absl-py as dependency for jax w/ gfbf/2023a + CUDA 12.1.1 + add Optax v0.2.2, Flax v0.8.4, scvi-tools v1.1.2#21039

Merged
boegel merged 3 commits intoeasybuilders:developfrom
lexming:20240724124338_new_pr_absl-py210
Sep 4, 2024
Merged

use stand-alone absl-py as dependency for jax w/ gfbf/2023a + CUDA 12.1.1 + add Optax v0.2.2, Flax v0.8.4, scvi-tools v1.1.2#21039
boegel merged 3 commits intoeasybuilders:developfrom
lexming:20240724124338_new_pr_absl-py210

Conversation

@lexming
Copy link
Copy Markdown
Contributor

@lexming lexming commented Jul 24, 2024 •
edited
Loading

(created using eb --new-pr)

Equivalent to #21038 for stack with CUDA

Adding new easyconfig for Flax, which deserves its own package; and for absl-py, which is used in many places already.

This clarifies the dependency tree of the Flax/JAX stack as:
absl-py > jax > Optax > Flax

Changelog:

  • simplify easyconfig of jax v0.4.25:
    • replace component on absl-py with regular dependency
    • define source_urls individually per each source sownloaded
  • add easyconfig for Optax v0.2.2 with CUDA 12 without those extensions already provided by its dependencies
  • add easyconfig for Flax v0.8.4 with CUDA 12
  • add easyconfig for scvi-tools v1.2.2 with CUDA 12 without those extensions already provided by its dependencies

This was referenced Jul 25, 2024
Closed
Closed
Closed
@lexming
Copy link
Copy Markdown
Contributor Author

lexming commented Jul 26, 2024

Test report by @lexming
SUCCESS
Build succeeded for 5 out of 5 (5 easyconfigs in total)
node402.hydra.os - Linux Rocky Linux 8.10, x86_64, AMD EPYC 7282 16-Core Processor (zen2), 1 x NVIDIA NVIDIA A100-PCIE-40GB, 550.90.07, Python 3.6.8
See https://gist.github.com/lexming/80f6e230f828d57a0dea58141c4566bd for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Aug 29, 2024

@boegelbot please test @ jsc-zen3-a100

@boegel boegel added this to the release after 4.9.2 milestone Aug 29, 2024
@boegel boegel mentioned this pull request Aug 29, 2024
Merged
@boegelbot
Copy link
Copy Markdown
Collaborator

boegelbot commented Aug 29, 2024

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=21039 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_21039 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 4774

Test results coming soon (I hope)...

Details

- notification for comment with ID 2317739963 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Copy Markdown
Collaborator

boegelbot commented Aug 29, 2024

Test report by @boegelbot
SUCCESS
Build succeeded for 5 out of 5 (5 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.18
See https://gist.github.com/boegelbot/222dfbe77bd170f61ff05b35c991d99b for a full test report.

@boegel boegel changed the title Flax/JAX stack in 2023a w/ CUDA 12.1.1: scvi-tools, Flax, Optax, jax, absl-py use stand-alone absl-py as dependency for jax w/ gfbf/2023a + CUDA 12.1.1 + add Optax v0.2.2, Flax v0.8.4, scvi-tools v1.1.2 Sep 4, 2024
boegel
boegel approved these changes Sep 4, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel
Copy link
Copy Markdown
Member

boegel commented Sep 4, 2024

Going in, thanks @lexming!

@boegel boegel merged commit fa1afbe into easybuilders:develop Sep 4, 2024
@lexming lexming deleted the 20240724124338_new_pr_absl-py210 branch September 5, 2024 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

change update

Projects

None yet

Milestone

4.9.3

Development

Successfully merging this pull request may close these issues.

3 participants

@lexming @boegel @boegelbot