Skip to content

take into account that binutils might be a filtered dependency for TensorFlow#3058

Merged
boegel merged 2 commits intoeasybuilders:developfrom
Flamefire:tensorflow-binutils
Dec 22, 2023
Merged

take into account that binutils might be a filtered dependency for TensorFlow#3058
boegel merged 2 commits intoeasybuilders:developfrom
Flamefire:tensorflow-binutils

Conversation

@Flamefire
Copy link
Copy Markdown
Contributor

E.g. when using --filter-deps binutils in combination with --rpath the could could fail to determine the files from binutils to collect as the "software root" won't be set.
Fall back to collecting all rpath wrappers and a list of known binutils

Followup to #3054 similar to #2218

Logic is:

  • If binutils root is found use those found in its "bin" folder
  • else collect all wrappers as they mostly are binutils and the others don't hurt as we don't put that new folder into PATH anyway. Additionally add a hard-coded list of known binutils which could be extended in case we observe failures due to other missing tools where no rpath wrapper is created. e.g. "ar"

While at it I double-checked the other uses of which and used the new list of binutils in patch_crosstool_files preserving the existing behavior of raising an error if one of those we currently have isn't found.

Would be great if @jfgrimm could verify this new version still works but I assume it doesn't change behavior for this case as it should basically do exactly the same.

cc @boegel

E.g. when using `--filter-deps binutils` in combination with `--rpath`
the could could fail to determine the files from binutils to collect as
the "software root" won't be set.
Fall back to collecting all rpath wrappers and a list of known binutils
@jfgrimm jfgrimm added this to the next release (4.9.0?) milestone Dec 21, 2023
@jfgrimm
Copy link
Copy Markdown
Member

jfgrimm commented Dec 21, 2023

I've kicked off a build

@boegel boegel changed the title take into account that binutils might not be a dependency for TensorFlow take into account that binutils might be a filtered dependency for TensorFlow Dec 21, 2023
@jfgrimm
Copy link
Copy Markdown
Member

jfgrimm commented Dec 21, 2023

Test report by @jfgrimm

Overview of tested easyconfigs (in order)

  • SUCCESS TensorFlow-2.11.0-foss-2022a-CUDA-11.7.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
gpu14.viking2.yor.alces.network - Linux Rocky Linux 8.8, x86_64, AMD EPYC 7643 48-Core Processor, 2 x NVIDIA NVIDIA A40, 535.86.10, Python 3.6.8
See https://gist.github.com/jfgrimm/db8ea3ee916ca672f89a1be1ed943bc8 for a full test report.

Comment thread easybuild/easyblocks/t/tensorflow.py Outdated
@boegel
Copy link
Copy Markdown
Member

boegel commented Dec 21, 2023

Test report by @boegel

Overview of tested easyconfigs (in order)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
node4112.gallade.os - Linux Debian GNU/Linux 11 (bullseye), x86_64, AMD EPYC 7773X 64-Core Processor, Python 3.11.4
See https://gist.github.com/boegel/57ea446ff397985715780e8db85dc200 for a full test report.

edit: ignore this, I need to use easyconfig from easybuilders/easybuild-easyconfigs#19268 when building TensorFlow in EESSI with EasyBuild 4.8.2, trying again...

@boegel
Copy link
Copy Markdown
Member

boegel commented Dec 21, 2023

Successful test in EESSI build environment, see test report at easybuilders/easybuild-easyconfigs#19268 (comment)

@boegel
Copy link
Copy Markdown
Member

boegel commented Dec 21, 2023

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS TensorFlow-2.13.0-foss-2023a.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3109.skitty.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/boegel/6a3d23c03c6a44929479642e8ed3348b for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Dec 21, 2023

Tested by @jfgrimm with RPATH enabled, tested in EESSI build environment, tested without RPATH or EESSI, so good to go, thanks @Flamefire!

@boegel boegel merged commit 5d386b6 into easybuilders:develop Dec 22, 2023
@Flamefire Flamefire deleted the tensorflow-binutils branch December 23, 2023 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants