Skip to content

use relative paths to object files when compiling shared libraries in the OpenFOAM easyblock#3388

Merged
ocaisa merged 1 commit intoeasybuilders:developfrom
bedroge:openfoam_objects_relative_paths
Jul 30, 2024
Merged

use relative paths to object files when compiling shared libraries in the OpenFOAM easyblock#3388
ocaisa merged 1 commit intoeasybuilders:developfrom
bedroge:openfoam_objects_relative_paths

Conversation

@bedroge
Copy link
Copy Markdown
Contributor

@bedroge bedroge commented Jul 9, 2024

In EESSI we ran into the following error when compiling OpenFOAM v2312 for Intel Skylake (see EESSI/software-layer#563 (comment)):

make: /bin/sh: Argument list too long
make: *** [/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/wmake/makefiles/general:210: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so] Error 127

This worked fine for other CPU targets, but due to the CPU target names that we use, Skylake has the longest installation path ("intel/skylake_avx512"). Apparently, this makes the entire command for building the shared library libOpenFOAM.so just a bit too long. This command includes a large number (~600) of absolute paths to object files:

OMPI_CXX="g++" mpicxx -m64 -pthread -DOPENFOAM=2312 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -Wno-attributes -Wno-unknown-pragmas -O2 -fno-tree-vectorize -march=native -fno-math-errno -std=c++14 -fuse-ld=bfd  -DNoRepository -ftemplate-depth-100  -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM -DHAVE_LIBZ -DHAVE_EXTRAE -iquote. -IlnInclude -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/src/OpenFOAM/lnInclude -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/src/OSspecific/POSIX/lnInclude   -fPIC -shared -Xlinker --add-needed -Xlinker --no-as-needed  
/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM/global/foamConfig.o
/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM/global/globals.o
<about 600 more of these /paths/to/some.o> 
-L/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib \
    /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOSspecific.o -lz -L/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/dummy -lPstream  -o /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so

This PR fixes the issue by replacing the absolute paths to all these object files by relative ones. It does that by replacing the $WM_PROJECT_DIR (main installation path of this OpenFOAM version) part of the path by the relative path (relative to the current working dir) to this main installation directory for every object file. This means that the ~600 paths will then look something like:

../../../build/linux64GccDPInt32Opt/src/Pstream/dummy/UPstream.o

And this greatly reduces the length of this entire command, which made it compile without issues for our Skylake installation path.

Since the same code seems to be part of both different OpenFOAM forks and in all the versions that I checked, I haven't restricted this to a specific fork/version.

@bedroge bedroge added bug fix EESSI Related to EESSI project labels Jul 9, 2024
@bedroge bedroge mentioned this pull request Jul 9, 2024
Merged
@bedroge
Copy link
Copy Markdown
Contributor Author

bedroge commented Jul 9, 2024 •
edited
Loading

Actually, this makefile was introduced in 4.x for the .org fork. For older versions, the file does not exist, but in that case the change just won't be applied (as it checks if the file exists), which should be fine.

For the .com version, this file should be present for all versions.

@bedroge
Copy link
Copy Markdown
Contributor Author

bedroge commented Jul 9, 2024

Successful build with the EESSI bot for skylake: EESSI/software-layer#563 (comment)

@bedroge
Copy link
Copy Markdown
Contributor Author

bedroge commented Jul 9, 2024

@boegelbot please test @ generoso
CORE_CNT=16
EB_ARGS="--installpath=/tmp/$USER/pr3388 OpenFOAM-8-foss-2020a.eb OpenFOAM-9-foss-2021a.eb OpenFOAM-10-foss-2022a.eb OpenFOAM-11-foss-2023a.eb OpenFOAM-v2106-foss-2021a.eb OpenFOAM-v2112-foss-2021b.eb OpenFOAM-v2206-foss-2022a.eb OpenFOAM-v2306-foss-2022b.eb OpenFOAM-v2312-foss-2023a.eb"

@boegelbot
Copy link
Copy Markdown

boegelbot commented Jul 9, 2024

@bedroge: Request for testing this PR well received on login1

PR test command 'EB_PR=3388 EB_ARGS="--installpath=/tmp/$USER/pr3388 OpenFOAM-8-foss-2020a.eb OpenFOAM-9-foss-2021a.eb OpenFOAM-10-foss-2022a.eb OpenFOAM-11-foss-2023a.eb OpenFOAM-v2106-foss-2021a.eb OpenFOAM-v2112-foss-2021b.eb OpenFOAM-v2206-foss-2022a.eb OpenFOAM-v2306-foss-2022b.eb OpenFOAM-v2312-foss-2023a.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --job-name test_PR_3388 --ntasks="16" ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 13909

Test results coming soon (I hope)...

Details

- notification for comment with ID 2218652517 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Copy Markdown

boegelbot commented Jul 10, 2024

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-8-foss-2020a.eb
  • SUCCESS OpenFOAM-9-foss-2021a.eb
  • SUCCESS OpenFOAM-10-foss-2022a.eb
  • SUCCESS OpenFOAM-11-foss-2023a.eb
  • SUCCESS OpenFOAM-v2106-foss-2021a.eb
  • SUCCESS OpenFOAM-v2112-foss-2021b.eb
  • SUCCESS OpenFOAM-v2206-foss-2022a.eb
  • SUCCESS OpenFOAM-v2306-foss-2022b.eb
  • SUCCESS OpenFOAM-v2312-foss-2023a.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total)
cnx1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/b5e53db5955ea2e03c35574f917e7ad3 for a full test report.

ocaisa
ocaisa approved these changes Jul 30, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@ocaisa ocaisa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ocaisa ocaisa merged commit 0cb2cc9 into easybuilders:develop Jul 30, 2024
@boegel boegel added this to the release after 4.9.2 milestone Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ocaisa ocaisa ocaisa approved these changes

Assignees

No one assigned

Labels

bug fix EESSI Related to EESSI project

Projects

None yet

Milestone

4.9.3

Development

Successfully merging this pull request may close these issues.

4 participants

@bedroge @boegelbot @ocaisa @boegel