Skip to content

enhance BuildEnv generic easyblock to copy RPATH wrapper scripts from build environment to installation directory (if available)#3661

Merged
boegel merged 7 commits intoeasybuilders:developfrom
ocaisa:buildenv_rpath
Apr 23, 2025
Merged

enhance BuildEnv generic easyblock to copy RPATH wrapper scripts from build environment to installation directory (if available)#3661
boegel merged 7 commits intoeasybuilders:developfrom
ocaisa:buildenv_rpath

Conversation

@ocaisa
Copy link
Copy Markdown
Member

@ocaisa ocaisa commented Mar 13, 2025 •
edited by boegel
Loading

ocaisa
ocaisa commented Mar 14, 2025
View reviewed changes
Comment thread easybuild/easyblocks/generic/buildenv.py Outdated
@ocaisa
Copy link
Copy Markdown
Member Author

ocaisa commented Mar 14, 2025

Just to show this in action:

ocaisa@LAPTOP-O6HF2IKC:~$ module load buildenv
ocaisa@LAPTOP-O6HF2IKC:~$ mpicc hello_world.c

ocaisa@LAPTOP-O6HF2IKC:~$ cat hello_world.c
/*The Parallel Hello World Program*/
#include <stdio.h>
#include <mpi.h>
int main(int argc, char **argv)
{
   int node;
   MPI_Init(&argc,&argv);
   MPI_Comm_rank(MPI_COMM_WORLD, &node);
   printf("Hello World from Node %d\n",node);
   MPI_Finalize();
}

ocaisa@LAPTOP-O6HF2IKC:~$ mpirun -n 2 a.out
Hello World from Node 1
Hello World from Node 0

ocaisa@LAPTOP-O6HF2IKC:~$ ldd a.out
        linux-vdso.so.1 (0x00007ffd0e779000)
        libmpi.so.40 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenMPI/4.1.6-GCC-13.2.0/lib/libmpi.so.40 (0x00007febae519000)
        libc.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib/../lib64/libc.so.6 (0x00007febae348000)
        libopen-rte.so.40 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenMPI/4.1.6-GCC-13.2.0/lib/libopen-rte.so.40 (0x00007febae28d000)
        libopen-pal.so.40 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenMPI/4.1.6-GCC-13.2.0/lib/libopen-pal.so.40 (0x00007febae1e0000)
        libhwloc.so.15 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/hwloc/2.9.2-GCCcore-13.2.0/lib64/libhwloc.so.15 (0x00007febae181000)
        libpciaccess.so.0 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libpciaccess/0.17-GCCcore-13.2.0/lib64/libpciaccess.so.0 (0x00007febae173000)
        libxml2.so.2 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libxml2/2.11.5-GCCcore-13.2.0/lib64/libxml2.so.2 (0x00007febae016000)
        libz.so.1 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/../lib64/libz.so.1 (0x00007febadffc000)
        liblzma.so.5 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/../lib64/liblzma.so.5 (0x00007febadfcc000)
        libevent_core-2.1.so.7 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libevent/2.1.12-GCCcore-13.2.0/lib64/libevent_core-2.1.so.7 (0x00007febadf97000)
        libevent_pthreads-2.1.so.7 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libevent/2.1.12-GCCcore-13.2.0/lib64/libevent_pthreads-2.1.so.7 (0x00007febadf90000)
        libm.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib/../lib64/libm.so.6 (0x00007febadeb0000)
        /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 (0x00007febae644000)

ocaisa@LAPTOP-O6HF2IKC:~$ readelf -d a.out | grep '(RPATH)'
 0x000000000000000f (RPATH)              Library rpath: [/cvmfs/software.eessi.io/host_injections/2023.06/software/linux/x86_64/intel/skylake_avx512/rpath_overrides/OpenMPI/system/lib:/cvmfs/software.eessi.io/host_injections/2023.06/software/linux/x86_64/intel/skylake_avx512/rpath_overrides/OpenMPI/system/lib64:/home/ocaisa/eessi/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/buildenv/default-foss-2023b/lib:/home/ocaisa/eessi/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/buildenv/default-foss-2023b/lib64:$ORIGIN:$ORIGIN/../lib:$ORIGIN/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenMPI/4.1.6-GCC-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/hwloc/2.9.2-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libevent/2.1.12-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/ScaLAPACK/2.2.0-gompi-2023b-fb/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/FFTW.MPI/3.3.10-gompi-2023b/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/FFTW/3.3.10-GCC-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/FlexiBLAS/3.3.1-GCC-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenBLAS/0.3.24-GCC-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/UCC/1.2.0-GCCcore-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/PMIx/4.2.6-GCCcore-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libfabric/1.19.0-GCCcore-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/UCX/1.15.0-GCCcore-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenSSL/1.1/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libpciaccess/0.17-GCCcore-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libxml2/2.11.5-GCCcore-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/numactl/2.0.16-GCCcore-13.2.0/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/GCCcore/13.2.0/lib/gcc/x86_64-pc-linux-gnu/13.2.0:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/GCCcore/13.2.0/lib/gcc/x86_64-pc-linux-gnu/13.2.0/../../../../lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/../lib64:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/GCCcore/13.2.0/lib/gcc/x86_64-pc-linux-gnu/13.2.0/../../..:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/ScaLAPACK/2.2.0-gompi-2023b-fb/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/FFTW.MPI/3.3.10-gompi-2023b/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/FFTW/3.3.10-GCC-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/FlexiBLAS/3.3.1-GCC-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenBLAS/0.3.24-GCC-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/UCC/1.2.0-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/PMIx/4.2.6-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libfabric/1.19.0-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/UCX/1.15.0-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenSSL/1.1/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libpciaccess/0.17-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libxml2/2.11.5-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/numactl/2.0.16-GCCcore-13.2.0/lib]

@ocaisa ocaisa mentioned this pull request Mar 17, 2025
Merged
@boegel boegel changed the base branch from 5.0.x to develop March 19, 2025 11:07
@boegel boegel added this to the 5.x milestone Mar 19, 2025
@boegel
Copy link
Copy Markdown
Member

boegel commented Mar 19, 2025

@ocaisa I changed to target branch in this PR from 5.0.x to develop, you should synchronize your PR branch with current develop branch (which has received a massive update after the release of EasyBuild v5.0.0, see #3670)

@boegel boegel modified the milestones: 5.x.x, 5.x Apr 9, 2025
Micket
Micket reviewed Apr 15, 2025
View reviewed changes
Comment thread easybuild/easyblocks/generic/buildenv.py Outdated
boegel
boegel reviewed Apr 23, 2025
View reviewed changes
Comment thread easybuild/easyblocks/generic/buildenv.py Outdated
@boegel boegel mentioned this pull request Apr 23, 2025
Merged
@boegel
set self.rpath_wrappers_dir to build dir + copy RPATH wrapper script …
2f1ffb4 
…to install dir in install step of buildenv easyblock
@boegel boegel mentioned this pull request Apr 23, 2025
Merged
@ocaisa
Merge pull request #19 from boegel/buildenv_rpath
3f60df6 
set `self.rpath_wrappers_dir` to build dir + copy RPATH wrapper script to install dir in install step of buildenv easyblock
@boegel boegel modified the milestones: 5.x, 5.0.1 Apr 23, 2025
@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 23, 2025 •
edited
Loading

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS buildenv-default-intel-2023a.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3507.doduo.os - Linux RHEL 9.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.9.18
See https://gist.github.com/boegel/bfbc0a3ccc7776b7c07681a4cd008a56 for a full test report.

edit: with RPATH enabled

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 23, 2025 •
edited
Loading

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS buildenv-default-intel-2023a.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3507.doduo.os - Linux RHEL 9.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.9.18
See https://gist.github.com/boegel/77a2b2874be4c24dea339fcf48483007 for a full test report.

edit: with RPATH disabled

@boegel boegel changed the title Support exporting rpath wrappers in buildenv easyblock enhance BuildEnv generic easyblock to copy RPATH wrapper scripts from build environment to installation directory (if available) Apr 23, 2025
@boegel boegel merged commit 5fda10d into easybuilders:develop Apr 23, 2025
17 checks passed
@ocaisa ocaisa mentioned this pull request Aug 27, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Micket Micket Micket left review comments

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

enhancement

Projects

None yet

Milestone

5.1.0

Development

Successfully merging this pull request may close these issues.

3 participants

@ocaisa @boegel @Micket