Skip to content

enable keeping of symbolic links by default when copying (set keepsymlinks easyconfig parameter to True)#4773

Merged
boegel merged 2 commits intoeasybuilders:5.0.xfrom
bartoldeman:keepsymlinks
Mar 2, 2025
Merged

enable keeping of symbolic links by default when copying (set keepsymlinks easyconfig parameter to True)#4773
boegel merged 2 commits intoeasybuilders:5.0.xfrom
bartoldeman:keepsymlinks

Conversation

@bartoldeman
Copy link
Copy Markdown
Contributor

Not keeping symlinks caused surprising behaviour with for example installing Julia using the Tarball easyblock: it gives copies instead of symbolic links for each shared library, when they are symbolic links inside the tarball. This reverses the default behaviour to be less surprising.

Not keeping symlinks caused surprising behaviour with for example
installing Julia using the Tarball easyblock: it gives copies instead
of symbolic links for each shared library, when they are symbolic
links inside the tarball. This reverses the default behaviour to be less
surprising.
@bartoldeman bartoldeman added this to the 5.0 milestone Feb 26, 2025
@bartoldeman bartoldeman marked this pull request as draft February 26, 2025 13:45
@bartoldeman bartoldeman marked this pull request as ready for review February 26, 2025 14:57
@bartoldeman
Copy link
Copy Markdown
Contributor Author

Added a simple test. There was nothing in easyblocks to change, a few custom easyblocks explicitly set keepsymlinks to True but they can keep doing that.

@boegel boegel changed the title Change keepsymlinks default from False to True enable keepsymlinks by default Feb 26, 2025
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

This makes sense to me.

It's a bit difficult to figure out why we have this disabled by default, this was introduced before we started using Git (in March 2012...).

We should at least test some easyblocks that take into account keepsymlinks to see if there's any unexpected fallout, but I would be surprised if there is.

@bartoldeman
Copy link
Copy Markdown
Contributor Author

bartoldeman commented Feb 27, 2025

Tested Blender (PackedBinary EasyBlock), which has copies instead of symbolic links in our installed modules:
https://gist.github.com/bartoldeman/3fcdbb93943f4bc102ecc49e4d961789

Test result

Build succeeded for 1 out of 1

Overview of tested easyconfigs (in order)

  • SUCCESS Blender-4.0.1-linux-x86_64-CUDA-12.1.1.eb

@bartoldeman
Copy link
Copy Markdown
Contributor Author

bartoldeman commented Feb 27, 2025

Tarball EasyBlock: Julia-1.11.3-linux-x86_64.eb (motivation for this PR)
https://gist.github.com/bartoldeman/bc124203a3e88d37fe004ddb3d2fc1d0

Test result

Build succeeded for 1 out of 1

Overview of tested easyconfigs (in order)

  • SUCCESS Julia-1.11.3-linux-x86_64.eb

@bartoldeman
Copy link
Copy Markdown
Contributor Author

Test result

Build succeeded for 1 out of 1

Overview of tested easyconfigs (in order)

  • SUCCESS NCCL-2.8.3-CUDA-11.1.1.eb

https://gist.github.com/bartoldeman/c28c907599ffe11c4575e14e6d375b39

Using MakeCp

before

-rwxr-xr--. 1 oldeman oldeman 158017160 Oct 26  2020 libnccl.so
-rwxr-xr--. 1 oldeman oldeman 158017160 Oct 26  2020 libnccl.so.2
-rwxr-xr--. 1 oldeman oldeman 158017160 Oct 26  2020 libnccl.so.2.8.3
-rw-r--r--. 1 oldeman oldeman 201656828 Oct 26  2020 libnccl_static.a
drwxr-xr-x. 2 oldeman oldeman         1 Oct 26  2020 pkgconfig

after

lrwxrwxrwx. 1 oldeman oldeman        12 Oct 26  2020 libnccl.so -> libnccl.so.2
lrwxrwxrwx. 1 oldeman oldeman        16 Oct 26  2020 libnccl.so.2 -> libnccl.so.2.8.3
-rwxr-xr--. 1 oldeman oldeman 158017160 Oct 26  2020 libnccl.so.2.8.3
-rw-r--r--. 1 oldeman oldeman 201656828 Oct 26  2020 libnccl_static.a
drwxr-xr-x. 2 oldeman oldeman         1 Oct 26  2020 pkgconfig

With these 3 I have all easyblocks that directly use keepsymlinks covered (Tarball, MakeCp, and Binary via PackedBinary)

@boegel boegel merged commit 19964d3 into easybuilders:5.0.x Mar 2, 2025
@boegel boegel removed this from EasyBuild v5.0 Mar 2, 2025
@boegel boegel changed the title enable keepsymlinks by default enable keeping of symbolic links by default when copying (set keepsymlinks easyconfig parameter to True) Mar 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants