patch Java binaries/libraries when using alternate sysroot to ensure correct glibc & co are picked up + add custom sanity check#2557
Conversation
…correct glibc & co are picked up + add basic custom sanity check for Java
|
Test report by @boegel Overview of tested easyconfigs (in order)
Build succeeded for 4 out of 4 (2 easyconfigs in total) edit: no |
|
Test report by @boegel Overview of tested easyconfigs (in order)
Build succeeded for 48 out of 48 (48 easyconfigs in total) edit: no |
|
Test report by @boegel Overview of tested easyconfigs (in order)
Build succeeded for 4 out of 4 (2 easyconfigs in total) edit: no |
…or ppc64le systems)
ocaisa
left a comment
There was a problem hiding this comment.
With a few changes the post install step could be made completely independent of the package and placed in framework instead.
| if elf_interp is None: | ||
| raise EasyBuildError("Failed to isolate ELF interpreter!") | ||
|
|
||
| bindir = os.path.join(self.installdir, 'bin') |
There was a problem hiding this comment.
Everything up until now has been independent of the actual package being installed. Rather than an explicit bin, can we just check all the dirs that are being added to PATH?
There was a problem hiding this comment.
Are we "guaranteed" to have the additions to PATH finalised by this point?
There was a problem hiding this comment.
Not really, since make_module_req_guess and make_module_extra are generally responsible for specifying what gets included in $PATH (and $LD_LIBRARY_PATH).
The way that's done currently is by emitting Tcl or Lua statements to update these environment variables directly, we would need to do some extra bookkeeping to know what's being added to $PATH and/or $LD_LIBRARY_PATH in a clean way in post_install_step already...
There was a problem hiding this comment.
make_module_req_guess just returns a dict which contains the guess for PATH and LIBRARY_PATH, so we can just call it? We also know that make_module_extra doesn't handle PATH or LIBRARY_PATH, unless there is a reason for an exception somewhere
There was a problem hiding this comment.
(I'm explicitly saying LIBRARY_PATH as we filter LD_LIBRARY_PATH in EESSI)
| curr_rpath, _ = run_cmd("patchelf --print-rpath %s" % path, simple=False, trace=False) | ||
| self.log.debug("RPATH for %s (after shrinking): %s" % (path, curr_rpath)) | ||
|
|
||
| libdir = os.path.join(self.installdir, 'lib') |
There was a problem hiding this comment.
Same here, can we just check all the paths that are being added to LIBRARY_PATH
|
@ocaisa I'm fine with leveraging some of the code being added here to framework, but I feel we can still do that later, maybe only once we have a second use case for using I would really like to see this PR merged as is, since it has been used to install |
|
Sure we can move it to framework later, but I think it is already good practice to leverage the additions to EDIT: |
Use generic approach to decide what to patchelf
|
Retested in EESSI pilot 2021.12, still works like a charm. |
|
Test report by @boegel Overview of tested easyconfigs (in order)
Build succeeded for 8 out of 8 (8 easyconfigs in total) |
Enhancement for custom easyblock for Java to fix problems that occur when using
Javain the context of an alternate sysroot and using RPATH (see EESSI/software-layer#123)Also adds basic custom sanity check for Java, including running
java -helpandjavac -help, which work for Java 1.7.x, 1.8.x, 11.x, 13.xThis is a draft PR because this should be cleaned up a bit, by adding support in EasyBuild framework for leveraging
patchelf, so it's a lot cleaner to patch a binary installation like this (we may also need this for other stuff, like CUDA, Intel tools, etc.)