Enhance sitecustomize.py script installed by Python easyblock to fix support for virtualenv#2483
Merged
boegel merged 3 commits intoeasybuilders:developfrom Jun 30, 2021
Conversation
99543f0 to
4d53333
Compare
boegel
requested changes
Jun 21, 2021
Member
boegel
left a comment
There was a problem hiding this comment.
@Flamefire Can you edit the PR description to mention the error messages that pop up unless this is included?
Member
|
Test report by @boegel Overview of tested easyconfigs (in order)
Build succeeded for 4 out of 4 (4 easyconfigs in total) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(created using
eb --new-pr)virtualenvseems to bundle a customsite.pywhich in versions prior to 20.something does not havesite.getsitepackages(), see e.g. pypa/virtualenv#737The error in that case is
AttributeError: module 'site' has no attribute 'getsitepackages'The workaround is a bit hacky but seems to work across various versions of Python/virtualenv.
Some problems considered/solved:
base_prefixis supposed to contain the python prefix, but does not in early virtualenv versions, it instead contains the virtualenv prefixreal_prefixwas used instead but later removedgetsitepackagessite-packagesHence the very conservative approach of trying getting the first of all possible prefixes and moving all paths starting with that after the EBPYTHONPREFIXES
I also added a sanity check to check the base case, i.e. site-customize does work at all and adds the new site-packages path before the core site-packages path. (conservative check, we might want to limit to moving paths only ending with site-packages)
Testing that this works with virtualenv is hard, as we'd actually need to install a virtualenv. Not sure if we really want to do that.
The script I used to verify the new site-customize.py: checkPythonModule.sh