Skip to content

Allow 'nosource: True' in PythonPackage extensions#3305

Merged
smoors merged 1 commit intoeasybuilders:developfrom
Flamefire:20240412170740_new_pr_pythonpackage
Apr 18, 2024
Merged

Allow 'nosource: True' in PythonPackage extensions#3305
smoors merged 1 commit intoeasybuilders:developfrom
Flamefire:20240412170740_new_pr_pythonpackage

Conversation

@Flamefire
Copy link
Copy Markdown
Contributor

@Flamefire Flamefire commented Apr 12, 2024 •
edited
Loading

(created using eb --new-pr)

To be used with easybuilders/easybuild-framework#4506

Motivation is a use case like: https://github.com/ComputeCanada/easybuild-easyconfigs/blob/5573c66a91fb2990723da630e7eb690957a59f46/easybuild/easyconfigs/d/DuckDB/DuckDB-0.10.1-GCCcore-12.3.0.eb

easyblock = 'CMakeNinja'

...

exts_defaultclass = 'PythonPackage'
exts_list = [
    ("%(namelower)s", version, {
        'nosource': True, # bundled already
        'start_dir': 'tools/pythonpkg/',
        'preconfigopts': 'export DUCKDB_BINARY_DIR=%(installdir)s &&',
        'use_pip': True,
    }),
]

I.e. we want to build using some build system which has a python package included or as a build artifact. Hence we don't need nor want to use a source.

With these 2 PRs it gets to the correct pip install . in the right directory. I couldn't test further as it fails to get the version using setuptools_scm which tries to use git on a source-only tarball.

@smoors
Copy link
Copy Markdown
Contributor

smoors commented Apr 18, 2024

I couldn't test further as it fails to get the version using setuptools_scm which tries to use git on a source-only tarball.

this can be worked around by setting SETUPTOOLS_SCM_PRETEND_VERSION envvar:

'preinstallopts': 'export SETUPTOOLS_SCM_PRETEND_VERSION=%(version)s && '

@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented Apr 18, 2024

this can be worked around by setting SETUPTOOLS_SCM_PRETEND_VERSION envvar:

'preinstallopts': 'export SETUPTOOLS_SCM_PRETEND_VERSION=%(version)s && '

Works, thanks!
This is the EC I used: DuckDB-0.10.1-GCCcore-12.3.0.eb

@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented Apr 18, 2024

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS DuckDB-0.10.1-GCCcore-12.3.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
n1493 - Linux RHEL 8.7 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (icelake), Python 3.8.13
See https://gist.github.com/Flamefire/5e2eef06a838a56bf94b74c61b65aaef for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Apr 18, 2024

I couldn't test further as it fails to get the version using setuptools_scm which tries to use git on a source-only tarball.

this can be worked around by setting SETUPTOOLS_SCM_PRETEND_VERSION envvar:

'preinstallopts': 'export SETUPTOOLS_SCM_PRETEND_VERSION=%(version)s && '

How insane would it be to let PythonPackage automatically set this environment variable when installing an extension?
Would that cause trouble?

@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented Apr 18, 2024

How insane would it be to let PythonPackage automatically set this environment variable when installing an extension? Would that cause trouble?

Hm, I can't think of any issues. We always want the version of the software as the version. And if this avoids even trying to use git (where either the binary or the .git folder may not exist) this would be two birds with one stone: Faster and less issues

@smoors
Copy link
Copy Markdown
Contributor

smoors commented Apr 18, 2024 •
edited
Loading

Hm, I can't think of any issues. We always want the version of the software as the version. And if this avoids even trying to use git (where either the binary or the .git folder may not exist) this would be two birds with one stone: Faster and less issues

i agree. there will be slight differences in exceptional cases with a git checkout of a commit. for example, for GBprocesS/4.0.0.post1-foss-2022a i get

>>> import gbprocess
>>> gbprocess.__version__
'4.0.0.post2.dev0+g26119ab.d20240418'

in gitlab this commit is tagged as 4.0.0.post1, so i'd say our version is more useful in this case

@smoors smoors added this to the release after 4.9.1 milestone Apr 18, 2024
smoors
smoors approved these changes Apr 18, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@smoors smoors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@smoors
Copy link
Copy Markdown
Contributor

smoors commented Apr 18, 2024

Going in, thanks @Flamefire!

@smoors smoors merged commit 9473538 into easybuilders:develop Apr 18, 2024
@Flamefire Flamefire deleted the 20240412170740_new_pr_pythonpackage branch April 18, 2024 15:59
@boegel boegel changed the title Allow 'nosource: True' in PythonPackage extensions' Allow 'nosource: True' in PythonPackage extensions Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@smoors smoors smoors approved these changes

Assignees

No one assigned

Labels

enhancement

Projects

None yet

Milestone

4.9.2

Development

Successfully merging this pull request may close these issues.

3 participants

@Flamefire @smoors @boegel