Skip to content

add support for injecting checksums for Rust crates#4661

Merged
boegel merged 5 commits intoeasybuilders:developfrom
Flamefire:cargo-checksums
Feb 11, 2026
Merged

add support for injecting checksums for Rust crates#4661
boegel merged 5 commits intoeasybuilders:developfrom
Flamefire:cargo-checksums

Conversation

@Flamefire
Copy link
Copy Markdown
Contributor

@Flamefire Flamefire commented Sep 27, 2024

Even though the crates are recognized as sources for Cargo easyconfigs the checksums are not written, at least when neither 'source_urls', 'sources' nor 'patches' are present.

Handle crates like sources and add test.

boegel
boegel reviewed Oct 9, 2024
View reviewed changes
Comment thread easybuild/framework/easyblock.py Outdated
# grab raw lines for source_urls, sources, patches
keys = ['patches', 'source_urls', 'sources']
# grab raw lines for the following params
keys = ['source_urls', 'sources', 'crates', 'patches']
Copy link
Copy Markdown
Member

@boegel boegel Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this approach is a bit weird, as it breaks the boundary between framework and easyblocks, since crates is not a general easyconfig parameter...

Thoughts on this @Micket?

Copy link
Copy Markdown
Member

@boegel boegel Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we come up with an approach that would also work for components (in Bundle easyblock)?

Copy link
Copy Markdown
Contributor Author

@Flamefire Flamefire Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the confcall I added a method src_parameter_names to the easyblock class.

In the Cargo easyblock we can override this to also return "crates".

For components I don't think this is useful. For now they look like:

components = [
    (name, version, {
        'source_urls': ['https://poppler.freedesktop.org/'],
        'sources': [SOURCE_TAR_XZ],
        'checksums': ['86b09e5a02de40081a3916ef8711c5128eaf4b1fc59d5f87d0ec66f04f595db4'],
        'configopts': "-DENABLE_BOOST=ON",
    }),
    ('poppler-data', '0.4.10', {
        'source_urls': ['https://poppler.freedesktop.org/'],
        'sources': [SOURCE_TAR_GZ],
        'checksums': ['6e2fcef66ec8c44625f94292ccf8af9f1d918b410d5aa69c274ce67387967b30'],
    }),
]

While the Bundle easyblock seemingly supports specifying checksums for everything at the top-level (outside the components) I don't think it actually does: It creates a full list of sources, patches and checksums during __init__ but the actual checksum check (at least the contrib check) requires them in each component as above.

However with this design it would be possible to add the checksum list after the components but then we'd need to include logic to remove them from each component

The general approach is different here anyway: crates are just another way to specify sources while components are similar to exts_list which yield actual easyblock instances.

@boegel boegel added this to the 4.x milestone Oct 9, 2024
@Flamefire Flamefire force-pushed the cargo-checksums branch 3 times, most recently from 367617f to 74f2842 Compare July 22, 2025 14:36
@Flamefire Flamefire force-pushed the cargo-checksums branch 3 times, most recently from 2d391e5 to bfa5835 Compare October 8, 2025 15:10
@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented Nov 18, 2025

Rebased to resolve conflicts with recent updates.

@boegel boegel modified the milestones: 5.x, release after 5.2.0 Nov 19, 2025
@Flamefire
add support for injecting checksums for cargo crates
9123ee3 
Even though the `crates` are recognized as sources for `Cargo`
easyconfigs the checksums are not written, at least when
neither 'source_urls', 'sources' nor 'patches' are present.

Handle `crates` like `sources` and add test-
@Flamefire
Fix tests
f2464b8 
The added EC and EasyBlock requires changes to the expected output in
some tests.
@Flamefire
Add customization point for easyblocks to specify additional paramete…
3f3c63f 
…rs for "sources"

Some easyblocks such as `Cargo` will use a custom EasyConfig parameters
to fill the `sources` list.
To avoid hard-coding them `inject_checksums` introduce a static method
`src_parameter_names` to return a list of parameters that contribute to
the final list of `sources` when parsing the EasyConfig.
@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented Feb 10, 2026

@boegel Rebased again. Anything left?

@boegel
Copy link
Copy Markdown
Member

boegel commented Feb 11, 2026

@Flamefire We need a companion PR for Cargo easyblock that redefines this new src_parameter_names static method, correct?

@boegel
Copy link
Copy Markdown
Member

boegel commented Feb 11, 2026

@Flamefire We need a companion PR for Cargo easyblock that redefines this new src_parameter_names static method, correct?

Ah, no, it's already there in Cargo, it snuck in through this PR:

@boegel
Copy link
Copy Markdown
Member

boegel commented Feb 11, 2026

tested with this develop easyblock using eb --inject-checksums skani-0.2.2-GCCcore-12.3.0.eb, and something isn't quite right (may need a fix in Cargo easyblock)?

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/kehoste/work/easybuild-framework/easybuild/main.py", line 859, in <module>
    main_with_hooks()
    ~~~~~~~~~~~~~~~^^
  File "/Users/kehoste/work/easybuild-framework/easybuild/main.py", line 844, in main_with_hooks
    exit_code: EasyBuildExit = main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))
                               ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kehoste/work/easybuild-framework/easybuild/main.py", line 795, in main
    is_successful = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
                                    hooks, do_build)
  File "/Users/kehoste/work/easybuild-framework/easybuild/main.py", line 574, in process_eb_args
    inject_checksums(ordered_ecs, options.inject_checksums)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kehoste/work/easybuild-framework/easybuild/framework/easyblock.py", line 5615, in inject_checksums
    keys = ['data_sources', 'source_urls'] + app.src_parameter_names() + ['patches']
                                             ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/kehoste/work/easybuild-easyblocks/easybuild/easyblocks/generic/cargo.py", line 174, in src_parameter_names
    return super().src_parameter_names() + ['crates']
           ~~~~~^^
RuntimeError: super(): no arguments

@Flamefire Flamefire mentioned this pull request Feb 11, 2026
Merged
@Flamefire
Copy link
Copy Markdown
Contributor Author

Flamefire commented Feb 11, 2026

Seems the arguments to super are required for static methods.
Fixed in easybuilders/easybuild-easyblocks#4062 which can be merged independently but before this one

boegel
boegel approved these changes Feb 11, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

boegel
boegel requested changes Feb 11, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Comment thread test/framework/sandbox/easybuild/easyblocks/generic/cargo.py Outdated
@boegel boegel merged commit af71154 into easybuilders:develop Feb 11, 2026
40 of 41 checks passed
@Flamefire Flamefire deleted the cargo-checksums branch February 12, 2026 08:44
@boegel boegel changed the title add support for injecting checksums for cargo crates add support for injecting checksums for Rust crates Feb 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@boegel boegel boegel approved these changes

Assignees

No one assigned

Labels

enhancement

Projects

None yet

Milestone

5.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@Flamefire @boegel