Skip to content

Also handle references to git tags in Cargo crates and download git repos recursively#4081

Merged
Micket merged 4 commits intoeasybuilders:developfrom
Flamefire:20260227142354_new_pr_cargo
Mar 3, 2026
Merged

Also handle references to git tags in Cargo crates and download git repos recursively#4081
Micket merged 4 commits intoeasybuilders:developfrom
Flamefire:20260227142354_new_pr_cargo

Conversation

@Flamefire
Copy link
Copy Markdown
Contributor

@Flamefire Flamefire commented Feb 27, 2026 •
edited
Loading

(created using eb --new-pr)

We already handle Cargo.toml files with "git+https://github.com/ctsa/rust-libbigwig.git?branch=feature" by adding branch = "feature" to the relevant entry of the lock file.

We need the same for tags: "git+https://github.com/ctsa/rust-libbigwig.git?tag=v0.4.0"

Maybe we shouldn't strip tags from existing lock files when automatically creating the crates easyconfig parameter, e.g. from:

[[package]]
name = "rust-libbigwig"
version = "0.4.0"
source = "git+https://github.com/ctsa/rust-libbigwig.git?tag=v0.4.0#1ce3d66a07a9b4d195984775c10bc82333afeab1"

cargo vendor seems to use the tag only:

[source."git+https://github.com/ctsa/rust-libbigwig.git?tag=v0.4.0"]
git = "https://github.com/ctsa/rust-libbigwig.git"
tag = "v0.4.0"
replace-with = "vendored-sources"

@Micket
Copy link
Copy Markdown
Contributor

Micket commented Feb 27, 2026

regex is broken, crashing easybuild

EasyBuild crashed! Please consider reporting a bug, this should not happen...

Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/main.py", line 862, in <module>
    main_with_hooks()
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/main.py", line 847, in main_with_hooks
    exit_code: EasyBuildExit = main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/main.py", line 798, in main
    is_successful = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/main.py", line 614, in process_eb_args
    ecs_with_res = build_and_install_software(
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/main.py", line 226, in build_and_install_software
    raise error
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/main.py", line 180, in build_and_install_software
    (ec_res['success'], app_log, err_msg, err_code) = build_and_install_one(ec, init_env)
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 5120, in build_and_install_one
    result = app.run_all_steps(run_test_cases=run_test_cases)
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 4932, in run_all_steps
    self.run_step(step_name, step_methods)
  File "/apps/Common/software/EasyBuild/5.2.1/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 4773, in run_step
    current_method()
  File "/dev/shm/included-easyblocks-xvrnwnd_/easybuild/easyblocks/generic/cargo.py", line 353, in extract_step
    self._setup_offline_config(git_sources)
  File "/dev/shm/included-easyblocks-xvrnwnd_/easybuild/easyblocks/generic/cargo.py", line 439, in _setup_offline_config
    git_refs = self._get_crate_git_repo_refs(crate_name)
  File "/dev/shm/included-easyblocks-xvrnwnd_/easybuild/easyblocks/generic/cargo.py", line 467, in _get_crate_git_repo_refs
    git_branch_spec = re.compile(r'(?<ref>branch|tag)\s*=\s*"(?<value>[^"]*)"', re.M)
  File "/usr/lib64/python3.9/re.py", line 252, in compile
    return _compile(pattern, flags)
  File "/usr/lib64/python3.9/re.py", line 304, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/lib64/python3.9/sre_compile.py", line 788, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/lib64/python3.9/sre_parse.py", line 955, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/lib64/python3.9/sre_parse.py", line 444, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/lib64/python3.9/sre_parse.py", line 749, in _parse
    raise source.error("unknown extension ?<" + char,
re.error: unknown extension ?<r at position 1

@Flamefire Flamefire changed the title Also handle references to git tags in Cargo crates Also handle references to git tags in Cargo crates and download git repos recursively Feb 27, 2026
Micket
Micket approved these changes Feb 27, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@Micket Micket left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Micket Micket added the bug fix label Feb 27, 2026
@Micket Micket added this to the next release (5.2.2?) milestone Feb 27, 2026
@Micket
Copy link
Copy Markdown
Contributor

Micket commented Feb 27, 2026

Working on easyconfig to test soon

@Micket Micket mentioned this pull request Feb 27, 2026
Merged
1 task
Micket
Micket reviewed Mar 2, 2026
View reviewed changes
Comment thread easybuild/easyblocks/generic/cargo.py
@Micket
Copy link
Copy Markdown
Contributor

Micket commented Mar 3, 2026

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS cargo-c-0.10.15-GCCcore-14.3.0.eb

  • SUCCESS cargo-pgo-0.3.0-GCCcore-14.3.0.eb

  • SUCCESS rav1e-0.8.1-GCCcore-14.3.0.eb

Build succeeded for 3 out of 3 (total: 5 mins 29 secs) (3 easyconfigs in total)
vera-icelake-build - Linux Rocky Linux 9.6, x86_64, Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz, Python 3.9.21
See https://gist.github.com/Micket/f45cf953994b6072a7046fb052c6fb98 for a full test report.

@Micket
Copy link
Copy Markdown
Contributor

Micket commented Mar 3, 2026

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS ruff-0.14.3-GCCcore-14.3.0.eb

  • SUCCESS bcrypt-4.3.0-GCCcore-14.3.0.eb

  • SUCCESS tlparse-0.4.3-GCCcore-14.3.0.eb

Build succeeded for 3 out of 3 (total: 10 mins 51 secs) (3 easyconfigs in total)
vera-icelake-build - Linux Rocky Linux 9.6, x86_64, Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz, Python 3.9.21
See https://gist.github.com/Micket/a41b71957e8eed20516a6c5e3611c97f for a full test report.

@Micket Micket merged commit 7059ac7 into easybuilders:develop Mar 3, 2026
22 checks passed
@Flamefire Flamefire deleted the 20260227142354_new_pr_cargo branch March 3, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Micket Micket Micket approved these changes

Assignees

No one assigned

Labels

bug fix

Projects

None yet

Milestone

5.3.0

Development

Successfully merging this pull request may close these issues.

2 participants

@Flamefire @Micket