Skip to content

Use uv for CI build and deploy on GitHub action and CircleCI#618

Closed
willingc wants to merge 20 commits intonapari:mainfrom
willingc:ci-uv-security
Closed

Use uv for CI build and deploy on GitHub action and CircleCI#618
willingc wants to merge 20 commits intonapari:mainfrom
willingc:ci-uv-security

Conversation

@willingc
Copy link
Copy Markdown
Contributor

@willingc willingc commented Mar 11, 2025 •
edited
Loading

Description

This PR improves observability and performance for GitHub action and CircleCI when building and deploying docs.

Pre-Doc build scripts

This PR:

  • edits the scripts for pre-doc build for maintainability and to notify more often that scripts have not stalled
  • creates a scripts logger for the pre-doc build scripts to improve observability and troubleshooting
  • sets the log level to INFO when running in CI; reset to DEBUG if needed locally for troubleshooting

GitHub Action

This PR:

  • Corrects errors found by zizmor tool for static security analysis of actions
  • Uses uv for build and deploy for faster dependency resolution
  • Reduces fetch-depth
  • Add logging to scripts to notify more often that scripts have not stalled
  • Use uv caching for small speedup

CircleCI

This makes similar improvements using uv that was done in the GitHub action.

@github-actions github-actions Bot added the maintenance CI, dependencies, and other maintenance label Mar 11, 2025
Czaki
Czaki reviewed Mar 11, 2025
View reviewed changes
Comment thread .github/workflows/build_and_deploy.yml
@willingc
Copy link
Copy Markdown
Contributor Author

willingc commented Mar 12, 2025

@Czaki I got this running green when using uv. Going to look at further optimizing things. Saves a few minutes so far on this run.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Mar 12, 2025
@willingc willingc changed the title Add uv ci proof of concept Use uv for CI build and deploy Mar 12, 2025
@psobolewskiPhD
Copy link
Copy Markdown
Member

psobolewskiPhD commented Mar 13, 2025

Do you want to port these changes to CircleCI workflow as well?

@psobolewskiPhD
Copy link
Copy Markdown
Member

psobolewskiPhD commented Mar 15, 2025 •
edited
Loading

Looks like circle may take some time: they just merged uv support:
CircleCI-Public/cimg-python#257
and this PR is still open: CircleCI-Public/python-orb#124

@willingc willingc changed the title Use uv for CI build and deploy Use uv for CI build and deploy on GitHub action and CircleCI Mar 15, 2025
@willingc
Copy link
Copy Markdown
Contributor Author

willingc commented Mar 15, 2025

@psobolewskiPhD I got it to pass on CircleCI by installing uv into the image until the circle image with uv is released.

@psobolewskiPhD
Copy link
Copy Markdown
Member

psobolewskiPhD commented Mar 15, 2025

Looks like it saves about a minute. Worth it I guess?
We'll need to make an issue to remember to update once uv is fully supported.

@willingc
Copy link
Copy Markdown
Contributor Author

willingc commented Mar 15, 2025 •
edited
Loading

I would say if it a minute is saved each run that it makes good sense if it runs 10x/day.

Czaki
Czaki reviewed Mar 17, 2025
View reviewed changes
Comment thread .circleci/config.yml
- run:
name: Install qt libs + xvfb
command: sudo apt-get update && sudo apt-get install -y xvfb libegl1 libdbus-1-3 libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xinput0 libxcb-xfixes0 x11-utils
# [TODO] Remove this step once uv is installed by default in the cimg/python:3.10 image
Copy link
Copy Markdown
Contributor

@Czaki Czaki Mar 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does uv is available on more modern images (3.11 or 3.12)?

Copy link
Copy Markdown
Contributor Author

@willingc willingc Mar 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR adding uv was merged into the image 3 days ago (thanks @psobolewskiPhD). Unfortunately, it has been over a month since the images have been updated on docker hub. There are a few other issues that may prevent quick release https://github.com/CircleCI-Public/cimg-python/issues so I think this is a reasonable approach until the release happens in the future.

@willingc
Copy link
Copy Markdown
Contributor Author

willingc commented Mar 17, 2025

Once this lands I will open a PR on napari/napari repo to keep circleci config in sync.

@psobolewskiPhD
Copy link
Copy Markdown
Member

psobolewskiPhD commented Apr 12, 2025

@willingc
looks like images on Circle might be updated:
CircleCI-Public/cimg-python#267 (comment)

Also do you think we could break out the uv/actions bit from the scripts changes (merge conflicts)?

@willingc
Copy link
Copy Markdown
Contributor Author

willingc commented Apr 15, 2025

@psobolewskiPhD I will revisit this PR once I get word from you that all of your scripts changes are up and merged. Thanks.

@willingc willingc mentioned this pull request Apr 15, 2025
Merged
@willingc
Copy link
Copy Markdown
Contributor Author

willingc commented Apr 15, 2025

PR #673 drops the build time on CircleCI approx 9% over prior week P95.

jni pushed a commit that referenced this pull request Apr 16, 2025
@willingc
Update circleci cimg base to 3.10.17 which includes uv (#673)
92d5f54 
# References and relevant issues
Split out from #618 

# Description
Updates circleci base image and uses uv for venv management
@willingc
Copy link
Copy Markdown
Contributor Author

willingc commented Jun 24, 2025

Closing as this is no longer necessary.

@willingc willingc closed this Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Czaki Czaki Czaki left review comments

Assignees

@willingc willingc

Labels

documentation Improvements or additions to documentation maintenance CI, dependencies, and other maintenance

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@willingc @psobolewskiPhD @Czaki