[DEV]: add pixi.lock and add pixi to pyproject.toml by jklymak · Pull Request #31384 · matplotlib/matplotlib

jklymak · 2026-03-25T20:21:42Z

Edit: apologies for the CI churn --- I needed to iterate on the Pixi workflow to get it working end-to-end.

Overview

This PR adds Pixi configuration to pyproject.toml and introduces a committed pixi.lock.

The main goal is to provide a project-committed, pinned dependency snapshot for Matplotlib, addressing #23548. In particular:

pyproject.toml defines the Pixi environment under [tool.pixi]
pixi.lock records the fully resolved dependency set
CI exercises that lockfile to verify that it can be used reproducibly

Why `pyproject.toml`

Pixi configuration is placed in pyproject.toml rather than a project-level pixi.toml.

The reasoning is:

pyproject.toml is already the canonical project configuration
it keeps the shared Pixi definition attached to the project itself
developers who want local customization can still create an untracked pixi.toml, which Pixi will prefer over pyproject.toml

This gives the project a canonical shared configuration while still allowing personal overrides.

Lockfile policy

A main goal of this PR is to commit a pinned dependency list, and pixi.lock is the mechanism used for that.

pixi.lock should therefore be treated as part of the repository state:

changes to Pixi dependency inputs should be accompanied by a regenerated pixi.lock
CI uses the lockfile to verify reproducibility
old commits retain the dependency snapshot that was current at that point in history

In that sense, pixi.lock is the committed pinned dependency snapshot for the project.

Developer workflow

Basic usage:

pixi install

pixi run python -m pip install \
  --no-build-isolation \
  --config-settings=setup-args="-Db_lto=false" \
  --editable .

After that, for example:

pixi run tests-core
pixi run build-doc

Dependency changes:

If dependencies change, then pixi.lock should be regenerated. It is ignored by default in .gitignore so developers who want to change pixi.lock will need to regenerate it, and force add it to a commit:

mv pixi.toml pixi.backup
pixi lock
git add -f pixi.lock

`tools/pixi.example.toml`

A template is provided for developers who want to customize their local Pixi setup.

For example:

cp tools/pixi.example.toml pixi.toml

This is intended for local experimentation or system-specific tuning. It is not the project's canonical Pixi configuration.

Key design decisions

Editable install is not managed directly by Pixi

This PR intentionally does not use:

    [pypi-dependencies]
    matplotlib = { path = ".", editable = true }

In testing, that led to lockfile instability (lock-file not up-to-date with the workspace), especially in CI.

Instead, responsibilities are split:

Pixi manages the environment and pinned dependencies
pip install -e . installs Matplotlib itself

This keeps pixi.lock stable while still supporting editable development installs.

Single environment

Although Pixi supports separate environments such as test and doc, this PR currently uses a single environment containing the relevant dependencies.

The main reason is developer convenience:

avoids repeated editable installs in separate environments
keeps the workflow simple
still allows the lockfile to capture the relevant dependency sets

This can be revisited later if stricter separation is desirable.

CI

This PR adds a minimal CI job that checks that the Pixi environment can be recreated from the committed lockfile.

At the moment the job is intentionally lightweight: it verifies that the lockfile works and that Matplotlib can be built/imported in that environment. It can be extended later to run tests and/or documentation builds.

Notes / future work

extend Pixi CI coverage to run more tests and build docs
decide whether additional environment separation is worthwhile
refine developer-facing documentation around local pixi.toml
overrides

AI disclosure

I went back and forth w/ chatGPT quite a bot on how to organize this and the various tradeoffs. It also helped with writing this summary, which is of course long-winded, but hopefully complete.

jklymak · 2026-03-26T00:04:45Z

.. I'm surprised the tests are running given I labelled this [ci doc]

tacaswell · 2026-03-26T03:59:13Z

tools/pixi.toml

+depends-on = ["clean-build", "dev-install"]
+description = "Clean and rebuild editable install"
+
+[tasks.install-test]


We should make features for tests + docs.

Yes we could do it that way, but I'm not sure of the pros and cons. It did not seem there was a way to make conditional install dependencies so I wasn't sure the advantage of having separate feature envs for a developer versus just having one env.

tools/pixi.toml

tacaswell · 2026-03-26T04:00:31Z

tools/pixi.toml

+description = "No-op on non-mac platforms"
+
+[target.osx-arm64.tasks.prepare-freetype]
+cmd = "cp subprojects/packagefiles/freetype-2.6.1-meson/src/gzip/zconf.h subprojects/freetype-2.6.1/src/gzip"


I also don't understand this, is something going wrong with unpacking the subproject? This seems like something we should be fixing in the meson files rather than here?

I don't think it's a general problem, but it's definitely a problem on macOS with the version of zconf

OK< I think this was not necessary, as it now works fine without it.

tacaswell · 2026-03-26T04:03:17Z

It is my understanding that if you have metadata about the build you are (or will soon be able) to point to a repo as a conda dependency and pixi-build will build the conda-package on the fly (the same way it does with pypi-deps). Given that we are going to want pixi.toml in the lop level eventually.

That said, I agree we do not want to check in the pixi.lock file.

tacaswell · 2026-03-26T05:00:10Z

I mean add a section like

[pypi-dependencies]
matplotlib = { path = "..", editable = true }

jklymak · 2026-03-26T05:13:15Z

Ah I see. No that didn't work as I don't think it's compatible with build isolation.

jklymak · 2026-03-26T23:10:37Z

OK< my comment above was incorrect. The new pixi.toml has removed the font copying things, and uses

[pypi-dependencies]
matplotlib = { path = ".", editable = true }

which indeed seems to be fine.

Still not sure if we should run with features or just a separate install steps?

.github/workflows/pixi_tests.yml

pyproject.toml

.github/workflows/pixi_tests.yml

doc/devel/development_setup.rst

timhoffm · 2026-04-01T14:48:50Z

Note: Mypy errors are based on a new mypy release yesterday, which improves type narrowing. https://mypy.readthedocs.io/en/stable/changelog.html

Unrelated to here, but needs investigation.

jklymak · 2026-04-01T15:22:38Z

Built docs are here: https://output.circle-artifacts.com/output/job/2a1cb0cd-b9e6-487d-8213-ccadbf52b0b5/artifacts/0/doc/build/html/devel/development_setup.html

Not sure about the Note... Tip... Note... I think I'll move the pyproject.toml note up to be a warning as we really don't want people to edit that unless absolutely necessary.

EDIT: combined the two tips, and made the note about pyproject.toml a warning.

https://output.circle-artifacts.com/output/job/7fa2bf9b-db3d-464c-aa8e-8e8c8829cdb9/artifacts/0/doc/build/html/devel/development_setup.html

timhoffm · 2026-04-01T17:04:26Z

Do we need the warning at all? / Do we need the warning here?

This is about dev environment setup. Somebody setting up is not likely to mess with existing configuration. We also don’t warn on the other environment setups.

jklymak · 2026-04-01T17:28:18Z

This is about dev environment setup. Somebody setting up is not likely to mess with existing configuration. We also don’t warn on the other environment setups.

I guess I'm not sure, partially out of ignorance for how others work.

The central tension here is between what a casual developer wants, which is a pixi.toml that they may want to edit, versus what we want for CI which is to check and create the canonical pixi.lock that we want to preserve for posterity.

If someone can't get pixi install to work out of the box, they may be tempted to change pyproject.toml because that is where pixi is getting the manifest. We'd rather they copy tools/pixi.toml locally, and modify that, and then do pixi install.

However, I'm very open to the idea that the way I've suggested we resolve this tension here is not the best one. You could imagine the CI-verified pixi.lock goes somewhere more discrete than the top level, and the burden to update it belongs to experienced developers who change the project requirements. That would be a bit of reorganization here, but may be cleaner.

timhoffm · 2026-04-01T17:55:03Z

My primary question would be: Isn’t the config we provide here good enough for the average developer? There could be expert use cases, but I’d trust that experts know enough to not mess up.

I would only discuss a custom pixi.toml if that is a typical use case: And then its usage should be explicitly explained somewhere around here.

Also, we have the PR review, so that we will catch improper updates.

Overall, I’d leave this out and only add the warning, if improper updates become reasonably common.

timhoffm · 2026-04-01T18:17:17Z

Note: I did a pixi setup yesterday on a fresh machine, and it went quite smooth with the preconfigured settings.

Running tests is possible.

Doc build is a bit fiddly because Sphinx crashes with awkward messages if the “external” dependencies Ghostscript and Latex are not available. That’s not a direct issue of pixi, but we could consider including the Ghostscript conda package (in a follow up PR).

jklymak · 2026-04-01T18:59:17Z

My primary question would be: Isn’t the config we provide here good enough for the average developer?

I think so, and it's encouraging that it works on at least three machines (mine, yours, and CI)

The goal with the warning was just to alert folks not to edit pyproject.toml. However, happy to remove that section and add it back if it becomes problematic.

timhoffm

Optional wording improvements.

doc/devel/development_setup.rst

timhoffm · 2026-04-01T20:34:02Z

doc/devel/development_setup.rst

+
+      A pixi configuration is included in ``pyproject.toml``.  The environment
+      includes the base Matplotlib dependencies, and documentation and testing
+      dependencies. To install the dependencies::


Optional: Some duplication with the paragraph before the tab group:

We recommend using one of the following options for a dedicated development environment because these options are configured to install the Python dependencies as part of their setup.

Maybe shorten the paragraph before to (or similar)

We recommend using one of the following options for a dedicated development environment because they contain the development dependencies.

And then leave out the sentence here ("The environment ... testing dependencies").

I think I simplified this to reduce the redundancy - the point was that the pixi install does all the dependencies (beyond the really big doc ones), not that it installs the base dependencies.

But pip install —group dev and the environment.yaml do that as well, AFAICS. So there‘s nothing special about pixi.

OK, so changed to

A `pixi`_ configuration is included in ``pyproject.toml`` that will install the development dependencies::

timhoffm · 2026-04-01T20:35:19Z

doc/devel/development_setup.rst

+      includes the base Matplotlib dependencies, and documentation and testing
+      dependencies. To install the dependencies::
+
+        pixi install


Do we need a dedicated pixi install? I assume pixi run would install the needed dependencies automatically? So we could simplify.

I suppose we could just have the one command, but I like keeping them separate so the user knows it was the pixi part that failed, not the pip install part. When these things fail they are pretty opaque...

Fair. OTOH the pixi install failing is quite unlikely. That‘d mean either an issue in the package infrastructure like a failing server) which is beyond our responsibility and should be rare) or it’s a configuration issue on our side. In the latter case we‘d also notice soon as this would also be broken in CI.

I leave this up to you.

I'm more thinking it may fail on machines we don't test...

doc/devel/development_setup.rst

…ci doc] Apply suggestions from code review Co-authored-by: Tim Hoffmann <[email protected]>

github-actions bot added the Documentation: devdocs files in doc/devel label Mar 25, 2026

jklymak changed the title ~~Dev add pixi toml~~ [DEV]: add tools/pixi.toml and doc how to have a development environment for pixi Mar 25, 2026

jklymak marked this pull request as ready for review March 25, 2026 23:41

jklymak mentioned this pull request Mar 26, 2026

[DOC] [BLD] Building / developing matplotlib with uv & pixi #30877

Open

tacaswell reviewed Mar 26, 2026

View reviewed changes

tools/pixi.toml Outdated Show resolved Hide resolved

tacaswell reviewed Mar 26, 2026

View reviewed changes

jklymak force-pushed the dev-add-pixi-toml branch from 7d73c47 to 27d86a1 Compare March 28, 2026 04:49

jklymak marked this pull request as draft March 28, 2026 04:58

jklymak force-pushed the dev-add-pixi-toml branch 2 times, most recently from aedf224 to 055029d Compare March 28, 2026 16:22

jklymak changed the title ~~[DEV]: add tools/pixi.toml and doc how to have a development environment for pixi~~ [DEV]: add pixi.lock and add pixi to pyproject.toml Mar 28, 2026

jklymak force-pushed the dev-add-pixi-toml branch from 055029d to 85257fe Compare March 28, 2026 16:32

jklymak marked this pull request as ready for review March 28, 2026 16:50

jklymak requested a review from tacaswell March 28, 2026 19:51

jklymak force-pushed the dev-add-pixi-toml branch 3 times, most recently from a0a2ff5 to c0467de Compare March 30, 2026 17:51

jklymak added the CI: testing CI configuration and testing label Mar 30, 2026

jklymak force-pushed the dev-add-pixi-toml branch from c0467de to f3e0de0 Compare March 30, 2026 18:09

timhoffm reviewed Mar 31, 2026

View reviewed changes

.github/workflows/pixi_tests.yml Outdated Show resolved Hide resolved

.github/workflows/pixi_tests.yml Show resolved Hide resolved

pyproject.toml Show resolved Hide resolved

timhoffm reviewed Apr 1, 2026

View reviewed changes

.github/workflows/pixi_tests.yml Outdated Show resolved Hide resolved

timhoffm reviewed Apr 1, 2026

View reviewed changes

.github/workflows/pixi_tests.yml Show resolved Hide resolved

timhoffm reviewed Apr 1, 2026

View reviewed changes

doc/devel/development_setup.rst Outdated Show resolved Hide resolved

doc/devel/development_setup.rst Outdated Show resolved Hide resolved

jklymak force-pushed the dev-add-pixi-toml branch from a40ae3f to 3ba6a65 Compare April 1, 2026 14:27

DEV: add pixi.toml

d32b629

jklymak force-pushed the dev-add-pixi-toml branch from 3ba6a65 to 313bf08 Compare April 1, 2026 14:28

jklymak force-pushed the dev-add-pixi-toml branch 2 times, most recently from 7b7c719 to 0ce998c Compare April 1, 2026 15:51

jklymak force-pushed the dev-add-pixi-toml branch 2 times, most recently from 965359f to 24087b2 Compare April 1, 2026 19:38

timhoffm approved these changes Apr 1, 2026

View reviewed changes

jklymak force-pushed the dev-add-pixi-toml branch from 24087b2 to 63bbd36 Compare April 2, 2026 00:40

TST: add test that pixi lock file is consistent with pyproject.toml […

093c210

…ci doc] Apply suggestions from code review Co-authored-by: Tim Hoffmann <[email protected]>

jklymak force-pushed the dev-add-pixi-toml branch from 63bbd36 to 093c210 Compare April 2, 2026 19:24

Uh oh!

Conversation

jklymak commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Why pyproject.toml

Lockfile policy

Developer workflow

Dependency changes:

tools/pixi.example.toml

Key design decisions

Editable install is not managed directly by Pixi

Single environment

CI

Notes / future work

AI disclosure

Uh oh!

jklymak commented Mar 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tacaswell commented Mar 26, 2026

Uh oh!

tacaswell commented Mar 26, 2026

Uh oh!

jklymak commented Mar 26, 2026

Uh oh!

jklymak commented Mar 26, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

timhoffm commented Apr 1, 2026

Uh oh!

jklymak commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timhoffm commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jklymak commented Apr 1, 2026

Uh oh!

timhoffm commented Apr 1, 2026

Uh oh!

timhoffm commented Apr 1, 2026

Uh oh!

jklymak commented Apr 1, 2026

Uh oh!

timhoffm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timhoffm Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

jklymak commented Mar 25, 2026 •

edited

Loading

Why `pyproject.toml`

`tools/pixi.example.toml`

jklymak commented Apr 1, 2026 •

edited

Loading

timhoffm commented Apr 1, 2026 •

edited

Loading

timhoffm Apr 2, 2026 •

edited

Loading