Skip to content

[DEV]: add pixi.lock and add pixi to pyproject.toml#31384

Open
jklymak wants to merge 1 commit intomatplotlib:mainfrom
jklymak:dev-add-pixi-toml
Open

[DEV]: add pixi.lock and add pixi to pyproject.toml#31384
jklymak wants to merge 1 commit intomatplotlib:mainfrom
jklymak:dev-add-pixi-toml

Conversation

@jklymak
Copy link
Copy Markdown
Member

@jklymak jklymak commented Mar 25, 2026

Edit: apologies for the CI churn --- I needed to iterate on the Pixi workflow to get it working end-to-end.

Overview

This PR adds Pixi configuration to pyproject.toml and introduces a committed pixi.lock.

The main goal is to provide a project-committed, pinned dependency snapshot for Matplotlib, addressing #23548. In particular:

  • pyproject.toml defines the Pixi environment under [tool.pixi]
  • pixi.lock records the fully resolved dependency set
  • CI exercises that lockfile to verify that it can be used reproducibly

Why pyproject.toml

Pixi configuration is placed in pyproject.toml rather than a project-level pixi.toml.

The reasoning is:

  • pyproject.toml is already the canonical project configuration
  • it keeps the shared Pixi definition attached to the project itself
  • developers who want local customization can still create an untracked pixi.toml, which Pixi will prefer over pyproject.toml

This gives the project a canonical shared configuration while still allowing personal overrides.

Lockfile policy

A main goal of this PR is to commit a pinned dependency list, and pixi.lock is the mechanism used for that.

pixi.lock should therefore be treated as part of the repository state:

  • changes to Pixi dependency inputs should be accompanied by a regenerated pixi.lock
  • CI uses the lockfile to verify reproducibility
  • old commits retain the dependency snapshot that was current at that point in history

In that sense, pixi.lock is the committed pinned dependency snapshot for the project.

Developer workflow

Basic usage:

pixi install

pixi run python -m pip install \
  --no-build-isolation \
  --config-settings=setup-args="-Db_lto=false" \
  --editable .

After that, for example:

pixi run tests-core
pixi run build-doc

Dependency changes:

If dependencies change, then pixi.lock should be regenerated. It is ignored by default in .gitignore so developers who want to change pixi.lock will need to regenerate it, and force add it to a commit:

mv pixi.toml pixi.backup
pixi lock
git add -f pixi.lock

tools/pixi.example.toml

A template is provided for developers who want to customize their local Pixi setup.

For example:

cp tools/pixi.example.toml pixi.toml

This is intended for local experimentation or system-specific tuning. It is not the project's canonical Pixi configuration.


Key design decisions

Editable install is not managed directly by Pixi

This PR intentionally does not use:

    [pypi-dependencies]
    matplotlib = { path = ".", editable = true }

In testing, that led to lockfile instability (lock-file not up-to-date with the workspace), especially in CI.

Instead, responsibilities are split:

  • Pixi manages the environment and pinned dependencies
  • pip install -e . installs Matplotlib itself

This keeps pixi.lock stable while still supporting editable development installs.

Single environment

Although Pixi supports separate environments such as test and doc, this PR currently uses a single environment containing the relevant dependencies.

The main reason is developer convenience:

  • avoids repeated editable installs in separate environments
  • keeps the workflow simple
  • still allows the lockfile to capture the relevant dependency sets

This can be revisited later if stricter separation is desirable.

CI

This PR adds a minimal CI job that checks that the Pixi environment can be recreated from the committed lockfile.

At the moment the job is intentionally lightweight: it verifies that the lockfile works and that Matplotlib can be built/imported in that environment. It can be extended later to run tests and/or documentation builds.


Notes / future work

  • extend Pixi CI coverage to run more tests and build docs
  • decide whether additional environment separation is worthwhile
  • refine developer-facing documentation around local pixi.toml
    overrides

AI disclosure

I went back and forth w/ chatGPT quite a bot on how to organize this and the various tradeoffs. It also helped with writing this summary, which is of course long-winded, but hopefully complete.

@github-actions github-actions bot added the Documentation: devdocs files in doc/devel label Mar 25, 2026
@jklymak jklymak changed the title Dev add pixi toml [DEV]: add tools/pixi.toml and doc how to have a development environment for pixi Mar 25, 2026
@jklymak jklymak marked this pull request as ready for review March 25, 2026 23:41
@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Mar 26, 2026

.. I'm surprised the tests are running given I labelled this [ci doc]

tools/pixi.toml Outdated
depends-on = ["clean-build", "dev-install"]
description = "Clean and rebuild editable install"

[tasks.install-test]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make features for tests + docs.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we could do it that way, but I'm not sure of the pros and cons. It did not seem there was a way to make conditional install dependencies so I wasn't sure the advantage of having separate feature envs for a developer versus just having one env.

tools/pixi.toml Outdated
description = "No-op on non-mac platforms"

[target.osx-arm64.tasks.prepare-freetype]
cmd = "cp subprojects/packagefiles/freetype-2.6.1-meson/src/gzip/zconf.h subprojects/freetype-2.6.1/src/gzip"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't understand this, is something going wrong with unpacking the subproject? This seems like something we should be fixing in the meson files rather than here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a general problem, but it's definitely a problem on macOS with the version of zconf

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK< I think this was not necessary, as it now works fine without it.

@tacaswell
Copy link
Copy Markdown
Member

It is my understanding that if you have metadata about the build you are (or will soon be able) to point to a repo as a conda dependency and pixi-build will build the conda-package on the fly (the same way it does with pypi-deps). Given that we are going to want pixi.toml in the lop level eventually.

That said, I agree we do not want to check in the pixi.lock file.

@tacaswell
Copy link
Copy Markdown
Member

I mean add a section like

[pypi-dependencies]
matplotlib = { path = "..", editable = true }

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Mar 26, 2026

Ah I see. No that didn't work as I don't think it's compatible with build isolation.

@jklymak
Copy link
Copy Markdown
Member Author

jklymak commented Mar 26, 2026

OK< my comment above was incorrect. The new pixi.toml has removed the font copying things, and uses

[pypi-dependencies]
matplotlib = { path = ".", editable = true }

which indeed seems to be fine.

Still not sure if we should run with features or just a separate install steps?

@jklymak jklymak force-pushed the dev-add-pixi-toml branch from 7d73c47 to 27d86a1 Compare March 28, 2026 04:49
@jklymak jklymak marked this pull request as draft March 28, 2026 04:58
@jklymak jklymak force-pushed the dev-add-pixi-toml branch 2 times, most recently from aedf224 to 055029d Compare March 28, 2026 16:22
@jklymak jklymak changed the title [DEV]: add tools/pixi.toml and doc how to have a development environment for pixi [DEV]: add pixi.lock and add pixi to pyproject.toml Mar 28, 2026
@jklymak jklymak force-pushed the dev-add-pixi-toml branch from 055029d to 85257fe Compare March 28, 2026 16:32
@jklymak jklymak marked this pull request as ready for review March 28, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Documentation: devdocs files in doc/devel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants