Skip to content

Narrow <V.postN pre-release exclusion to match spec#1140

Merged
notatallshaw merged 1 commit intopypa:mainfrom
notatallshaw:fix/less-than-post-prerelease-spec
Apr 5, 2026
Merged

Narrow <V.postN pre-release exclusion to match spec#1140
notatallshaw merged 1 commit intopypa:mainfrom
notatallshaw:fix/less-than-post-prerelease-spec

Conversation

@notatallshaw
Copy link
Copy Markdown
Member

@notatallshaw notatallshaw commented Mar 28, 2026

Discovered while looking for edge cases for #1119.

The spec says:

The exclusive ordered comparison <V MUST NOT allow a pre-release of the specified version unless the specified version is itself a pre-release.

For <1.0.post1 the specified version is 1.0.post1, so only 1.0.post1.devN should be excluded. The current code uses _base_version which also excludes 1.0.dev0, 1.0a1, etc. For example Specifier("<1.0.post1").contains(Version("1.0.dev0")) returns False but should return True.

The fix replaces the _base_version check with a comparison against _earliest_prerelease(spec) (i.e. spec.__replace__(dev=0)). For <2.0 this changes nothing. For <1.0.post1 it stops excluding pre-releases of 1.0.

I don't love this change. The current behavior feels intuitive, but the spec as written means <2.0 excludes 2.0b1 while <2.0.post1 does not. The current behavior also creates problems for interval-based specifier representations (#1119, #1120), where <V.postN requires N+2 intervals to model the _base_version exclusion correctly.

Subsumes #1139.

@notatallshaw
Copy link
Copy Markdown
Member Author

notatallshaw commented Mar 29, 2026 •
edited
Loading

@henryiii I'm trying to assess the impact of this, do you have any way to query dependencies on PyPI? rather than just versions.

You would only be impacted in if you wrote <V.postN and there was a pre-release of V and there's a chance that the final release V wouldn't match but a pre-release of V would. It's fairly obscure, but I did some GitHub code searches:

Even after using the GitHub option to remove duplicates (you have to click on it), several of these are just forks, and some of them are under tool.poetry.dependencies which has extended syntax and is parsed by Poetry directly.

I found no requirement that would in practice produce a different result, e.g. kaleido = ">=0.2.0, <0.2.1.post" doesn't have a pre-release of 0.2.1 nor fvcore = "<0.1.5.post20221220" has a pre-release of 0.1.5.

So while I think this change isn't very intuitive, it does match the spec, it makes interval representation significantly simpler, and it appears to have practically no impact on real world scenarios.

@henryiii
Copy link
Copy Markdown
Contributor

henryiii commented Mar 29, 2026

Yes, I can, I can try to check this either later today or tomorrow.

@notatallshaw notatallshaw mentioned this pull request Mar 29, 2026
Merged
notatallshaw added a commit to notatallshaw/packaging that referenced this pull request Apr 2, 2026
@notatallshaw
Add property-based tests for PEP 440
0848b9a 
325 Hypothesis tests that verify algebraic invariants
from the version specifiers specification. Each test
class quotes the spec paragraph it validates.

Tests are excluded from coverage and run as a separate
CI job via `nox -s property_tests`.

Two tests are expected to fail until pypa#1140 and pypa#1141
are merged (exclusive comparison exclusion rules).
@notatallshaw notatallshaw mentioned this pull request Apr 2, 2026
Merged
notatallshaw added a commit to notatallshaw/packaging that referenced this pull request Apr 2, 2026
@notatallshaw
Add property-based tests for PEP 440
396d476 
325 Hypothesis tests that verify algebraic invariants
from the version specifiers specification. Each test
class quotes the spec paragraph it validates.

Tests are excluded from coverage and run as a separate
CI job via `nox -s property_tests`.

Two tests are expected to fail until pypa#1140 and pypa#1141
are merged (exclusive comparison exclusion rules).
notatallshaw added a commit to notatallshaw/packaging that referenced this pull request Apr 2, 2026
@notatallshaw
Add property-based tests for PEP 440
fe35090 
325 Hypothesis tests that verify algebraic invariants
from the version specifiers specification. Each test
class quotes the spec paragraph it validates.

Tests are excluded from coverage and run as a separate
CI job via `nox -s property_tests`.

Two tests are expected to fail until pypa#1140 and pypa#1141
are merged (exclusive comparison exclusion rules).
notatallshaw added a commit to notatallshaw/packaging that referenced this pull request Apr 2, 2026
@notatallshaw
Add property-based tests for PEP 440
4307dbe 
325 Hypothesis tests that verify algebraic invariants
from the version specifiers specification. Each test
class quotes the spec paragraph it validates.

Tests are excluded from coverage and run as a separate
CI job via `nox -s property_tests`.

Two tests are expected to fail until pypa#1140 and pypa#1141
are merged (exclusive comparison exclusion rules).
notatallshaw added a commit to notatallshaw/packaging that referenced this pull request Apr 2, 2026
@notatallshaw
Add property-based tests for PEP 440
a30c2bf 
325 Hypothesis tests that verify algebraic invariants
from the version specifiers specification. Each test
class quotes the spec paragraph it validates.

Tests are excluded from coverage and run as a separate
CI job via `nox -s property_tests`.

Two tests are expected to fail until pypa#1140 and pypa#1141
are merged (exclusive comparison exclusion rules).
notatallshaw added a commit to notatallshaw/packaging that referenced this pull request Apr 2, 2026
@notatallshaw
Add property-based tests for PEP 440
5ca5ac7 
325 Hypothesis tests that verify algebraic invariants
from the version specifiers specification. Each test
class quotes the spec paragraph it validates.

Tests are excluded from coverage and run as a separate
CI job via `nox -s property_tests`.

Two tests are expected to fail until pypa#1140 and pypa#1141
are merged (exclusive comparison exclusion rules).
@notatallshaw
Copy link
Copy Markdown
Member Author

notatallshaw commented Apr 2, 2026

The tests I'm proposing in #1144 show that this fixes an edge case where the current behavior breaks the monotonic invariant, that given versions v1 and v2 and v2 < v1 then if the specifier <V matches v1 it should also match v2.

@henryiii
Copy link
Copy Markdown
Contributor

henryiii commented Apr 3, 2026

I'm having to use an older file, as the place I was getting them from is gone. I used copilot CLI and gpt-5-mini to scan through the sqlite file and produce the output below the line.

There were about 10K comparisons with a post release (that's every version of every package, only a fraction of that number is unique packages). Looking at just upper bound caps, this is 100 rows, which is 24 unique packages.

Summary of steps performed:

  • Queried projects.requires_dist for "post" followed by a digit.
  • Extracted version comparisons that include ".post" from requires_dist.
  • Removed exact-equality (==) comparisons; kept non-== operators.
  • Filtered to explicit upper-bound comparisons (< or <=) against .post versions.
  • Reduced to the most recent release per package.

Table of packages (package, version, comparisons):

Package Version Comparisons
ai-python 0.0.4 <=3.7.4.post0;>=3.7.4.post0;<=3.7.4.post0;>=3.7.4.post0;<=1.21.0.post2;>=1.21.0.post2;<=1.21.0.post2;>=1.21.0.post2
angr 9.2.32 <=2.0.1.post1
askbot-selimgul 0.11.0 <=1.9.0.post1
ax 0.52.0 <=0.9.9.post4
confluent-kafka-helpers 0.8.0 <=1.0.0.post1
dataloop-upipe 0.2.0 <=3.7.4.post0
dnntools 0.1.4 <=3.1.0.post1
dreamai 0.12.0 <=0.1.0.post0
EthicML 0.1.0a7 <=1.1.0.post2
evalml 0.81.1 <=2.5.1.post0
fastapi-crud-code-generator 0.0.25 <=1.1.3.post0
gdptools 0.0.32 <1.8.5.post1
hydrotools.-restclient 3.0.5 <=3.7.4.post0
jwst-gtvt 0.1.1.post3 <=4.0.1.post1
metalearn 0.6.1 <=0.22.2.post1
micropipelines 0.1.8 <=3.7.4.post0
musket-core 0.498 <=1.6.4.post2;>=1.6.4.post1
neptune-detectron2 1.0.0 <0.1.5.post20221220
pydgn 1.3.1 <=2.1.0.post1
quara 0.3.0 <=0.1.0.post1
redbrick-sdk 2.6.0 <=1.3.0.post1;<=1.3.0.post1;<=1.3.0.post1
skytemple-ssb-debugger 1.2.3 <0.0.3.post2
tamu-axolotl 2021.4.8 <=1.0.1.post1
tamu-d3m 2021.2.12 <=0.22.2.post1

@notatallshaw
Copy link
Copy Markdown
Member Author

notatallshaw commented Apr 3, 2026 •
edited
Loading

Thanks for the analysis!

Filtered to explicit upper-bound comparisons (< or <=) against .post versions.

This PR doesn't affect <= behavior at all, so on your list only gdptools, neptune-detectron2, and skytemple-ssb-debugger are affected, and only if the thing they are requiring have a release a pre-release of that exact V where they list <V.postN.

Looking at gdptools their requirement is Shapely (<1.8.5.post1), this change would match pre-release of 1.8.5 and 1.8.5.post0, but continue to not match pre-release of 1.8.5.post1, however there are no pre-releases of 1.8.5 or 1.8.5.post0 so this has no practical impact: https://pypi.org/project/shapely/#history

The more I've thought about this change the more confident I am that it's absolutely correct, spec wise and mathematically, and that it will have almost no practical impact.

@notatallshaw notatallshaw merged commit 7740476 into pypa:main Apr 5, 2026
56 checks passed
@notatallshaw notatallshaw mentioned this pull request Apr 11, 2026
Merged
This was referenced Apr 13, 2026
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@henryiii henryiii henryiii approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@notatallshaw @henryiii