Skip to content

[ty] Introduce fast path for protocol non-assignability#23952

Merged
charliermarsh merged 2 commits intomainfrom
charlie/proto-opt
Mar 14, 2026
Merged

[ty] Introduce fast path for protocol non-assignability#23952
charliermarsh merged 2 commits intomainfrom
charlie/proto-opt

Conversation

@charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Mar 13, 2026 •
edited
Loading

Summary

This PR adds a fast path to first verify that every required protocol member is definitely present.

Closes astral-sh/ty#3026.

@astral-sh-bot astral-sh-bot bot added the ty Multi-file analysis & type inference label Mar 13, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 13, 2026 •
edited
Loading

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 85.29%. The percentage of expected errors that received a diagnostic held steady at 78.13%. The number of fully passing files held steady at 64/132.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 13, 2026 •
edited
Loading

mypy_primer results

Changes were detected when running on open source projects 
pydantic (https://github.com/pydantic/pydantic)
- pydantic/_internal/_core_metadata.py:87:54: error[invalid-assignment] Invalid assignment to key "pydantic_js_extra" with declared type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | ((dict[str, int | float | str | ... omitted 3 union elements], type[Any], /) -> None)` on TypedDict `CoreMetadata`: value of type `dict[object, object]`
+ pydantic/_internal/_core_metadata.py:87:54: error[invalid-assignment] Invalid assignment to key "pydantic_js_extra" with declared type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | ((dict[str, Divergent], type[Any], /) -> None)` on TypedDict `CoreMetadata`: value of type `dict[object, object]`
- pydantic/fields.py:949:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:949:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:989:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:989:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1032:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1032:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1072:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1072:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1115:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1115:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1154:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1154:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1194:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
+ pydantic/fields.py:1194:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
- pydantic/fields.py:1573:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`, found `dict[str, int | float | str | ... omitted 3 union elements] | dict[Never, Never] | (((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`
+ pydantic/fields.py:1573:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`, found `dict[str, Divergent] | dict[Never, Never] | (((dict[str, Divergent], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 13, 2026 •
edited
Loading

Memory usage report

Summary

Project Old New Diff Outcome
prefect 705.20MB 706.23MB +0.15% (1.04MB) ⏫
sphinx 265.25MB 265.95MB +0.27% (722.71kB) ⏫
trio 117.75MB 118.03MB +0.24% (283.91kB) ⏫
flake8 47.89MB 48.05MB +0.33% (162.07kB) ⏫

Significant changes

Click to expand detailed breakdown

prefect

Name Old New Diff Outcome
Type<'db>::member_lookup_with_policy_ 15.41MB 15.89MB +3.14% (495.63kB) ⏫
infer_expression_types_impl 60.77MB 60.90MB +0.22% (138.47kB) ⏫
Type<'db>::member_lookup_with_policy_::interned_arguments 5.53MB 5.66MB +2.25% (127.56kB) ⏫
Type<'db>::class_member_with_policy_ 17.32MB 17.43MB +0.64% (114.10kB) ⏫
is_equivalent_to_object_inner 206.36kB 102.91kB -50.13% (103.45kB) ⏫
infer_definition_types 88.89MB 88.99MB +0.11% (96.58kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_ 9.74MB 9.79MB +0.45% (45.14kB) ⏫
infer_expression_type_impl 14.34MB 14.39MB +0.29% (43.12kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_::interned_arguments 5.15MB 5.19MB +0.62% (32.62kB) ⏫
Type<'db>::class_member_with_policy_::interned_arguments 9.37MB 9.40MB +0.32% (30.67kB) ⏫
infer_scope_types_impl 52.94MB 52.96MB +0.05% (24.66kB) ⏫
all_narrowing_constraints_for_expression 7.00MB 7.01MB +0.21% (15.14kB) ⏫
all_negative_narrowing_constraints_for_expression 2.60MB 2.61MB +0.36% (9.56kB) ⏫
CallableType 1.89MB 1.88MB -0.48% (9.35kB) ⏫
is_redundant_with_impl 5.57MB 5.57MB -0.14% (7.80kB) ⏫
... 29 more

sphinx

Name Old New Diff Outcome
Type<'db>::member_lookup_with_policy_ 6.10MB 6.50MB +6.58% (410.77kB) ⏫
Type<'db>::member_lookup_with_policy_::interned_arguments 2.54MB 2.64MB +3.90% (101.56kB) ⏫
Type<'db>::class_member_with_policy_ 7.55MB 7.63MB +1.03% (79.60kB) ⏫
infer_expression_types_impl 21.52MB 21.59MB +0.32% (70.65kB) ⏫
is_equivalent_to_object_inner 116.43kB 50.67kB -56.48% (65.77kB) ⏫
infer_definition_types 24.03MB 24.07MB +0.14% (33.80kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_ 2.37MB 2.40MB +1.20% (29.18kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_::interned_arguments 1.90MB 1.93MB +1.27% (24.66kB) ⏫
Type<'db>::class_member_with_policy_::interned_arguments 3.99MB 4.01MB +0.52% (21.23kB) ⏫
infer_scope_types_impl 15.58MB 15.59MB +0.09% (14.70kB) ⏫
infer_expression_type_impl 3.21MB 3.22MB +0.30% (10.00kB) ⏫
all_narrowing_constraints_for_expression 2.34MB 2.35MB +0.24% (5.72kB) ⏫
is_redundant_with_impl 1.81MB 1.81MB -0.23% (4.23kB) ⏫
CallableType 1.08MB 1.07MB -0.35% (3.87kB) ⏫
FunctionType<'db>::signature_ 2.28MB 2.27MB -0.12% (2.72kB) ⏫
... 34 more

trio

Name Old New Diff Outcome
Type<'db>::member_lookup_with_policy_ 1.66MB 1.82MB +9.35% (159.38kB) ⏫
Type<'db>::member_lookup_with_policy_::interned_arguments 868.87kB 909.90kB +4.72% (41.03kB) ⏫
is_equivalent_to_object_inner 68.93kB 34.14kB -50.46% (34.78kB) ⏫
Type<'db>::class_member_with_policy_ 1.97MB 2.00MB +1.46% (29.54kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_ 730.54kB 751.04kB +2.81% (20.50kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_::interned_arguments 581.81kB 599.34kB +3.01% (17.53kB) ⏫
infer_expression_types_impl 7.06MB 7.08MB +0.23% (16.56kB) ⏫
infer_definition_types 7.57MB 7.58MB +0.14% (11.18kB) ⏫
Type<'db>::class_member_with_policy_::interned_arguments 1.10MB 1.11MB +0.70% (7.92kB) ⏫
infer_expression_type_impl 1.43MB 1.43MB +0.23% (3.34kB) ⏫
infer_scope_types_impl 4.79MB 4.79MB +0.06% (2.92kB) ⏫
FunctionType 1.50MB 1.50MB +0.14% (2.19kB) ⏫
is_redundant_with_impl 479.86kB 477.89kB -0.41% (1.97kB) ⏫
enum_metadata 241.61kB 243.22kB +0.66% (1.61kB) ⏫
StringLiteralType 524.07kB 525.39kB +0.25% (1.33kB) ⏫
... 21 more

flake8

Name Old New Diff Outcome
Type<'db>::member_lookup_with_policy_ 408.31kB 484.19kB +18.58% (75.88kB) ⏫
Type<'db>::class_member_with_policy_ 545.16kB 566.15kB +3.85% (20.99kB) ⏫
Type<'db>::member_lookup_with_policy_::interned_arguments 207.09kB 227.09kB +9.66% (20.01kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_ 293.53kB 312.52kB +6.47% (18.99kB) ⏫
StaticClassLiteral<'db>::implicit_attribute_inner_::interned_arguments 238.41kB 254.62kB +6.80% (16.22kB) ⏫
is_equivalent_to_object_inner 26.32kB 11.81kB -55.13% (14.51kB) ⏫
infer_definition_types 1.86MB 1.87MB +0.45% (8.54kB) ⏫
Type<'db>::class_member_with_policy_::interned_arguments 299.61kB 304.79kB +1.73% (5.18kB) ⏫
FunctionType 433.78kB 437.06kB +0.76% (3.28kB) ⏫
infer_expression_types_impl 1.07MB 1.07MB +0.21% (2.30kB) ⏫
OverloadLiteral 118.10kB 119.74kB +1.39% (1.64kB) ⏫
infer_scope_types_impl 1002.21kB 1003.61kB +0.14% (1.39kB) ⏫
StringLiteralType 192.82kB 193.92kB +0.57% (1.09kB) ⏫
FunctionLiteral 63.19kB 64.12kB +1.48% (960.00B) ⏫
is_redundant_with_impl 140.37kB 139.85kB -0.37% (528.00B) ⏫
... 7 more

@codspeed-hq
Copy link

codspeed-hq bot commented Mar 13, 2026 •
edited
Loading

Merging this PR will improve performance by 47.6%

⚡ 4 improved benchmarks
✅ 22 untouched benchmarks
⏩ 30 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ WallTime static_frame 30.8 s 28.3 s +9.12%
⚡ WallTime pandas 85.1 s 80.8 s +5.24%
⚡ WallTime colour_science 122 s 82.7 s +47.6%
⚡ WallTime freqtrade 8.7 s 8.2 s +6.39%

Comparing charlie/proto-opt (c8b01b0) with main (7dfefd9)

Open in CodSpeed

Footnotes

  1. 30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@AlexWaygood AlexWaygood added the performance Potential performance improvement label Mar 13, 2026
@charliermarsh charliermarsh marked this pull request as ready for review March 14, 2026 01:43
@charliermarsh charliermarsh marked this pull request as draft March 14, 2026 01:44
@charliermarsh charliermarsh marked this pull request as ready for review March 14, 2026 01:49
carljm
carljm approved these changes Mar 14, 2026
View reviewed changes
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

crates/ty_python_semantic/src/types/instance.rs Outdated
/// Protocol compatibility can only succeed if every required member is present. Check that
/// necessary condition up front so we can avoid expensive per-member type comparisons and
/// generic protocol solving when the actual type is plainly missing a member.
fn has_all_protocol_members_defined(
Copy link
Contributor

@carljm carljm Mar 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit: should this be a free function in protocol_class.rs (or a method on ProtocolInstanceType?) rather than an associated function on TypeRelationChecker?

@charliermarsh charliermarsh enabled auto-merge (squash) March 14, 2026 02:42
@charliermarsh charliermarsh merged commit 93a16bd into main Mar 14, 2026
50 checks passed
@charliermarsh charliermarsh deleted the charlie/proto-opt branch March 14, 2026 02:47
carljm added a commit that referenced this pull request Mar 16, 2026
@carljm
Merge branch 'main' into charlie/metaclass
f02f39b 
* main: (131 commits)
  [ty] Fixup examples in `invalid-key` docs (#23968)
  [ty] Fix compiler warning about unused variable (#23967)
  [ty] Sync vendored typeshed stubs (#23963)
  Add a `.git-blame-ignore-revs` file (#23959)
  Revert "[ty] Completely remove the `NoReturn` shortcut optimization" (#23955)
  [ty] Completely remove the `NoReturn` shortcut optimization (#23378)
  [ty] Introduce fast path for protocol non-assignability (#23952)
  Bump typing conformance suite SHA (#23951)
  Minor followup to severity display - use preview function in server instead of checking preview disabled directly (#23950)
  Document editor features for markdown code formatting (#23924)
  [ty] Add `with_recursion_guard()` helpers to `relation.rs` (#23945)
  [ty] Remove `check_optional_method_pair` methods (#23947)
  [ty] Remove unused `CycleDetector::try_visit` method (#23944)
  [ty] Ensure TypedDict subscripts for unknown keys return Unknown (#23926)
  [ty] Fix variance of frozen dataclass-transform models (#23931)
  Display output severity in preview (#23845)
  Revert "[`ruff`] use `bitcode` instead of `bincode`" (#23935)
  Fix shell injection via `shell=True` in subprocess calls (#23894)
  [ty] Refactor `relation.rs` to store state on a struct rather than passing around 7 arguments every time we recurse (#23837)
  Don't return code actions for non-Python documents (#23905)
  ...
@dcreager
Copy link
Member

dcreager commented Mar 16, 2026

Oh this is clever, nicely done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@carljm carljm carljm approved these changes

@AlexWaygood AlexWaygood Awaiting requested review from AlexWaygood AlexWaygood is a code owner

@sharkdp sharkdp Awaiting requested review from sharkdp sharkdp is a code owner

@dcreager dcreager Awaiting requested review from dcreager dcreager is a code owner

@ibraheemdev ibraheemdev Awaiting requested review from ibraheemdev ibraheemdev is a code owner

Assignees

@carljm carljm

Labels

performance Potential performance improvement ty Multi-file analysis & type inference

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Converting a tuple of tuples into a dict takes too much time and memory

4 participants

@charliermarsh @dcreager @carljm @AlexWaygood