Skip to content

gh-146480: Override the exception in _PyErr_SetKeyError()#146486

Merged
vstinner merged 4 commits intopython:mainfrom
vstinner:pyerr_setkeyerror
Mar 27, 2026
Merged

gh-146480: Override the exception in _PyErr_SetKeyError()#146486
vstinner merged 4 commits intopython:mainfrom
vstinner:pyerr_setkeyerror

Conversation

@vstinner
Copy link
Copy Markdown
Member

@vstinner vstinner commented Mar 26, 2026

If _PyErr_SetKeyError() is called with an exception set, it now replaces the current exception with KeyError (as expected), instead of setting a SystemError or failing with a fatal error (in debug mode).

If _PyErr_SetKeyError() is called with an exception set, it now
replaces the current exception with KeyError (as expected), instead
of setting a SystemError or failing with a fatal error (in debug
mode).
@vstinner
Copy link
Copy Markdown
Member Author

cc @serhiy-storchaka

with self.assertRaises(KeyError) as cm:
# Test _PyErr_SetKeyError() with an exception set to check
# that the function overrides the current exception
_pyerr_setkeyerror("key")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is worth to test also the case when the argument is a tuple and when the argument is a KeyError instance. I think this was the reason of introducing _PyErr_SetKeyError() instead of simply calling PyErr_SetObject().

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I added tests on a tuple and a KeyError.

I think this was the reason of introducing _PyErr_SetKeyError() instead of simply calling PyErr_SetObject().

Ah right, PyErr_SetObject(exc_type, value) has an unsual API: if value is a tuple, it unpacks arguments by calling exc_type(*value). Otherwise, it calls exc_type(value).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if the value is already an instance of the specified exception type, it just uses it.

Python/errors.c Outdated
@@ -248,11 +248,19 @@ PyErr_SetObject(PyObject *exception, PyObject *value)

/* Set a key error with the specified argument, wrapping it in a
* tuple automatically so that tuple keys are not unpacked as the
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is outdated, because the code no longer explicitly wraps the key in a tuple. But it has roughtly the same effect, the reasons why it does not simply use PyErr_SetObject() is still valid. If we keep the current code, we need other wording here.

I wonder if partial reversion of aa36f83 will fix the issue. Would it have performance impact? This is just a curiosity, not suggestion.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the comment by removing the part about wrapping into a tuple.

I wonder if partial reversion of aa36f83 will fix the issue.

The old implementation can be called with an exception set:

void
_PyErr_SetKeyError(PyObject *arg)
{
    PyThreadState *tstate = _PyThreadState_GET();
    PyObject *tup = PyTuple_Pack(1, arg);
    if (!tup) {
        /* caller will expect error to be set anyway */
        return;
    }
    _PyErr_SetObject(tstate, PyExc_KeyError, tup);
    Py_DECREF(tup);
}

Would it have performance impact? This is just a curiosity, not suggestion.

According to my comment, I rewrote _PyErr_SetKeyError() to be able to reuse the faster BaseException_vectorcall(). I don't know if it has a significant impact on performance. This vectorcall function has to create a tuple of arguments using _PyTuple_FromArray(args, PyVectorcall_NARGS(nargsf)).

Update also outdated comment.
@vstinner
Copy link
Copy Markdown
Member Author

@serhiy-storchaka: I updated the PR to address your review.

@vstinner vstinner added the needs backport to 3.14 bugs and security fixes label Mar 26, 2026
@vstinner
Copy link
Copy Markdown
Member Author

IMO we should backport the fix to the 3.14 stable branch which is also affected by the issue.

Python 3.13 is not affected, it uses the old _PyErr_SetKeyError() implementation which uses PyTuple_Pack().

Copy link
Copy Markdown
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

But could you add other explanation why do we need _PyErr_SetKeyError() and cannot simply use PyErr_SetObject() in the comment?

It is worth to backport the test to 3.13.

with self.assertRaises(KeyError) as cm:
# Test _PyErr_SetKeyError() with an exception set to check
# that the function overrides the current exception
_pyerr_setkeyerror("key")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if the value is already an instance of the specified exception type, it just uses it.

* Elaborate _PyErr_SetKeyError() comment to explain the usage of this
  function.
* Explain the purpopose of the KeyError(KeyError('arg')) test.
* Move the test to test_capi.test_exceptions.
@vstinner
Copy link
Copy Markdown
Member Author

But could you add other explanation why do we need _PyErr_SetKeyError() and cannot simply use PyErr_SetObject() in the comment?

Ok, done.

@vstinner
Copy link
Copy Markdown
Member Author

I was lazy and didn't run the test suite, and I got catched by the CI: I forgot _testinternalcapi import when I moved the test to test_capi/test_exceptions.py. I should be fixed now.

@vstinner vstinner enabled auto-merge (squash) March 27, 2026 11:00
@vstinner vstinner merged commit d4153a9 into python:main Mar 27, 2026
67 checks passed
@vstinner vstinner deleted the pyerr_setkeyerror branch March 27, 2026 11:20
@miss-islington-app
Copy link
Copy Markdown

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Mar 27, 2026
…onGH-146486)

If _PyErr_SetKeyError() is called with an exception set, it now
replaces the current exception with KeyError (as expected), instead
of setting a SystemError or failing with a fatal error (in debug
mode).
(cherry picked from commit d4153a9f76736128306c4af01776729da846d926)

Co-authored-by: Victor Stinner <[email protected]>
@bedevere-app
Copy link
Copy Markdown

bedevere-app bot commented Mar 27, 2026

GH-146511 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Mar 27, 2026
@bedevere-app
Copy link
Copy Markdown

bedevere-app bot commented Mar 27, 2026

GH-146512 is a backport of this pull request to the 3.13 branch.

@vstinner
Copy link
Copy Markdown
Member Author

It is worth to backport the test to 3.13.

Done in #146512.

vstinner added a commit that referenced this pull request Mar 27, 2026
…146486) (#146511)

gh-146480: Override the exception in _PyErr_SetKeyError() (GH-146486)

If _PyErr_SetKeyError() is called with an exception set, it now
replaces the current exception with KeyError (as expected), instead
of setting a SystemError or failing with a fatal error (in debug
mode).
(cherry picked from commit d4153a9)

Co-authored-by: Victor Stinner <[email protected]>
vstinner added a commit that referenced this pull request Mar 27, 2026
gh-146480: Add tests on _PyErr_SetKeyError() (#146486)

(cherry picked from commit d4153a9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants