gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe#120165
Merged
Fidget-Spinner merged 9 commits intopython:mainfrom Jun 12, 2024
Merged
gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe#120165Fidget-Spinner merged 9 commits intopython:mainfrom
Fidget-Spinner merged 9 commits intopython:mainfrom
Conversation
Contributor
|
Do you have a test / stack trace of the reported race? I don't think we should be modifying an object's type ( |
Member
Author
|
Example code: class Foo:
pass
class Bar:
pass
import threading
import os
X = Foo()
def work():
foo = X
for _ in range(10000):
foo.__class__ = Bar
type(foo)
foo.__class__ = Foo
type(foo)
threads = []
for i in range(os.cpu_count() - 1):
thread = threading.Thread(target=work)
thread.start()
threads.append(thread)
work()
for thread in threads:
thread.join()TSAN output: Py_SET_TYPE: |
Member
Author
|
Wow... the test case actually causes a segfault even with the fixes, not just a tsan warning. |
Member
Well, Py_TYPE() returns a borrow reference which is a bad thing :-/ But it shouldn't matter in this case, since the type is not deleted by the test. |
Member
Author
|
Can't merge this until we fix the actual crasher code #120198. It seems even default build is affected. |
Fidget-Spinner
commented
Jun 8, 2024
picnixz
reviewed
Jun 9, 2024
Co-authored-by: Bénédikt Tran <[email protected]>
vstinner
reviewed
Jun 12, 2024
| } | ||
| Py_BEGIN_CRITICAL_SECTION(self); | ||
| if (newto->tp_flags & Py_TPFLAGS_HEAPTYPE) { | ||
| Py_INCREF(newto); |
Member
There was a problem hiding this comment.
Is it important to do the INCREF(newto) and DECREF(oldto) inside the critical section?
Or it it possible to restrict the critical section to the following code?
oldto = Py_TYPE(self);
Py_SET_TYPE(self, newto);
Member
Author
There was a problem hiding this comment.
You're right. I can shrink it!
Co-Authored-By: Nadeshiko Manju <[email protected]>
vstinner
approved these changes
Jun 12, 2024
Member
vstinner
left a comment
vstinner
left a comment
There was a problem hiding this comment.
LGTM.
Since my previous review, object_set_class() got a critical section and it LGTM.
Member
Author
|
Thanks for the review Victor! |
|
Thanks @Fidget-Spinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13. |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Jun 12, 2024
…20165) (cherry picked from commit e16aed6) Co-authored-by: Ken Jin <[email protected]> Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
|
GH-120403 is a backport of this pull request to the 3.13 branch. |
Fidget-Spinner
added a commit
that referenced
this pull request
Jun 12, 2024
GH-120403) gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165) (cherry picked from commit e16aed6) Co-authored-by: Ken Jin <[email protected]> Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
Contributor
|
I think we should address
The thread-safety guarantees should be something like:
@Fidget-Spinner, would you like to work on this? |
Member
Author
|
@colesbury yeah I can revert the atomics in Py_TYPE and PY_SET_TYPE and stop the world instead. |
mrahtz
pushed a commit
to mrahtz/cpython
that referenced
this pull request
Jun 30, 2024
…20165) Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
noahbkim
pushed a commit
to hudson-trading/cpython
that referenced
this pull request
Jul 11, 2024
…20165) Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
estyxx
pushed a commit
to estyxx/cpython
that referenced
this pull request
Jul 17, 2024
…20165) Co-authored-by: Bénédikt Tran <[email protected]> Co-authored-by: Nadeshiko Manju <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.