Skip to content

Fix GC TOCTOU race in collect_inner referent traversal#7511

Merged
youknowone merged 1 commit intoRustPython:mainfrom
youknowone:fix/dict-thread-safety
Mar 27, 2026
Merged

Fix GC TOCTOU race in collect_inner referent traversal#7511
youknowone merged 1 commit intoRustPython:mainfrom
youknowone:fix/dict-thread-safety

Conversation

@youknowone
Copy link
Member

@youknowone youknowone commented Mar 27, 2026

Pre-compute referent pointers once per object in step 3 and reuse them in step 4 (BFS reachability). Previously, gc_get_referent_ptrs() was called independently in both steps. If a dict's write lock state changed between the two calls (e.g., held by another thread during one traversal but not the other), the two traversals could return different results. This caused live objects to be incorrectly classified as unreachable and cleared by GC.

Summary by CodeRabbit

  • Refactor
    • Optimized garbage collection performance by reducing redundant pointer computation.

Pre-compute referent pointers once per object in step 3 and reuse
them in step 4 (BFS reachability). Previously, gc_get_referent_ptrs()
was called independently in both steps. If a dict's write lock state
changed between the two calls (e.g., held by another thread during
one traversal but not the other), the two traversals could return
different results. This caused live objects to be incorrectly
classified as unreachable and cleared by GC.
@youknowone youknowone marked this pull request as ready for review March 27, 2026 01:54
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 27, 2026

📝 Walkthrough

Walkthrough

This change optimizes the garbage collection algorithm in GcState::collect_inner by pre-computing referent pointers once in step 3 and caching them in a referents_map, then reusing this cache in step 4's reachability traversal with fallback computation for objects not included in step 3.

Changes

Cohort / File(s) Summary
Referent Pointer Caching
crates/vm/src/gc_state.rs
Added referents_map to pre-compute and cache object referent pointers in step 3, reused during step 4's BFS traversal with fallback to on-demand computation for skipped objects.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Whiskers twitch with delight so bright,
We cache the referents, once, just right,
No wasteful recomputing today,
The garbage collector hops with glee—hooray! 🎉

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and specifically identifies the main change: fixing a TOCTOU (time-of-check-time-of-use) race condition in the GC's collect_inner referent traversal by pre-computing referent pointers.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/vm/src/gc_state.rs (1)

465-473: Reduce avoidable cloning in the GC hot path.

At Line 473 and Lines 501-504, referent vectors are cloned multiple times. This adds extra allocations during collection and can be trimmed without changing behavior.

♻️ Proposed refactor (same behavior, fewer allocations)
-        let mut referents_map: std::collections::HashMap<GcPtr, Vec<NonNull<PyObject>>> =
+        let mut referents_map: std::collections::HashMap<GcPtr, Vec<NonNull<PyObject>>> =
             std::collections::HashMap::new();
         for &ptr in &collecting {
             let obj = unsafe { ptr.0.as_ref() };
             if obj.strong_count() == 0 {
                 continue;
             }
             let referent_ptrs = unsafe { obj.gc_get_referent_ptrs() };
-            referents_map.insert(ptr, referent_ptrs.clone());
-            for child_ptr in referent_ptrs {
+            for &child_ptr in &referent_ptrs {
                 let gc_ptr = GcPtr(child_ptr);
                 if collecting.contains(&gc_ptr)
                     && let Some(refs) = gc_refs.get_mut(&gc_ptr)
                 {
                     *refs = refs.saturating_sub(1);
                 }
             }
+            referents_map.insert(ptr, referent_ptrs);
         }
@@
-                let referent_ptrs = referents_map
-                    .get(&ptr)
-                    .cloned()
-                    .unwrap_or_else(|| unsafe { obj.gc_get_referent_ptrs() });
-                for child_ptr in referent_ptrs {
+                let referent_ptrs = referents_map
+                    .get(&ptr)
+                    .map(|v| std::borrow::Cow::Borrowed(v.as_slice()))
+                    .unwrap_or_else(|| {
+                        std::borrow::Cow::Owned(unsafe { obj.gc_get_referent_ptrs() })
+                    });
+                for &child_ptr in referent_ptrs.iter() {
                     let gc_ptr = GcPtr(child_ptr);
                     if collecting.contains(&gc_ptr) && reachable.insert(gc_ptr) {
                         worklist.push(gc_ptr);
                     }
                 }

Also applies to: 501-505

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/gc_state.rs` around lines 465 - 473, The GC hot path currently
clones referent vectors unnecessarily: when iterating over collecting you call
obj.gc_get_referent_ptrs() and insert referents_map.insert(ptr,
referent_ptrs.clone()), then later clone again (lines ~501-505). Change this to
move the original Vec<NonNull<PyObject>> into the map (avoid .clone()) and, when
later iterating, borrow or drain the stored Vec as needed instead of cloning it
(update uses around where referents_map.get/entry is accessed). Specifically
modify the code around referents_map, the loop over collecting, and usages of
gc_get_referent_ptrs()/referents_map so that the Vec returned by
gc_get_referent_ptrs() is inserted by value and reused rather than cloned.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@crates/vm/src/gc_state.rs`:
- Around line 465-473: The GC hot path currently clones referent vectors
unnecessarily: when iterating over collecting you call
obj.gc_get_referent_ptrs() and insert referents_map.insert(ptr,
referent_ptrs.clone()), then later clone again (lines ~501-505). Change this to
move the original Vec<NonNull<PyObject>> into the map (avoid .clone()) and, when
later iterating, borrow or drain the stored Vec as needed instead of cloning it
(update uses around where referents_map.get/entry is accessed). Specifically
modify the code around referents_map, the loop over collecting, and usages of
gc_get_referent_ptrs()/referents_map so that the Vec returned by
gc_get_referent_ptrs() is inserted by value and reused rather than cloned.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: bed67f50-2606-4c55-8469-84d0f8b546cc

📥 Commits

Reviewing files that changed from the base of the PR and between 9282a87 and e88e298.

📒 Files selected for processing (1)
  • crates/vm/src/gc_state.rs

@youknowone youknowone merged commit af0c252 into RustPython:main Mar 27, 2026
15 checks passed
@youknowone youknowone deleted the fix/dict-thread-safety branch March 27, 2026 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant