fix(pvc): handle FileSystemResizePending condition to prevent pod cre…#9981
Merged
mnencia merged 3 commits intocloudnative-pg:mainfrom Feb 21, 2026
Merged
Conversation
Contributor
|
❗ By default, the pull request is configured to backport to all release branches.
|
f70df38 to
8964d8d
Compare
1 task
Member
|
/test depth=push |
Contributor
|
@sxd, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/22146955316 |
8201b29 to
ead99f2
Compare
armru
approved these changes
Feb 19, 2026
…ation deadlock When a PVC has the FileSystemResizePending condition, the volume has been resized at the storage layer but the filesystem resize is pending - it requires a pod to mount the volume to complete the resize. Previously, classifyPVC() would return "resizing" for such PVCs even when no pod was attached. This caused a deadlock: 1. PVC needs a pod mount to complete filesystem resize 2. CNPG sees PVC as "resizing" so doesn't create a pod 3. Filesystem resize never completes → cluster stuck The fix reorders the classification logic to: 1. First check if a pod exists - if so, resizing can complete 2. If no pod and FileSystemResizePending condition is present, classify as "dangling" to trigger pod recreation 3. If no pod but still waiting for volume resize at storage layer, continue to classify as "resizing" This ensures pods are recreated when needed to complete filesystem resizes, while still respecting storage-layer resize operations in progress. Signed-off-by: Jeff Mealo <[email protected]> Assisted-by: Claude Opus 4.5
… logging Check that the FileSystemResizePending condition has ConditionTrue status before triggering PVC reclassification as dangling. Add an Info-level log message when this reclassification occurs to aid debugging. Signed-off-by: Armando Ruocco <[email protected]>
ead99f2 to
9861b18
Compare
Signed-off-by: Marco Nenciarini <[email protected]>
7f19a15 to
34c94b0
Compare
Member
|
/test |
Contributor
|
@mnencia, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/22235398611 |
mnencia
approved these changes
Feb 21, 2026
cnpg-bot
pushed a commit
that referenced
this pull request
Feb 21, 2026
After a volume resize completes at the storage layer, the filesystem resize still needs a running pod to mount the volume. If the pod is missing, the instance was stuck indefinitely because no new pod was created. Now the operator detects this condition and recreates the pod to complete the resize. Closes #9980 Signed-off-by: Jeff Mealo <[email protected]> Signed-off-by: Armando Ruocco <[email protected]> Signed-off-by: Marco Nenciarini <[email protected]> Co-authored-by: Armando Ruocco <[email protected]> Co-authored-by: Marco Nenciarini <[email protected]> (cherry picked from commit 0c0fa23)
cnpg-bot
pushed a commit
that referenced
this pull request
Feb 21, 2026
After a volume resize completes at the storage layer, the filesystem resize still needs a running pod to mount the volume. If the pod is missing, the instance was stuck indefinitely because no new pod was created. Now the operator detects this condition and recreates the pod to complete the resize. Closes #9980 Signed-off-by: Jeff Mealo <[email protected]> Signed-off-by: Armando Ruocco <[email protected]> Signed-off-by: Marco Nenciarini <[email protected]> Co-authored-by: Armando Ruocco <[email protected]> Co-authored-by: Marco Nenciarini <[email protected]> (cherry picked from commit 0c0fa23)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(pvc): handle FileSystemResizePending condition to prevent pod creation deadlock
Closes #9980
Problem
When a PVC has the
FileSystemResizePendingcondition, CloudNativePG incorrectly classifies it as "resizing" even when no pod is attached. This causes a deadlock:FileSystemResizePendingcondition (filesystem resize needs pod mount)Root Cause
In
classifyPVC(), the resizing check happens before the pod existence check:The
FileSystemResizePendingcondition specifically indicates the volume resize is done at the storage layer and requires a pod mount to complete the filesystem resize. Returningresizingprevents pod creation, creating a deadlock.Solution
Reorder the classification logic to:
FileSystemResizePending- classify asdanglingto trigger pod recreationResizing- classify asresizing(storage resize still in progress)Changes
pkg/reconciler/persistentvolumeclaim/resources.go: AddisFileSystemResizePending()helper functionpkg/reconciler/persistentvolumeclaim/status.go: Reorder classification logic inclassifyPVC()pkg/reconciler/persistentvolumeclaim/resources_test.go: Add comprehensive testsTesting
isFileSystemResizePending()function (4 tests)FileSystemResizePending(3 tests)Compatibility
AI Assistance
This fix was developed with assistance from Claude Opus 4.5 while developing the Dynamic Storage feature. The issue was discovered while running E2E tests on Azure AKS 1.34.2.