Skip to content

fix(roles): prevent postgres from leaking role passwords in logs#9950

Merged
mnencia merged 11 commits intomainfrom
dev/9799
Feb 21, 2026
Merged

fix(roles): prevent postgres from leaking role passwords in logs#9950
mnencia merged 11 commits intomainfrom
dev/9799

Conversation

@jsilvela
Copy link
Collaborator

@jsilvela jsilvela commented Feb 10, 2026

When PostgreSQL executes CREATE/ALTER ROLE with a cleartext password, it
can log the password via log_statement and log_min_error_statement. Wrap
password-carrying role statements in a transaction that suppresses
logging with SET LOCAL, preventing leakage in both success and failure
scenarios.

Closes #9799

@jsilvela jsilvela requested a review from a team as a code owner February 10, 2026 12:29
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Feb 10, 2026
@cnpg-bot cnpg-bot added backport-requested ◀️ This pull request should be backported to all supported releases release-1.25 release-1.27 release-1.28 labels Feb 10, 2026
@github-actions
Copy link
Contributor

❗ By default, the pull request is configured to backport to all release branches.

  • To stop backporting this pr, remove the label: backport-requested ◀️ or add the label 'do not backport'
  • To stop backporting this pr to a certain release branch, remove the specific branch label: release-x.y

@jsilvela jsilvela changed the title chore: prevent postgress logging passwords in cleartext chore: prevent postgress logging role passwords in cleartext Feb 10, 2026
@dosubot dosubot bot added the chore Intangible work to reduce technical debt label Feb 10, 2026
@jsilvela jsilvela marked this pull request as draft February 10, 2026 12:29
@jsilvela
Copy link
Collaborator Author

/test depth=push

@github-actions
Copy link
Contributor

@jsilvela, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/21868436248

@jsilvela jsilvela changed the title chore: prevent postgress logging role passwords in cleartext chore: prevent postgres from logging role passwords in cleartext Feb 10, 2026
@jsilvela jsilvela changed the title chore: prevent postgres from logging role passwords in cleartext chore: prevent postgres from logging role passwords during CREATE/ALTER ROLE Feb 10, 2026
@cnpg-bot cnpg-bot added the ok to merge 👌 This PR can be merged label Feb 10, 2026
@jsilvela
Copy link
Collaborator Author

jsilvela commented Feb 11, 2026

Manually tested.
Forcing errors by using ALTER to create a new role, so that postgres errors on trying to update a non-existing role. We get only this in the logs, with the "query" field not even present:

{"level":"info","ts":"2026-02-11T08:47:52.86477304Z","msg":"Triggered a managed role reconciliation",
  "logger":"instance-manager","logging_pod":"cluster-example-with-roles-1",<- snip ->}
{"level":"info","ts":"2026-02-11T08:47:52.902391931Z",
  "logger":"postgres","msg":"record",
  "logging_pod":"cluster-example-with-roles-1",
  "record":{
    "log_time":"2026-02-11 08:47:52.902 UTC",
    "user_name":"postgres","database_name":"postgres", <- snip ->,
    "command_tag":"ALTER ROLE" <-snip->,
    "error_severity":"ERROR","sql_state_code":"42704",
    "message":"role \"dante\" does not exist","application_name":"cnpg-instance-manager",<- snip ->,
    "query_id":"0"
  }
}

and in the Cluster CR's Status, we don't have any password leakage.

    ....
    managedRolesStatus:
      byStatus:
        pending-reconciliation:
        - dante
      ...
      cannotReconcile:
        dante:
        - 'could not perform ALTER on role dante: role "dante" does not exist'

The same is verified when ensuring errors by forcing CREATE when updating a role.

@jsilvela jsilvela changed the title chore: prevent postgres from logging role passwords during CREATE/ALTER ROLE chore: prevent postgres from leaking role passwords in logs Feb 11, 2026
@jsilvela jsilvela marked this pull request as ready for review February 11, 2026 10:03
@jsilvela
Copy link
Collaborator Author

And testing on replica clusters, as per the original Issue, note the "query" field is absent:

{"level":"info","ts":"2026-02-11T13:57:39.049732289Z",
  "logger":"postgres","msg":"record","logging_pod":"cluster-replica-tls-1",
  "record":{"log_time":"2026-02-11 13:57:39.049 UTC",
    "user_name":"postgres","database_name":"postgres","process_id":"156","connection_from":"[local]",
    "session_id":"698c8ad3.9c","session_line_num":"1",
    "command_tag":"ALTER ROLE","session_start_time":"2026-02-11 13:57:39 UTC",
    "virtual_transaction_id":"26/2","transaction_id":"0",
    "error_severity":"ERROR",
    "sql_state_code":"25006",
    "message":"cannot execute ALTER ROLE in a read-only transaction",
    "application_name":"cnpg-instance-manager","backend_type":"client backend",
    "query_id":"0"}
}

Copy link
Contributor

@maxlengdell maxlengdell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good fix!

@jsilvela jsilvela changed the title chore: prevent postgres from leaking role passwords in logs chore(roles): prevent postgres from leaking role passwords in logs Feb 12, 2026
@armru armru changed the title chore(roles): prevent postgres from leaking role passwords in logs fix(roles): prevent postgres from leaking role passwords in logs Feb 12, 2026
@armru
Copy link
Member

armru commented Feb 12, 2026

promoting to fix

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Feb 16, 2026
jsilvela and others added 9 commits February 21, 2026 17:59
Signed-off-by: Jaime Silvela <[email protected]>
Signed-off-by: Jaime Silvela <[email protected]>
Signed-off-by: Jaime Silvela <[email protected]>
…_error_statement

Fixed three issues:
1. SetUserPassword used db.Exec instead of tx.Exec, executing the ALTER
   ROLE outside the transaction and completely bypassing the SET LOCAL
   protection
2. Only log_min_error_statement was suppressed (error case), but
   log_statement (success case with ddl/mod/all) was not, leaving
   passwords exposed in normal statement logging
3. In Create(), the debug log was emitted after appending the password
   to the query, leaking it in application-level logs

Signed-off-by: Armando Ruocco <[email protected]>
@mnencia mnencia merged commit 983bc6b into main Feb 21, 2026
35 of 37 checks passed
@mnencia mnencia deleted the dev/9799 branch February 21, 2026 20:11
@github-project-automation github-project-automation bot moved this from Todo to Done in Security Map Feb 21, 2026
@dosubot
Copy link

dosubot bot commented Feb 21, 2026

Related Documentation

Checked 0 published document(s) in 1 knowledge base(s). No updates required.

How did I do? Any feedback?  Join Discord

cnpg-bot pushed a commit that referenced this pull request Feb 21, 2026
When PostgreSQL executes CREATE/ALTER ROLE with a cleartext password, it
can log the password via log_statement and log_min_error_statement. Wrap
password-carrying role statements in a transaction that suppresses
logging with SET LOCAL, preventing leakage in both success and failure
scenarios.

Closes #9799

Signed-off-by: Jaime Silvela <[email protected]>
Signed-off-by: Armando Ruocco <[email protected]>
Signed-off-by: Marco Nenciarini <[email protected]>
Co-authored-by: Armando Ruocco <[email protected]>
Co-authored-by: Marco Nenciarini <[email protected]>
(cherry picked from commit 983bc6b)
cnpg-bot pushed a commit that referenced this pull request Feb 21, 2026
When PostgreSQL executes CREATE/ALTER ROLE with a cleartext password, it
can log the password via log_statement and log_min_error_statement. Wrap
password-carrying role statements in a transaction that suppresses
logging with SET LOCAL, preventing leakage in both success and failure
scenarios.

Closes #9799

Signed-off-by: Jaime Silvela <[email protected]>
Signed-off-by: Armando Ruocco <[email protected]>
Signed-off-by: Marco Nenciarini <[email protected]>
Co-authored-by: Armando Ruocco <[email protected]>
Co-authored-by: Marco Nenciarini <[email protected]>
(cherry picked from commit 983bc6b)
cnpg-bot pushed a commit that referenced this pull request Feb 21, 2026
When PostgreSQL executes CREATE/ALTER ROLE with a cleartext password, it
can log the password via log_statement and log_min_error_statement. Wrap
password-carrying role statements in a transaction that suppresses
logging with SET LOCAL, preventing leakage in both success and failure
scenarios.

Closes #9799

Signed-off-by: Jaime Silvela <[email protected]>
Signed-off-by: Armando Ruocco <[email protected]>
Signed-off-by: Marco Nenciarini <[email protected]>
Co-authored-by: Armando Ruocco <[email protected]>
Co-authored-by: Marco Nenciarini <[email protected]>
(cherry picked from commit 983bc6b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-requested ◀️ This pull request should be backported to all supported releases chore Intangible work to reduce technical debt lgtm This PR has been approved by a maintainer ok to merge 👌 This PR can be merged release-1.25 release-1.27 release-1.28 security 👮 size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug]: If managed user creation/alteration fails for whatever reason, postgres logs the user's password

5 participants