Background

Recently, we got notified about a customer that the RDS instance was not reachable, with the logs in LocalStack:

postgresql_proxy           : PostgreSQL proxy quit unexpectedly:
Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/postgresql_proxy/proxy.py", line 222, in listen
    self.service_connection(key, mask)
  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/postgresql_proxy/proxy.py", line 170, in service_connection
    conn.received(recv_data)
  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/postgresql_proxy/connection.py", line 51, in received
    self.process_inbound_packet(header, body)
  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/postgresql_proxy/connection.py", line 67, in process_inbound_packet
    self.redirect_conn.out_bytes += message
AttributeError: 'NoneType' object has no attribute 'out_bytes'

I was able to reproduce this issue locally, by running the following psql command for a RDS-instance created in LocalStack:

psql -d test -U test -p 4510 -h localhost

• As we need a password here to connect to the instance, the command should normally include the -W option. When this option is included, the password prompt is shown instantly, and the connection succeeds
• If the -W option is omitted, the psql still shows (a slightly different) password prompt, but before we can see some messages being send to the proxy:

  ----> Received message. Relaying. Speaker: proxy_2, message:
  b'\x00\x00\x00\n\x04\xd2\x16/\x00\x00'
  ----> Received message. Relaying. Speaker: postgresql_2, message:
  b'N'
  ----> Received message. Relaying. Speaker: proxy_2, message:
  b'\x00\x00\x00L\x00\x03\x00\x00client_encoding\x00UTF8\x00user\x00test\x00database\x00test\x00application_name\x00psql\x00\x00'
  ----> Received message. Relaying. Speaker: postgresql_2, message:
  b'R\x00\x00\x00\x08\x00\x00\x00\x03'
  ----> resetting connection proxy_2
  ----> Received message. Relaying. Speaker: postgresql_2, message:
  b'E\x00\x00\x00nSFATAL\x00VFATAL\x00C08P01\x00Mexpected password response, got message type 88\x00Fauth.c\x00L671\x00Rrecv_password_packet\x00\x00'
  ---> error, redirect_conn is not set
  

  • the last line is where the self.redirect_conn is None, and the connection to the proxy is broken with the error message from above. Somewhere in between the connection seems to be reset here.

Solution

In order to not terminate the proxy, we need a check in the process_inbound_packet that ensures the self.redirect_conn is still available.