Skip to content

_SelectorTransport.get_write_buffer_size() is O(n), causing quadratic behavior under write pressure #146507

@Suor

Description

@Suor

Bug report

What happened?

_SelectorTransport.get_write_buffer_size() recalculates the total buffer size on every call by iterating the entire _buffer deque:

def get_write_buffer_size(self):
    return sum(map(len, self._buffer))

This method is called from _maybe_pause_protocol() after every write() / writelines(), and from _maybe_resume_protocol() during _write_ready(). When the transport cannot drain fast enough and chunks accumulate in the buffer, this creates O(n²) behavior — each new write iterates over all previously buffered chunks.

In our production system (aiohttp WebSocket server sending many small binary messages), this caused get_write_buffer_size to consume ~90% CPU under load.

Other transports already use O(1) tracking

_SelectorTransport is the only transport with this problem. Even within the same module, _SelectorDatagramTransport already maintains a running self._buffer_size counter — incrementing on write, decrementing on drain — and returns it directly from get_write_buffer_size() in O(1).

The same O(1) pattern is used in:

  • _ProactorBaseWritePipeTransport (self._buffer_size)
  • SSLProtocol (self._write_buffer_size)

Suggested fix

Maintain a running total of buffered bytes in _SelectorTransport, consistent with _SelectorDatagramTransport in the same module.

CPython versions tested on

3.13 (but the master branch here has the same code for _SelectorTransport.get_write_buffer_size().

Operating system

Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directorytopic-asyncio

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions