-
-
Notifications
You must be signed in to change notification settings - Fork 34.3k
Description
Bug report
What happened?
_SelectorTransport.get_write_buffer_size() recalculates the total buffer size on every call by iterating the entire _buffer deque:
def get_write_buffer_size(self):
return sum(map(len, self._buffer))This method is called from _maybe_pause_protocol() after every write() / writelines(), and from _maybe_resume_protocol() during _write_ready(). When the transport cannot drain fast enough and chunks accumulate in the buffer, this creates O(n²) behavior — each new write iterates over all previously buffered chunks.
In our production system (aiohttp WebSocket server sending many small binary messages), this caused get_write_buffer_size to consume ~90% CPU under load.
Other transports already use O(1) tracking
_SelectorTransport is the only transport with this problem. Even within the same module, _SelectorDatagramTransport already maintains a running self._buffer_size counter — incrementing on write, decrementing on drain — and returns it directly from get_write_buffer_size() in O(1).
The same O(1) pattern is used in:
_ProactorBaseWritePipeTransport(self._buffer_size)SSLProtocol(self._write_buffer_size)
Suggested fix
Maintain a running total of buffered bytes in _SelectorTransport, consistent with _SelectorDatagramTransport in the same module.
CPython versions tested on
3.13 (but the master branch here has the same code for _SelectorTransport.get_write_buffer_size().
Operating system
Linux
Linked PRs
Metadata
Metadata
Assignees
Labels
Projects
Status