Skip to content

Tags: nicois/llama.cpp

Tags

b8476

Toggle b8476's commit message

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
server : add SSE headers to fix streaming behind reverse proxy

Root cause: nginx enables proxy_buffering by default. For SSE (Server-
Sent Events) streams, this means nginx buffers the entire response
before forwarding any bytes to the client. Since SSE streams are
long-lived, the client never receives the first chunk and appears to
hang indefinitely.

The fix adds two response headers to all streaming (chunked) responses:

  X-Accel-Buffering: no  — instructs nginx to disable response buffering
  Cache-Control: no-cache — prevents caching of the event stream

These are the standard headers that SSE endpoints should emit for
compatibility with reverse proxies. They have no effect when the server
is accessed directly.