Skip to content

Query Service 0.14.3: WebSocket connection fails with extended_connect_disabled behind HTTP/2 proxies #100

@decebal

Description

@decebal

Description

After upgrading Query Service from 0.14.1 to 0.14.3, the WebSocket connection to Core fails with:

Mint.WebSocketError{reason: :extended_connect_disabled}

The QS health endpoint returns 503 because it can't establish the WebSocket stream to Core.

Environment

  • Hosting: Fly.io (both Core and Query Service)
  • Core: 0.14.3 — healthy, WebSocket endpoint works fine over HTTP/1.1
  • Query Service: 0.14.3 — starts but health check returns 503
  • Connection URL: wss://alphasigmapro-allsource-core.fly.dev/api/v1/events/stream

Root cause

The Mint HTTP client in QS 0.14.3 attempts an HTTP/2 extended CONNECT for the WebSocket upgrade. Fly.io's reverse proxy (and likely other proxies like Cloudflare, ALBs) does not support HTTP/2 extended CONNECT — only standard HTTP/1.1 Upgrade: websocket.

Verified that Core's WebSocket endpoint works correctly over HTTP/1.1:

curl -sv --http1.1 \
  -H "Connection: Upgrade" \
  -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" \
  -H "Sec-WebSocket-Key: dGVzdA==" \
  "https://alphasigmapro-allsource-core.fly.dev/api/v1/events/stream"

# Returns: HTTP/1.1 101 Switching Protocols ✓

Expected behavior

QS should connect to Core's WebSocket endpoint using HTTP/1.1 upgrade, as it did in 0.14.1.

Suggested fix

Either:

  1. Configure the Mint WebSocket client to use :http1 protocol mode (not :http2)
  2. Add a CORE_WS_PROTOCOL env var to let operators choose
  3. Fall back to HTTP/1.1 when HTTP/2 extended CONNECT fails

Logs

[CoreWebSocketClient] Connection failed: %Mint.WebSocketError{reason: :extended_connect_disabled}, retry 16 in 28531ms
WebSocket reconnection attempt url="wss://..." attempt=16
GET /api/health → Sent 503

This repeats every ~30s indefinitely.

Workaround

Rolled back Query Service to 0.14.1 which connects successfully.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions