Problem
proxy/proxy.go:144 sets ResponseHeaderTimeout: 5 * time.Minute on the shared transport, which bounds only time to response headers. Once headers arrive there is no deadline on the body, so an upstream that opens a stream and then produces nothing holds the connection until the client gives up.
Observed
Debugging a stalling Anthropic-compatible gateway, requests behaved like this:
HTTP 200, Content-Type: text/event-stream, headers in ~2s
event: message_start …"usage":{"input_tokens":0,"output_tokens":0}
event: content_block_start
event: content_block_delta …{"delta":{"text":""}} ← empty keep-alive
<nothing for 181–361 seconds>
33 such requests. usage never populated, zero tokens generated. From the proxy's perspective everything is healthy — headers arrived quickly — so nothing fires, nothing is logged as an error, and the entry lands in network.jsonl as a 200 with a 476-byte body.
Why it matters
- The failure is invisible. A stalled stream is indistinguishable from a slow-but-working one in the logs, and there is no signal to act on. Diagnosing this took hours largely because the proxy reported success.
- Resources are held. The connection, and any per-run bookkeeping, stay live for the full client timeout.
- Downstream clients invent their own behaviour. Claude Code waits ~4 minutes and then retries, which is the only reason these ever terminated.
Suggestion
An idle deadline on body reads — reset on each byte forwarded, so long legitimate generations are unaffected and only genuine silence trips it. On expiry, terminate the stream and surface it as an error rather than a 200, so RequestLogData records something a user can see.
Value is worth weighing: a configurable idle timeout would turn a silent multi-minute hang into a fast, legible failure. Default should be generous (a minute or more of complete silence) since reasoning models can be slow to first token.
Problem
proxy/proxy.go:144setsResponseHeaderTimeout: 5 * time.Minuteon the shared transport, which bounds only time to response headers. Once headers arrive there is no deadline on the body, so an upstream that opens a stream and then produces nothing holds the connection until the client gives up.Observed
Debugging a stalling Anthropic-compatible gateway, requests behaved like this:
33 such requests.
usagenever populated, zero tokens generated. From the proxy's perspective everything is healthy — headers arrived quickly — so nothing fires, nothing is logged as an error, and the entry lands innetwork.jsonlas a200with a 476-byte body.Why it matters
Suggestion
An idle deadline on body reads — reset on each byte forwarded, so long legitimate generations are unaffected and only genuine silence trips it. On expiry, terminate the stream and surface it as an error rather than a
200, soRequestLogDatarecords something a user can see.Value is worth weighing: a configurable idle timeout would turn a silent multi-minute hang into a fast, legible failure. Default should be generous (a minute or more of complete silence) since reasoning models can be slow to first token.