Cancelling requests under pool contention permanently leaks connection slots (AsyncConnectionPool → PoolTimeout)

[seanmuth]

Summary

When requests to AsyncConnectionPool are cancelled while the pool is under contention, connections can be left in a state that the pool never reclaims. They accumulate until every slot is consumed, after which all subsequent requests fail with PoolTimeout even though the origin server is healthy and the event loop is fine. Only recreating the pool (process restart) recovers.

This reproduces on httpcore 1.0.9 (latest) and appears to be the mechanism behind #658 (cancellation not releasing connections) and #830 (pool hangs after a cancelled connection).

Reproduction

Pure httpcore + stdlib asyncio (tiny local HTTP/1.1 server), no httpx:

import asyncio, contextlib, httpcore

MAX_CONNECTIONS, CONCURRENCY, CHURN_SECONDS = 10, 20, 5
TIMEOUT = {"connect": 2.0, "read": 2.0, "write": 2.0, "pool": 3.0}

async def _serve(reader, writer):
    with contextlib.suppress(Exception):
        while True:
            head = b""
            while b"\r\n\r\n" not in head:
                chunk = await reader.read(4096)
                if not chunk:
                    return
                head += chunk
            writer.write(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\n{}")
            await writer.drain()

async def _request(pool, port):
    return await pool.request(
        b"GET", f"http://127.0.0.1:{port}/",
        headers=[(b"Host", f"127.0.0.1:{port}".encode())],
        extensions={"timeout": TIMEOUT})

def _census(pool):
    conns = pool.connections
    stuck = sum(1 for c in conns if not c.is_idle() and not c.is_closed())
    return f"connections={len(conns)} (stuck={stuck}), waiting requests={len(pool._requests)}"

async def _trial(port, *, cancel):
    pool = httpcore.AsyncConnectionPool(max_connections=MAX_CONNECTIONS)
    deadline = asyncio.get_event_loop().time() + CHURN_SECONDS
    async def worker():
        while asyncio.get_event_loop().time() < deadline:
            task = asyncio.create_task(_request(pool, port))
            if cancel:
                await asyncio.sleep(0.005)   # start the request, then cancel mid-flight
                task.cancel()
                with contextlib.suppress(BaseException):
                    await task
            else:
                with contextlib.suppress(BaseException):
                    await task
    with contextlib.suppress(Exception):
        await asyncio.wait_for(asyncio.gather(*[worker() for _ in range(CONCURRENCY)],
                                              return_exceptions=True), timeout=CHURN_SECONDS + 10)
    await asyncio.sleep(2)   # let the pool settle; a healthy pool goes fully idle
    label = "cancelled" if cancel else "control  "
    try:
        r = await asyncio.wait_for(_request(pool, port), timeout=8)
        await r.aclose(); probe = f"HTTP {r.status}"
    except Exception as exc:
        probe = type(exc).__name__
    print(f"[{label}] {_census(pool)} -> probe: {probe}")
    with contextlib.suppress(Exception):
        await asyncio.wait_for(pool.aclose(), timeout=3)

async def main():
    print(f"httpcore {httpcore.__version__}")
    server = await asyncio.start_server(_serve, "127.0.0.1", 0)
    port = server.sockets[0].getsockname()[1]
    await _trial(port, cancel=False)
    await _trial(port, cancel=True)
    server.close()
    with contextlib.suppress(Exception):
        await asyncio.wait_for(server.wait_closed(), timeout=3)

asyncio.run(main())

Output

httpcore 1.0.9
[control  ] connections=10 (stuck=0), waiting requests=0  -> probe: HTTP 200
[cancelled] connections=10 (stuck=10), waiting requests=3 -> probe: PoolTimeout

Identical load; the only difference is that the cancelled trial cancels each in-flight request. The control pool returns fully idle and serves the probe; the cancelled pool is wedged at 10/10 stuck connections and the probe (and every future request) raises PoolTimeout.

Expected vs actual

• Expected: cancelling a request releases (or eventually reclaims) its pool slot, as it does for a serial/uncontended cancel.
• Actual: under contention the slot is never reclaimed; the pool degrades monotonically to permanent PoolTimeout.

Root cause analysis

Dumping the wedged pool, the stuck connections are overwhelmingly in the CONNECTING state (AsyncHTTPConnection with _connection is None and _connect_failed is False):

<AsyncHTTPConnection [CONNECTING]>   x7
<AsyncHTTPConnection ['http://127.0.0.1:...', HTTP/1.1, ACTIVE, Request Count: 2]>   x2
<AsyncHTTPConnection ['http://127.0.0.1:...', HTTP/1.1, NEW, Request Count: 0]>      x1

These arise when _assign_requests_to_connections() creates a connection and assigns it to a pooled request, but that request's task is cancelled before it ever calls connection.handle_async_request() — so _connect() never starts and never fails.

Two properties then combine to make the slot unrecoverable:

1. Not reapable. In _async/connection.py, for a not-yet-established connection, is_idle(), is_closed(), and has_expired() all return self._connect_failed (i.e. False). _async/connection_pool.py::_assign_requests_to_connections() only removes connections that are is_closed() / has_expired() / surplus-is_idle() — a CONNECTING orphan matches none of these, so it is never removed from self._connections.
2. Not reusable. For the same not-yet-established connection, is_available() returns False for HTTP/1.1 (only HTTP/2 is treated as available while connecting). So the orphan cannot be handed to a queued request either.

Net: the slot is held forever by a connection that will never connect, never fail, never expire, and can never be reused.

There is also a secondary effect that defeats naive pool-side reaping: cancelled requests can remain in self._requests, so if orphaned connections are forcibly reclaimed, the freed slots are immediately re-consumed by those stranded requests.

Notes for whoever fixes this (attempts that do not work)

We explored several fixes against the repro; sharing so they're not re-tried:

• Closing the owning connection in AsyncConnectionPool.handle_async_request's except BaseException — deadlocks: await conn.aclose() blocks when the connection is cancelled mid-connect (live socket / in-flight connect).
• Merely removing the owning connection from self._connections in that except — insufficient: the excepting request rarely owns the connection at cancel time, and stranded requests re-consume the freed slot.
• Extending AsyncShieldCancellation to cover the whole of PoolByteStream.aclose() — no effect; the dominant orphan is CONNECTING, not a stream-close race.

A correct fix seems to need both (a) making an abandoned not-yet-established connection reclaimable (e.g. a distinct state, or is_closed()/reaper awareness), with a guaranteed non-blocking teardown for the never-started case, and (b) ensuring a cancelled request is removed from self._requests so reclaimed slots aren't re-consumed. Happy to help test candidate fixes against the repro.

HTTP/1.1 vs HTTP/2

This reproduces on an HTTP/1.1 pool and appears largely HTTP/1.1-specific:

• For a not-yet-established connection, is_available() returns False for HTTP/1.1 but True when HTTP/2 is enabled (self._http2 and (https or not http1) ...). So an HTTP/2-capable CONNECTING connection can still be handed to queued requests, whereas the HTTP/1.1 one is stuck.
• HTTP/1.1 is one-request-per-connection, so a cancelled request wedges a whole slot and the pool must grow many connections to stay busy (contention). HTTP/2 multiplexes, so the pool rarely reaches contention and a cancellation affects a stream rather than a connection.

I have not tested whether an HTTP/2 pool has a rarer analogue, so I'm not claiming HTTP/2 is immune, just that the exhaustion demonstrated here is an HTTP/1.1 pool phenomenon.

Environment

• httpcore 1.0.9 (httpcore[asyncio]), anyio/asyncio backend
• Python 3.12
• Reproduces deterministically with the script above; real-world trigger is a long-lived HTTP/1.1 pool under concurrent load where callers routinely cancel in-flight requests (e.g. client disconnects / task cancellation), which slowly accumulates orphans until the pool wedges.

---

🤖 Investigated and drafted with Claude Code (Claude Opus 4.8, 1M context).