Skip to content

HTTP/2 Connection Multiplexing with httpx

A service calls one internal API with 300 concurrent requests per burst. Over HTTP/1.1 each in-flight request needs its own TCP connection, so the client either opens 300 sockets — 300 TLS handshakes, 300 slots in the server's accept queue, a spike of ephemeral ports — or caps the pool and queues requests behind PoolTimeout. HTTP/2 removes that trade-off by carrying many concurrent request/response streams over one connection. httpx supports it with a single flag, but "turn on http2=True" is where most teams stop, and then discover that nothing changed because the server never negotiated h2, or that one connection reset now fails 100 requests at once. This guide enables HTTP/2 properly, proves how many connections are really open, sizes the limits for multiplexing, and handles the failure modes that are unique to sharing a connection.

Prerequisites

Concurrency over HTTP/1.1 versus HTTP/2 2 columns contrasting HTTP/1.1 pool, HTTP/2 multiplexing. Concurrency over HTTP/1.1 versus HTTP/2 HTTP/1.1 pool one request per connection 300 in flight = 300 sockets a TLS handshake per socket loss affects one request limits count requests HTTP/2 multiplexing many streams per connection 300 in flight = 3 sockets a handshake per 100 streams loss stalls every stream limits count connections Multiplexing trades connection overhead for shared fate on each connection.

1. Enable HTTP/2 and confirm it was negotiated

http2=True offers HTTP/2; the server decides. Over TLS the choice happens during the handshake via ALPN, and a server, load balancer or proxy that does not advertise h2 silently gives you HTTP/1.1. Always check response.http_version instead of assuming.

# pip install "httpx[http2]"
import asyncio
import httpx


async def check_protocol(url: str) -> str:
    async with httpx.AsyncClient(http2=True) as client:
        response = await client.get(url)
        return response.http_version        # "HTTP/2" or "HTTP/1.1"


async def main() -> None:
    print(await check_protocol("https://www.example.com/"))

    # Plain-text internal service that speaks h2c with prior knowledge:
    async with httpx.AsyncClient(http1=False, http2=True) as client:
        r = await client.get("http://api.internal:8080/health")
        print(r.http_version)


asyncio.run(main())

Verify: the first call prints HTTP/2 for an h2-capable HTTPS endpoint. If it prints HTTP/1.1, check whether a TLS-terminating proxy sits in the path — many load balancers negotiate h2 with the client but not with the upstream, or the reverse.

2. Count the connections that are actually opened

The benefit of multiplexing is fewer connections, so measure connections, not requests. httpx exposes httpcore's trace hook through the trace request extension, which reports every TCP connect and TLS handshake as it happens.

import asyncio
import collections
import httpx

events: collections.Counter[str] = collections.Counter()


async def trace(event_name: str, info: dict) -> None:
    if event_name in ("connection.connect_tcp.complete", "connection.start_tls.complete"):
        events[event_name] += 1


async def burst(client: httpx.AsyncClient, url: str, n: int) -> None:
    async with asyncio.TaskGroup() as tg:
        for _ in range(n):
            tg.create_task(client.get(url, extensions={"trace": trace}))


async def main(url: str) -> None:
    for http2 in (False, True):
        events.clear()
        limits = httpx.Limits(max_connections=400, max_keepalive_connections=400)
        async with httpx.AsyncClient(http2=http2, limits=limits) as client:
            await burst(client, url, 300)
        print(f"http2={http2}: tcp={events['connection.connect_tcp.complete']} "
              f"tls={events['connection.start_tls.complete']}")

Verify: with http2=False the TCP count climbs towards the burst size; with http2=True it collapses to one or a few connections — one per SETTINGS_MAX_CONCURRENT_STREAMS worth of simultaneous requests. If both runs show the same count, step 1's negotiation check failed for this URL.

Connections needed for a 300-request burst 4 bars comparing HTTP/1.1 TCP connections with the others. Connections needed for a 300-request burst HTTP/1.1 TCP connections 300 HTTP/1.1 TLS handshakes 300 HTTP/2 TCP connections 3 HTTP/2 TLS handshakes 3 Computed: connections = ceil(concurrent requests / streams per connection), with 100 streams for HTTP/2. Every avoided handshake is a round trip and a burst of CPU on both sides.

The chart is the arithmetic of a single 300-request burst: HTTP/1.1 needs one connection and one TLS handshake per concurrent request, while HTTP/2 needs one connection per block of streams the server allows. Most servers advertise 100 or more concurrent streams, so the handshake bill falls by two orders of magnitude.

3. Size the limits for streams, not sockets

httpx.Limits counts connections. Under HTTP/2, each connection carries up to the server's advertised stream limit, and httpcore opens another connection only when the existing ones are full. That changes what the numbers mean.

import asyncio
import httpx

SERVER_MAX_STREAMS = 100          # read from the server's SETTINGS frame or its docs
TARGET_CONCURRENCY = 300

limits = httpx.Limits(
    max_connections=TARGET_CONCURRENCY // SERVER_MAX_STREAMS + 1,   # 4 connections
    max_keepalive_connections=TARGET_CONCURRENCY // SERVER_MAX_STREAMS + 1,
    keepalive_expiry=60.0,        # keep warm connections longer: each one is worth 100 streams
)
timeout = httpx.Timeout(connect=3.0, read=10.0, write=10.0, pool=2.0)
client = httpx.AsyncClient(http2=True, limits=limits, timeout=timeout)

# Cap in-flight work explicitly so a burst beyond the stream budget waits in the
# application, where it is observable, instead of inside the pool.
inflight = asyncio.Semaphore(TARGET_CONCURRENCY)


async def fetch(url: str) -> bytes:
    async with inflight:
        response = await client.get(url)
        response.raise_for_status()
        return response.content

A long keepalive_expiry matters more under HTTP/2 than HTTP/1.1: closing one idle h2 connection throws away the capacity of a hundred concurrent requests, and the next burst pays a fresh handshake before any of them can start. The semaphore is the same concurrency cap you would use with HTTP/1.1; here it keeps the queue visible in your metrics.

Verify: under a sustained burst of TARGET_CONCURRENCY, the trace counter from step 2 stays at or below max_connections, and httpx.PoolTimeout never fires. A pool timeout with few connections open means the server's real stream limit is lower than SERVER_MAX_STREAMS.

4. Retry safely when a shared connection dies

Multiplexing concentrates risk. When a server restarts it sends GOAWAY, and when a connection is reset, every stream on it fails together — a hundred requests, not one. httpx surfaces these as httpx.RemoteProtocolError or httpx.ReadError. Idempotent requests can be retried on a fresh connection; non-idempotent ones need an idempotency key first.

import asyncio
import random
import httpx

RETRYABLE = (httpx.RemoteProtocolError, httpx.ReadError, httpx.ConnectError)
IDEMPOTENT = {"GET", "HEAD", "PUT", "DELETE", "OPTIONS"}


async def request_with_retry(client: httpx.AsyncClient, method: str, url: str,
                             attempts: int = 3, **kwargs) -> httpx.Response:
    for attempt in range(1, attempts + 1):
        try:
            return await client.request(method, url, **kwargs)
        except RETRYABLE:
            if method not in IDEMPOTENT and "Idempotency-Key" not in kwargs.get("headers", {}):
                raise                                   # unsafe to replay
            if attempt == attempts:
                raise
            # jitter so a hundred failed streams do not reconnect in the same instant
            await asyncio.sleep(random.uniform(0, 0.1 * 2 ** attempt))
    raise AssertionError("unreachable")

The jitter is not optional here. Without it, every stream that failed on the dead connection retries at the same moment, the pool opens several new connections simultaneously, and the thundering herd lands on the instance that just came back — the pattern described in exponential backoff with jitter.

Verify: restart one backend instance during a load test. Requests that were in flight fail once, are retried with spread-out delays, and succeed on a new connection; the error rate returns to zero within one retry window.

5. Know when to stay on HTTP/1.1

HTTP/2 is not a universal upgrade. It removes connection overhead but keeps TCP-level head-of-line blocking: one lost packet stalls every stream on that connection until it is retransmitted. It also adds per-stream flow control, which can slow very large transfers.

import httpx

# Small, frequent, latency-sensitive calls to one origin: multiplex.
api_client = httpx.AsyncClient(http2=True, limits=httpx.Limits(max_connections=4))

# Large downloads or uploads: separate HTTP/1.1 client so a 2 GB transfer
# neither shares a congestion window with API calls nor fights h2 flow control.
bulk_client = httpx.AsyncClient(http2=False, limits=httpx.Limits(max_connections=8),
                                timeout=httpx.Timeout(10.0, read=300.0))

Verify: compare p99 latency of small API calls with and without a concurrent bulk transfer on the same client. If the tail rises during transfers, split the traffic onto two clients as shown. For streaming large bodies without buffering, see streaming large responses with httpx.

Which protocol for this traffic? A decision on What does the traffic look like with 3 outcomes. Which protocol for this traffic? What does the traffic look like? many small calls, one origin HTTP/2 client few connections, long keep-alive large uploads or downloads separate HTTP/1.1 client isolated congestion window proxy speaks only HTTP/1.1 no gain from h2 measure on the server side Split clients by traffic shape rather than forcing one protocol on everything.

Verification

HTTP/2 multiplexing is configured correctly when:

  • Negotiation is confirmed: response.http_version == "HTTP/2" is asserted in a startup smoke test for each upstream, not assumed.
  • Connection count is low and stable: trace counters show connections in single digits during bursts that previously opened hundreds.
  • No pool timeouts under target load: max_connections × server stream limit comfortably exceeds peak concurrency, and the application-level semaphore holds any overflow.
  • Connection loss is survivable: a backend restart produces a brief, jittered retry burst for idempotent calls and no user-visible errors.
  • Bulk traffic is isolated: large transfers do not move the latency tail of small API calls.

Pitfalls & edge cases

  • Proxies that downgrade. Corporate proxies and some service meshes terminate TLS and speak HTTP/1.1 upstream. The client sees h2, the server sees 300 connections. Measure on the server side too.
  • Keep-alive expiry shorter than the server's idle timeout. If the server closes idle h2 connections after 30 seconds and the client keeps them for 60, the first request after a lull hits a dead connection. Set keepalive_expiry just below the server's idle timeout.
  • Assuming stream limits. Servers can advertise 10 concurrent streams, not 100, and can lower the value mid-connection. Treat the limit as data from the server, and alarm on pool timeouts rather than hard-coding capacity.
  • Non-idempotent requests on a GOAWAY. Streams with IDs above the server's last-processed ID were not handled and are safe to retry; others may have been. Without idempotency keys, never blindly replay a POST after a protocol error.
  • Missing h2 dependency. Installing plain httpx and passing http2=True raises an ImportError at client creation; pin httpx[http2] in requirements so a rebuild does not break it.

Frequently Asked Questions

How do I enable HTTP/2 in httpx?

Install the extra with pip install "httpx[http2]" and create the client with httpx.AsyncClient(http2=True). Over HTTPS, HTTP/2 is negotiated with ALPN during the TLS handshake, so confirm it by checking response.http_version. For plain-text services that support prior-knowledge h2c, use AsyncClient(http1=False, http2=True).

Does httpx max_connections limit requests or connections under HTTP/2?

It limits connections. Each HTTP/2 connection can carry as many concurrent streams as the server advertises, commonly 100, so a limit of four connections can serve roughly 400 simultaneous requests. httpcore opens an additional connection only when existing ones have no free streams, up to max_connections.

Why does my httpx client still use HTTP/1.1 with http2=True?

The server or a proxy in front of it did not negotiate HTTP/2. ALPN lets either side decline h2, and many load balancers or TLS-terminating proxies only speak HTTP/1.1 on one side. The request succeeds silently over HTTP/1.1, which is why checking response.http_version is essential.

Is HTTP/2 always faster than HTTP/1.1 for async Python clients?

No. HTTP/2 removes per-request connection and TLS handshake overhead, which helps many small concurrent requests to one origin. It keeps TCP head-of-line blocking, so packet loss stalls every stream on a connection, and its flow control can slow very large transfers. Large uploads and downloads are often better on a separate HTTP/1.1 client.