Skip to content

Handling Cookies in Async HTTP Clients

An async HTTP client usually lives for the whole process and serves many concurrent tasks, while its cookie jar is a single piece of shared state. That combination produces three failures that do not occur in a one-request-at-a-time script. Measured on Python 3.14 with httpx 0.28.1 and aiohttp 3.14.3 against a local aiohttp server: aiohttp's default cookie jar discarded a session cookie set by http://127.0.0.1, so the next request was anonymous, while httpx kept it; CookieJar(unsafe=True) fixed it. Two users logging in concurrently through one shared httpx client got 100 of 200 answers for the other user, because each login overwrote the jar's session cookie; a client per user, or aiohttp's DummyCookieJar with explicit Cookie headers, gave 0 wrong answers. And against a server that issues a new token on every response, 20 concurrent requests presented an already-superseded token 333 times out of 500 with httpx and 263 with aiohttp, against 0 when the same requests ran one at a time. This guide sets cookie handling to match how the client is shared.

Prerequisites

1. Check that cookies are kept at all

Both clients have a cookie jar on by default, but their acceptance rules differ. A login that sets a cookie followed by a request that echoes it shows the difference:

async with aiohttp.ClientSession("http://127.0.0.1:8000") as s:
    async with s.get("/login", params={"user": "alice"}):
        pass
    async with s.get("/whoami") as r:
        print(await r.json())          # {"session": None}

Measured: against http://127.0.0.1, httpx sent the cookie back and the server saw alice; aiohttp's default jar did not store it, and the server saw no session. aiohttp's CookieJar rejects cookies from hosts given as IP addresses unless created with unsafe=True. Against http://localhost, both kept the cookie. The failure appears in tests and internal services addressed by IP, and disappears in production behind a hostname — or the reverse.

session = aiohttp.ClientSession(cookie_jar=aiohttp.CookieJar(unsafe=True))

Verify: a login-then-echo test against the exact base URL the service uses shows the cookie arriving.

Session cookie after a login, by client and host A grid of 3 rows by 3 columns. Session cookie after a login, by client and host client http://127.0.0.1 http://localhost httpx.AsyncClient kept kept aiohttp, default CookieJar dropped kept aiohttp, CookieJar(unsafe=True) kept kept httpx 0.28.1, aiohttp 3.14.3.

A shared client is right for connection reuse and wrong for per-user cookies. Measured with one httpx client: in each of 100 rounds, two users logged in concurrently and then asked the server who they were:

async def flow(client, user):
    await client.get("/login", params={"user": user})
    await asyncio.sleep(random.uniform(0, 0.005))
    return user, (await client.get("/whoami")).json()["session"]

results = await asyncio.gather(flow(shared, "alice"), flow(shared, "bob"))

100 of the 200 answers named the other user — in every round, whichever login finished second overwrote the jar's session cookie, and both users then used it. In a service that logs in to a downstream API on behalf of its own users, this is a data leak, not a flaky test. A client per identity removed it: 0 of 200 wrong. When identities are many and short-lived, keep one client for connections and pass cookies explicitly, with the jar disabled.

Verify: a concurrent test with two identities through the production client code never returns one identity's data to the other.

3. Pass per-request cookies explicitly

To keep one connection pool while serving many identities, disable the jar and send the Cookie header yourself from per-identity storage:

session = aiohttp.ClientSession(BASE, cookie_jar=aiohttp.DummyCookieJar())

async def login(user) -> str:
    async with session.get("/login", params={"user": user}) as r:
        return r.cookies["session"].value          # read from the response, not a jar

async def whoami(token: str):
    async with session.get("/whoami", headers={"Cookie": f"session={token}"}) as r:
        return await r.json()

Measured with the same 100 rounds of two concurrent users: 0 of 200 wrong. DummyCookieJar stores nothing, so responses cannot overwrite each other's cookies. With httpx, per-request cookies= is deprecated; set the Cookie header, or use a short-lived AsyncClient per identity that shares a transport with the main client. Store the per-identity cookie where the identity lives — the user's own session record — not in a process-wide dictionary.

Verify: the shared client's jar is a DummyCookieJar (aiohttp) or never used for identity cookies (httpx), and each request's cookie comes from the caller.

One pool, many identities A flow of 5 stages. One pool, many identities Shared client DummyCookieJar, pooled connections Login read the cookie from the response Store with the user's own session record Request Cookie header from that store Result 0 of 200 crossed answers Connections are shared; cookies are not.

4. Serialize requests when tokens rotate per response

Some servers issue a new session or CSRF token in every response and expect the next request to present it. Concurrent requests cannot all do that: each was sent with the token current at send time, and by the time it arrives others have replaced it. Measured with a server that rotated a token cookie on every response and counted requests presenting an older one:

rotation_lock = asyncio.Lock()

async def call_rotating(client, path):
    async with rotation_lock:                       # one in flight per identity
        return await client.get(path)

With 20 concurrent requests, 333 of 500 presented a superseded token using httpx, and 263 of 500 using aiohttp; with the requests serialized, 0 of 500. Whether the server rejects superseded tokens or tolerates a few recent ones is its policy, but a client cannot be correct under concurrency unless it serializes per identity. Most APIs avoid this design for exactly this reason; when one requires it, the lock is the price of using it.

Verify: against a rotating-token server, a concurrent test produces no rejected tokens, and the lock is per identity, not global.

Requests sent with a superseded token, of 500 3 horizontal bars comparing httpx, 20 concurrent with the others. Requests sent with a superseded token, of 500 httpx, 20 concurrent 333 aiohttp, 20 concurrent 263 serialized with a lock 0 The server issued a new token in every response. Concurrency and per-response rotation do not mix.

5. Treat the jar as process state

Whatever the jar holds is sent with every matching request for the life of the client. Clear it when the identity it represents ends, and do not let it grow without bound:

async def logout(client: httpx.AsyncClient):
    await client.post("/logout")
    client.cookies.clear()                          # aiohttp: session.cookie_jar.clear()

For a scraper or crawler visiting many sites, a single jar accumulates cookies from every domain it touches; use DummyCookieJar, or a jar per site that is dropped when the site is done, as in building an async web crawler. For tests, assert on the jar's contents directly — client.cookies in httpx, session.cookie_jar in aiohttp — so cookie behaviour is checked rather than inferred from server responses.

Verify: after logout, the jar no longer contains the identity's cookies, and long-running clients do not accumulate cookies from unrelated hosts.

Verification

Cookie handling is correct when:

  • Cookies are kept where expected, including for IP-addressed hosts with aiohttp (unsafe=True).
  • No jar is shared between identities: a client per identity, or a disabled jar with explicit headers.
  • Rotating-token APIs are called one request at a time per identity.
  • Jars are cleared at logout and do not accumulate across unrelated hosts.

Diagnostic Hook: when users of a service occasionally see each other's data from a downstream API, look for a shared HTTP client whose jar stores a session cookie. Concurrent logins through one jar mixed up 100 of 200 answers in this test.

Pitfalls & edge cases

  • aiohttp against IP addresses. Cookies were dropped without unsafe=True.
  • One jar for many users. Measured: 100 of 200 crossed answers.
  • Concurrent calls to a rotating-token API. Measured: 333 of 500 superseded tokens.
  • Deprecated per-request cookies= in httpx. Set the header instead.

Frequently Asked Questions

Why does aiohttp not send cookies back?

Its default CookieJar ignores cookies from hosts given as IP addresses. A cookie from http://127.0.0.1 was dropped; CookieJar(unsafe=True) kept it.

Can I share one httpx client between users with different sessions?

Not with the cookie jar. Concurrent logins through one client crossed 100 of 200 answers. Use a client per identity, or pass Cookie headers explicitly.

How do I disable cookies in aiohttp?

Create the session with cookie_jar=aiohttp.DummyCookieJar(). It stores nothing, so cookies must be sent with explicit Cookie headers.

Why do concurrent requests fail with rotating session tokens?

Each request carries the token current when it was sent, and others replace it in flight: 333 of 500 were superseded with 20 concurrent. Serialize per identity.