Skip to content

Reusing aiohttp ClientSession Across Requests

A service that opens async with aiohttp.ClientSession() as session: inside every request handler looks correct, passes tests, and quietly destroys throughput in production. Each ClientSession owns a TCPConnector — a connection pool, a DNS cache, and a TLS session cache. Constructing one per request means every call pays a fresh TCP three-way handshake and a full TLS negotiation, the DNS cache is empty every time, and connections are never reused. Under load you also leak connectors (the Unclosed connector warning) and flood the kernel with TIME_WAIT sockets until you hit the file-descriptor limit. The fix is one sentence: create one ClientSession for the lifetime of the process and share it across all requests. This guide shows the anti-pattern, the shared-session pattern, lifecycle management, connector tuning, and clean shutdown.

Prerequisites

Session per request versus one shared session The per-request approach builds and tears down a connector for every call so no connection is reused; the shared session keeps one pool of keep-alive connections reused across all requests. Session per request (bug) One shared session request 1 request 2 request 3 new conn + TLS new conn + TLS new conn + TLS no reuse, leaked connectors request 1 request 2 request 3 shared pool keep-alive DNS cache TLS resume connections reused, no leaks

Step 1: See the anti-pattern (session per request)

This handler creates and destroys a session on every call. It is the version that looks idiomatic — it even uses async with correctly — but it is the bug.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# pip install aiohttp
import aiohttp


async def fetch(url: str) -> dict:
    # ANTI-PATTERN: a new ClientSession (and connector, DNS cache, TLS cache)
    # is built and torn down for every single call.
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            resp.raise_for_status()
            return await resp.json()

Verify the problem: run this in a loop against the same host while watching ss -tn | grep <host>. You will see a new connection (and a growing pile of TIME-WAIT sockets) for every call — no reuse. With debug logging on, you may also see Unclosed connector warnings if any call is cancelled before the async with exits.

Step 2: Create one shared session for the process

Build the session once and pass it to every caller. The session is just an object holding the pool — there is nothing per-request about it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import aiohttp

# Created once, at startup, and reused everywhere.
_session: aiohttp.ClientSession | None = None


def get_session() -> aiohttp.ClientSession:
    assert _session is not None, "session not initialised"
    return _session


async def fetch(url: str) -> dict:
    session = get_session()                 # the SAME session every call
    async with session.get(url) as resp:    # reuses a pooled connection
        resp.raise_for_status()
        return await resp.json()

Verify: issue several requests to the same host and watch ss -tn — after the first connection, subsequent requests reuse it (the connection count stays flat, TIME-WAIT does not grow). That is keep-alive working.

Step 3: Manage the session lifecycle (startup / shutdown)

The session must be created and closed around the application's life. In a plain script, wrap everything in one async with. In a long-running service, hook into the framework's lifespan so the session opens at startup and closes at shutdown.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import aiohttp

# Plain script: one session for the whole run.
async def main(urls: list[str]) -> None:
    global _session
    async with aiohttp.ClientSession() as _session:
        results = [await fetch(u) for u in urls]
    # session is closed here; pool drained.


# aiohttp server: tie the session to the app's lifecycle.
from aiohttp import web

async def on_startup(app: web.Application) -> None:
    app["session"] = aiohttp.ClientSession()

async def on_cleanup(app: web.Application) -> None:
    await app["session"].close()            # graceful drain on shutdown

def make_app() -> web.Application:
    app = web.Application()
    app.on_startup.append(on_startup)
    app.on_cleanup.append(on_cleanup)
    return app

Verify: the session is created exactly once per process (log a line in on_startup) and closed exactly once (log in on_cleanup). For an ASGI framework, the equivalent hooks are the lifespan startup/shutdown events.

Step 4: Configure the TCPConnector limits

A shared session still needs a bounded pool. Pass a tuned TCPConnector: limit caps total connections, limit_per_host caps fan-out to any single host, and ttl_dns_cache controls how long resolved addresses are cached. Match these to your concurrency and the OS file-descriptor ceiling — the methodology is in connection pooling and keep-alive.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import aiohttp

connector = aiohttp.TCPConnector(
    limit=100,            # total pooled connections across all hosts
    limit_per_host=20,    # cap per host so one slow host can't starve others
    ttl_dns_cache=300,    # cache DNS for 5 min; avoids a lookup per connect
    keepalive_timeout=30, # keep idle connections 30s; must be < server's idle
)
session = aiohttp.ClientSession(
    connector=connector,
    timeout=aiohttp.ClientTimeout(total=10, connect=2),
)

Verify: under sustained fan-out, the connection count plateaus at limit rather than climbing without bound. If you see Too many open files, your limit exceeds ulimit -n — lower it or raise the OS limit.

Step 5: Close cleanly to avoid "Unclosed" warnings

ClientSession and its connector own kernel sockets. If the process exits without closing the session, aiohttp emits Unclosed client session / Unclosed connector warnings and leaks the sockets. Always close in a finally or lifecycle hook, and give the transports a moment to finish closing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import asyncio
import aiohttp

async def run() -> None:
    session = aiohttp.ClientSession()
    try:
        async with session.get("https://example.com") as resp:
            await resp.read()
    finally:
        await session.close()               # closes connector + sockets
        await asyncio.sleep(0)              # let transports finish closing

Verify: run with python -W error::ResourceWarning (or PYTHONWARNINGS=error) — a clean shutdown produces zero Unclosed warnings. Any warning means a session or response escaped its async with/close().

Verification

After switching to a shared session, the signals to confirm:

  • Connection reuse ratio drops toward zero. New connections opened divided by requests issued should fall from ~1.0 (one per request) to near zero once the pool is warm.
  • No leaked connectors. Running with ResourceWarning promoted to errors produces no Unclosed warnings at exit.
  • TIME-WAIT sockets stop accumulating. ss -tn | grep -c TIME-WAIT against the target host stays flat instead of climbing with request volume.
  • Throughput rises and tail latency falls, because requests skip the TCP+TLS handshake on every call.

Pitfalls & Edge Cases

  • The session is bound to one event loop. A ClientSession created under one loop cannot be used from another. Do not create it at import time (module level) before a loop exists, and do not reuse it across asyncio.run() calls — each run() creates and destroys a loop, orphaning the session.
  • It is not for cross-thread sharing. A session belongs to its loop's thread. If you run multiple event loops (e.g. one per worker thread), give each its own session; never call one session's methods from another thread.
  • Closing too early breaks in-flight requests. Closing the session while requests are still running cancels them. Drain or await outstanding work before close(), ideally as part of graceful shutdown.
  • Per-request timeouts still matter. A shared session has a default timeout, but you can and should override it per call with the timeout= argument or asyncio.timeout() for requests that need a different deadline. See timeouts and deadlines.
  • One global default header set. Headers set on the session apply to every request. Put only truly global headers (User-Agent, auth for a single upstream) on the session; pass request-specific headers per call.

Frequently Asked Questions

Why is creating an aiohttp ClientSession per request a problem?

Each ClientSession owns a connector with a connection pool, DNS cache, and TLS session cache. Creating one per request means every call pays a fresh TCP handshake and TLS negotiation, never reuses a connection, and can leak connectors and flood the kernel with TIME-WAIT sockets until file descriptors are exhausted.

Can I share one aiohttp ClientSession across threads?

No. A ClientSession is bound to the event loop it was created on and belongs to that loop's thread. If you run multiple event loops, give each its own session; never call one session's methods from a different thread.

How do I avoid the Unclosed client session warning?

Close the session explicitly with await session.close() inside a finally block or your framework's shutdown hook, and yield once (await asyncio.sleep(0)) so transports finish closing. Run with python -W error::ResourceWarning to confirm no Unclosed warnings appear at exit.