Skip to content

Caching DNS Lookups in Async HTTP Clients

Every new connection starts with a name lookup, and in asyncio that lookup is getaddrinfo — a blocking C call that the loop runs in a thread pool. On a warm pool it is invisible, because connections are reused. It becomes visible exactly when things are already going badly: a burst of new connections after a pool was recycled, a deploy that restarts every worker, a retry storm. Then each connect waits for the resolver, the default executor fills with DNS threads, and unrelated to_thread work queues behind them. The fix is a small cache with the properties DNS itself has — a TTL, negative entries, and one lookup per name even when fifty callers ask at once. The risk is caching too long: a failover that moves a service to a new address is invisible to a client holding a stale answer. This guide builds the cache, measures it, and sets the TTL rules that keep failover working.

Prerequisites

Where the lookup happens on a connect 5 stages from client connect to request. Where the lookup happens on a connect client connect needs an address loop.getaddrinfo blocking call default executor a thread per lookup TCP + TLS the visible cost request finally sent The lookup shares the executor with every other offloaded blocking call.

1. Measure what a lookup costs your connects

Resolution cost varies by orders of magnitude: an /etc/hosts entry or a local caching daemon answers in microseconds, while a remote resolver over a congested network takes tens of milliseconds. Measure both the real cost and the shape of the problem.

import asyncio
import socket
import statistics
import time

lookups = {"n": 0}


async def slow_resolver(host: str, port: int, **kwargs):
    """Stands in for a remote resolver round trip."""
    lookups["n"] += 1
    await asyncio.sleep(0.02)
    return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("10.0.0.1", port))]


async def measure_real() -> None:
    loop = asyncio.get_running_loop()
    times = []
    for _ in range(200):
        started = time.perf_counter()
        await loop.getaddrinfo("localhost", 80, type=socket.SOCK_STREAM)
        times.append((time.perf_counter() - started) * 1000)
    print(f"local getaddrinfo: median {statistics.median(times):.3f} ms, "
          f"max {max(times):.2f} ms")


async def measure_remote() -> None:
    lookups["n"] = 0
    started = time.perf_counter()
    for _ in range(20):
        await slow_resolver("api.example.com", 443)
    per_connect = (time.perf_counter() - started) / 20 * 1000
    print(f"remote resolver: {per_connect:.1f} ms per connect, {lookups['n']} lookups")


asyncio.run(measure_real())
asyncio.run(measure_remote())

Locally, getaddrinfo measured a median of 0.027 ms with a worst case of 1.94 ms — the worst case being the thread hop, not the lookup. Against a resolver 20 ms away, every connect pays 20 ms. The rule that follows: cache when your resolver is remote or your connection churn is high, and leave it alone when a local caching daemon already answers in microseconds.

Verify: your local median is well under a millisecond, and the maximum shows the executor hop cost.

2. Cache with a TTL and coalesce concurrent lookups

The cache stores the resolver's answer with an expiry, and a per-key in-flight task so that fifty simultaneous callers for the same name trigger one lookup rather than fifty.

import asyncio
import socket
import time


class CachingResolver:
    def __init__(self, ttl: float = 30.0, negative_ttl: float = 1.0) -> None:
        self.ttl, self.negative_ttl = ttl, negative_ttl
        self.cache: dict[tuple, tuple[float, object]] = {}
        self._inflight: dict[tuple, asyncio.Task] = {}
        self.hits = self.misses = self.lookups = 0

    async def resolve(self, host: str, port: int, family: int = socket.AF_UNSPEC):
        key = (host, port, family)
        entry = self.cache.get(key)
        if entry is not None and entry[0] > time.monotonic():
            self.hits += 1
            if isinstance(entry[1], Exception):
                raise entry[1]                              # negative cache hit
            return entry[1]
        self.misses += 1
        task = self._inflight.get(key)
        if task is None:                                    # no await before the insert
            task = self._inflight[key] = asyncio.create_task(self._lookup(key))
            task.add_done_callback(lambda _t, k=key: self._inflight.pop(k, None))
        return await asyncio.shield(task)                   # one caller's cancel ≠ everyone's

    async def _lookup(self, key: tuple):
        host, port, family = key
        self.lookups += 1
        loop = asyncio.get_running_loop()
        try:
            infos = await loop.getaddrinfo(host, port, family=family, type=socket.SOCK_STREAM)
        except socket.gaierror as exc:
            self.cache[key] = (time.monotonic() + self.negative_ttl, exc)
            raise
        self.cache[key] = (time.monotonic() + self.ttl, infos)
        return infos


async def main() -> None:
    resolver = CachingResolver(ttl=30.0)
    await asyncio.gather(*(resolver.resolve("localhost", 80) for _ in range(50)))
    print(f"50 concurrent callers -> {resolver.lookups} actual lookup(s)")
    await resolver.resolve("localhost", 80)
    print(f"hits={resolver.hits} misses={resolver.misses}")


asyncio.run(main())

Coalescing is the same single-flight pattern applied to name resolution, and it matters most in exactly the situation that makes DNS visible: a burst of connects to the same host after a pool reset.

Verify: fifty concurrent callers trigger one lookup, and a later call is served from the cache.

3. Cache failures briefly, successes for the record's TTL

A resolver failure that is not cached at all turns one outage into a lookup storm; one that is cached for minutes keeps a service down after DNS recovers. Negative entries need a short, separate TTL.

import asyncio
import socket
import time


async def main() -> None:
    resolver = CachingResolver(ttl=30.0, negative_ttl=1.0)

    try:
        await resolver.resolve("no-such-host.invalid", 80)
    except socket.gaierror as exc:
        print("first failure:", type(exc).__name__)

    started = time.perf_counter()
    try:
        await resolver.resolve("no-such-host.invalid", 80)
    except socket.gaierror:
        print(f"second failure served from cache in {(time.perf_counter() - started) * 1e3:.3f} ms")

    await asyncio.sleep(1.1)                                # negative TTL expires
    try:
        await resolver.resolve("no-such-host.invalid", 80)
    except socket.gaierror:
        print("after the negative TTL, the resolver was asked again:", resolver.lookups, "lookups")


asyncio.run(main())

One second is a reasonable negative TTL: long enough to absorb a burst, short enough that a name which starts resolving is picked up almost immediately. For successful answers, the honest TTL is the one in the DNS record — which getaddrinfo does not expose. That is the argument for aiodns, which surfaces record TTLs, or for a conservative fixed TTL that is shorter than the shortest record TTL you depend on.

Verify: the second failure is served from the cache in microseconds, and after the negative TTL expires the resolver is consulted again.

What each TTL controls A grid of 4 rows by 2 columns. What each TTL controls setting too short too long positive TTL more lookups failover delayed negative TTL lookup storm on outage outage prolonged refresh-ahead point extra background lookups requests wait at expiry pooled connection lifetime more handshakes old address kept in use The last row matters most: an open connection never consults DNS again.

4. Refresh ahead of expiry so no request pays the lookup

With a plain TTL, the unlucky request that arrives just after expiry waits for the resolver. Refreshing in the background shortly before expiry keeps the cache warm and moves the cost off the request path.

import asyncio
import socket
import time


class RefreshingResolver(CachingResolver):
    def __init__(self, ttl: float = 30.0, refresh_at: float = 0.8, **kwargs) -> None:
        super().__init__(ttl=ttl, **kwargs)
        self.refresh_at = refresh_at                        # fraction of the TTL
        self._refreshers: set[asyncio.Task] = set()

    async def resolve(self, host: str, port: int, family: int = socket.AF_UNSPEC):
        key = (host, port, family)
        entry = self.cache.get(key)
        if entry is not None and entry[0] > time.monotonic():
            age_remaining = entry[0] - time.monotonic()
            if age_remaining < self.ttl * (1 - self.refresh_at) and key not in self._inflight:
                task = asyncio.create_task(self._lookup(key))    # refresh ahead, off the path
                self._refreshers.add(task)
                task.add_done_callback(self._refreshers.discard)
                task.add_done_callback(lambda t: t.exception())   # never-retrieved guard
        return await super().resolve(host, port, family)

    async def aclose(self) -> None:
        for task in list(self._refreshers):
            task.cancel()
        await asyncio.gather(*self._refreshers, return_exceptions=True)


async def main() -> None:
    resolver = RefreshingResolver(ttl=0.2, refresh_at=0.5)
    await resolver.resolve("localhost", 80)
    await asyncio.sleep(0.15)                               # past the refresh point
    await resolver.resolve("localhost", 80)                 # served from cache, refresh started
    await asyncio.sleep(0.05)
    print("lookups after refresh-ahead:", resolver.lookups, "| hits:", resolver.hits)
    await resolver.aclose()


asyncio.run(main())

Refresh-ahead trades a few extra background lookups for a cache that never makes a request wait. Keep the refresh task's exception retrieved — a failing background refresh should log, not raise at garbage collection — and cancel the refreshers during shutdown so they do not outlive the client.

Verify: the second call is a cache hit while the lookup counter increases in the background.

5. Keep failover working

Caching addresses is caching a routing decision. The TTL is the upper bound on how long the client keeps sending traffic to an address that has been moved away — and connection reuse extends it further, because an established connection ignores DNS entirely.

import asyncio
import time


class FailoverAwareResolver(RefreshingResolver):
    """Drops cached entries for a host after connection failures, so failover is picked up."""

    def __init__(self, *args, failures_before_flush: int = 2, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.failures: dict[str, int] = {}
        self.failures_before_flush = failures_before_flush
        self.flushes = 0

    def note_connect_failure(self, host: str) -> None:
        count = self.failures.get(host, 0) + 1
        self.failures[host] = count
        if count >= self.failures_before_flush:
            for key in [k for k in self.cache if k[0] == host]:
                del self.cache[key]                         # next resolve asks the resolver
            self.failures[host] = 0
            self.flushes += 1

    def note_connect_success(self, host: str) -> None:
        self.failures.pop(host, None)


async def main() -> None:
    resolver = FailoverAwareResolver(ttl=300.0)
    await resolver.resolve("localhost", 80)
    resolver.note_connect_failure("localhost")
    resolver.note_connect_failure("localhost")              # two failures: flush the entry
    print("cache flushed:", resolver.flushes, "| entries left:", len(resolver.cache))
    await resolver.resolve("localhost", 80)
    print("re-resolved, lookups:", resolver.lookups)
    await resolver.aclose()


asyncio.run(main())

Three rules keep failover honest: a TTL no longer than the shortest record TTL you rely on (30–60 seconds for a service behind a failover-capable name), flushing a host's entry after repeated connection failures, and bounding connection lifetime in the pool so long-lived connections eventually re-resolve. Without the last one, a pool of hour-old connections keeps using the old address regardless of what the cache says.

Verify: two connection failures flush the host's entries, and the next resolve consults the resolver even though the TTL had not expired.

Should this client cache DNS? A decision on What does resolution cost here with 3 outcomes. Should this client cache DNS? What does resolution cost here? local daemon, microseconds no client cache pool reuse is enough remote resolver, milliseconds cache with a short TTL plus coalescing library has one built in configure it aiohttp ttl_dns_cache Measure first: caching a cheap lookup only adds a failover risk.

Verification

DNS caching is set up correctly when:

  • The cost is known: resolution latency is measured for the deployment's actual resolver, and caching is applied only where it pays.
  • Bursts cause one lookup: concurrent callers for the same name coalesce.
  • Failures are cached briefly: a negative TTL of about a second absorbs storms without prolonging outages.
  • Requests do not wait for refreshes: entries are refreshed ahead of expiry, and refresh failures are logged, not raised.
  • Failover still works: TTLs are short, entries are flushed after repeated connect failures, and pooled connections have a bounded lifetime.

Pitfalls & edge cases

  • Caching forever "because DNS is slow". A long TTL breaks failover; if DNS is genuinely slow, run a local caching daemon instead.
  • Ignoring record TTLs. getaddrinfo does not expose them; aiodns does. A fixed TTL must be shorter than the records you depend on.
  • Caching only the first address. Names resolve to several addresses for a reason; keep the whole list and let the connector try them in order, preserving Happy Eyeballs behaviour for IPv6 and IPv4.
  • Per-client caches. A cache inside each client object multiplies lookups by the number of clients; share one resolver per process.
  • Forgetting that aiohttp already does this. aiohttp.TCPConnector has a built-in TTL cache (ttl_dns_cache); check your client's features before writing your own.

Frequently Asked Questions

Does asyncio cache DNS lookups?

No. Each connection resolves the name through loop.getaddrinfo, which runs the blocking system resolver in the default thread pool. Any caching comes from the operating system, a local caching daemon, or the client library — aiohttp's TCPConnector caches with a TTL, while httpx does not cache by default.

How long should an async client cache DNS answers?

No longer than the shortest TTL of the records it depends on, typically 30 to 60 seconds for names used for failover, and cache failures for about a second. Longer TTLs delay failover, and combined with long-lived pooled connections they can keep traffic going to a withdrawn address for a long time.

How do I avoid a DNS lookup storm when connections are re-established?

Coalesce concurrent lookups for the same name into a single in-flight resolution and share its result, and cache negative answers briefly so a failing name is not queried by every caller. Refreshing entries shortly before they expire also keeps bursts off the request path.

Why do DNS lookups affect unrelated asyncio work?

Because getaddrinfo is blocking and runs in the loop's default thread pool, which is shared with asyncio.to_thread and other offloaded work. A burst of slow lookups occupies those threads, so unrelated blocking calls queue behind them. Caching, or a pure-async resolver such as aiodns, removes that contention.