Skip to content

Handling 429 and Retry-After in Async Clients

A single 429 Too Many Requests is easy to handle: sleep, retry, move on. Sixty concurrent tasks each receiving a 429 from the same API is a different problem, and the naive per-task retry makes it worse. Each task sleeps independently, wakes at nearly the same moment, and fires another synchronised burst into a quota that has not recovered — so the upstream returns 429 again, your retry budget drains, and the error rate stays pinned while the actual request rate oscillates between zero and far too much.

The fix is to treat the quota as what it is: shared state. When one caller learns the limit has been hit, every caller for that host or account must back off, and they must resume with enough jitter that they do not re-synchronise. This guide implements that: parsing Retry-After in both of its formats, pausing at the right scope, adding jitter, bounding the retry budget, and feeding the signal back into the local limiter so the next burst never happens.

Prerequisites

  • Python 3.11+ for asyncio.timeout() and TaskGroup. The examples use httpx:
pip install httpx
Per-task retry versus a shared pause Three lanes contrasting synchronised retry bursts with a jittered shared-gate recovery. Per-task retry versus a shared pause per-task retry burst all sleep 1s burst again 429 again shared gate burst all callers paused jittered ramp upstream quota recovering within limit seconds after the first 429 Independent backoff re-synchronises; a shared gate plus jitter does not.

1. Parse Retry-After in both formats

RFC 9110 allows Retry-After to be either a delay in seconds or an HTTP date. Servers use both, and a client that understands only the integer form silently falls back to guessing against the ones that send a date.

from email.utils import parsedate_to_datetime
from datetime import datetime, timezone


def parse_retry_after(value: str | None) -> float | None:
    """Return seconds to wait, or None when the header is absent/unusable."""
    if not value:
        return None
    value = value.strip()
    if value.isdigit():                                   # delay-seconds form
        return float(value)
    try:                                                  # HTTP-date form
        when = parsedate_to_datetime(value)
    except (TypeError, ValueError):
        return None
    if when.tzinfo is None:
        when = when.replace(tzinfo=timezone.utc)
    return max((when - datetime.now(timezone.utc)).total_seconds(), 0.0)

Clamp the result before using it: a server that says "retry in 3600 seconds" is telling you to stop, not to hold a request open for an hour. Decide a ceiling (typically your request budget) and fail the call above it rather than pinning a task for the duration. Verify: unit-test both forms plus a malformed value; the malformed case must return None, not raise.

2. Pause the whole quota scope, not one caller

The scope of a 429 is whatever the quota applies to — usually the host, sometimes the API key or tenant. Model it as a shared "paused until" instant that every caller consults before it sends.

import asyncio
import time


class QuotaGate:
    """One gate per quota scope (host, account, tenant)."""

    def __init__(self) -> None:
        self._resume_at = 0.0
        self._event = asyncio.Event()
        self._event.set()                       # open

    async def wait(self) -> None:
        while not self._event.is_set():
            await self._event.wait()

    def pause(self, seconds: float) -> None:
        resume = time.monotonic() + seconds
        if resume <= self._resume_at:
            return                              # an existing, longer pause wins
        self._resume_at = resume
        self._event.clear()
        asyncio.get_running_loop().call_later(seconds, self._reopen)

    def _reopen(self) -> None:
        if time.monotonic() >= self._resume_at - 0.01:
            self._event.set()

asyncio.Event is exactly the right primitive: a pause blocks every waiter, and one set() releases them all. Keeping the longest pause avoids the bug where a later, shorter 429 shortens a pause the server asked to be long. Verify: with ten concurrent callers, trigger one 429 with Retry-After: 2 and confirm all ten stop sending for two seconds — not just the one that received it.

3. Resume with jitter so the herd disperses

Every caller released by the same set() runs in the same loop iteration, which recreates the burst that caused the 429. Spread them.

import random


async def wait_for_quota(gate: QuotaGate, spread: float = 0.5) -> None:
    await gate.wait()
    await asyncio.sleep(random.uniform(0, spread))    # decorrelate the wakeups

The spread should be roughly one concurrency-window's worth of time: with 20 concurrent callers and a 100/s quota, 0.2–0.5 s of jitter is enough to turn a spike into a ramp. This is the same decorrelation principle as jittered backoff, applied to release rather than retry. Verify: log send timestamps after a pause; they should be spread across the jitter window rather than clustered in a single millisecond.

The path of one throttled request A left-to-right pipeline of cooperative 429 handling. The path of one throttled request await the gate shared per host send counts against quota 429 + Retry-After parse both formats pause everyone longest pause wins resume jittered ramp, not spike The response is interpreted once, centrally, and applied to every concurrent caller.

4. Put it together in the client wrapper

One place enforces the gate, sends the request, interprets the response, and updates shared state.

import asyncio
import logging
import random
import time
import httpx

log = logging.getLogger("quota")


class QuotaAwareClient:
    def __init__(self, client: httpx.AsyncClient, max_attempts: int = 4,
                 max_pause: float = 60.0) -> None:
        self._client = client
        self._gates: dict[str, QuotaGate] = {}
        self.max_attempts, self.max_pause = max_attempts, max_pause
        self.stats = {"sent": 0, "throttled": 0, "paused_seconds": 0.0}

    def _gate(self, host: str) -> QuotaGate:
        return self._gates.setdefault(host, QuotaGate())

    async def request(self, method: str, url: str, **kwargs) -> httpx.Response:
        host = httpx.URL(url).host
        gate = self._gate(host)
        for attempt in range(self.max_attempts):
            await wait_for_quota(gate)
            self.stats["sent"] += 1
            response = await self._client.request(method, url, **kwargs)
            if response.status_code != 429:
                return response

            self.stats["throttled"] += 1
            delay = parse_retry_after(response.headers.get("Retry-After"))
            if delay is None:                       # no guidance: back off ourselves
                delay = min(2 ** attempt, 30) * random.uniform(0.5, 1.0)
            if delay > self.max_pause:
                raise httpx.HTTPStatusError(
                    f"{host} asked for {delay:.0f}s, above the {self.max_pause:.0f}s ceiling",
                    request=response.request, response=response)
            log.warning("429 from %s — pausing all callers for %.1fs (attempt %d)",
                        host, delay, attempt + 1)
            self.stats["paused_seconds"] += delay
            gate.pause(delay)                       # everyone waits, not just us
        raise httpx.HTTPStatusError("rate limited beyond retry budget",
                                    request=response.request, response=response)

Only retry idempotent requests automatically. A 429 on a POST that creates a resource is usually safe to retry (the request never reached the handler), but not always — if the API is not idempotent and does not support an idempotency key, surface the error and let the caller decide. Verify: point the client at a stub that returns 429 twice then 200, and confirm exactly three sends with the correct pauses in between.

5. Feed the signal back into your own limiter

A 429 means your configured rate is above the real one. Retrying politely is a bandage; lowering the local limit is the fix.

class AdaptiveRate:
    """Multiplicative decrease on 429, slow additive recovery afterwards."""

    def __init__(self, rate: float, floor: float = 1.0) -> None:
        self.target = self.rate = rate
        self.floor = floor

    def on_throttled(self) -> None:
        self.rate = max(self.floor, self.rate * 0.5)          # halve immediately
        log.warning("reducing local rate to %.1f/s after 429", self.rate)

    def on_success_window(self) -> None:
        if self.rate < self.target:                            # recover gently
            self.rate = min(self.target, self.rate + self.target * 0.05)

Additive-increase/multiplicative-decrease is the same control law TCP uses for the same reason: back off fast enough to stop the damage, recover slowly enough not to re-trigger it. Wire on_throttled() into the 429 branch above and call on_success_window() from a periodic task after each clean minute. Verify: with a stub enforcing 50/s while the client is configured for 100/s, the effective rate should settle just under 50 within a minute and the 429 count should stop growing.

6. Alert on the rate, not the event

A handled 429 is not an error, but a sustained stream of them is a misconfiguration.

# Prometheus-style
# rate(http_client_throttled_total[5m]) / rate(http_client_requests_total[5m]) > 0.01
#   → more than 1% of requests rejected for quota: lower the local limit

Track throttled / sent as a ratio, and total paused seconds as a gauge for how much throughput the quota is costing you. Below roughly 0.1% the cooperative handling is doing its job; above 1% you are pacing yourself with retries instead of a limiter, which wastes a round trip per rejection and, on many APIs, counts against the quota anyway. Verify: the ratio drops to zero after the adaptive limiter settles; if it does not, the quota is shared with another consumer you have not accounted for.

Additive increase, multiplicative decrease Three lanes showing adaptive rate control against a fixed real quota. Additive increase, multiplicative decrease configured rate 100/s 50/s after 429 recovering +5%/min 429 responses spike real quota 50/s — unknown to the client minutes after deploy Halve on rejection, recover slowly: the client converges just under the real quota.

Verification

  • One 429 pauses everyone. Under concurrency, a single rejection stops all callers for the same scope; the send-timestamp log shows a gap, not a trickle.
  • Resumes are spread. After a pause, send timestamps are distributed across the jitter window rather than clustered.
  • The ratio trends to zero. Over a full traffic peak, throttled responses fall below 0.1% of requests as the adaptive rate settles below the real quota.

Pitfalls and edge cases

  • Per-task retry on a shared quota. Each task backing off alone produces a synchronised second burst. Pause at the quota's scope.
  • Ignoring the HTTP-date form of Retry-After. A date-formatted header parsed as an integer yields nothing, so the client falls back to guessing and often retries far too early.
  • Unbounded pause. A server asking for 3600 s should fail the request, not hold a task (and its memory, and its connection) for an hour.
  • Retrying non-idempotent writes blindly. Use an idempotency key where the API offers one; otherwise surface the 429 to the caller.
  • Counting a 429 as success in the limiter. A rejected request usually still consumes quota; count it as a send, not a free retry.
  • No jitter on resume. Every waiter released by one Event.set() runs in the same iteration — the herd re-forms immediately.
  • Per-instance adaptation only. Ten replicas each halving their own rate converge slowly and unevenly. Where it matters, share the limit through Redis or a sidecar rather than letting each instance discover it separately.

Frequently Asked Questions

How should concurrent async tasks share a 429 backoff?

Through shared state scoped to the quota — usually per host or per API key. Model it as an asyncio.Event that every caller awaits before sending: when any task receives a 429, it clears the event for the computed delay so all callers pause together, and a single set() releases them. Add per-caller jitter on resume so they do not re-synchronise into another burst.

What formats can the Retry-After header use?

Two: a non-negative integer number of seconds, or an HTTP-date. Parse both — use email.utils.parsedate_to_datetime for the date form and subtract the current UTC time. Clamp the result to a ceiling, and treat anything above that ceiling as a request to stop rather than a delay to wait out.

Should a 429 be retried automatically?

For idempotent methods, yes, within a bounded attempt budget and honouring Retry-After. For non-idempotent writes, retry only when the API supports an idempotency key; otherwise surface the error, because a 429 does not always guarantee the request had no effect.

How do I stop getting 429s at all?

Lower the local limit until the upstream stops rejecting you, and adapt automatically: halve the configured rate on each 429 and recover it slowly during clean windows. A sustained 429 rate above about 1% of requests means the local limit is set above the real quota — commonly because several replicas were each configured with the full published quota.