Skip to content

Propagating Deadlines with contextvars

An API promises a two-second response. The handler calls a service client with a two-second timeout, which calls a cache with a one-second timeout, which falls back to a database query with a three-second timeout. When the cache is slow, the request fails at two seconds for the user while the database query keeps running for another three, holding a connection nobody will read from. Every layer picked a sensible timeout for itself; none of them knew how much time the request had left. A deadline — an absolute point in time by which the whole request must finish — fixes this if every layer can see it. Passing a deadline argument through every function signature works until the first library callback or middleware drops it. A ContextVar carries it implicitly, per request, through tasks and thread hops. This guide builds deadline scopes that can only tighten, enforces them with asyncio.timeout_at() on the loop clock, fails fast when the budget is already spent, and forwards the remaining time across thread and service boundaries.

Prerequisites

Nested scopes can only tighten 4 stacked layers from Request edge to Scope exit. Nested scopes can only tighten Request edge deadline = now + 2.0 s set in a ContextVar Layer asks for 5 s min(outer, proposed) still the 2 s deadline Sub-call asks for 0.5 s min(outer, proposed) tightened for this call Scope exit reset(token) outer deadline restored min() is the contract: inner code may shorten the budget, never extend it.

1. Store an absolute deadline on the loop clock

Store the absolute deadline, not a duration: time spent in earlier layers is then subtracted automatically. Use the loop's clock, loop.time(), so the value works directly with asyncio.timeout_at() and with virtual-time tests.

import asyncio
import contextlib
import contextvars

_deadline: contextvars.ContextVar[float | None] = contextvars.ContextVar("deadline", default=None)


def remaining() -> float | None:
    """Seconds left in the current request, or None when no deadline is set."""
    deadline = _deadline.get()
    if deadline is None:
        return None
    return deadline - asyncio.get_running_loop().time()


@contextlib.contextmanager
def deadline_scope(seconds: float):
    """Set a deadline `seconds` from now — but never later than an existing one."""
    now = asyncio.get_running_loop().time()
    proposed = now + seconds
    current = _deadline.get()
    token = _deadline.set(proposed if current is None else min(current, proposed))
    try:
        yield _deadline.get()
    finally:
        _deadline.reset(token)


async def main() -> None:
    with deadline_scope(2.0):                          # the request budget
        await asyncio.sleep(0.05)
        with deadline_scope(5.0):                      # a layer asks for more: ignored
            print(f"inner remaining: {remaining():.2f}s")    # about 1.95
        with deadline_scope(0.5):                      # a layer asks for less: honoured
            print(f"tighter remaining: {remaining():.2f}s")  # about 0.50
    print("outside:", remaining())                     # None


asyncio.run(main())

min() is the whole contract: an inner scope may shorten the deadline for its sub-operation but can never extend the request's budget. reset(token) restores the outer deadline when the inner scope ends, even on error or cancellation.

Verify: the inner request for five seconds still reports about 1.95 seconds remaining, the tighter scope reports about 0.5, and the deadline is gone outside the scope.

2. Enforce it with asyncio.timeout_at

A deadline only helps if calls actually stop at it. asyncio.timeout_at(when) takes an absolute loop time, so wrapping any awaited call in a helper that reads the context variable enforces the request deadline without the call site knowing the number.

import asyncio
import contextlib


@contextlib.asynccontextmanager
async def within_deadline():
    """Bound the enclosed awaits by the current request deadline, if any."""
    deadline = _deadline.get()
    if deadline is None:
        yield
        return
    async with asyncio.timeout_at(deadline):
        yield


async def query_database(sql: str) -> str:
    async with within_deadline():
        await asyncio.sleep(3.0)                       # a slow query
        return "rows"


async def handler() -> str:
    with deadline_scope(0.2):
        try:
            return await query_database("SELECT ...")
        except TimeoutError:
            return "deadline exceeded: query cancelled at the request deadline"


print(asyncio.run(handler()))

The query is cancelled at 200 ms — the request budget — instead of running for three seconds. Cancelling the await matters beyond latency: a well-behaved database client releases its connection back to the pool when its await is cancelled, which is exactly the resource the introduction's orphaned query was holding.

Verify: the handler returns the deadline message after about 200 ms, not three seconds.

3. Fail fast when the budget is already spent

Starting work that cannot finish in time wastes downstream capacity. Before an expensive call, check the remaining budget against the call's typical latency, and fail immediately — or take a cheaper path — when it cannot fit.

import asyncio


class DeadlineTooShort(TimeoutError):
    pass


def require_budget(minimum: float) -> None:
    left = remaining()
    if left is not None and left < minimum:
        raise DeadlineTooShort(f"{left * 1000:.0f} ms left, call needs about {minimum * 1000:.0f} ms")


async def recommendations(user_id: str) -> list[str]:
    try:
        require_budget(minimum=0.15)                   # p50 latency of the ML service
    except DeadlineTooShort:
        return ["bestsellers"]                         # cheap fallback instead of a doomed call
    async with within_deadline():
        await asyncio.sleep(0.15)
        return ["personalised-1", "personalised-2"]


async def page(budget: float) -> list[str]:
    with deadline_scope(budget):
        await asyncio.sleep(0.1)                       # earlier work used part of the budget
        return await recommendations("u-7")


async def main() -> None:
    print("generous budget:", await page(0.5))
    print("tight budget:   ", await page(0.2))


asyncio.run(main())

With half a second, the personalised call fits; with 200 ms, 100 of which were already used, the fallback is returned immediately and the recommendation service never sees a request that would have timed out anyway. The same check before retries prevents the retry storms described in retry budgets to prevent retry storms.

Verify: the generous budget returns personalised results and the tight budget returns ['bestsellers'] without waiting.

Enough budget left for this call? A decision on How much of the request budget remains with 3 outcomes. Enough budget left for this call? How much of the request budget remains? more than typical latency make the call inside timeout_at less than typical latency skip it return a fallback no deadline set use call defaults set one at the edge A call that cannot finish in time only adds load downstream.

4. Carry the deadline into threads and child tasks

Child tasks copy the context, so they inherit the deadline automatically. Threads started with asyncio.to_thread() inherit it too — but blocking code there cannot use asyncio.timeout_at, and the loop clock is not the thread's clock. Convert to a relative budget at the boundary and pass it to the blocking API's own timeout.

import asyncio
import socket
import time


def blocking_lookup(host: str, budget: float | None) -> str:
    """Blocking client: honours the budget through its own timeout parameter."""
    started = time.monotonic()
    sock_timeout = None if budget is None else max(0.001, budget)
    time.sleep(min(0.05, sock_timeout or 0.05))         # stand-in for socket.create_connection(..., timeout)
    return f"{host} resolved in {time.monotonic() - started:.2f}s (timeout {sock_timeout})"


async def resolve(host: str) -> str:
    budget = remaining()                                 # computed on the loop, in loop time
    return await asyncio.to_thread(blocking_lookup, host, budget)


async def main() -> None:
    with deadline_scope(1.0):
        async with asyncio.TaskGroup() as tg:
            child = tg.create_task(asyncio.sleep(0, result=remaining()))   # inherits the deadline
            lookup = tg.create_task(resolve("db.internal"))
        print(f"child task saw {child.result():.2f}s remaining")
        print(lookup.result())


asyncio.run(main())

Computing remaining() on the loop and passing the number as an argument is deliberate: inside the worker thread there is no running loop to call loop.time() on, and the blocking library needs a plain timeout in seconds anyway. Remember that cancelling the awaiting coroutine does not stop the thread — the library's own timeout is what bounds it.

Verify: the child task reports just under one second remaining, and the thread receives a timeout just under one second.

5. Forward the remaining time to downstream services

Across a network hop, send the remaining duration, not the absolute deadline — clocks on different hosts disagree, while durations survive the trip with only the network delay added. Subtract a small margin for that delay, and on the receiving side open a new deadline scope from the header.

import asyncio

HEADER = "x-request-timeout-ms"
NETWORK_MARGIN = 0.02


def outbound_headers() -> dict[str, str]:
    left = remaining()
    if left is None:
        return {}
    return {HEADER: str(max(0, int((left - NETWORK_MARGIN) * 1000)))}


async def downstream_handler(headers: dict[str, str]) -> str:
    budget = int(headers.get(HEADER, "30000")) / 1000   # server default when absent
    with deadline_scope(budget):
        return f"downstream starts with {remaining():.3f}s"


async def main() -> None:
    with deadline_scope(0.5):
        await asyncio.sleep(0.1)
        headers = outbound_headers()
        print(headers)                                   # about 380 ms
        print(await downstream_handler(headers))


asyncio.run(main())

The downstream service now works within the caller's remaining budget minus the margin, and its own nested scopes can only tighten it further. gRPC does this natively with its grpc-timeout metadata, covered in gRPC deadlines and cancellation; for HTTP, an outbound hook like the one in propagating request IDs with contextvars can add the header automatically.

Verify: the header carries roughly 380 ms, and the downstream handler starts with about 0.38 seconds of budget.

The deadline across boundaries 4 stages from request context to network hop. The deadline across boundaries request context absolute loop time child tasks inherit via copy thread hop relative seconds network hop remaining ms - margin Absolute inside the process, relative across every boundary out of it.

Verification

Deadline propagation works when:

  • One deadline per request: it is set once at the edge as an absolute loop time in a context variable.
  • Scopes only tighten: nested scopes use min() and restore the outer value on exit.
  • Awaits are bounded by it: I/O calls run inside asyncio.timeout_at(deadline) and are cancelled at the request deadline.
  • Doomed work is skipped: expensive calls check the remaining budget first and fall back when it is too short.
  • Boundaries convert correctly: threads receive relative timeouts, and downstream services receive remaining milliseconds minus a margin.

Pitfalls & edge cases

  • Storing durations instead of deadlines. A duration in context is the same for every layer, so time already spent is never subtracted.
  • Wall-clock deadlines. time.time() jumps with clock adjustments; use the loop's monotonic clock inside a process and durations across processes.
  • Background work inheriting a request deadline. A task spawned during a request that should outlive it inherits an already-expired deadline; start it with a clean context, as in avoiding contextvar leaks in background tasks.
  • Catching TimeoutError too broadly. A fallback that catches every TimeoutError also swallows the request deadline itself. Distinguish the request-level timeout from a sub-operation's, or re-raise when remaining() is non-positive.
  • Deadlines shorter than connection setup. A tiny remaining budget can make every call fail at connect. Fail fast on the budget check instead of attempting connections that cannot succeed.

Frequently Asked Questions

How do I propagate a request deadline through async Python code?

Store the absolute deadline in a contextvars.ContextVar at the start of the request, using the event loop's clock. Tasks created during the request inherit it automatically, and any layer can compute the remaining time or wrap awaits in asyncio.timeout_at(deadline) so they are cancelled when the request's budget runs out.

Should a deadline be stored as an absolute time or a duration?

Inside one process, store an absolute time on a monotonic clock, so time spent by earlier layers is subtracted automatically. Across a network boundary, send the remaining duration instead, because different hosts' clocks disagree, and let the receiver convert it back into its own absolute deadline.

What is asyncio.timeout_at?

It is the absolute-time variant of asyncio.timeout, added in Python 3.11. It takes a deadline expressed in the event loop's time, as returned by loop.time(), and raises TimeoutError in the enclosed block if that time is reached, which makes it a natural fit for request deadlines stored as absolute loop times.

How do I apply a request deadline to blocking code in a thread?

Compute the remaining seconds on the event loop before offloading, pass that number to the thread as an argument, and give it to the blocking library's own timeout parameter. Cancelling the awaiting coroutine does not stop a running thread, so the library timeout is what actually bounds the work.