Skip to content

Context Variables & Request Context in Asyncio

Every production service has data that belongs to the current request rather than to any function: the request ID that ties log lines together, the authenticated user and tenant, the remaining time budget, the tracing span. In a threaded server that data lived in threading.local(), because one thread handled one request at a time. An event loop breaks that model completely. A single thread interleaves thousands of requests, switching between them at every await, so a thread-local — or a module-level global — holds whichever request happened to write it last. The symptom is quiet and expensive: log lines stamped with the wrong request ID, audit records attributed to another tenant, and a trace that stitches two unrelated requests together.

The contextvars module is the language-level fix, and asyncio is built on it. Each task runs inside its own Context, a mapping of context variables to values that is copied when the task is created and switched in and out by the event loop whenever the task runs. This section covers how that mechanism works on the loop, where propagation happens automatically and where it silently stops — notably at thread pools — and the patterns that keep request data correct through task groups, background work, executors and libraries that were written before contextvars existed. The parent section, Asyncio Fundamentals & Event Loop Architecture, covers the scheduler this mechanism is woven into.

Scope of this section:

  • The copy-on-create model: what a task inherits, and why changes never flow back to the parent.
  • Which asyncio and concurrency APIs propagate context, and which drop it.
  • Request-scoped patterns for IDs, principals, deadlines and tracing without passing arguments through every layer.
  • Context in threads, executors and callbacks, including Python 3.14's thread context options.
  • Diagnosing context bugs: leaked values, mutable state shared by accident, and missing IDs in logs.

Architectural principles

  • Context is copied, never shared. A task snapshots the creating context at create_task() time. Setting a variable inside the task changes only the task's copy; the parent and sibling tasks never see it. Design data flow to go down the call tree, and return results explicitly.
  • Store immutable values. The copy is shallow. A dict or list stored in a context variable is the same object in every copied context, so mutating it leaks across tasks. Store frozen dataclasses, tuples and strings, and replace rather than mutate.
  • Set and reset in the same scope. ContextVar.set() returns a token; reset(token) restores the previous value. Pair them with try/finally or a context manager so a value never outlives the request that set it, even on error or cancellation.
  • Context is for cross-cutting data, not business inputs. Request IDs, deadlines, tenants and trace spans are legitimate. An order ID or a payment amount belongs in function arguments, where the dependency is visible and testable.
  • Every boundary that leaves the task must be checked. Tasks, asyncio.to_thread() and loop callbacks carry context. run_in_executor(), raw threads, process pools and many third-party thread pools do not. Assume a boundary drops context until you have verified otherwise.
Context flows down, never back up 4 stacked layers from Request scope to Back in the parent. Context flows down, never back up Request scope request_id=req-aaa tenant=acme deadline set TaskGroup child copied at create_task reads req-aaa sets are local asyncio.to_thread copied into thread reads req-aaa logs stamped Back in the parent child sets invisible return values instead reset at scope end Each arrow is a copy: cheap, isolated, and one-directional.

Execution model: how the loop switches contexts

A contextvars.Context is an immutable mapping with copy-on-write semantics, so copying one is cheap — constant time regardless of how many variables it holds. When you call asyncio.create_task(coro), the task calls contextvars.copy_context() and stores the copy. From then on, every time the loop runs a step of that task, it does so through context.run(task_step), which makes the task's context current for the duration of the step and restores the previous one afterwards. Reading request_id.get() inside the coroutine therefore always sees the task's own values, no matter which other tasks ran in between.

Callbacks follow the same rule. loop.call_soon(), call_later() and call_at() capture the current context when the callback is scheduled and run the callback inside it, which is why a done-callback added in a request handler still sees that request's ID. Since Python 3.11, create_task() also accepts an explicit context= argument, letting you start a task in a context you prepared — useful for background work that must not inherit request data, or for work that must run with a specific context captured earlier.

What the loop does not do is merge anything back. When a child task finishes, its context is simply discarded. That is the property that makes context variables safe on an event loop, and the property that surprises people who expect a child to "set the user" for its parent. The same copy semantics apply in task scheduling and lifecycle: a task is an independent unit, and so is its context.

One thread, two requests, two contexts 3 lanes over time. One thread, two requests, two contexts loop thread req-aaa req-bbb req-aaa req-bbb req-aaa req-bbb threading.local aaa bbb bbb! bbb bbb! bbb ContextVar aaa bbb aaa bbb aaa bbb await points → A thread-local keeps the last writer; a context variable follows the task.

Pattern catalogue

Request-scoped variables with a scope manager

The foundation: declare variables at module level, set them once at the edge of the request, and restore them when the request ends.

import contextvars
import uuid
from contextlib import contextmanager

request_id: contextvars.ContextVar[str] = contextvars.ContextVar("request_id", default="-")


@contextmanager
def request_scope(incoming_id: str | None = None):
    token = request_id.set(incoming_id or uuid.uuid4().hex[:12])
    try:
        yield request_id.get()
    finally:
        request_id.reset(token)


async def handle(raw_request: dict) -> dict:
    with request_scope(raw_request.get("x-request-id")) as rid:
        result = await process(raw_request)          # everything below reads request_id.get()
        return {"request_id": rid, "result": result}

Use this in middleware or at the top of a message handler. The trade-off is implicitness: functions that read request_id.get() depend on something not visible in their signature, so keep the number of variables small and their names explicit. The complete version, including propagation to downstream HTTP calls, is in propagating request IDs with contextvars.

Logging enrichment with a filter

The highest-value use of request context is logs. A logging.Filter reads the variables when each record is created, so every log call anywhere in the request carries the ID without passing a logger adapter through the stack.

import logging


class ContextFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        record.request_id = request_id.get()
        return True


handler = logging.StreamHandler()
handler.addFilter(ContextFilter())
handler.setFormatter(logging.Formatter("%(asctime)s %(request_id)s %(levelname)s %(message)s"))
logging.getLogger().addHandler(handler)

Attach the filter to the handler, not only to a logger, so records from third-party libraries' loggers are enriched too. The trade-off: the filter runs for every record, so keep it to cheap get() calls — no formatting or lookups.

Deadlines that follow the request

A remaining-time budget is naturally request-scoped, and storing the absolute deadline in a context variable lets any layer derive its own timeout without every function accepting a timeout argument.

import asyncio
import contextvars
import time

deadline: contextvars.ContextVar[float | None] = contextvars.ContextVar("deadline", default=None)


def remaining(floor: float = 0.0) -> float:
    d = deadline.get()
    return float("inf") if d is None else max(floor, d - time.monotonic())


async def call_inventory(sku: str) -> int:
    async with asyncio.timeout(remaining()):         # never outlives the request budget
        await asyncio.sleep(0.01)
        return 7

Store the absolute monotonic deadline rather than a duration, so time spent in earlier layers is automatically subtracted. When the budget must cross a service boundary, serialise the remaining time into a header as described in propagating deadlines across async service calls.

Immutable principals and trace state

Authenticated identity and tracing metadata tend to be structured. Store them as frozen objects, and derive new ones instead of mutating.

import contextvars
from dataclasses import dataclass, replace


@dataclass(frozen=True)
class Principal:
    user_id: str
    tenant: str
    scopes: frozenset[str]


principal: contextvars.ContextVar[Principal | None] = contextvars.ContextVar("principal", default=None)


def require_scope(scope: str) -> Principal:
    p = principal.get()
    if p is None or scope not in p.scopes:
        raise PermissionError(scope)
    return p


def impersonate(tenant: str) -> contextvars.Token:
    current = principal.get()
    return principal.set(replace(current, tenant=tenant))   # new object, old one untouched

The trade-off of immutability is a small allocation per change, which is negligible next to the class of bug it removes: a mutable dict of "request attributes" stored once and edited by concurrent child tasks.

Crossing into threads

asyncio.to_thread() copies the current context into the worker thread; loop.run_in_executor() does not. When you must use an executor directly — a process-wide pool, a custom thread pool — capture the context and run the function inside it.

import asyncio
import contextvars
from functools import partial


async def run_in_pool_with_context(executor, fn, *args):
    loop = asyncio.get_running_loop()
    ctx = contextvars.copy_context()
    return await loop.run_in_executor(executor, partial(ctx.run, fn, *args))

A copied context should be run by one thread at a time: Context.run() raises RuntimeError if the same context object is entered concurrently. Copy per call, as above, rather than sharing one captured context between jobs. The details, including Python 3.14's threading.Thread(context=...), are in carrying contextvars across threads and executors.

Which boundaries carry context A grid of 6 rows by 2 columns. Which boundaries carry context API copies context? what to do asyncio.create_task / TaskGroup yes, at creation nothing loop.call_soon / call_later yes, at scheduling nothing asyncio.to_thread yes nothing loop.run_in_executor no ctx.run via partial threading.Thread no by default context= on 3.14 ProcessPoolExecutor never pass values as args Treat every unlisted library thread pool as a "no" until a test proves otherwise.

Resource boundaries

Context variables are cheap, but they are not free, and their lifetime is tied to objects you do not always see.

Concern What holds it How to bound it
Memory per task One copied Context per task, sharing structure with its parent Negligible unless values are large; never store request bodies or result sets
Value lifetime Every context that captured the value, including pending callbacks and long-lived tasks Reset at the end of the scope; start background tasks with context=contextvars.Context()
Leaks into background work Tasks created inside a request inherit its values for their entire life Create long-lived tasks at startup, or with an empty context
Variable count Module-level ContextVar objects, created once Declare at import time; never create ContextVar objects per request
Thread pools Contexts entered by worker threads for the duration of a job Copy per job; never share one context object across concurrent jobs
Cross-process work Nothing — process pools start with a fresh interpreter state Pass request ID and deadline explicitly as arguments

The subtle one is the background-task row. A cache-refresh task started lazily by the first request that needed it runs forever with that request's ID, tenant and deadline in its context. Its logs are misattributed, and a deadline read from context has long expired. Start such tasks at application startup or give them a fresh context explicitly.

Integrated production example

The service below handles two requests concurrently. Each gets its own request ID, principal and deadline; log lines are enriched through a filter; a TaskGroup fans out to an async call and a blocking PDF renderer in a thread; and all request data is restored when the scope ends.

import asyncio
import contextvars
import logging
import time
import uuid
from contextlib import contextmanager
from dataclasses import dataclass

request_id: contextvars.ContextVar[str] = contextvars.ContextVar("request_id", default="-")
deadline: contextvars.ContextVar[float | None] = contextvars.ContextVar("deadline", default=None)


@dataclass(frozen=True)                           # immutable: safe to share across tasks
class Principal:
    user_id: str
    tenant: str


principal: contextvars.ContextVar[Principal | None] = contextvars.ContextVar("principal", default=None)


class ContextFilter(logging.Filter):
    """Stamp every record with the current request's context — no logger plumbing."""

    def filter(self, record: logging.LogRecord) -> bool:
        record.request_id = request_id.get()
        p = principal.get()
        record.tenant = p.tenant if p else "-"
        return True


handler = logging.StreamHandler()
handler.addFilter(ContextFilter())
handler.setFormatter(logging.Formatter("%(asctime)s %(request_id)s %(tenant)s %(message)s"))
log = logging.getLogger("svc")
log.addHandler(handler)
log.setLevel(logging.INFO)


@contextmanager
def request_scope(user: Principal, budget_s: float, incoming_id: str | None = None):
    tokens = [
        request_id.set(incoming_id or uuid.uuid4().hex[:12]),
        principal.set(user),
        deadline.set(time.monotonic() + budget_s),
    ]
    try:
        yield
    finally:
        for var, token in zip((request_id, principal, deadline), tokens):
            var.reset(token)                     # restore, even on error or cancellation


def remaining() -> float:
    d = deadline.get()
    return float("inf") if d is None else max(0.0, d - time.monotonic())


def render_pdf(order_id: str) -> str:            # blocking library call
    log.info("rendering %s in a worker thread", order_id)
    time.sleep(0.05)
    return f"{order_id}.pdf"


async def fetch_line_items(order_id: str) -> list[str]:
    async with asyncio.timeout(remaining()):     # budget follows the request implicitly
        await asyncio.sleep(0.02)
        log.info("fetched line items for %s", order_id)
        return ["sku-1", "sku-2"]


async def handle(order_id: str, user: Principal, incoming_id: str | None = None) -> str:
    with request_scope(user, budget_s=0.5, incoming_id=incoming_id):
        log.info("start order %s", order_id)
        async with asyncio.TaskGroup() as tg:    # children copy the context at creation
            items = tg.create_task(fetch_line_items(order_id))
            pdf = tg.create_task(asyncio.to_thread(render_pdf, order_id))  # to_thread copies it too
        log.info("done: %d items, %s", len(items.result()), pdf.result())
        return pdf.result()


async def main() -> None:
    await asyncio.gather(
        handle("o-1", Principal("u-1", "acme"), incoming_id="req-aaa"),
        handle("o-2", Principal("u-2", "globex"), incoming_id="req-bbb"),
    )
    log.info("outside any request")


asyncio.run(main())

Running it prints interleaved lines from both requests — req-aaa acme and req-bbb globex — each correctly stamped, including the lines logged from inside the worker threads, followed by - - outside any request once both scopes have reset. Three details carry the design: tokens are reset in a finally so cancellation cannot leave values behind; the thread hop uses asyncio.to_thread(), which copies the context; and the principal is a frozen dataclass, so a child task cannot alter what its siblings see.

Diagnostic Hook — are log lines carrying the right request?

Export two counters from the logging filter: records without a request ID (the variable returned its default) broken down by logger name, and records whose request ID is not active (compare against a set of in-flight IDs maintained by the middleware). The first spikes when a boundary drops context — usually a run_in_executor() or a library thread pool — and the logger name points at the culprit. The second, which should always be zero, reveals values leaking into background tasks or a missing reset(). Alert on any non-zero rate of the second counter, and on the first exceeding a small baseline after a deploy.

Reading the two context health counters 3 bars comparing records without ID: small baseline with the others. Reading the two context health counters records without ID: small baseline startup, background records without ID: rising after deploy a boundary dropped context record with inactive request ID leak or missing reset Bar length encodes urgency, not a measured quantity. Missing IDs point at a boundary; inactive IDs point at a leak.

Failure modes

Failure mode Root cause Detection Fix
Log lines carry another request's ID Request data stored in a module global or threading.local() Two request IDs interleaved within one trace Move the value into a ContextVar set per request
Request ID missing in thread-pool logs run_in_executor() or a library pool does not copy context Default value appears only in logs from worker threads Use asyncio.to_thread() or ctx.run via copy_context()
Parent never sees a value set by a child Tasks copy context; changes do not flow back Value is correct inside the child and stale after it Return the value from the child explicitly
State leaks between concurrent requests Mutable object stored in a context variable and mutated Request-specific keys appear in unrelated requests Store immutable values; replace instead of mutate
Background task logs a stale request forever Long-lived task created inside a request inherits its context One request ID appears for hours after the request ended Create background tasks at startup or with context=contextvars.Context()
RuntimeError: cannot enter context One captured Context run by two threads at once Exception from Context.run under concurrent jobs Copy the context per job, not once per request
Deadline read from context already expired Absolute deadline inherited by work that outlives the request Immediate TimeoutError in deferred jobs Clear or reset the deadline for deferred work

Frequently Asked Questions

Why can't I use threading.local for request data in asyncio?

An event loop runs many requests on one thread, switching between them at every await. A thread-local holds a single value per thread, so whichever request wrote it last wins, and other requests read the wrong data. ContextVar values are tracked per task instead, because the loop runs each task step inside that task's own context.

Do asyncio tasks inherit context variables from the code that creates them?

Yes. create_task copies the current context at the moment the task is created, and every step of the task runs inside that copy. Values set later in the parent are not visible to the task, and values the task sets are not visible to the parent or its siblings. Since Python 3.11 you can pass context= to use a different context.

Why is my ContextVar empty inside run_in_executor?

loop.run_in_executor submits the function to the executor without copying the current context, so the worker thread runs with its own default context. asyncio.to_thread does copy it. For a custom executor, capture contextvars.copy_context() and submit functools.partial(ctx.run, fn, *args), copying once per call.

Can a child task set a context variable for its parent?

No. Context changes made in a task stay in that task's copy and are discarded when it finishes. This isolation is intentional and is what makes context variables safe under concurrency. If the parent needs a value computed by a child, return it from the child's coroutine or pass an explicit result object.

Is it safe to store a dict in a ContextVar?

It is safe only if nobody mutates it. Copying a context is shallow, so every task that inherited the variable points at the same dict, and a mutation in one task is visible in all of them. Store immutable values such as frozen dataclasses or tuples, and set a new value rather than editing the existing one.