Skip to content

Running Callbacks in a Copied Context

Context variables follow coroutines through tasks automatically, but callbacks and threads are where request context gets lost or, worse, ends up attached to the wrong request. asyncio's rule is consistent once you know it: call_soon, call_later, call_at and add_done_callback capture a copy of the context at the moment the callback is scheduled — verified on 3.14, a callback scheduled while rid was "at-schedule" still saw that value after the variable had been changed to "later". Plain threading.Thread targets see none of the caller's context (they read the default), while asyncio.to_thread copies it in. And every scheduling call accepts context= to override the capture. This guide maps each case and shows when to pass a context explicitly.

Prerequisites

1. Know what each scheduling path captures

import asyncio
import contextvars
import threading

rid = contextvars.ContextVar("rid", default="-")


async def main() -> None:
    loop = asyncio.get_running_loop()
    seen: list[str] = []

    rid.set("at-schedule")
    loop.call_soon(lambda: seen.append(rid.get()))
    rid.set("later")
    await asyncio.sleep(0)
    print(seen)                                         # ['at-schedule']

    rid.set("in-loop")
    t = threading.Thread(target=lambda: seen.append(rid.get()))
    t.start(); t.join()
    print(seen[-1])                                     # '-'  (default: no context)

    print(await asyncio.to_thread(rid.get))             # 'in-loop'


asyncio.run(main())

The full table, all verified on Python 3.14:

Path Context the code sees
create_task(coro) copy taken at task creation
loop.call_soon / call_later / call_at(cb) copy taken at scheduling
fut.add_done_callback(cb) copy taken when the callback is added
asyncio.to_thread(fn) copy of the caller's context
loop.run_in_executor(None, fn) empty context unless you wrap fn
threading.Thread(target=fn) empty context (default build)

Verify: run the snippet and confirm the three outputs.

Which context does the code run in? A grid of 5 rows by 3 columns. Which context does the code run in? path context captured when create_task copy task creation call_soon, call_later copy scheduling add_done_callback copy callback registration asyncio.to_thread copy the call run_in_executor, Thread empty never Everything asyncio schedules gets a copy; raw executors and threads start empty.

2. Wrap run_in_executor and threads with copy_context

asyncio.to_thread is implemented as run_in_executor(None, functools.partial(ctx.run, func, ...)) with ctx = contextvars.copy_context(). Direct run_in_executor calls and threads you start yourself need the same wrapping:

import contextvars
import functools


async def hash_file(path: str) -> str:
    loop = asyncio.get_running_loop()
    ctx = contextvars.copy_context()
    return await loop.run_in_executor(None, functools.partial(ctx.run, _hash_file, path))


def start_worker_thread(fn, *args) -> threading.Thread:
    ctx = contextvars.copy_context()
    t = threading.Thread(target=ctx.run, args=(fn, *args), daemon=True)
    t.start()
    return t

This matters for logging and tracing: a log line emitted from inside _hash_file carries the request id only if the context came along. Process pools are different — contexts cannot be pickled — so pass the values you need as arguments instead, as described in carrying contextvars across threads and executors.

Python 3.14 adds sys.flags.thread_inherit_context, which makes new threads start with a copy of the creating thread's context. It is enabled by default only on free-threaded builds; on the standard 3.14 build used here it was 0, which is why the plain thread above saw the default.

Verify: log from inside the executor function; the request id appears.

3. Choose the context explicitly with context=

Every asyncio scheduling call — call_soon, call_later, call_at, call_soon_threadsafe, create_task, add_done_callback — accepts context=. Use it when the capture-at-scheduling default is wrong:

async def on_message(msg) -> None:
    # The broker client schedules our handler from ITS reader task, whose context
    # has no request id. Build a context for this message explicitly.
    ctx = contextvars.Context()
    ctx.run(rid.set, msg.headers.get("x-request-id", "-"))
    asyncio.create_task(process(msg), context=ctx, name=f"msg:{msg.id}")

contextvars.Context() is an empty context; ctx.run(var.set, value) populates it without touching the current one. This is the right pattern for any code that receives work from a long-lived reader loop — message consumers, websocket dispatchers, protocol callbacks — where the ambient context belongs to the connection, not the message. It complements the request-id patterns in propagating request IDs with contextvars.

Verify: inside process, rid.get() returns the message's header value, and the reader task's own context is unchanged.

Giving each message its own context A sequence of 4 messages between 3 participants. Giving each message its own context reader task new Context handler task receive message, header rid=abc Context() then ctx.run(rid.set, abc) create_task(process(msg), context=ctx) logs carry rid=abc The reader's context belongs to the connection; each message gets one of its own.

4. Run code in a clean context on purpose

The opposite problem: code that should not inherit the current request's context. A background job scheduled from inside a request inherits that request's id, user and tenant, so its logs and metrics are attributed to a request that ended long ago. Start it in a fresh context:

def spawn_detached(coro, *, name: str) -> asyncio.Task:
    """Start work that must not carry the caller's request context."""
    return asyncio.create_task(coro, name=name, context=contextvars.Context())


async def handle_upload(req):
    await store(req.file)
    spawn_detached(rebuild_thumbnails(req.file.id), name=f"thumbs:{req.file.id}")
    return Response(202)

Starting from an empty context also prevents subtle leaks of large objects referenced by context variables — a per-request database session, a request body — that a long-running background task would otherwise keep alive. That failure mode is covered in avoiding ContextVar leaks in background tasks.

Verify: inside the detached task, rid.get() returns the default, and the task does not keep the request's objects alive.

5. Watch context in done-callbacks

Done-callbacks capture the context at add_done_callback time, not at completion time. A metrics callback added in a request handler runs with that request's context even if the future completes during another request:

def record(fut: asyncio.Future) -> None:
    metrics.inc("upstream_calls", tenant=tenant.get())      # tenant at registration time


async def call_upstream(request):
    fut = shared_client.submit(request)
    fut.add_done_callback(record)                           # captures THIS request's tenant
    return await fut

That is usually what you want for per-request attribution. When a callback is registered once on a long-lived object — a connection's close future, a shared pool's events — its captured context is whatever request happened to register it, which is almost never right. Pass context=contextvars.Context() for such registrations so they never carry a stray request's values.

Verify: for every callback registered on a shared, long-lived object, check which context it captures.

Which context should this callback or task get? A decision on Whose work is it with 3 outcomes. Which context should this callback or task get? Whose work is it? the current request's default capture nothing to do an item from a shared reader Context() + set its ids context= per item detached background work empty Context() no stray request data The default is right for request work; shared readers and background jobs need an explicit choice.

Verification

Context propagation through callbacks is correct when:

  • Logs from executors and threads carry the request id, via to_thread or explicit ctx.run wrapping.
  • Per-message work started from shared readers runs in a per-message context.
  • Background jobs run in an empty context and never report a stale request id.
  • Long-lived callbacks are registered with an explicit empty context.

Diagnostic Hook: add the request id to every log record via a logging filter that reads the context variable, and count records whose id is the default value per logger. A logger that suddenly emits many default-id records has lost context at some boundary — a new executor call or a thread — and that logger's module is where to look.

Pitfalls & edge cases

  • run_in_executor without wrapping. The function runs in an empty context; logs lose their ids.
  • Process pools. Contexts do not pickle; pass values explicitly.
  • Callbacks registered on shared objects. They carry whichever request registered them.
  • Building contexts with copy_context() inside a reader. That copies the reader's state; start from Context() when per-item isolation is the goal.

Frequently Asked Questions

What context does loop.call_soon run a callback in?

A copy of the context current at the moment call_soon was called. Later changes to context variables in the scheduling code do not affect it. Pass context= to choose a different one.

Do threads inherit contextvars?

Plain threading.Thread targets start with an empty context on standard builds, so they see defaults. asyncio.to_thread copies the caller's context into the thread. Python 3.14 adds a thread_inherit_context flag, enabled by default only on free-threaded builds.

Does run_in_executor propagate contextvars?

No. Wrap the function with contextvars.copy_context().run, as asyncio.to_thread does internally, or use to_thread for thread pools.

How do I run a background task without the current request's context?

Pass context=contextvars.Context() to create_task. The task starts with an empty context, so it does not report or keep alive the request's values.

What is the difference between copy_context() and Context()?

contextvars.copy_context() returns a copy of the current context, with every variable the caller can see. contextvars.Context() returns an empty context in which every variable reads its default. Use a copy to carry request state into a thread or callback, and an empty context for work that must not inherit it.

Does call_soon_threadsafe capture the calling thread's context?

Yes: like call_soon, it copies the context current in the thread that calls it. A worker thread usually has an empty context, so pass context= explicitly if the callback needs the request's values.