Avoiding contextvar Leaks in Background Tasks

A cache-refresh loop has logged every line under request ID req-8c1f for three days. That request — the first one to need the cache after a deploy, which started the refresher lazily — finished in 40 milliseconds on Monday. Since then the refresher has also been running with that request's tenant, its authenticated principal, and a deadline that expired on Monday afternoon, so every refresh that checks the deadline times out immediately. And the 10 MB upload body the request stored in context for an audit hook is still in memory. Nothing is wrong with context variables here; they did exactly what they promise. A task copies the context of the code that creates it, for its whole life. Request-scoped tasks should inherit; work that outlives the request must not. This guide reproduces each symptom, then detaches background work with a clean context, carries over only the values it genuinely needs, and adds a guard that catches accidental inheritance in tests.

Prerequisites

• Python 3.11+ for create_task(..., context=...); standard library only.
• The context model from Context Variables & Request Context.
• Request IDs and deadlines from propagating request IDs with contextvars and propagating deadlines with contextvars.

1. Reproduce the stale-context background task

A lazily started background task is the typical source. The first request that needs the service starts the task, and the task copies that request's context.

import asyncio
import contextvars

request_id = contextvars.ContextVar("request_id", default="-")
deadline = contextvars.ContextVar("deadline", default=None)
seen: list[str] = []


class PriceCache:
    def __init__(self) -> None:
        self._refresher: asyncio.Task | None = None

    def ensure_started(self) -> None:
        if self._refresher is None:                    # lazy start inside a request: the bug
            self._refresher = asyncio.create_task(self._refresh_forever())

    async def _refresh_forever(self) -> None:
        loop = asyncio.get_running_loop()
        while True:
            left = None if deadline.get() is None else round(deadline.get() - loop.time(), 2)
            seen.append(f"refresh request_id={request_id.get()} deadline_left={left}")
            await asyncio.sleep(0.05)


async def handle(cache: PriceCache, rid: str) -> None:
    loop = asyncio.get_running_loop()
    rid_token = request_id.set(rid)
    dl_token = deadline.set(loop.time() + 0.03)        # a 30 ms request budget
    try:
        cache.ensure_started()
        await asyncio.sleep(0.01)
    finally:
        request_id.reset(rid_token)
        deadline.reset(dl_token)


async def main() -> None:
    cache = PriceCache()
    await handle(cache, "req-8c1f")
    await handle(cache, "req-99aa")                    # later requests change nothing
    await asyncio.sleep(0.12)
    cache._refresher.cancel()
    print("\n".join(seen))


asyncio.run(main())

Every refresh logs request_id=req-8c1f, and the deadline it inherited is already negative from the second refresh onwards. Resetting the variables in the request's finally did not help: reset() changes the request task's context, while the refresher holds its own copy made at create_task() time.

Verify: all refresh lines carry req-8c1f, and deadline_left becomes negative after the first iteration.

2. Measure the memory a leaked context retains

Context values are references. A background task's copied context keeps every object the request stored in context alive for as long as the task runs — request bodies, parsed documents, database sessions.

import asyncio
import contextvars
import gc
import weakref

current_request = contextvars.ContextVar("current_request", default=None)


class Request:
    def __init__(self, rid: str) -> None:
        self.rid = rid
        self.body = bytearray(10_000_000)              # a 10 MB upload kept for an audit hook


async def poll_forever() -> None:
    while True:
        await asyncio.sleep(0.01)


async def handle(detach: bool) -> tuple[asyncio.Task, weakref.ref]:
    req = Request("req-1")
    token = current_request.set(req)
    try:
        ctx = contextvars.Context() if detach else None
        task = asyncio.create_task(poll_forever(), context=ctx)
        return task, weakref.ref(req)
    finally:
        current_request.reset(token)


async def main() -> None:
    for detach in (False, True):
        task, ref = await handle(detach)
        await asyncio.sleep(0.05)
        gc.collect()
        print(f"detach={detach}: request object alive while task runs -> {ref() is not None}")
        task.cancel()
        await asyncio.gather(task, return_exceptions=True)


asyncio.run(main())

With an inherited context, the 10 MB request object stays alive for as long as the background task does; with context=contextvars.Context(), it is freed as soon as the request returns. A handful of lazily started tasks per deploy is a small leak; a pattern that starts a long-lived task per request is an unbounded one, and it shows up as memory that grows with request count, as tracked in tracking task growth in long-running services.

Verify: the first line reports the request object alive, the second reports it freed.

3. Detach long-lived tasks with a clean context

The fix is explicit ownership of context at task creation. Background work gets a fresh, empty contextvars.Context(), or — better — is started at application startup, where there is no request context to inherit in the first place.

import asyncio
import contextvars


def spawn_detached(coro, *, name: str) -> asyncio.Task:
    """Start work that must not inherit the caller's request context."""
    return asyncio.create_task(coro, name=name, context=contextvars.Context())


class PriceCacheFixed:
    def __init__(self) -> None:
        self._refresher: asyncio.Task | None = None

    async def start(self) -> None:                     # called from application startup
        self._refresher = spawn_detached(self._refresh_forever(), name="cache.refresher")

    def ensure_started(self) -> None:                  # still safe if called lazily
        if self._refresher is None:
            self._refresher = spawn_detached(self._refresh_forever(), name="cache.refresher")

    async def _refresh_forever(self) -> None:
        while True:
            seen.append(f"refresh request_id={request_id.get()} deadline={deadline.get()}")
            await asyncio.sleep(0.05)


async def main() -> None:
    seen.clear()
    cache = PriceCacheFixed()
    await handle(cache, "req-8c1f")                    # lazy start inside a request
    await asyncio.sleep(0.12)
    cache._refresher.cancel()
    print("\n".join(seen))                             # request_id=- deadline=None


asyncio.run(main())

The refresher now logs request_id=- and sees no deadline, even though it was still started lazily inside a request. loop.call_soon(), call_later() and call_at() accept the same context= argument for callbacks that should not run under the caller's context.

Verify: every refresh line shows request_id=- and deadline=None.

4. Carry over only what the work needs

Some deferred work legitimately belongs to the request that created it — sending a confirmation email after a response, writing an audit record — and needs its request ID for correlation, but not its deadline, and not its request body. Build a curated context: start from empty and set only the chosen variables.

import asyncio
import contextvars

tenant = contextvars.ContextVar("tenant", default=None)

CARRY_OVER = (request_id, tenant)                      # correlation and ownership: yes
# deliberately excluded: deadline, current_request, database session


def curated_context(variables=CARRY_OVER) -> contextvars.Context:
    ctx = contextvars.Context()
    for var in variables:
        value = var.get()
        ctx.run(var.set, value)                        # set inside the new context only
    return ctx


def spawn_follow_up(coro, *, name: str) -> asyncio.Task:
    return asyncio.create_task(coro, name=name, context=curated_context())


async def send_receipt(order_id: str) -> None:
    await asyncio.sleep(0.05)                          # outlives the request's 30 ms deadline
    print(f"receipt for {order_id}: request_id={request_id.get()} "
          f"tenant={tenant.get()} deadline={deadline.get()}")


async def checkout() -> asyncio.Task:
    loop = asyncio.get_running_loop()
    request_id.set("req-42")
    tenant.set("acme")
    deadline.set(loop.time() + 0.03)
    return spawn_follow_up(send_receipt("order-7"), name="email.receipt:order-7")


async def main() -> None:
    task = await asyncio.create_task(checkout())
    await task                                         # prints request_id=req-42 tenant=acme deadline=None


asyncio.run(main())

The follow-up keeps the request ID and tenant, so its logs correlate with the request, while the expired deadline stays behind. Keeping the allow-list explicit in one place makes the decision reviewable: adding a new context variable does not silently start flowing into background work.

Verify: the receipt line shows request_id=req-42, tenant=acme and deadline=None.

5. Guard against accidental inheritance in tests

A test that fails when a long-lived task starts inside a request context catches the regression before production does. Wrap task creation during tests and flag tasks that inherit a request ID while being named as background work.

import asyncio
import contextvars

BACKGROUND_PREFIXES = ("cache.", "periodic.", "consumer.")


def inheritance_guard(loop, coro, **kwargs):
    ctx = kwargs.get("context")
    if ctx is None:                                    # an empty Context is falsy: test for None
        ctx = contextvars.copy_context()
    task = asyncio.Task(coro, loop=loop, **kwargs)
    inherited_id = ctx.get(request_id, "-")
    if task.get_name().startswith(BACKGROUND_PREFIXES) and inherited_id != "-":
        task.cancel()
        raise AssertionError(f"background task {task.get_name()} inherited request {inherited_id}")
    return task


async def main() -> None:
    asyncio.get_running_loop().set_task_factory(inheritance_guard)
    request_id.set("req-1")
    try:
        asyncio.create_task(asyncio.sleep(1), name="cache.refresher")        # the bug
    except AssertionError as exc:
        print("caught:", exc)
    ok = spawn_detached(asyncio.sleep(0), name="cache.refresher")             # the fix
    await ok
    print("detached task allowed")


asyncio.run(main())

The guard relies on the naming convention from naming and tracking tasks for observability: background kinds share recognisable prefixes. Install it in the test suite's loop factory; production does not need the check once the tests enforce it.

Verify: creating cache.refresher inside a request raises the assertion, and the detached version is allowed.

Verification

Background tasks are free of request context when:

• Long-lived tasks start clean: they are created at startup or with context=contextvars.Context().
• Logs show no stale IDs: background log lines carry - or their own IDs, never a finished request's.
• Expired deadlines stay behind: deferred work does not inherit request deadlines.
• Memory is released with the request: objects stored in request context are freed when the request ends.
• Carry-over is explicit: follow-up work receives a curated context from one allow-list, and a test guard catches inheritance.

Pitfalls & edge cases

• Resetting variables in the request does not fix it. reset() only affects the current context; tasks already created keep their copy.
• Mutable values shared by reference. A curated context that copies a mutable object still shares it with the request. Carry immutable values only.
• Framework background task helpers. Utilities that run work "after the response" often create tasks inside the request context. Check how they create tasks, as discussed for running background tasks in FastAPI safely.
• Thread pools with inherited context. Pool threads created during a request can retain its context under Python 3.14's thread inheritance settings; copy context per job, as in carrying contextvars across threads and executors.
• Legitimately request-scoped children. Children in a request's TaskGroup should inherit; do not detach them. The rule applies only to work that outlives the request.

Frequently Asked Questions

Why does my background task log an old request ID?

It was created while a request's context was active, so it copied that context, including the request ID, when create_task was called. It keeps that copy for its entire life, even after the request finishes and resets its own variables. Create long-lived tasks at startup or pass context=contextvars.Context().

How do I start an asyncio task without inheriting context variables?

Since Python 3.11, pass a context explicitly: asyncio.create_task(coro, context=contextvars.Context()) starts the task in a fresh, empty context. The loop's call_soon, call_later and call_at methods accept the same context argument for plain callbacks.

Can context variables cause memory leaks in asyncio?

Yes, indirectly. A task's copied context references every value that was set when it was created. A long-lived task created during a request keeps objects stored in that request's context alive, such as request bodies or sessions, until the task finishes. Detaching the task from request context avoids it.

How do I pass only some context variables to a background task?

Create a new contextvars.Context, then for each variable to carry over call ctx.run(var.set, var.get()) so the value is set only inside the new context, and pass it to create_task with context=ctx. Keep the list of carried variables in one place so additions are deliberate.

Related

• Context Variables & Request Context — up to the topic overview for context copying and boundaries.
• Using contextvars with async generators — another place where context is not what it seems.
• Asyncio Fundamentals & Event Loop Architecture — the section overview for tasks and their lifecycle.