Skip to content

Resetting ContextVars with Tokens

ContextVar.set() returns a Token, and ContextVar.reset(token) restores the variable to exactly what it was before that set() — including "not set at all". That pair is how you scope a context change to a block: override the tenant for one admin operation, raise the log level for one request, swap the database session inside a nested transaction. It has one hard rule — the token can only be reset in the same context it was created in — and violating it raised ValueError: <Token …> was created in a different Context in a test where a child task tried to reset its parent's token. Python 3.14 made the common case shorter: a token is now a context manager, so with var.set(value): resets on exit, verified on 3.14.

Prerequisites

1. Pair every set() with reset() in finally

The scoped override is a three-line pattern:

import contextvars

tenant = contextvars.ContextVar("tenant", default=None)


async def as_tenant(name: str, fn):
    token = tenant.set(name)
    try:
        return await fn()
    finally:
        tenant.reset(token)          # back to exactly the previous state

reset(token) restores the previous value, not the default. If the variable was unset before, it becomes unset again, and tenant.get() falls back to the default (or raises LookupError if there is none). That distinction matters for nested overrides: an inner override's reset returns to the outer override's value, not to the top-level default. Calling tenant.set(old_value) instead of reset looks equivalent and is not — it leaves the variable "set" even if it was unset before, and it adds a new binding rather than unwinding one.

Verify: nest two overrides; after the inner block, get() returns the outer value; after the outer block, the default.

Nested overrides unwind in order 3 lanes over time. Nested overrides unwind in order tenant value default outer inner outer again default outer token held by the outer block inner token inner block time → Each reset restores what was there before its own set(), so nesting needs no bookkeeping.

2. Use the token as a context manager on 3.14

From Python 3.14, Token supports the context manager protocol, and the pattern collapses to one line:

async def handle_admin_action(action):
    with tenant.set("system"):                 # resets on exit, including on exceptions
        await audit_log.write(action)
        await action.run()
    # tenant is back to the request's value here

Verified on 3.14: inside the block get() returned the new value; after it, the previous one. On earlier versions, a tiny helper gives the same shape:

from contextlib import contextmanager


@contextmanager
def scoped(var: contextvars.ContextVar, value):
    token = var.set(value)
    try:
        yield
    finally:
        var.reset(token)

Note that this is a synchronous context manager used around awaits — that is fine, because the with block runs entirely inside one task and one context. There is no need for async with.

Verify: raise inside the block; the variable is still restored.

3. Never carry a token across a task boundary

A token belongs to the context in which set() ran. Every task has its own context copy, so a token created in one task cannot be reset in another:

async def main() -> None:
    token = tenant.set("x")

    async def reset_elsewhere():
        tenant.reset(token)          # ValueError: <Token …> was created in a different Context

    await asyncio.create_task(reset_elsewhere())

This bites in callback-heavy code: a library calls a "before" hook in one task and an "after" hook in another, and the hooks pass a token between them. The fix is to keep set() and reset() in the same coroutine frame — a try/finally or with block — so they cannot drift apart. Where a framework forces split hooks (some ASGI middleware, some ORMs' event systems), store the previous value instead and set() it back in the after-hook, accepting that "unset" cannot be restored.

Verify: grep for tokens stored on objects or in dicts; each is a candidate for crossing a context boundary.

Where a token may be reset A grid of 5 rows by 3 columns. Where a token may be reset reset happens in allowed why same coroutine, finally block yes same context an awaited coroutine, same task yes same context another task ValueError each task has its own copy a done-callback or call_soon ValueError runs in a captured copy a thread via to_thread ValueError the thread gets a copy Keep set() and reset() in one frame and this table never matters.

4. Handle tokens in async generators carefully

An async generator that sets a context variable and yields is the trickiest case. Each anext() call runs the generator's next step in the caller's context at that moment, so a set() before a yield changes the consumer's context, and a reset() after the yield must run in that same context — which it does, as long as the generator is driven from one task:

async def tenant_scoped_rows(name: str, query):
    with tenant.set(name):                 # 3.14; use scoped() on older versions
        async for row in query:
            yield row                      # the CONSUMER now sees tenant == name

The surprise is the first comment-worthy line: while the generator is suspended at yield, the consumer's code runs with the generator's override in effect. That is rarely intended. If the generator is driven from different tasks, or closed by the garbage collector in another context, the reset raises ValueError. Set context around the work inside the generator, not across the yield:

async def tenant_scoped_rows(name: str, query):
    async for row in query:
        with tenant.set(name):
            enriched = await enrich(row)   # override only while the generator works
        yield enriched

The wider topic is covered in using contextvars with async generators.

Verify: in the consumer's loop body, tenant.get() returns the consumer's own value, not the generator's.

5. Test the unwinding

Context bugs are invisible until two requests interleave, so test the reset explicitly — including the error paths:

import pytest


async def test_override_is_scoped():
    assert tenant.get() is None
    with pytest.raises(RuntimeError):
        with scoped(tenant, "acme"):
            assert tenant.get() == "acme"
            raise RuntimeError("boom")
    assert tenant.get() is None                    # restored despite the exception


async def test_concurrent_overrides_do_not_mix():
    async def worker(name):
        with scoped(tenant, name):
            await asyncio.sleep(0.01)
            return tenant.get()
    assert await asyncio.gather(worker("a"), worker("b")) == ["a", "b"]

The second test passes because gather runs each worker in its own task and context — the isolation described in propagating request IDs with contextvars.

Verify: both tests pass; removing the finally from scoped makes the first fail.

The scoped-override pattern A flow of 4 stages. The scoped-override pattern token = var.set(v) override begins run the block awaits are fine finally / with exit reset(token) previous state back including unset One frame owns both ends of the override; that is the entire rule.

Verification

Token usage is correct when:

  • Every set() meant to be temporary is paired with reset(token) in finally or a with block.
  • No token is stored and reset elsewhere — no ValueError: created in a different Context in logs.
  • Async generators do not hold an override across yield.
  • Tests cover exceptions and concurrency for each scoped override.

Diagnostic Hook: log ValueError exceptions whose message contains "created in a different Context" with a dedicated counter; each one is a token crossing a task or callback boundary. For long-lived overrides such as tenant or log level, include the variable's value in structured logs so a value that "leaks" past its block shows up as an unexpected label in later log lines.

Pitfalls & edge cases

  • Restoring with set(old) instead of reset(token). It cannot restore "unset" and accumulates bindings.
  • Forgetting the finally. An exception leaves the override in place for the rest of the task.
  • Resetting the same token twice. The second call raises a RuntimeError saying the token has already been used once.
  • Overrides across yield. The consumer runs under the generator's override.

Frequently Asked Questions

What does ContextVar.reset(token) do?

It restores the variable to the state it had before the set() call that produced the token — the previous value, or unset if it had none. It must be called in the same context where set() ran.

Why do I get 'Token was created in a different Context'?

The token was reset in a different task, callback or thread from the one that called set(). Each task runs in its own copy of the context, so keep set() and reset() in the same coroutine, typically in try/finally.

Can I use a ContextVar token as a context manager?

On Python 3.14 and later, yes: with var.set(value): resets the variable when the block exits. On earlier versions, write a small contextmanager helper that calls set and reset.

Is it OK to use a sync with block for a ContextVar inside async code?

Yes. The block runs inside one task and one context even if it contains awaits, so a plain with is correct; async with is not needed.