Skip to content

Closing Async Generators with aclosing

A database cursor wrapped in an async generator streams rows to a report builder. The builder stops after the first thousand rows it needs, and the connection that the generator's finally block was supposed to release stays checked out. Sometimes it comes back a few milliseconds later; sometimes only when the process shuts down; and its release log line carries no request ID, because it runs in a task nobody created. This is not a bug in the generator. When a caller breaks out of async for, the generator is simply left suspended at its last yield. Python cannot run its finally synchronously the way it does for ordinary generators, because cleanup may need to await, so the work is deferred to the event loop's garbage-collection hooks — or to loop shutdown. This guide shows exactly when that deferred cleanup runs, why it runs in the wrong task and context, and how contextlib.aclosing() and explicit aclose() make it happen at the right moment.

Prerequisites

Where generator cleanup actually runs 3 lanes over time. Where generator cleanup actually runs aclosing break finally, caller caller continues dropped ref break caller continues finally, Task-2 kept ref break caller continues, returns finally at shutdown time → Only aclosing runs cleanup in the caller, with the caller's context, on time.

1. Observe when an unclosed generator cleans up

A generator with an awaiting finally block, a caller that breaks early, and a context variable carrying a request ID are enough to see all three problems.

import asyncio
import contextvars
import gc

events: list[str] = []
request_id = contextvars.ContextVar("request_id", default="-")


async def rows(tag: str):
    try:
        for i in range(10):
            yield i
            await asyncio.sleep(0)
    finally:
        task = asyncio.current_task().get_name()
        events.append(f"{tag}: cleanup in task={task} request_id={request_id.get()}")
        await asyncio.sleep(0)                           # cleanup that needs to await


async def break_early(keep_reference: bool):
    request_id.set("req-1")
    gen = rows("dropped" if not keep_reference else "kept")
    async for i in gen:
        if i == 2:
            break
    events.append("caller continues")
    if not keep_reference:
        del gen
        gc.collect()                                     # finaliser schedules aclose() in a new task
    await asyncio.sleep(0.01)
    return gen if keep_reference else None


async def main() -> None:
    asyncio.current_task().set_name("main")
    await break_early(keep_reference=False)
    global survivor
    survivor = await break_early(keep_reference=True)    # still referenced when main returns
    events.append("main returns")


asyncio.run(main())
for line in events:
    print(line)

The output tells the story. For the dropped generator, caller continues comes before the cleanup, and the cleanup runs in Task-2 — a task the loop created to call aclose() after garbage collection — although it still happened to see req-1. For the generator that was still referenced, cleanup did not happen until asyncio.run() shut down async generators after main returns, in yet another task, with request_id=-.

Verify: your output shows caller continues before any cleanup line, the cleanup task names are not main, and the kept generator's cleanup appears after main returns with the default request ID.

2. Close deterministically with contextlib.aclosing

contextlib.aclosing(gen) is an async context manager that calls await gen.aclose() when the block exits, however it exits. aclose() throws GeneratorExit into the generator at its suspended yield, so its finally runs right there — in the caller's task, with the caller's context, before the next line of the caller.

import asyncio
import contextlib


async def report(limit: int) -> list[int]:
    request_id.set("req-2")
    collected = []
    async with contextlib.aclosing(rows("aclosing")) as gen:
        async for i in gen:
            collected.append(i)
            if len(collected) == limit:
                break
    events.append("caller continues")
    return collected


async def main() -> None:
    asyncio.current_task().set_name("main")
    events.clear()
    print(await report(limit=3))
    print(events)


asyncio.run(main())

Now the events read aclosing: cleanup in task=main request_id=req-2 followed by caller continues. The same guarantee holds if the loop body raises or the task is cancelled: the async with exit awaits aclose() before the exception continues upward. This is the pattern writing async iterators for paginated APIs relies on to stop paging and release connections on break.

Verify: the cleanup line names task=main and request_id=req-2, and it appears before caller continues.

3. Write generators whose cleanup survives being closed

aclose() delivers GeneratorExit at the yield. Generator code must let it propagate: catching it and yielding again raises RuntimeError: async generator ignored GeneratorExit. Cleanup may await, but should not start new long operations, and a generator that is closed while it is running — mid-await in another task — raises RuntimeError: aclose(): asynchronous generator is already running.

import asyncio
import contextlib


class FakeCursor:
    def __init__(self) -> None:
        self.closed = False

    async def fetch(self, n: int) -> list[int]:
        await asyncio.sleep(0)
        return list(range(n))

    async def close(self) -> None:
        await asyncio.sleep(0)
        self.closed = True


async def stream_rows(cursor: FakeCursor, batch: int = 100):
    try:
        while rows_batch := await cursor.fetch(batch):
            for row in rows_batch:
                yield row                                  # GeneratorExit arrives here on aclose()
    finally:
        await asyncio.wait_for(cursor.close(), timeout=2)  # bounded async cleanup


async def bad_generator():
    try:
        yield 1
    except GeneratorExit:
        yield 2                                            # wrong: never yield during close


async def main() -> None:
    cursor = FakeCursor()
    async with contextlib.aclosing(stream_rows(cursor)) as rows_iter:
        async for row in rows_iter:
            if row == 5:
                break
    print("cursor closed:", cursor.closed)                 # True

    gen = bad_generator()
    await anext(gen)
    try:
        await gen.aclose()
    except RuntimeError as exc:
        print("close failed:", exc)                        # async generator ignored GeneratorExit


asyncio.run(main())

Bounding cleanup with a timeout matters because aclose() waits for the finally block to finish; a cleanup call that hangs would hang the caller's async with exit. Keep the rule simple: finally for cleanup, never except GeneratorExit with a yield.

Verify: the cursor reports closed: True immediately after the loop, and the bad generator's aclose() raises async generator ignored GeneratorExit.

Inside a generator being closed A grid of 4 rows by 2 columns. Inside a generator being closed inside the generator allowed? result await in finally yes cleanup completes bounded cleanup timeout yes close cannot hang yield after GeneratorExit no RuntimeError aclose while running elsewhere no already running Cleanup belongs in finally; the yield is where GeneratorExit arrives.

4. Close generators you pass around or store

aclosing fits generators consumed inside one block. Generators that are handed to another component, stored on an object, or consumed partially across several calls need an explicit owner that calls aclose() — typically at the end of a request or during shutdown, with an AsyncExitStack when there are several.

import asyncio
import contextlib


class ChangeFeed:
    """Holds a long-lived async generator across method calls."""

    def __init__(self) -> None:
        self._stack = contextlib.AsyncExitStack()
        self._events = None

    async def open(self) -> None:
        gen = rows("feed")
        self._events = await self._stack.enter_async_context(contextlib.aclosing(gen))

    async def next_batch(self, n: int) -> list[int]:
        batch = []
        async for item in self._events:
            batch.append(item)
            if len(batch) == n:
                break                                      # generator stays open for next call
        return batch

    async def close(self) -> None:
        await self._stack.aclose()                         # closes the generator exactly once


async def main() -> None:
    events.clear()
    feed = ChangeFeed()
    await feed.open()
    print(await feed.next_batch(2), await feed.next_batch(2))   # [0, 1] [2, 3]
    await feed.close()
    print(events)                                              # cleanup ran at close()


asyncio.run(main())

Breaking out of async for does not close the generator — which is exactly what lets next_batch resume where it stopped. The owner's close() is the single place cleanup happens. The stack pattern is covered in depth in managing dynamic resource sets with AsyncExitStack.

Who closes this generator? A decision on How is the generator consumed with 3 outcomes. Who closes this generator? How is the generator consumed? within one block contextlib.aclosing closed at block exit across several calls owner calls aclose() exit stack or close() by a framework framework closes it check middleware Shutdown hooks catch the rest, late and in the wrong context.

Verify: the two batches continue from each other, and the generator's cleanup event appears only after feed.close().

5. Rely on loop shutdown only as a safety net

asyncio.run() and asyncio.Runner call loop.shutdown_asyncgens() before closing the loop, which closes every async generator that is still alive. Code that manages loops manually must do the same, or generators left open at exit are never finalised and emit warnings. Treat this as the last line of defence and measure how often it fires.

import asyncio
import sys


def finalizer_counter():
    counts = {"finalized_late": 0}
    original = sys.get_asyncgen_hooks()

    def finalizer(agen):
        counts["finalized_late"] += 1                      # a generator nobody closed explicitly
        if original.finalizer is not None:
            original.finalizer(agen)

    return counts, finalizer


async def main() -> None:
    counts, finalizer = finalizer_counter()
    hooks = sys.get_asyncgen_hooks()
    sys.set_asyncgen_hooks(firstiter=hooks.firstiter, finalizer=finalizer)
    gen = rows("leaky")
    await anext(gen)
    del gen                                                # dropped while suspended
    await asyncio.sleep(0.01)
    print("generators finalised by GC:", counts["finalized_late"])


asyncio.run(main())

The asyncio loop installs its own firstiter and finalizer hooks when it starts; wrapping the finaliser, as above, counts generators that reached garbage collection without being closed. In a healthy service that number stays near zero, and a rising count points at a call site missing aclosing.

Verify: the script reports one generator finalised by GC; wrapping the iteration in aclosing brings it to zero.

Verification

Async generator cleanup is deterministic when:

  • Cleanup runs before the caller continues: generators consumed in a block are wrapped in contextlib.aclosing.
  • Cleanup runs in the right task and context: request IDs and cancellation apply to the finally block.
  • Long-lived generators have an owner: an object or exit stack calls aclose() exactly once.
  • Generators follow the close protocol: no yield after GeneratorExit, and cleanup is bounded by a timeout.
  • Late finalisation is rare and measured: the GC-finalised count stays near zero, and manually managed loops call shutdown_asyncgens().

Pitfalls & edge cases

  • Assuming break closes the generator. It only stops iteration; the generator remains suspended until closed or collected.
  • Closing from another task while it runs. aclose() on a generator that is currently executing in another task raises already running. Close generators from the task that consumes them.
  • Cleanup that needs the request. A finally that logs, emits a span or checks permissions sees the wrong context when run by the finaliser. Close explicitly so it runs in the caller.
  • Generators created by frameworks. Streaming response bodies are async generators too; frameworks usually close them, but a custom middleware that iterates a body must use aclosing itself.
  • asynccontextmanager generators. These are closed by the context manager protocol already; wrapping them in aclosing is unnecessary. Their own rules are in building async context managers with asynccontextmanager.

Frequently Asked Questions

Does breaking out of async for close an async generator?

No. Breaking stops the iteration but leaves the generator suspended at its last yield, so its finally block has not run. It runs later when the generator is garbage-collected, when the event loop shuts down async generators, or immediately if you call aclose, for example through contextlib.aclosing.

What does contextlib.aclosing do?

It is an async context manager that awaits the object's aclose method when the async with block exits, whether normally, by exception or by cancellation. For an async generator this throws GeneratorExit at the suspended yield so its finally block runs immediately in the caller's task and context.

Why does my async generator's cleanup run in a different task?

When an unclosed async generator is garbage-collected, asyncio's finalizer hook schedules its aclose in a new task, and at loop shutdown shutdown_asyncgens closes remaining generators in yet another task. That task does not share the consumer's context, so context variables such as request IDs and the consumer's cancellation do not apply.

What causes async generator ignored GeneratorExit?

The generator caught GeneratorExit, usually with a broad except clause, and then yielded another value while being closed. Generators must let GeneratorExit propagate. Put cleanup in a finally block, which may await, and never yield from inside it or from an except block handling GeneratorExit.