Skip to content

Using asyncio.wait with FIRST_COMPLETED and FIRST_EXCEPTION

asyncio.wait() is the lowest-level way to wait on a group of tasks, and the one with the fewest opinions. It returns two sets — done and pending — and does nothing else: it does not raise the tasks' exceptions, it does not cancel anything on timeout, and it does not care what you do with the leftovers. That makes it the right tool for the cases gather() and TaskGroup handle badly — reacting to whichever of several tasks finishes first, multiplexing a fixed set of long-running loops, or bounding a wait without cancelling the work. It also makes it the easiest way to leak tasks. In a test with three tasks and a 120 ms timeout, two came back in done, one in pending, and the pending one was still running — cancelled() was False — until it was explicitly cancelled.

Prerequisites

1. Know the contract

done, pending = await asyncio.wait(tasks, timeout=None, return_when=asyncio.ALL_COMPLETED)
  • tasks must be an iterable of tasks or futures. Since 3.11 coroutines are rejected.
  • return_when is ALL_COMPLETED (default), FIRST_COMPLETED, or FIRST_EXCEPTION.
  • timeout bounds the wait. When it expires, wait() returns normally — no TimeoutError — with whatever has not finished in pending.
  • Exceptions inside tasks are not raised. A failed task is simply in done; you find out by calling task.result() or task.exception().
  • Nothing is cancelled, ever. The tasks in pending keep running.
import asyncio


async def sleep_for(d: float) -> float:
    await asyncio.sleep(d)
    return d


async def main() -> None:
    tasks = [asyncio.create_task(sleep_for(d)) for d in (0.05, 0.1, 0.2)]
    done, pending = await asyncio.wait(tasks, timeout=0.12)
    print(len(done), len(pending), [t.cancelled() for t in pending])   # 2 1 [False]
    for t in pending:
        t.cancel()                                   # your job, not wait()'s
    await asyncio.gather(*pending, return_exceptions=True)


asyncio.run(main())

Verify: without the final cancel loop, the 200 ms task completes after wait() returns — add a print inside it and watch it appear.

Three ways to wait for a group of tasks A grid of 5 rows by 4 columns. Three ways to wait for a group of tasks behaviour asyncio.wait gather TaskGroup task raises stays in done first error raised others cancelled, group raised timeout expires returns normally needs wait_for needs asyncio.timeout unfinished tasks left running left running always cancelled result order unordered sets input order you hold the tasks first-finished logic yes no no wait() is the only one that lets you react to the first completion; it is also the only one that never cleans up.

2. Multiplex long-running loops with FIRST_COMPLETED

The canonical use is a small, fixed set of concurrent loops where whichever finishes first decides what happens next — a websocket connection with a reader loop, a writer loop, and a shutdown signal:

async def run_connection(ws, outbox: asyncio.Queue, stop: asyncio.Event) -> None:
    reader = asyncio.create_task(read_loop(ws), name="reader")
    writer = asyncio.create_task(write_loop(ws, outbox), name="writer")
    stopper = asyncio.create_task(stop.wait(), name="stop")
    tasks = {reader, writer, stopper}
    try:
        done, _ = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
        for t in done:
            if t is not stopper and not t.cancelled() and t.exception():
                log.warning("%s ended with %r", t.get_name(), t.exception())
    finally:
        for t in tasks:
            t.cancel()
        await asyncio.gather(*tasks, return_exceptions=True)

Whichever ends first — the peer closes (reader returns), a send fails (writer raises), or the server is shutting down — the other two are cancelled and awaited in the finally. A TaskGroup cannot express "stop the others when one returns normally", because only an exception cancels its siblings; wait(FIRST_COMPLETED) can.

Verify: close the socket from the peer side and check that the writer task is cancelled and nothing is left in asyncio.all_tasks() for this connection.

One connection, three loops, first one out ends it A flow of 4 stages. One connection, three loops, first one out ends it start 3 tasks reader, writer, stop wait(FIRST_COMPLETED) returns 1+ done inspect done log real failures finally cancel and gather all The finally block is the part that makes this safe; wait() itself leaves the other two running.

3. Process results as they finish, in a loop

When you have many tasks and want to handle each result as soon as it is ready while still being able to add new tasks, loop on wait(FIRST_COMPLETED) and feed the pending set back in:

async def crawl(start_urls: list[str], limit: int = 20) -> None:
    queue = list(start_urls)
    running: set[asyncio.Task] = set()
    while queue or running:
        while queue and len(running) < limit:
            running.add(asyncio.create_task(fetch(queue.pop())))
        done, running = await asyncio.wait(running, return_when=asyncio.FIRST_COMPLETED)
        for t in done:
            if t.exception() is not None:
                log.warning("fetch failed: %r", t.exception())
                continue
            queue.extend(extract_links(t.result()))

This is a bounded, dynamic work set — the shape of a crawler or any recursive fan-out — without a semaphore. as_completed() cannot do this, because its input set is fixed when you call it. Note the reassignment: running becomes the returned pending set, so finished tasks drop out naturally.

One cost to know: each wait() call attaches a done-callback to every task in the set and removes it afterwards, so looping over a set of N tasks is O(N) per iteration. For a few dozen tasks this is irrelevant; for tens of thousands, use a queue and fixed workers instead, as in building an async worker pool with TaskGroup.

Verify: len(running) never exceeds limit, and the loop exits when both the queue and the running set are empty.

4. Fail fast with FIRST_EXCEPTION

FIRST_EXCEPTION returns when any task raises, or when all complete if none do. It is the closest wait() gets to TaskGroup semantics, minus the automatic cancellation:

async def load_all(sources) -> list:
    tasks = [asyncio.create_task(load(s), name=f"load:{s}") for s in sources]
    done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_EXCEPTION)
    failed = [t for t in done if not t.cancelled() and t.exception() is not None]
    if failed:
        for t in pending:
            t.cancel()
        await asyncio.gather(*pending, return_exceptions=True)
        raise failed[0].exception()
    return [t.result() for t in tasks]

If you find yourself writing exactly this, use a TaskGroup instead — it does the cancellation and the waiting for you and reports every failure in an ExceptionGroup rather than the first. FIRST_EXCEPTION earns its place when you want to inspect the failure and decide whether to cancel the rest, for example tolerating failures from optional sources while aborting on a required one. That decision is covered from the other side in returning partial results from a fan-out before a deadline.

Verify: with one task failing at 50 ms and another running for 200 ms, wait() returns at about 50 ms with one done and one pending.

Is asyncio.wait the right call here? A decision on What should happen when one task finishes with 3 outcomes. Is asyncio.wait the right call here? What should happen when one task finishes? stop the rest, or grow the set asyncio.wait you own cleanup nothing; all must succeed TaskGroup cancels on failure handle it; the set is fixed as_completed results in finish order Reach for wait() when the first completion changes what you do next.

5. Never return with pending tasks

Every pattern above ends the same way: whatever is still pending is cancelled and awaited before the function returns. Wrap that in a helper so it is impossible to forget:

async def cancel_and_wait(tasks: set[asyncio.Task], timeout: float = 5.0) -> None:
    for t in tasks:
        t.cancel()
    if tasks:
        done, still = await asyncio.wait(tasks, timeout=timeout)
        for t in still:
            log.error("task %s ignored cancellation for %.1fs", t.get_name(), timeout)

Using wait() with a timeout here is deliberate: a task that swallows CancelledError would make gather() hang forever, whereas this logs the offender and returns. Tasks that ignore cancellation are a bug worth finding — the patterns for writing cleanup that cannot swallow it are in preventing CancelledError leaks in cleanup.

Verify: after the function returns, asyncio.all_tasks() contains none of the tasks it created.

Verification

Your asyncio.wait() usage is safe when:

  • Every call site handles pending, either by looping on it or by cancelling and awaiting it.
  • Every task in done has its exception retrieved, so nothing is reported later as "never retrieved".
  • No coroutines are passed directly — the code runs clean on Python 3.11+.
  • Tasks per connection or request drop to zero after the connection or request ends.

Diagnostic Hook: tag tasks with a name prefix per owner (conn:1234:reader) and periodically count asyncio.all_tasks() by prefix. Any owner whose count stays above zero after it has finished is a leaked pending set; the prefix tells you which code path forgot its cleanup.

Pitfalls & edge cases

  • Treating the timeout as a deadline on the work. It bounds only the wait; the work continues unless you cancel it.
  • Passing a list and expecting ordered results. done is a set; keep your own list of tasks if order matters.
  • Calling wait() with an empty set. It raises ValueError; guard the loop condition.
  • Ignoring results in done. An exception in a task you never inspect is logged at GC time with no context.
  • Re-wrapping the same coroutine. Creating a new task from an already-awaited coroutine raises RuntimeError: cannot reuse already awaited coroutine.

Frequently Asked Questions

Does asyncio.wait cancel pending tasks on timeout?

No. On timeout it returns normally with the unfinished tasks in the pending set, and they keep running. Cancel and await them yourself if the work should stop.

What is the difference between asyncio.wait and asyncio.gather?

gather returns results in input order and raises the first exception; wait returns done and pending sets, raises nothing, and supports FIRST_COMPLETED and FIRST_EXCEPTION. Use wait when the first completion should change what you do next.

Why does asyncio.wait raise TypeError about coroutines?

Since Python 3.11 asyncio.wait only accepts tasks and futures, because passing coroutines silently created tasks you could not match against the returned sets. Wrap each coroutine with asyncio.create_task first.

How do I wait for the first task to finish and cancel the rest?

Call asyncio.wait with return_when=FIRST_COMPLETED, read the result from the done set, then cancel every task in the pending set and await them, ideally in a finally block.

When should I use FIRST_EXCEPTION instead of a TaskGroup?

When you want to inspect the first failure and decide whether to cancel the rest — for example tolerating optional sources. If any failure should cancel everything, a TaskGroup does it automatically and reports every failure.