Skip to content

Returning Partial Results from a Fan-Out Before a Deadline

A price-comparison endpoint queries eight shops concurrently and has 200 ms to answer. Written with TaskGroup, one shop that refuses connections cancels the seven that were about to succeed and the user gets an error. Written with asyncio.gather(return_exceptions=True), the failure is tolerated but the slowest shop now sets the response time, and the endpoint answers in two seconds instead of 200 ms. Neither is what the product wants, which is: return whatever arrived in time, say what is missing, and never wait past the budget. That is a different contract from "all or nothing", and it needs to be built deliberately — with failures turned into values, a hard deadline, an optional early exit once enough results are in, and a clear separation between optional calls that may be dropped and required ones that must still fail fast.

Prerequisites

Three fan-out contracts 3 columns contrasting TaskGroup, gather, return_exceptions, settle + deadline. Three fan-out contracts TaskGroup all or nothing first failure cancels the rest latency: slowest call fits required calls gather, return_exceptions everything, eventually failures become list items latency: slowest call no budget at all settle + deadline what arrived in time failures become values latency: the budget fits optional calls Pick the contract per dependency, not per request.

1. Settle each call into a result value

The first job is to stop one failure from affecting its siblings. Wrap every call so that an ordinary exception becomes a value — an Err — while cancellation still propagates. Cancellation is how the deadline will stop slow calls in the next step, so it must never be turned into a result.

import asyncio
from dataclasses import dataclass
from typing import Any, Awaitable, Generic, TypeVar

T = TypeVar("T")


@dataclass(frozen=True)
class Ok(Generic[T]):
    key: str
    value: T


@dataclass(frozen=True)
class Err:
    key: str
    error: BaseException


async def settle(key: str, call: Awaitable[T]) -> Ok[T] | Err:
    """Turn a failure into a value so it cannot cancel sibling tasks."""
    try:
        return Ok(key, await call)
    except asyncio.CancelledError:
        raise                                   # cancellation is control flow, not a result
    except Exception as exc:
        return Err(key, exc)

This wrapper is safe to run inside a TaskGroup too: because settle never raises an ordinary exception, the group never sees a failure and never cancels the other children. It is also more explicit than gather(return_exceptions=True), which mixes values and exceptions in one list and hands back a cancelled child's CancelledError as just another list item.

Verify: run settle around a coroutine that raises ConnectionError; it returns Err(key, ConnectionError(...)). Cancel a task running settle and awaiting it raises CancelledError, not a result.

2. Stop at the deadline and keep what finished

With failures settled, the deadline decides everything else. asyncio.wait() with a timeout returns the tasks that finished and those still running, without cancelling anything and without raising. Cancel the stragglers yourself, give them a moment to clean up, and build the response from what completed.

import asyncio
import random
from typing import Awaitable, Callable


async def fetch_price(shop: str) -> float:
    rng = random.Random(shop)
    await asyncio.sleep(rng.uniform(0.01, 0.3))
    if shop.endswith("3"):
        raise ConnectionError(f"{shop} refused")
    return round(rng.uniform(10, 20), 2)


async def gather_until(deadline: float,
                       calls: dict[str, Callable[[], Awaitable[T]]]
                       ) -> tuple[dict[str, T], dict[str, str]]:
    tasks = {asyncio.create_task(settle(k, fn()), name=f"fetch:{k}"): k
             for k, fn in calls.items()}
    done, pending = await asyncio.wait(tasks, timeout=deadline)
    for t in pending:
        t.cancel()
    if pending:
        await asyncio.wait(pending)              # let cancelled calls release connections
    values: dict[str, T] = {}
    missing: dict[str, str] = {tasks[t]: "deadline" for t in pending}
    for t in done:
        r = t.result()
        if isinstance(r, Ok):
            values[r.key] = r.value
        else:
            missing[r.key] = type(r.error).__name__
    return values, missing


async def main() -> None:
    shops = {f"shop-{i}": (lambda s=f"shop-{i}": fetch_price(s)) for i in range(8)}
    values, missing = await gather_until(0.2, shops)
    print(sorted(values), missing)


asyncio.run(main())

Waiting for the cancelled tasks after cancelling them matters in production. A cancelled HTTP call still has to return its connection to the pool; skipping the wait leaves that cleanup racing the next request. If a call can ignore cancellation, bound this second wait as described in cancelling a task and waiting for it to finish.

Verify: the run prints three or four shop names and a missing dictionary where slow shops are marked deadline and failing ones carry their exception type. Total runtime stays close to 200 ms regardless of how slow the slowest shop is.

Eight calls, one 200 ms budget 6 lanes over time. Eight calls, one 200 ms budget shop-1 ok shop-3 error shop-4 ok shop-5 ok shop-0 still waiting cancel shop-6 still waiting cancel 0 ms 200 ms 300 ms time → (deadline at 200 ms) Completed values are kept, the error is reported, stragglers are cancelled.

3. Return early once a quorum succeeds

Sometimes the product needs enough results rather than all available ones: three quotes to show a comparison, two replicas agreeing on a value, the first healthy mirror. Iterate over asyncio.as_completed() inside the deadline and stop as soon as the quorum is met, then cancel the rest.

import asyncio
from typing import Awaitable, Callable


async def first_n(n: int, calls: dict[str, Callable[[], Awaitable[T]]],
                  deadline: float) -> dict[str, T]:
    tasks = {asyncio.create_task(settle(k, fn())) for k, fn in calls.items()}
    values: dict[str, T] = {}
    try:
        async with asyncio.timeout(deadline):
            for next_done in asyncio.as_completed(tasks):
                r = await next_done
                if isinstance(r, Ok):
                    values[r.key] = r.value
                    if len(values) >= n:
                        break                    # quorum reached: stop waiting
    except TimeoutError:
        pass                                     # fewer than n arrived; caller decides
    finally:
        for t in tasks:
            t.cancel()
        await asyncio.gather(*tasks, return_exceptions=True)
    return values

The finally block runs on every exit path — quorum, timeout, or the caller being cancelled — so no fetch outlives the function. Failed calls are skipped rather than counted, which means a quorum of three can still be met when two of eight shops are down.

Verify: await first_n(3, shops, deadline=1.0) returns exactly three prices, typically well under the deadline, and asyncio.all_tasks() afterwards contains no leftover fetch tasks.

4. Tell the caller what is missing

A partial response that looks complete is worse than an error: the user sees three prices and assumes there were only three shops. Make incompleteness part of the response contract, and record it as a metric so a degraded upstream shows up on a dashboard instead of only in support tickets.

import asyncio
import collections
from dataclasses import dataclass, field

missing_counter: collections.Counter[tuple[str, str]] = collections.Counter()


@dataclass
class PriceResponse:
    prices: dict[str, float]
    partial: bool
    missing: dict[str, str] = field(default_factory=dict)   # shop -> reason


async def compare_prices(shops: dict, budget: float = 0.2) -> PriceResponse:
    values, missing = await gather_until(budget, shops)
    for shop, reason in missing.items():
        missing_counter[(shop, reason)] += 1                 # export per shop and reason
    return PriceResponse(prices=values, partial=bool(missing), missing=missing)

Distinguishing deadline from an exception type in missing is what makes the metric actionable: a shop that is consistently deadline needs a latency fix or a smaller share of the budget, while a shop that is consistently ConnectionError needs a circuit breaker so it stops consuming a task slot on every request.

Verify: the serialised response includes "partial": true and a reason per missing shop whenever fewer than all shops answered, and the counter increments by shop and reason.

5. Keep required calls fail-fast

Not every call in a request is optional. The prices are optional; the user's currency and the product record are not — without them there is nothing meaningful to return. Put required calls in a TaskGroup, where any failure cancels the rest and propagates, and run the optional fan-out alongside them.

import asyncio


async def load_product(product_id: str) -> dict:
    await asyncio.sleep(0.02)
    return {"id": product_id, "name": "Widget"}


async def load_currency(user_id: str) -> str:
    await asyncio.sleep(0.01)
    return "EUR"


async def product_page(product_id: str, user_id: str, shops: dict) -> dict:
    async with asyncio.timeout(0.5):                      # whole-request budget
        async with asyncio.TaskGroup() as tg:
            product = tg.create_task(load_product(product_id))    # required
            currency = tg.create_task(load_currency(user_id))     # required
            prices = tg.create_task(compare_prices(shops, budget=0.2))  # optional inside
    return {
        "product": product.result(),
        "currency": currency.result(),
        "prices": prices.result(),
    }

This is the shape to aim for: required work keeps all-or-nothing semantics, including an ExceptionGroup if both required calls fail, while the optional fan-out has its own smaller budget nested inside the request budget and can never cause the request to fail.

Verify: make load_currency raise and the request fails with an ExceptionGroup containing that error; make every shop fail instead and the request succeeds with an empty, clearly partial price list.

Required and optional work in one request 4 stacked layers from Request budget to Response. Required and optional work in one request Request budget asyncio.timeout(0.5) one deadline for all Required calls TaskGroup product record user currency Optional fan-out budget 0.2 s settled results partial flag Response product currency prices + missing Required failures fail the request; optional failures only shrink the answer.

Verification

Partial results are implemented correctly when:

  • Latency is bounded by the budget: response time tracks the deadline, not the slowest dependency, under fault injection that makes one call hang.
  • One failure never removes other results: a raising dependency appears in missing while every other completed value is returned.
  • Nothing outlives the request: after a response, no fan-out tasks remain in asyncio.all_tasks(), and connection pools show no leaked checkouts.
  • Incompleteness is explicit: responses carry partial and per-key reasons, and the missing-by-reason metric is exported.
  • Required dependencies still fail the request: errors in required calls propagate through the TaskGroup rather than being settled away.

Pitfalls & edge cases

  • Settling BaseException. Catching BaseException in the wrapper converts CancelledError and KeyboardInterrupt into results, so the deadline can no longer stop tasks and shutdown hangs. Catch Exception only.
  • Per-call timeouts that add up. Giving each of eight calls its own 200 ms timeout and awaiting them in sequence is a 1.6-second worst case. The budget must be enforced once, around the whole fan-out.
  • Nested budgets that exceed the outer one. An optional fan-out with a 400 ms budget inside a 300 ms request timeout is cut off by the outer timeout and returns nothing. Derive inner budgets from the remaining outer time, as in propagating deadlines across async service calls.
  • Caching partial results as complete. If the response is cached, a partial answer can be served long after the upstream recovered. Skip caching or use a much shorter TTL when partial is true.
  • Unbounded fan-out width. Querying every shop for every request multiplies upstream load by the fan-out width. Cap concurrency per dependency with per-dependency semaphores so a traffic spike does not become an outage upstream.

Frequently Asked Questions

How do I get partial results when some asyncio tasks fail?

Wrap each call so ordinary exceptions are returned as values instead of raised, while CancelledError still propagates. Run the wrapped calls as tasks, collect the successful values, and record failures separately. Because the wrapper never raises, one failing call cannot cancel its siblings, even inside a TaskGroup.

How do I return whatever results finished before a timeout?

Create the tasks, then call asyncio.wait(tasks, timeout=budget). It returns the finished and pending sets without raising. Cancel the pending tasks, wait briefly for their cleanup, and build the response from the finished ones, marking the rest as missing because of the deadline.

Should I use gather(return_exceptions=True) or TaskGroup for partial results?

Neither alone gives a deadline. gather with return_exceptions tolerates failures but waits for the slowest call. TaskGroup cancels everything on the first failure. Wrap calls in a settle function and combine them with asyncio.wait and a timeout, and keep TaskGroup for calls that are genuinely required.

How do I stop waiting once enough tasks have succeeded?

Iterate over asyncio.as_completed inside an asyncio.timeout block, count successful results, and break when the quorum is reached. In a finally block cancel all remaining tasks and gather them with return_exceptions=True so none of them outlive the function on any exit path.