Skip to content

Cancelling Futures and Their Callbacks

Cancelling a future is a single call, but its effects spread: every done-callback runs, every coroutine awaiting it gets CancelledError, and any structure still referring to it keeps it alive. Measured on Python 3.14: the first future.cancel("shutting down") returned True, a second False, and cancelling a future that already had a result also returned False. Done-callbacks did not run inside cancel(); they ran on the next loop iteration, and a callback that called f.result() without checking f.cancelled() raised inside the loop, which logged "Exception in callback ..." — while a careful callback recorded the cancellation. Awaiting the future raised CancelledError carrying the message ('shutting down',). Cancelling a task that was awaiting a bare future also cancelled the future; awaiting it through asyncio.shield left the future untouched. A request/response map that registered a future per request and never removed it held 10,000 cancelled futures after 10,000 timed-out requests; removing the entry in a finally left 0. Cancelled timer handles stayed in the loop's schedule — 100,000 entries with 40,000 cancelled — until more than half were cancelled, when it shrank to 60,000. This guide covers those behaviours.

Prerequisites

1. Know what cancel() returns and when callbacks run

cancel() moves a pending future to the cancelled state and returns True; on a future that is already done — result, exception or cancelled — it does nothing and returns False:

fut = loop.create_future()
fut.add_done_callback(on_done)

fut.cancel("shutting down")     # True
fut.cancel()                    # False: already cancelled

done = loop.create_future()
done.set_result(1)
done.cancel()                   # False: a result cannot be taken back

Measured: exactly those return values, and no callback had run when cancel() returned — the list the callbacks appended to was empty. Callbacks are scheduled with call_soon and run on the next loop iteration, in registration order. That gap matters: code that cancels a future and then inspects state the callbacks are supposed to update sees the old state until it yields. Use the return value: False means someone else already completed the future, and the cancellation was lost.

Verify: code that cancels futures checks the return value, and does not expect callbacks to have run before its next await.

2. Write callbacks that expect cancellation

A done-callback receives the future and is called for every outcome, including cancellation. Calling result() on a cancelled future raises CancelledError — inside the event loop's callback machinery:

def careless(fut):
    record(fut.result())                       # raises CancelledError if cancelled

def careful(fut):
    if fut.cancelled():
        record_cancelled()
        return
    if (exc := fut.exception()) is not None:
        record_failure(exc)
        return
    record(fut.result())

Measured: after cancellation, the careful callback recorded ("careful", "cancelled"); the careless one raised, and the loop logged asyncio ERROR Exception in callback main.<locals>.careless() at cancel.py:7 with the traceback. The future and its other callbacks were unaffected, but the careless callback's work — updating a metric, releasing a resource — silently did not happen. Check cancelled() first, then exception(), then result().

Verify: every done-callback handles the cancelled, failed and successful cases explicitly.

What cancellation did, measured A grid of 8 rows by 2 columns. What cancellation did, measured action result cancel(), then cancel() again True, then False cancel() on a future with a result False done-callbacks run on the next loop iteration callback calls result() on a cancelled future 'Exception in callback' logged await cancel('shutting down')'d future CancelledError('shutting down',) cancel the task awaiting a future the future is cancelled too cancel the task awaiting shield(future) the future stays pending 10,000 timed-out requests, no cleanup 10,000 cancelled futures kept Python 3.14.

3. Expect cancellation to travel from task to future

A task waiting on a future is waiting on that future. When the task is cancelled, asyncio cancels the future it is blocked on, which is how the cancellation reaches the task's coroutine:

fut = loop.create_future()

async def waiter():
    return await fut

task = asyncio.create_task(waiter())
await asyncio.sleep(0)
task.cancel()
# fut.cancelled() is now True

Measured: after cancelling the task, fut.cancelled() was True. For a future owned by one waiter, that is correct. For a future shared by several — a cached result, a batch shared by many callers, a connection's "ready" future — one waiter's cancellation cancels it for everyone. Await shared futures through asyncio.shield:

async def wait_for_shared(shared_future):
    return await asyncio.shield(shared_future)     # cancelling this waiter leaves the future alone

Measured: with the shield, cancelling the waiting task left the future pending. The pattern, and why the batcher in batching individual calls with futures returns shielded futures, is the same.

Verify: futures awaited by more than one task are awaited through shield.

Cancelling a task that awaits a future A sequence of 5 messages between 4 participants. Cancelling a task that awaits a future caller task future callbacks task.cancel() cancel the awaited future call_soon(each callback) wake: CancelledError next iteration: run callbacks Through shield(), only the outer wrapper future is cancelled.

4. Remove futures from registries when their waiter gives up

Request/response protocols keep a map from request ID to the future that the response will resolve. When the waiting caller times out or is cancelled, the future is cancelled — but the map still holds it:

pending: dict[int, asyncio.Future] = {}

async def request(conn, payload) -> bytes:
    rid = next_id()
    fut = asyncio.get_running_loop().create_future()
    pending[rid] = fut
    try:
        await conn.send(rid, payload)
        async with asyncio.timeout(5):
            return await fut
    finally:
        pending.pop(rid, None)                  # on success, timeout and cancellation alike

Measured with 10,000 requests whose responses never arrived and a 1 ms timeout: without the finally, the map held 10,000 futures, all cancelled — a leak of one future per lost response, for the life of the connection; with it, the map was empty. The response handler must also tolerate an ID that is no longer there or a future that is already cancelled: check fut.done() before set_result, as described in avoiding InvalidStateError when setting future results, and the full correlation pattern is in correlating requests and responses with futures.

Verify: after a burst of timed-out requests, the pending map's size returns to zero.

5. Cancel timers, and know what stays behind

loop.call_later and loop.call_at return a TimerHandle; handle.cancel() stops the callback from running. Cancelling also drops the handle's references to its callback and arguments, so large arguments are freed at once — but the handle itself stays in the loop's schedule until asyncio removes it:

handle = loop.call_later(30, expire_session, session_id)
...
handle.cancel()                                  # callback and args released now

Measured with 60,000 live timers due in 60 s and 40,000 cancelled timers due in an hour, each created with a 10 KB argument: the schedule held 100,000 handles, and resident memory rose by only 17 MiB rather than the 400 MB the arguments would have needed — the cancelled handles no longer referenced them. After 30,000 more were cancelled, taking the cancelled share above half, asyncio purged the schedule to 60,000. Cancelled timers at the front of the schedule are discarded as soon as they come due; ones behind live timers wait for that purge. A service that creates and cancels a timer per request — a common way to implement timeouts — therefore carries a schedule several times its live size, which costs heap operations on every timer, but not the memory of what the timers referred to. Prefer asyncio.timeout, whose deadline is rescheduled in place, as in Timeouts & Deadlines.

Verify: per-request timers are cancelled when the request ends, and the size of the loop's timer heap stays bounded under load.

What needs handling when a future is cancelled? A decision on Where does the future live with 4 outcomes. What needs handling when a future is cancelled? Where does the future live? it has done-callbacks check cancelled() first else 'Exception in callback' several tasks await it await shield(fut) one cancel won't cancel all in a request-id map pop in finally 10,000 leaked to 0 it's a TimerHandle cancel(); args freed handle stays until purge Cancellation is an outcome every holder of the future must handle.

Verification

Future cancellation is handled correctly when:

  • cancel() return values are checked, and callbacks are not assumed to have run yet.
  • Every done-callback handles cancellation before reading a result.
  • Shared futures are awaited through shield, so one waiter cannot cancel them for all.
  • Registries of futures are cleaned in finally, and timers are cancelled when their purpose ends.

Diagnostic Hook: export the size of every future registry — pending requests per connection, in-flight batch keys — and the length of the loop's timer schedule. Registries that grow with traffic and never shrink, or a timer schedule many times larger than the number of live timeouts, show where cancelled work is being left behind.

Pitfalls & edge cases

  • result() in a callback without checking. Measured: "Exception in callback" in the logs.
  • Unshielded shared futures. Measured: one task's cancellation cancelled the future.
  • Registries without finally cleanup. Measured: 10,000 dead futures held.
  • Expecting callbacks inside cancel(). They run on the next loop iteration.

Frequently Asked Questions

Does future.cancel() run the done callbacks immediately?

No: it schedules them with call_soon, so they run on the next loop iteration. In testing, no callback had run when cancel() returned.

Why does cancelling my task cancel a future other tasks are waiting on?

A task awaiting a future is cancelled by cancelling that future. Await shared futures through asyncio.shield, which left the future pending in testing.

Why do I see 'Exception in callback' after cancelling a future?

A done-callback called result() on the cancelled future, which raises CancelledError. Check fut.cancelled() first.

Do cancelled call_later timers leak memory?

Their callbacks and arguments are released on cancel(); the handles stay in the loop's schedule until asyncio purges them. 40,000 cancelled timers with 10 KB arguments added only 17 MiB.