Skip to content

Implementing Cooperative Cancellation in CPU Loops

Cancellation in asyncio is delivered at an await. A coroutine that computes for half a second without awaiting anything cannot be cancelled during that half second — the request is recorded and nothing happens until the next suspension point, which in a tight loop means after the work has finished. Measured below on a 0.63-second loop, a cancel requested 50 ms in took effect 648.9 ms later, which is to say: not at all. The same property starves everything else in the process; a heartbeat task ticking every 10 ms recorded zero ticks while that loop ran. Both problems have the same one-line fix, and the interesting part is sizing it.

Prerequisites

  • Python 3.11+. The loop-level behaviour described here is the same on every version.
  • Cancellation delivery from Cancellation Patterns — task.cancel() schedules a CancelledError at the task's next suspension point.
  • Offloading options from CPU-Bound Task Offloading, because chunking and offloading solve different halves of the problem.
How a cancel reaches a chunked loop 5 stages from task.cancel() to finally runs. How a cancel reaches a chunked loop task.cancel() flag set on the task chunk finishes up to one chunk of delay await sleep(0) control goes to the loop CancelledError raised at the yield finally runs checkpoint or discard Between the request and the yield, nothing at all happens: the CPU loop owns the thread.

1. Confirm the loop is uncancellable

A synchronous loop inside a coroutine holds the thread:

async def total(n: int) -> int:
    s = 0
    for i in range(n):                                 # no await anywhere
        s += i * i
    return s

Cancelling that task 50 ms into a 0.63 s run produced no cancellation at all — the loop ran to completion and the result was computed. The CancelledError would have been raised at the next await, and there was not one.

The same run with a heartbeat task awaiting asyncio.sleep(0.01) recorded 0 ticks during the loop and 34 ticks once yields were added. Everything else — health checks, timeouts, socket reads, the metrics exporter — was equally frozen. A CPU loop in a request handler is therefore both an availability problem and a latency problem, independent of whether anyone tries to cancel it.

Verify: run a heartbeat task beside your handler and count missed ticks while it works.

2. Chunk the loop and yield

Split the range and hand control back at the chunk boundary:

async def total(n: int, chunk: int = 100_000) -> int:
    s = 0
    for start in range(0, n, chunk):
        s += sum(i * i for i in range(start, min(start + chunk, n)))
        await asyncio.sleep(0)                         # a cancellation point
    return s

await asyncio.sleep(0) yields to the loop and resumes on the next iteration — the standard way to create a suspension point with no delay. Cancellation latency falls immediately:

yield interval cancellation took effect after
never 648.9 ms (ran to completion)
every 1,000,000 76.0 ms
every 100,000 5.7 ms
every 10,000 0.7 ms

The pattern is exactly what you would expect: the worst-case delay is one chunk. What the table does not show is that the event loop is now responsive throughout, which usually matters more than the cancellation itself.

Verify: cancellation latency is within one chunk's runtime, and the heartbeat ticks throughout.

How long a cancel takes to land 4 bars comparing no yielding with the others. How long a cancel takes to land no yielding 648.9 ms (never cancelled) every 1,000,000 76.0 ms every 100,000 5.7 ms every 10,000 0.7 ms Time from the cancel request until the task actually stopped, on a 0.63 s loop. Without a yield the cancel simply never arrives: the loop runs to completion.

3. Size the chunk by time, not by count

Yields are not free — each one is a trip through the scheduler. On the same 20-million-iteration loop:

yield interval overhead
every 10,000,000 +1.0%
every 100,000 +5.8%
every 1,000 +11.5%
every 100 +58.8%

Yielding every hundred iterations more than halved throughput to buy responsiveness nobody can perceive. The useful way to choose is to pick a time budget and convert:

CHUNK_SECONDS = 0.005                                  # 5 ms of work per chunk

start = time.perf_counter()
processed = 0
for item in items:
    process(item)
    processed += 1
    if time.perf_counter() - start >= CHUNK_SECONDS:
        await asyncio.sleep(0)
        start = time.perf_counter()

The time check itself costs roughly 50 ns, negligible against a 5 ms chunk, and it adapts automatically to items of varying cost — which a fixed iteration count does not. For a request-serving loop, 1–5 ms keeps p99 latency intact; for a background job, 50–100 ms costs almost nothing and is still responsive to a shutdown signal.

Verify: measure one chunk's duration under load; it should match your budget within a factor of two.

What the yields cost 4 bars comparing every 10,000,000 with the others. What the yields cost every 10,000,000 +1.0% every 100,000 +5.8% every 1,000 +11.5% every 100 +58.8% Same 20 million iteration loop; overhead is the cost of scheduling, not of the work. Yield on a time budget of a few milliseconds, not on a round iteration count.

4. Leave the partial work in a defined state

Once the loop is cancellable, it will be cancelled mid-computation, and the CancelledError propagates from the await. Anything the loop has mutated is now half-done:

async def reindex(documents, index) -> None:
    done = 0
    try:
        for batch in batched(documents, 500):
            index.add_all(batch)
            done += len(batch)
            await asyncio.sleep(0)
    except asyncio.CancelledError:
        logging.info("reindex cancelled after %d documents", done)
        index.mark_incomplete(checkpoint=done)         # record where we stopped
        raise                                          # always re-raise

Two choices to make deliberately. Whether partial results are usable — for an aggregation, usually not; for an incremental index or a resumable export, very much so, in which case a checkpoint turns cancellation into a pause. And whether cleanup can itself be interrupted: a finally that awaits can be cancelled again, so protect a critical flush with asyncio.shield as described in Cancellation Patterns.

Verify: cancel the loop mid-run and confirm the state it leaves is either clean or resumable, never silently partial.

5. Know when chunking is the wrong tool

Yielding makes a CPU loop interruptible; it does not make it concurrent. The work still runs on the loop's thread, still uses the whole core, and still delays every other task by one chunk at a time. If the loop takes seconds, it belongs off the loop entirely.

  • A thread with a stop flag. asyncio.to_thread moves the work off the loop; a threading.Event the function checks makes it stoppable, as covered in timing out blocking calls in threads. Under the GIL this still competes for the interpreter with your loop, unless the work is in a C extension that releases it.
  • A process pool. Real parallelism and a worker that can be killed outright, at the cost of pickling and a pool rebuild after a kill.
  • Neither. Work measured in microseconds does not need any of this. Chunking a loop that takes 200 µs adds overhead and complexity to fix a problem nobody has.
Three places CPU work can be made interruptible A grid of 4 rows by 2 columns. Three places CPU work can be made interruptible approach interrupts in the catch await sleep(0) in the loop one chunk the loop still owns the CPU thread + stop flag one check interval the GIL still shares one core process + kill immediately pickling and a pool rebuild no interruption at all never fine for work under a millisecond Yielding fixes responsiveness; only a process gets the work off this interpreter.

Verify: compare the loop's duration against your latency budget — if one chunk cannot fit inside it, chunking is not enough.

How big should a chunk be? A decision on How responsive must this be with 3 outcomes. How big should a chunk be? How responsive must this be? a shared request loop chunks of 1-5 ms p99 latency stays intact a background job chunks of 50-100 ms overhead near zero sub-millisecond work no chunking just call it Measure a chunk once, then divide: chunk size is a time budget, not an iteration count.

Verification

A CPU loop is cooperatively cancellable when:

  • It yields on a time budget, not on an arbitrary iteration count.
  • Cancellation lands within one chunk, measured, not assumed.
  • A concurrent heartbeat keeps ticking while the loop runs.
  • The overhead is known: a measured percentage, chosen deliberately.
  • Partial state is defined: cancellation leaves the work clean or resumable, with CancelledError re-raised.

Pitfalls & edge cases

  • await asyncio.sleep(0.001) instead of sleep(0). A non-zero sleep goes through a timer and adds at least a millisecond per chunk, often much more; sleep(0) reschedules immediately.
  • Yielding inside a lock. Every yield is a place another task can run, so a loop holding a lock now holds it across suspension points — with the deadlock risks that implies.
  • Assuming a yield reduces total CPU. It does not: the same work is done, plus scheduler overhead. What changes is who else gets to run.
  • Chunking generators lazily consumed elsewhere. If the consumer awaits between items, the yields are already there; adding more is pure overhead.
  • Numpy and C extensions. A single vectorised call cannot be chunked from Python — split the array instead, which also happens to be faster.
  • Free-threaded builds. Removing the GIL lets threads run CPU work in parallel, but a coroutine on the loop's thread still blocks that loop; chunking is still required there.

Frequently Asked Questions

Why can't I cancel a CPU-bound coroutine in asyncio?

Because cancellation is delivered at an await. A loop that never awaits has no suspension point, so the CancelledError has nowhere to be raised and the task runs to completion. Measured, a cancel requested 50 ms into a 0.63 s loop took effect only when the loop finished.

How do I make a long computation cancellable in asyncio?

Break it into chunks and put await asyncio.sleep(0) at each boundary. That creates a suspension point where cancellation can be delivered, so the worst-case delay is one chunk — 5.7 ms with chunks of 100,000 iterations in the measurements here.

How often should I yield inside a CPU loop?

Aim for a few milliseconds of work per chunk rather than a fixed iteration count. Yielding every 100,000 iterations cost about 6% on the loop measured here, while every 100 iterations cost 59% for responsiveness nobody can observe. Use a time check to adapt to items of varying cost.

Does await asyncio.sleep(0) actually let other tasks run?

Yes. It schedules the coroutine's resumption with call_soon and returns control to the loop, so every task already in the ready queue runs before it resumes. It is the standard cooperative yield, and a heartbeat task ticked 34 times during a loop where it had previously ticked zero.

Should I chunk the loop or move it to a thread?

Chunk it when the whole computation is short enough that one chunk fits inside your latency budget. Move it off the loop when the work takes seconds — a thread with a stop flag, or a process pool worker you can kill. Chunking makes a loop interruptible; it does not stop it from using the loop's CPU.