Skip to content

Running asyncio Code Inside Jupyter Notebooks

The first time most people run async code in a notebook, they paste the script's last line — asyncio.run(main()) — into a cell and get RuntimeError: asyncio.run() cannot be called from a running event loop. The kernel is itself an asyncio application: ipykernel runs an event loop for the whole life of the kernel, and every cell executes inside it. That changes three things compared with a script: you await directly instead of calling asyncio.run(), tasks you start keep running after the cell finishes, and libraries that call asyncio.run() internally break. This guide covers each, and the workaround — running such a library in a thread — that avoids patching asyncio.

Prerequisites

1. Use top-level await instead of asyncio.run

IPython's autoawait feature lets a cell contain a bare await at the top level. The cell is compiled as a coroutine and awaited on the kernel's running loop:

import asyncio
import httpx

async with httpx.AsyncClient() as client:          # top-level async with works too
    responses = await asyncio.gather(
        *(client.get(f"https://httpbin.org/delay/{i % 3}") for i in range(6))
    )
[r.status_code for r in responses]

async with and async for work at the top level as well. What you get is the same as await main() inside a script's main() — with the important difference that the loop does not stop when the cell ends.

To check what you are running on, ask the loop directly:

loop = asyncio.get_running_loop()
type(loop).__name__, loop.is_running()          # ('_UnixSelectorEventLoop', True)

Verify: a cell with await asyncio.sleep(1) completes in one second with no error, and asyncio.run(asyncio.sleep(1)) in the next cell raises the RuntimeError.

What runs where inside a Jupyter kernel 4 stacked layers. What runs where inside a Jupyter kernel your cell await, async with, async for at top level IPython autoawait compiles the cell into a coroutine kernel event loop already running, lives as long as the kernel kernel process one thread runs the loop and your cells Your code is a guest on a loop that is already running, which is why asyncio.run refuses to start another.

2. Call libraries that use asyncio.run internally

Some libraries expose a synchronous API implemented as asyncio.run(self._async_impl()). Called from a cell, they fail with the same RuntimeError, because the cell's thread already has a running loop. Run them in a worker thread, which has no loop of its own:

def sync_lib_call():
    return some_library.fetch_everything()        # internally: asyncio.run(...)

result = await asyncio.to_thread(sync_lib_call)

Verified with a stand-in library: calling it directly raised asyncio.run() cannot be called from a running event loop; calling it through asyncio.to_thread returned its result. The thread creates, uses and closes its own private loop, and the kernel's loop stays free while it runs.

The same applies to any sync wrapper built with asyncio.Runner. It does not apply to sync APIs that run their own loop on a background thread — those already work in notebooks.

Verify: the cell returns the library's result, and the kernel remains responsive to other cells while it runs.

3. Treat background tasks as kernel-lifetime objects

In a script, asyncio.run() cancels every remaining task when main() returns. In a notebook nothing ever returns: the loop runs until the kernel shuts down. A task created in a cell keeps running after the cell finishes, and keeps running if you re-execute the cell — now twice:

async def poll(interval: float = 5.0):
    while True:
        print("tick", await fetch_status())
        await asyncio.sleep(interval)

poller = asyncio.create_task(poll(), name="poller")      # keep the handle!

# a later cell
poller.cancel()

Keep a handle to every task you start, give it a name, and cancel it explicitly. To find strays started by a cell you have since re-run, list them:

[t.get_name() for t in asyncio.all_tasks() if not t.done()]

The list also contains the kernel's own tasks; your named ones are the ones to cancel. Prints from background tasks appear under whichever cell's output area is active when they print, which is confusing but harmless — prefer logging to a file or collecting results into a list for long-running pollers.

Verify: after poller.cancel(), poller.cancelled() becomes True after the next await asyncio.sleep(0), and the ticks stop.

A background task outlives the cell that created it 3 lanes over time. A background task outlives the cell that created it cells cell 1 cell 2 cell 1 re-run cancel poller 1 still polling after cell 1 ended poller 2 duplicate from the re-run time → Nothing cancels notebook tasks for you; re-running a cell is how duplicates appear.

4. Avoid nest_asyncio unless you have no choice

nest_asyncio.apply() patches asyncio so that run_until_complete() and asyncio.run() can be called while a loop is already running, by re-entering the running loop. It makes the error go away, and it is common in notebook tutorials. It also changes the semantics of every asyncio program in the process: a nested run_until_complete() runs the inner loop iteration inside a callback of the outer one, so tasks that the outer code assumed were paused can make progress mid-statement, and ordering guarantees that ordinary asyncio code relies on no longer hold. It patches asyncio internals, so it can break on new Python releases.

Approach Changes asyncio globally Kernel stays responsive Works for
top-level await no yes your own async code
await asyncio.to_thread(sync_call) no yes libraries that call asyncio.run
nest_asyncio.apply() yes, process-wide no, nested run blocks anything, with risks

Prefer the first two. If a library forces nesting on you, use it in a separate process or script rather than patching the kernel you also use for other work.

Verify: search notebooks shared in your team for nest_asyncio; each use should be replaceable by one of the first two approaches.

How to run this async code in a notebook A decision on Who calls asyncio.run with 3 outcomes. How to run this async code in a notebook Who calls asyncio.run? nobody, it is my coroutine top-level await no setup needed a library, internally await to_thread(call) it gets its own loop code that needs nesting separate process keep the kernel clean The kernel's loop is shared state; keep it unpatched.

5. Move notebook code into a script cleanly

Code written with top-level await needs one change to run as a script: wrap the cells in an async def main() and call asyncio.run(main()) once. Background tasks created in cells need an owner, because asyncio.run() will cancel them when main() returns:

import asyncio


async def main() -> None:
    async with asyncio.TaskGroup() as tg:
        tg.create_task(poll(), name="poller")
        await run_analysis()
        raise SystemExit                           # or cancel poller explicitly


if __name__ == "__main__":
    asyncio.run(main())

Notebooks hide two classes of bug that the script will expose: tasks that were never awaited (the kernel kept them alive) and resources that were never closed (the kernel never exited). Running the script with PYTHONASYNCIODEBUG=1 once surfaces both, using the techniques in finding blocking calls with asyncio debug mode.

Verify: the script exits cleanly with no "Task was destroyed but it is pending" or "Unclosed client session" warnings.

Verification

Async code is notebook-safe when:

  • No cell calls asyncio.run() or run_until_complete() directly.
  • Libraries that call asyncio.run() are invoked through asyncio.to_thread.
  • Every background task has a named handle and is cancelled before the kernel is reused for something else.
  • The kernel is not patched with nest_asyncio in shared environments.

Diagnostic Hook: add a utility cell that prints non-kernel tasks — [t for t in asyncio.all_tasks() if t.get_name().startswith(("poller", "job"))] with a naming convention — and run it before long analyses. Duplicate names mean a cell was re-run without cancelling its previous task.

Pitfalls & edge cases

  • Blocking calls in cells. A time.sleep() or a sync HTTP call in a cell blocks the kernel loop and every background task on it.
  • Interrupting a cell. The interrupt stops the awaited cell; tasks you created earlier keep running.
  • Objects bound to a loop that no longer exists. Restarting the kernel invalidates every client and pool; re-run the cells that create them.
  • Different kernels, different loops. Copying an async object between notebooks via pickling does not move its connections.

Frequently Asked Questions

Why does asyncio.run fail in Jupyter?

The Jupyter kernel already runs an asyncio event loop in the same thread as your cells, and asyncio.run refuses to start a second loop while one is running. Use top-level await in the cell instead.

How do I run async code in a Jupyter notebook?

Write await directly in a cell — IPython compiles the cell into a coroutine and awaits it on the kernel's loop. async with and async for also work at the top level.

How do I call a library that uses asyncio.run from Jupyter?

Run the library call in a worker thread with await asyncio.to_thread(call). The thread has no running loop, so the library's asyncio.run creates its own, and the kernel stays responsive.

Should I use nest_asyncio?

Avoid it where possible. It patches asyncio process-wide to allow re-entering a running loop, which changes ordering guarantees for all async code in the kernel. Top-level await and to_thread cover most cases without patching.