Skip to content

Running Multiple Event Loops in Separate Threads

Most asyncio services should run exactly one event loop. There are, however, legitimate reasons to run several in one process: an embedded third-party component that insists on owning its own loop, a noisy dependency client you want isolated so its stalls cannot delay request handling, a sync application that hosts an async subsystem, or a free-threaded Python build where several loops can genuinely use several cores. Each of those turns into a debugging session the first time an object created on one loop is used from another — an asyncio.Queue whose reader wakes seconds late or never, or a future that raises got Future attached to a different loop. This guide sets up one loop per thread correctly, shows which objects are bound to their loop, uses the thread-safe entry points for communication between loops, serves one port from several loops with SO_REUSEPORT, and stops everything in order.

Prerequisites

One process, several loops 4 stacked layers from Thread loop-edge to Shared by all. One process, several loops Thread loop-edge own event loop own server socket own locks, queues Thread loop-backend own event loop own DB client own futures Between threads run_coroutine_threadsafe call_soon_threadsafe plain data only Shared by all one process one GIL (regular build) one address space Objects stay on their loop; only values move between loops.

1. Give each thread its own loop

A loop belongs to the thread that runs it. The cleanest way to create one per thread is to call asyncio.run() — or an asyncio.Runner — inside the thread's target function, so the loop's whole life, including cleanup, happens on that thread. Publish a handle to the loop only after it is running.

import asyncio
import threading
from dataclasses import dataclass, field


@dataclass
class LoopWorker:
    name: str
    loop: asyncio.AbstractEventLoop | None = None
    ready: threading.Event = field(default_factory=threading.Event)
    stop: asyncio.Event | None = None

    def run(self) -> None:
        asyncio.run(self._main())                  # the loop lives and dies on this thread

    async def _main(self) -> None:
        self.loop = asyncio.get_running_loop()
        self.stop = asyncio.Event()                # created on, and bound to, this loop
        self.ready.set()                           # publish only once the loop is running
        await self.stop.wait()


def start_worker(name: str) -> tuple[LoopWorker, threading.Thread]:
    worker = LoopWorker(name)
    thread = threading.Thread(target=worker.run, name=f"loop-{name}", daemon=True)
    thread.start()
    worker.ready.wait()
    return worker, thread

Waiting on a threading.Event before using worker.loop avoids a race in which another thread reads the attribute before the loop exists. Everything the worker needs — clients, queues, locks — should be created inside _main(), on the loop that will use them.

Verify: start two workers and print worker.loop for each; they are different objects, and threading.enumerate() shows loop-a and loop-b threads.

2. Keep loop-bound objects on their own loop

Futures, tasks, queues, locks, events, streams and nearly every async client bind to a loop when they are first used. Using them from another loop fails in one of two ways, and the quiet one is worse.

import asyncio
import threading
import time

shared: dict[str, object] = {}
ready = threading.Event()


def owner_thread() -> None:
    async def main() -> None:
        loop = asyncio.get_running_loop()
        shared["future"] = loop.create_future()
        shared["queue"] = asyncio.Queue()
        ready.set()
        await asyncio.sleep(0.5)
        shared["queue"].put_nowait("hello")          # wrong: wakes a reader on another loop
        shared["put_at"] = time.monotonic()
    asyncio.run(main())


async def foreign_loop() -> None:
    try:
        await shared["future"]
    except RuntimeError as exc:
        print("loud:", str(exc).split(" got ")[-1])  # Future ... attached to a different loop
    loop = asyncio.get_running_loop()
    loop.call_later(3.0, lambda: None)               # the only other reason this loop wakes up
    item = await shared["queue"].get()
    print(f"quiet: {item!r} delivered {time.monotonic() - shared['put_at']:.2f}s after the put")


t = threading.Thread(target=owner_thread)
t.start()
ready.wait()
asyncio.run(foreign_loop())
t.join()

Awaiting the foreign future raises immediately. The queue is subtler. The reader parks on its loop, and the writer's put_nowait() resolves the reader's waiter using the non-thread-safe call_soon() of a loop it does not run on. The reading loop is asleep in its selector and nobody wakes it, so the item sits undelivered until something else happens to wake that loop — here a timer 2.5 seconds later. Without that timer the get() would never return. No exception, no log line; only a stuck or strangely slow coroutine. Treat every asyncio object as owned by one loop, and move data between loops, never the objects.

Verify: the script prints the attached to a different loop error and then reports the item delivered about 2.5 seconds after it was put, the delay set entirely by the unrelated timer.

3. Communicate between loops with thread-safe entry points

Two calls are safe from any thread: asyncio.run_coroutine_threadsafe(coro, loop) schedules a coroutine on the target loop and returns a concurrent.futures.Future, and loop.call_soon_threadsafe(callback, *args) schedules a plain callback. Build cross-loop communication from those, and when a coroutine on one loop needs a result from another, wrap the returned future so it can be awaited without blocking.

import asyncio


async def ask(target: asyncio.AbstractEventLoop, coro) -> object:
    """Run `coro` on another loop and await its result from the current one."""
    cf_future = asyncio.run_coroutine_threadsafe(coro, target)
    return await asyncio.wrap_future(cf_future)


def post(target_queue: asyncio.Queue, target: asyncio.AbstractEventLoop, item: object) -> None:
    """Deliver an item into a queue owned by another loop."""
    target.call_soon_threadsafe(target_queue.put_nowait, item)


async def lookup_in_isolated_loop(key: str) -> str:
    await asyncio.sleep(0.01)                     # e.g. a call through a client owned by that loop
    return f"value-for-{key}"


async def request_handler(isolated: asyncio.AbstractEventLoop) -> str:
    return await ask(isolated, lookup_in_isolated_loop("user:42"))

asyncio.wrap_future() keeps the calling loop responsive while the other loop works, and cancelling the awaiting coroutine cancels the task on the target loop — the behaviour described in awaiting concurrent.futures Futures in asyncio. post() puts the put_nowait call itself on the owning loop, so the queue's waiters are woken by the loop they belong to.

Verify: calling request_handler(worker_b.loop) from worker A's loop returns value-for-user:42, and a queue owned by worker B receives items posted from worker A.

Asking another loop for a result 4 stages from loop A coroutine to loop A awaits. Asking another loop for a result loop A coroutine needs a value schedule on B threadsafe call loop B task does the work loop A awaits wrapped future Loop A keeps serving while loop B works; nothing blocks either thread.

4. Serve one port from several loops with SO_REUSEPORT

On Linux, SO_REUSEPORT lets several sockets bind the same address, with the kernel spreading incoming connections between them. Each loop thread can then run its own server for the same port.

import asyncio
import threading


def serve_on_loop(worker_id: int, port_box: dict, started: threading.Event, seconds: float) -> None:
    async def main() -> None:
        async def handle(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
            writer.write(f"served by loop {worker_id}\n".encode())
            await writer.drain()
            writer.close()

        server = await asyncio.start_server(handle, "127.0.0.1", port_box.get("port", 0),
                                            reuse_port=True)
        port_box["port"] = server.sockets[0].getsockname()[1]
        started.set()
        async with server:
            await asyncio.sleep(seconds)

    asyncio.run(main())


async def probe(port: int, attempts: int = 30) -> set[str]:
    seen = set()
    for _ in range(attempts):
        reader, writer = await asyncio.open_connection("127.0.0.1", port)
        seen.add((await reader.readline()).decode().strip())
        writer.close()
    return seen


box: dict = {}
threads = []
for n in range(3):
    started = threading.Event()
    th = threading.Thread(target=serve_on_loop, args=(n, box, started, 1.0))
    th.start()
    started.wait()
    threads.append(th)
print(sorted(asyncio.run(probe(box["port"]))))
for th in threads:
    th.join()

Thirty connections were spread across all three loops. Be precise about what this buys. On a regular CPython build the three loops still share one GIL, so Python-level request handling does not run in parallel; the gain is isolation — one loop stalled by a slow handler does not delay connections accepted by the others. On a free-threaded build the loops can use several cores. For CPU parallelism on a regular build, run several processes behind SO_REUSEPORT instead.

Verify: the probe prints three distinct served by loop N values; stalling one loop with a blocking sleep in its handler slows only the connections the kernel routed to it.

5. Stop every loop in a defined order

Daemon threads die abruptly at interpreter exit, skipping finally blocks and leaving connections half-closed. Signal each loop to finish from the controlling thread with call_soon_threadsafe, then join the threads with a timeout, stopping loops that accept work before loops that serve it.

import asyncio
import threading


def request_stop(worker: LoopWorker) -> None:
    if worker.loop is not None and not worker.loop.is_closed():
        worker.loop.call_soon_threadsafe(worker.stop.set)    # set the Event on its own loop


def shutdown(workers_in_order: list[tuple[LoopWorker, threading.Thread]], grace: float = 5.0) -> list[str]:
    stuck = []
    for worker, thread in workers_in_order:                  # e.g. edge loops before backend loops
        request_stop(worker)
        thread.join(grace)
        if thread.is_alive():
            stuck.append(worker.name)
    return stuck


workers = [start_worker("edge"), start_worker("backend")]
print("stuck loops:", shutdown(workers))

Because asyncio.run() owns each loop, returning from _main() cancels leftover tasks, shuts down async generators and the default executor, and closes the loop on its own thread. The controlling thread only has to ask and wait. Set the stop event with call_soon_threadsafe rather than calling stop.set() directly, since asyncio.Event is bound to its loop just like the queue in step 2.

Stopping several loops 5 ordered steps. Stopping several loops stop edge loops first no new work accepted join with a timeout bounded wait per thread stop backend loops finish work already accepted join with a timeout asyncio.run cleans up report stuck threads by loop name Each loop is asked on its own thread; nothing is torn down from outside.

Verify: shutdown returns an empty list, both threads have exited, and no "Task was destroyed but it is pending" warnings appear.

Verification

Multiple loops are set up correctly when:

  • Each loop is created and closed on its own thread, by asyncio.run() or a Runner inside the thread target.
  • No asyncio object crosses a loop boundary: only plain data moves between loops, through run_coroutine_threadsafe or call_soon_threadsafe.
  • Cross-loop waits are non-blocking: coroutines await wrapped futures instead of calling .result().
  • Shutdown is ordered and bounded: every loop is asked to stop, threads are joined with a timeout, and stragglers are reported by name.
  • Expectations match the build: the design relies on isolation on a GIL build and on parallelism only on a free-threaded one.

Pitfalls & edge cases

  • Creating a client at import time. A module-level client binds to whichever loop first uses it. Create clients inside each loop's main coroutine and keep them there.
  • asyncio.get_event_loop() from a thread. In a non-main thread without a running loop it raises; do not use it to "find" a loop. Pass loop references explicitly.
  • Signal handlers. loop.add_signal_handler() works only on the main thread's loop. Handle signals there and forward a stop request to the other loops.
  • Blocking cross-loop calls. Calling run_coroutine_threadsafe(...).result() from inside a coroutine blocks the calling loop, and deadlocks if the target loop is waiting on it in return. Always await the wrapped future.
  • Per-loop exception handlers. Handlers set with set_exception_handler() apply to one loop; install one on each, as described in installing a custom exception handler on the event loop.

Frequently Asked Questions

Can a Python process run more than one asyncio event loop?

Yes, one per thread. Each thread can run its own loop, typically by calling asyncio.run or using asyncio.Runner inside the thread's target function. A single thread can only run one loop at a time, and objects such as futures, queues and locks remain bound to the loop that first used them.

Why does my asyncio.Queue hang when used from another thread's loop?

The queue's waiting reader is parked on its own loop, and a put performed on a different loop schedules the wake-up without waking that loop, so get returns only when the loop happens to wake for another reason, or never, and no error is raised. Keep each queue on one loop and deliver items from other threads with loop.call_soon_threadsafe(queue.put_nowait, item) on the owning loop.

How do I call a coroutine on another event loop and await the result?

Schedule it with asyncio.run_coroutine_threadsafe(coro, target_loop), which returns a concurrent.futures.Future, then await asyncio.wrap_future(that_future) from the calling coroutine. The calling loop stays responsive, and cancelling the awaiting coroutine cancels the task running on the target loop.

Do multiple event loops in threads use multiple CPU cores?

Not for Python code on a regular CPython build, because all threads share the global interpreter lock; the benefit is isolation between loops. On a free-threaded build the loops can execute Python in parallel. For CPU parallelism on a regular build, run several processes, for example behind SO_REUSEPORT.