Skip to content

Calling Async Code from Sync Library Callbacks

Some synchronous libraries call back into your code — a parser's per-record hook, an SDK's credential provider, a framework's event handler — and your implementation needs something that is only available as a coroutine: an async database client, an async cache, an HTTP call. A synchronous function cannot await, and the obvious ways of running a coroutine from inside one fail when the event loop is already running in the same thread. Measured on Python 3.14 with a synchronous parser that called back once per record and needed each callback's return value: calling asyncio.run(...) inside the callback raised "asyncio.run() cannot be called from a running event loop"; loop.run_until_complete(...) raised "This event loop is already running"; and asyncio.run_coroutine_threadsafe(...).result() from a callback on the loop's own thread blocked until its 2-second timeout, because the loop it was waiting for was the thread it was blocking. Running the library itself in asyncio.to_thread and bridging each callback back with run_coroutine_threadsafe processed 500 records in 0.58 s with loop lag under 1.1 ms; the bridge alone cost 34 µs per callback. When the library did not need the callback's result, scheduling tasks from the callback and awaiting them afterwards processed 500 records in 6 ms. This guide picks between those.

Prerequisites

1. See why you cannot start a loop inside the callback

When async code calls a synchronous library directly, the library — and therefore your callback — runs on the event loop's thread, inside a running loop. Neither entry point for running a coroutine works there:

def on_record(record):
    return asyncio.run(enrich(record))                  # RuntimeError
    return loop.run_until_complete(enrich(record))      # RuntimeError

async def handler():
    return SyncParser(on_record).run(500)               # the library runs on the loop thread

Measured: asyncio.run raised "asyncio.run() cannot be called from a running event loop", and run_until_complete raised "This event loop is already running". Both are deliberate: one thread can run one loop at a time, and a nested run would let other tasks execute in the middle of your synchronous call. Packages that patch asyncio to allow re-entrant loops exist, but they make every coroutine in the process subject to re-entrancy it was not written for; avoid them in services. The workable options all move something to another thread or another time.

Verify: no callback calls asyncio.run or run_until_complete, and you know which thread each library callback runs on.

2. Do not block the loop thread waiting for the loop

asyncio.run_coroutine_threadsafe schedules a coroutine on a loop from another thread and returns a concurrent.futures.Future to wait on. Called from the loop's own thread, waiting on it is a deadlock:

def on_record(record):
    future = asyncio.run_coroutine_threadsafe(enrich(record), loop)
    return future.result(timeout=2)        # blocks the loop thread; the loop cannot run enrich()

Measured: the callback blocked for exactly its 2-second timeout and then raised TimeoutError; without a timeout it would have blocked forever, freezing the whole service. The coroutine was scheduled on the loop, but the loop was stuck inside the synchronous call waiting for it. Any blocking wait for loop work from the loop's thread has this shape, as covered in avoiding deadlocks when threads wait on the loop. A defensive check catches the mistake in development:

def bridge(coro, loop):
    try:
        running = asyncio.get_running_loop()
    except RuntimeError:
        running = None
    if running is loop:
        coro.close()
        raise RuntimeError("bridge() called on the loop thread: run the library in a thread")
    return asyncio.run_coroutine_threadsafe(coro, loop).result()

Verify: the bridge refuses to wait when called from the loop's own thread.

Ways to run a coroutine inside a sync callback A grid of 5 rows by 3 columns. Ways to run a coroutine inside a sync callback approach where the callback runs result asyncio.run(coro) loop thread RuntimeError: cannot be called from a running event loop loop.run_until_complete(coro) loop thread RuntimeError: This event loop is already running run_coroutine_threadsafe(...).result() loop thread blocked 2.00 s until timeout library in to_thread + bridge worker thread 500 records in 0.58 s, lag 1.1 ms create_task in callback, await after loop thread 500 records in 0.006 s Python 3.14; each callback's async work awaits 1 ms.

3. Run the library in a thread and bridge each callback

When the library needs the callback's return value, move the library off the loop thread. Run the whole synchronous call in asyncio.to_thread; the callback then runs in that worker thread, and can block it while the loop does the async work:

async def parse_with_enrichment(n: int):
    loop = asyncio.get_running_loop()

    def on_record(record):
        future = asyncio.run_coroutine_threadsafe(enrich(record), loop)
        return future.result(timeout=5)              # blocks this worker thread, not the loop

    return await asyncio.to_thread(SyncParser(on_record).run, n)

Measured: 500 records in 0.58 s — about 1.15 ms each, of which 1 ms was the awaited work itself — with the event loop's maximum lag at 1.1 ms throughout. With a no-op coroutine, the bridge's round trip cost 34 µs per callback: scheduling onto the loop, running, and waking the waiting thread. Always pass a timeout to result(), so a stuck coroutine produces an error in the library's thread rather than a silently stuck thread pool slot. The same structure works for long-lived synchronous frameworks that own their own thread, using a loop running in a background thread, as in running an event loop in a background thread.

Verify: the library call runs in a worker thread, callbacks return correct values, and loop lag stays low during the call.

A callback bridged from a worker thread A sequence of 6 messages between 3 participants. A callback bridged from a worker thread coroutine worker thread (library) event loop to_thread(parser.run) on_record(record) run_coroutine_threadsafe(enrich) await enrich: 1 ms result (34 us bridge overhead) all records parsed Only the worker thread blocks; the loop keeps serving other tasks.

4. Defer the async work when the library does not need results

Many callbacks are notifications — "record parsed", "file uploaded", "progress changed" — whose return value the library ignores. Then the callback can stay on the loop thread and just schedule the async work, to be awaited once the synchronous call returns:

async def parse_and_store(n: int):
    loop = asyncio.get_running_loop()
    pending: list[asyncio.Task] = []

    def on_record(record):
        pending.append(loop.create_task(store(record)))    # schedule; do not wait

    SyncParser(on_record).run(n)                            # stays on the loop thread
    return await asyncio.gather(*pending)

Measured: 500 records in 6 ms, because the tasks only ran once the synchronous parse returned and then all ran concurrently. Two cautions: the synchronous call still runs on the loop thread, so it must be short — a long parse here would block the loop, in which case go back to step 3; and the list of pending tasks grows with the input, so for large inputs bound it with a semaphore or hand records to a bounded queue, as in bounding the number of live tasks.

Verify: notification callbacks never block, and every scheduled task is awaited or tracked.

5. Prefer an async-native boundary when you control the library

The bridges above are adapters around a mismatch. When you maintain the library, or can choose another, remove the mismatch: let the library accept async callbacks, or return records instead of calling back. An iterator-shaped API is the easiest to consume from both worlds:

for record in parser.records(source):       # sync caller
    handle(record)

async for record in async_records(parser, source):   # async caller, via one thread
    await handle(record)

A synchronous iterator can be adapted to an async one by pulling batches in a thread, as shown in adapting blocking iterators to async iterators, and that keeps every await in your own code rather than inside a callback the library controls. For callbacks that must accept either kind, the library side of the problem is covered in accepting sync or async callbacks.

Verify: new integrations expose iterators or async callbacks rather than sync callbacks that need async work.

How should this callback reach async code? A decision on What does the library need from the callback with 4 outcomes. How should this callback reach async code? What does the library need from the callback? a return value library in to_thread + run_coroutine_threadsafe 34 us bridge nothing (notification) create_task, await afterwards keep the sync call short you control the library iterator or async callbacks no bridge at all any never asyncio.run / block on the loop thread RuntimeError or deadlock Move the library or the work; never nest the loop.

Verification

Async work inside sync callbacks is handled correctly when:

  • No callback calls asyncio.run or run_until_complete.
  • Callbacks that need results run in a worker thread, bridged with run_coroutine_threadsafe and a timeout.
  • Notification callbacks only schedule work, which is awaited and bounded.
  • The bridge refuses to block on the loop's own thread.

Diagnostic Hook: if a service freezes completely — no requests served, no timers firing, CPU idle — take a stack dump and look for concurrent.futures result() on the loop's thread. A thread waiting on a future that only its own loop could complete is the deadlock in step 2, and it stops everything at once.

Pitfalls & edge cases

  • asyncio.run inside a callback. Measured: RuntimeError while a loop is running.
  • run_coroutine_threadsafe(...).result() on the loop thread. Measured: blocked until the timeout; forever without one.
  • Long synchronous calls on the loop thread with deferred tasks. The deferral does not unblock the loop.
  • Re-entrant loop patches. They expose all coroutines to unexpected re-entrancy.

Frequently Asked Questions

How do I call an async function from a synchronous callback?

If the callback runs in a worker thread, use asyncio.run_coroutine_threadsafe(coro, loop).result(timeout=...). If it runs on the loop thread, either run the library in asyncio.to_thread first or schedule the work with loop.create_task and await it later.

Why does asyncio.run fail inside my callback?

The callback runs inside a running event loop; asyncio.run raised "cannot be called from a running event loop" in testing.

Why does run_coroutine_threadsafe hang?

Called and waited on from the loop's own thread, it blocks the loop that would run the coroutine; it blocked until a 2 s timeout in testing.

How much does bridging from a thread to the loop cost?

About 34 µs per round trip with run_coroutine_threadsafe in testing; 500 callbacks with 1 ms of async work each took 0.58 s with loop lag under 1.1 ms.