Skip to content

Running FastAPI Endpoints Without Blocking the Loop

FastAPI makes one decision for you that has more effect on tail latency than any other: whether your endpoint runs on the event loop or in a thread. Declare a handler async def and it runs directly on the loop, where a single synchronous call inside it freezes every other request in the process. Declare the same handler def and Starlette runs it in a bounded threadpool, where blocking is safe but concurrency is capped by that pool's size. Neither choice is wrong; choosing without knowing the rule is how services end up with 40 ms p50 and 3 s p99.

The rule is simple to state and easy to violate accidentally: async def handlers must never block, and any handler that does block should be def. This guide covers how to tell which handlers are violating it, how to fix them (offload, replace, or re-declare), how to size the threadpool that def handlers share, and how to check that dependencies and middleware are not blocking behind your back.

Prerequisites

  • Python 3.11+, FastAPI and an ASGI server:
pip install "fastapi[standard]" uvicorn httpx
Where does this handler run? A decision diamond on handler declaration with three outcome cards. Where does this handler run? Is the handler declared async def? yes on the event loop must never block, ever no, plain def in the threadpool blocking is safe and expected async def that blocks the worst case stalls every other request Starlette inspects the callable; the declaration is the routing decision.

1. Know which path each handler takes

Starlette inspects the callable: coroutine functions run on the loop, plain functions are dispatched to anyio's worker threadpool.

from fastapi import FastAPI
import time, asyncio, httpx

app = FastAPI()
client = httpx.AsyncClient()


@app.get("/good-async")            # runs ON the loop — must never block
async def good_async():
    r = await client.get("https://upstream.internal/data")
    return r.json()


@app.get("/bad-async")             # runs ON the loop — and blocks it for 300 ms
async def bad_async():
    time.sleep(0.3)                # every other request in the process waits
    return {"ok": True}


@app.get("/fine-sync")             # runs in the threadpool — blocking is expected here
def fine_sync():
    time.sleep(0.3)                # only occupies one of the pool's threads
    return {"ok": True}

The asymmetry is the point: bad_async and fine_sync do exactly the same work, but one stalls the whole process and the other costs one thread. Verify: run all three under hey -c 20 and compare p99 of an unrelated endpoint; only /bad-async should degrade it.

2. Find the blocking handlers you already have

Rather than auditing by eye, let asyncio tell you.

import asyncio, logging
from fastapi import FastAPI

app = FastAPI()


@app.on_event("startup")
async def tighten_slow_callback_threshold() -> None:
    loop = asyncio.get_running_loop()
    loop.slow_callback_duration = 0.05        # log any callback over 50 ms
    logging.getLogger("asyncio").setLevel(logging.WARNING)

Start the server with PYTHONASYNCIODEBUG=1 uvicorn app:app and drive your normal traffic: every async def handler that blocks appears as a slow-callback warning naming its source line. Common offenders are requests, psycopg2, boto3, open() on network storage, pandas/numpy transforms, template rendering, and password hashing. Verify: the warning list matches the handlers you suspect — and includes at least one you did not.

3. Fix it: re-declare, offload, or replace

Three fixes, chosen by what the handler actually does.

import asyncio
from fastapi import FastAPI
import httpx

app = FastAPI()


@app.get("/report")                 # (a) mostly-blocking handler -> plain def
def report(year: int):
    return render_heavy_report(year)          # threadpool absorbs it


@app.get("/profile/{uid}")          # (b) one blocking call inside async logic
async def profile(uid: str):
    prefs = await cache.get(uid)              # genuinely async
    legacy = await asyncio.to_thread(sdk.fetch_profile, uid)    # offloaded
    return prefs | legacy


@app.get("/upstream")               # (c) blocking client -> async client
async def upstream():
    r = await client.get("https://upstream.internal/data")      # httpx, not requests
    return r.json()

Prefer (c) whenever a maintained async client exists — it is always cheaper than a thread hop. Use (a) when the handler is dominated by synchronous work; the threadpool is exactly the right place for it. Use (b) for one stubborn call inside otherwise-async logic, as covered in running blocking SDK calls with asyncio.to_thread. Verify: after the change, the slow-callback warning for that handler disappears and unrelated p99 returns to baseline.

Which fix for which handler Four-row matrix of blocking work and its remedy. Which fix for which handler fix when mostly synchronous work declare it plain def reports, template rendering one blocking call in async logic asyncio.to_thread a legacy SDK call blocking HTTP client swap for httpx/aiohttp requests, urllib heavy CPU work process pool executor parsing, compression, hashing Replacing the library beats a thread hop; a thread beats blocking the loop.

4. Size the threadpool that def handlers share

All def handlers, plus every run_in_threadpool call, share one anyio limiter — 40 tokens by default. Once it is full, further requests queue invisibly.

import anyio.to_thread
from fastapi import FastAPI

app = FastAPI()


@app.on_event("startup")
async def size_threadpool() -> None:
    limiter = anyio.to_thread.current_default_thread_limiter()
    limiter.total_tokens = 80        # concurrent blocking handlers allowed

Size it as (requests per second through def handlers) × (their p99 duration), then check it against memory: each thread costs ~8 MB of stack plus whatever the handler allocates, so 80 threads is not free. Raising it without measuring simply moves the queue into the database's connection pool. Verify: under load on your def endpoints, active threads should stay below the limit; if they pin at it, latency is now queueing for a thread.

5. Check dependencies, middleware and lifespan too

Handlers are the obvious place to look; these three are where blocking hides.

from fastapi import Depends, FastAPI, Request
import asyncio

app = FastAPI()


def sync_dependency():             # def dependency -> threadpool (safe)
    return legacy_config_lookup()


async def async_dependency():      # async dependency -> loop (must not block!)
    return await config_store.get()


@app.middleware("http")
async def timing(request: Request, call_next):
    # Anything synchronous here runs on the loop for EVERY request.
    response = await call_next(request)
    return response


@app.on_event("startup")
async def warm() -> None:
    await asyncio.to_thread(load_model_from_disk)     # 4 s of I/O, off the loop

FastAPI applies the same def/async def rule to dependencies, so a synchronous dependency is dispatched to the threadpool automatically — but an async def dependency that calls something blocking is a loop stall on every request that uses it. Middleware always runs on the loop. And startup work that reads a large model or warms a cache blocks the loop before the server ever accepts a connection, delaying readiness rather than serving traffic. Verify: with debug mode on, no slow-callback warning names a middleware, dependency, or startup frame.

p99 of an unrelated /health endpoint under load Three bars showing how a blocking handler inflates unrelated endpoint latency. p99 of an unrelated /health endpoint under load all handlers correct 4 ms one def handler, threadpool full 120 ms one async def handler blocking 300 ms 1.4 s 20 concurrent requests against the slow endpoint; health endpoint measured separately. A blocking async handler damages endpoints that never call it.

Verification

  • Loop lag stays flat. Under load on the heaviest endpoints, p99 event loop lag stays in single-digit milliseconds.
  • Unrelated endpoints are unaffected. A fast health endpoint keeps its p99 while a slow endpoint is hammered.
  • Threads are bounded and used. Active worker threads stay under the limiter's tokens, and blocking handlers do not queue for them.

Pitfalls and edge cases

  • async def with a blocking library. The most common FastAPI performance bug by a wide margin; requests, psycopg2 and most SDK clients are synchronous.
  • Making everything async def for speed. An async def handler that blocks is strictly worse than the def version, which at least runs in a thread.
  • Ignoring the shared limiter. All def handlers plus run_in_threadpool compete for the same 40 tokens by default.
  • Blocking in middleware. It runs on the loop for every request, so its cost is multiplied by your whole traffic.
  • async def dependencies that block. They stall the loop just as handler bodies do, and are easier to overlook.
  • CPU work in the threadpool. Threads do not escape the GIL for pure Python; heavy computation belongs in a process pool.
  • Multiple uvicorn workers as a fix. More processes hide a blocking handler at low load and fail identically under high load.

Frequently Asked Questions

Should FastAPI endpoints be def or async def?

Use async def when the handler awaits genuinely asynchronous work and never calls anything blocking. Use plain def when the handler does synchronous work — Starlette then runs it in a worker threadpool, where blocking is contained to one thread. The dangerous combination is an async def handler containing a blocking call, which stalls every request in the process.

How do I find blocking calls in a FastAPI application?

Run the server with PYTHONASYNCIODEBUG=1 and set loop.slow_callback_duration to about 50 ms at startup. Every async def handler, dependency or middleware that overruns the threshold is logged with its source location. Confirm the finding by stubbing the suspect call and re-running the same load test.

How many threads does FastAPI use for def endpoints?

All def endpoints and run_in_threadpool calls share one anyio limiter with 40 tokens by default. Adjust it at startup via anyio.to_thread.current_default_thread_limiter().total_tokens, sizing from requests per second times p99 duration, and remember each thread costs roughly 8 MB of stack plus whatever the handler allocates.

Does adding more uvicorn workers fix a blocking endpoint?

No — it multiplies the number of loops that can stall. More workers raise total capacity, so the problem hides at low load, but each worker still freezes entirely while its blocking call runs. Fix the handler first; scale workers for CPU parallelism afterwards.