Skip to content

Sizing uvicorn Workers for Async Services

--workers 4 is copied from a Gunicorn tutorial more often than it is derived from a measurement, and for an async service the number frequently matters less than people expect — or matters enormously, depending entirely on what the handlers do. Two experiments on the same application make the point. For a handler that awaits 20 ms, one worker served 808 requests per second and four served 670 — extra processes made it slightly worse. For a handler that burns 20 ms of CPU, one worker served 58 rps and eight served 364, a 6.3x gain. Same server, same client, same concurrency; the only difference is where the time goes.

Prerequisites

Workers against throughput for a CPU-bound handler 4 bars comparing 1 worker with the others. Workers against throughput for a CPU-bound handler 1 worker 58 rps, p50 1,061 ms 2 workers 102 rps, p50 528 ms 4 workers 192 rps, p50 249 ms 8 workers 364 rps, p50 122 ms 50 concurrent clients against a handler that burns 20 ms of CPU on the loop. CPU-bound work scales with processes, because one loop is one core.

1. Find out where your handlers spend time

Everything downstream depends on this one measurement, and it is the step most sizing exercises skip. A handler awaiting the database is idle, and one loop can hold thousands of idle requests. A handler parsing a large JSON document, rendering a template, hashing a password or serialising a big response is using a core, and one loop can use exactly one.

The practical test is the loop lag measurement under load: if lag rises with traffic, the handlers are CPU-bound and more workers will help. If lag stays flat while latency rises, the bottleneck is downstream — a database, an upstream service, a connection pool — and more workers will only send more concurrent requests to the thing that is already slow.

The CPU-bound scaling curve measured here is close to linear:

workers throughput p50
1 58 rps 1,061 ms
2 102 rps 528 ms
4 192 rps 249 ms
8 364 rps 122 ms

Each worker is a whole interpreter with its own loop, so CPU work genuinely parallelises. The I/O-bound case did not improve at all, and its p99 got three times worse (149 ms to 432 ms) as the workers competed for the same cores.

Verify: run the same load test at one and at four workers; if throughput does not rise, the bottleneck is not CPU.

The same experiment for an I/O-bound handler 2 bars comparing 1 worker with the others. The same experiment for an I/O-bound handler 1 worker 808 rps, p99 149 ms 4 workers 670 rps, p99 432 ms Same client and concurrency; the handler awaits instead of computing. One loop already had headroom, so extra processes only added scheduling contention.

2. Count the memory before choosing the number

Each worker is a separate process with its own interpreter, imports, connection pools and caches. Measured on this application, the worker processes cost about 34 MiB each — 143.5 MiB for four, 271.8 MiB for eight, on top of a master around 26 MiB.

That is the floor. A real service adds its own per-worker overhead: a machine-learning model loaded at startup, a big in-memory cache, or a connection pool whose size is per worker. Which is the trap worth flagging explicitly — max_size=20 with 8 workers is 160 connections to the database, not 20. Divide pool sizes by the worker count, or discover the limit when the database refuses connections.

pool = await asyncpg.create_pool(DSN, max_size=POOL_SIZE_TOTAL // WORKERS)

Memory is also why a fork-based worker model does not help as much as it looks: copy-on-write shares pages at first, but Python's reference counting touches object headers, so most of the "shared" memory is unshared within minutes.

Verify: total resident memory scales roughly linearly with the worker count, and pool sizes are divided accordingly.

What decides the worker count A grid of 4 rows by 2 columns. What decides the worker count factor implication measured here handler profile CPU scales, I/O often does not 6.3x versus none cores available more workers than cores just queues per-container limits memory per worker each is a full interpreter about 34 MiB each orchestrator containers may replace workers one worker per pod Measure your own handlers: the right number follows from where the time goes.

3. Prefer replicas to workers under an orchestrator

If something already supervises processes — Kubernetes, ECS, Nomad — running a second supervisor inside the container duplicates work and loses signal:

  • Health checks become ambiguous. The orchestrator probes the container; if one of four workers is wedged, the probe may still pass.
  • Restarts are coarse. A crashed worker is invisible outside the container, while a crashed replica is a restart the orchestrator records.
  • Autoscaling loses resolution. Per-pod CPU is the scaling signal, and four workers per pod make each pod's utilisation a blend.
  • Rolling deploys get slower. Each pod must drain four workers instead of one.

The usual recommendation — one worker per container, scale with replicas — follows from that, not from performance. On a plain VM without an orchestrator, --workers is exactly the right tool, and the count should come from the measurement in step 1.

Verify: in a containerised deployment, each pod runs one worker and scaling happens by replica count.

Two ways to use more cores 2 columns contrasting --workers N, N replicas. Two ways to use more cores --workers N one container, N processes shared image and page cache one lifespan per worker a crash takes one worker no per-worker health signal N replicas N containers, one process each the orchestrator restarts them independent readiness probes per-instance metrics and logs more memory overhead Replicas are the better default under an orchestrator; workers suit a plain VM.

4. Do not use workers to hide blocking code

A worker count chosen to compensate for a blocking call is a decision to pay for the bug forever. A handler that calls requests.get() or a synchronous database driver blocks its entire loop, so each worker serves one request at a time and you need as many workers as concurrent requests — which is how an async service ends up with a thread-per-request cost model and none of the benefits.

The fix is the blocking call, not the worker count:

result = await asyncio.to_thread(legacy_client.fetch, key)   # off the loop

or an async client, or an explicit executor for heavy work as in CPU-bound task offloading. The diagnostic is straightforward: if one worker's throughput is close to 1 / handler_latency, something is serialising requests, and adding workers will scale that inefficiency linearly rather than fixing it.

Verify: a single worker's concurrency is far above one — many in-flight requests per worker at any moment.

5. Configure the rest of the worker settings deliberately

Three uvicorn settings interact with the worker count:

  • --limit-concurrency rejects requests above a threshold with 503 rather than queueing them unboundedly. It is the front-door version of load shedding, and it is per worker.
  • --timeout-keep-alive (default 5 s) governs how long idle connections are held. With many workers and many clients, idle connections are file descriptors that add up.
  • --timeout-graceful-shutdown caps how long in-flight requests have during a deploy. Without it, one long request delays every rolling restart.

Also remember that the lifespan runs once per worker. Anything that must happen once per deployment — schema migrations, a cache warm that another worker can share, registering a singleton — must not live there, or four workers will do it four times, concurrently.

Verify: the concurrency limit, keep-alive and shutdown timeouts are set explicitly rather than left at defaults you have not read.

How many workers should this service run? A decision on What runs the service with 3 outcomes. How many workers should this service run? What runs the service? Kubernetes or similar one worker per pod scale replicas instead a VM, CPU-bound handlers workers near core count measure, then set a VM, I/O-bound handlers start at 2, measure often no gain past that Containers already have a process supervisor; a second one inside the pod rarely helps.

Verification

The worker count is right when:

  • It follows a measurement, not a formula copied from a tutorial.
  • CPU-bound scaling is confirmed: throughput rises roughly linearly with workers.
  • Memory is budgeted: total RSS and per-worker pool sizes both account for the count.
  • Blocking code is fixed, not compensated for with more processes.
  • Containers run one worker, with scaling by replica.
  • Per-worker settings are explicit, including concurrency and shutdown timeouts.

Pitfalls & edge cases

  • Pool sizes not divided by workers. max_size=20 with 8 workers opens up to 160 connections.
  • --reload in production. It runs a file watcher and is single-worker; it is a development tool only.
  • Workers plus threads plus asyncio. Three concurrency mechanisms multiply; cap the executor per worker too.
  • In-process state. Caches, rate limiters and circuit breakers are per worker, so eight workers have eight independent views — see sharing circuit breaker state.
  • More workers than cores. They do not add throughput and they do add context switching and memory.
  • Benchmarking from one client machine. The load generator becomes the bottleneck; if throughput does not move, check the client before the server.

Frequently Asked Questions

How many uvicorn workers should I run?

It depends entirely on whether handlers are CPU-bound. For a handler burning 20 ms of CPU, throughput scaled from 58 rps at one worker to 364 at eight. For a handler awaiting 20 ms, one worker served 808 rps and four served 670 — no gain at all. Measure your own handlers before choosing.

Does adding uvicorn workers make an async app faster?

Only if a single event loop is the bottleneck, which means CPU work in handlers. If the constraint is a database or an upstream service, more workers send more concurrent requests to the same slow thing and make tail latency worse.

How much memory does each uvicorn worker use?

Each is a full interpreter. Measured on a small Starlette app, about 34 MiB per worker — 271.8 MiB for eight — before the application's own caches, models or pools. Remember connection pool sizes are per worker, so divide the intended total by the worker count.

Should I use uvicorn workers or more container replicas?

Under an orchestrator, replicas: they give per-instance health checks, restarts, metrics and autoscaling resolution that workers inside one container hide. On a plain VM, workers are exactly the right tool, sized from a measurement.

Can more workers fix blocking code in an async handler?

They will hide it at a linear cost. A blocking call means one request per worker at a time, so you need as many workers as concurrent requests. Move the call off the loop with asyncio.to_thread or an async client instead.