Production Python · Async & Concurrency

Concurrent Python that survives real traffic

Diagnostics-first references on event loops, worker pools, network I/O, throttling, cancellation and resilience — written for engineers operating systems under real load, not tutorials.

Start with the fundamentals Diagnose a slow loop

This knowledge base focuses on the decisions and failure modes that surface after systems reach real traffic: event loop saturation, queue backpressure, upstream rate limits, cancellation bugs, shutdown races, and cross-runtime coordination between async tasks, threads and processes. Over two hundred pages across four sections, each one built from a specific production symptom rather than an API tour, and each number in them measured on real hardware rather than quoted.

Explore the library

⚙️

Asyncio Fundamentals & Event Loop Architecture

Event loop internals, coroutines, tasks, futures, scheduling, request context with contextvars, AnyIO and trio interop — and the profiling that makes a stall explainable.

🧵

Concurrent Execution & Worker Patterns

Threads vs processes vs asyncio, free-threaded Python and subinterpreters, worker pools, queues, background job systems, caching, and rate limiting that respects upstream quotas.

🌐

Network I/O & Protocol Handling

HTTP clients and servers, HTTP/2 multiplexing, WebSockets, database drivers, message brokers, ASGI servers, gRPC, subprocesses and file I/O.

🛡️

Resilience, Cancellation & Error Handling

Timeouts, cancellation, idempotent retries, exception groups, graceful shutdown, circuit breakers, observability, leak hunting — and deterministic tests that prove them.

Start here for a specific symptom

• Latency spikes with low CPU — a blocking call is freezing the loop: finding blocking calls with asyncio debug mode, then measuring event loop lag in production so it never surprises you again.
• Memory grows until the container dies — an unbounded queue or send buffer: bounded asyncio.Queue with backpressure and drain and write backpressure in asyncio streams.
• The upstream keeps returning 429 — your limiter is above their real quota: token bucket rate limiter and handling 429 and Retry-After.
• Every deploy drops requests — the shutdown sequence is out of order: draining in-flight requests before shutdown and handling SIGTERM in asyncio services.
• One slow dependency takes everything down — nothing is contained: bulkhead isolation with per-dependency semaphores and implementing an async circuit breaker.
• Background work silently disappears — the loop only holds a weak reference: preventing task garbage collection with strong references.
• Log lines carry the wrong request ID — request data is stored per thread, not per task: propagating request IDs with contextvars and carrying contextvars across threads and executors.
• Orphaned child processes pile up — a timeout stopped the wait, not the process: running subprocesses with asyncio.create_subprocess_exec and streaming subprocess output without deadlocks.
• Retries charged the customer twice — the timeout hid a completed side effect: idempotency keys for safe async retries.
• The service dies every few hours with no error — something is leaking: tracking task growth in long-running services and detecting leaked sockets and file descriptors.
• Everything is slow but the dependencies are fine — you are saturated, not slow: measuring queue wait and service time separately and load shedding when the event loop is overloaded.
• A popular cache key expires and the database falls over — 50 concurrent misses became 50 origin calls: preventing cache stampedes in asyncio.
• The event was never published, or was published twice — a dual write with no atomicity: the transactional outbox pattern in asyncio.
• The deploy discarded work that was still running — nobody owned the task: running background tasks in FastAPI safely and building a durable job queue on Postgres.
• The async test suite is slow and flaky — it waits on the wall clock and on luck: controlling time in asyncio tests and testing asyncio code with pytest-asyncio.

What you will get

• Practical patterns for timeout, retry, cancellation, throttling and graceful shutdown behaviour.
• Trade-off guidance for selecting asyncio, threads, processes, or hybrid models, with the arithmetic behind each choice.
• Diagnostics-first examples for tracing starvation, deadlocks, contention, pool exhaustion and leaked resources.
• Production-oriented references for I/O scaling, protocol design, connection reuse and throughput tuning.

Audience

• Python engineers running web services, gateways, streaming systems and data pipelines.
• Teams modernizing legacy concurrency stacks with minimal operational risk.
• Developers who want architecture-level context and implementation-level examples in one place.

How to navigate the content

• Use the four overview pages for mental models, boundaries, and system trade-offs.
• Use the section overviews beneath them for the patterns and failure modes of one specific area.
• Use the deep-dive articles for step-by-step implementation, verification steps and diagnostic hooks.
• Follow the inline links across topics to connect a design choice to its operational behaviour.

New in this edition

• Message Brokers & Event Streams — Kafka, RabbitMQ and Redis Streams consumers verified against live brokers, plus the transactional outbox that makes publishing atomic with the database write.
• ASGI Servers & Frameworks — lifespan-owned resources, streaming that cut peak memory from 14.6 MiB to 0.9 MiB, and a uvicorn worker count derived from measurement.
• gRPC & RPC — grpc.aio services, streaming calls, interceptors, and a deadline that propagated across a two-hop chain and cancelled the leaf.
• Background Jobs & Task Queues — Celery, arq and taskiq compared on the same workload, and a Postgres job queue measured at 2,001 jobs per second.
• Async Caching & Deduplication — stampede protection that took 50 origin calls to one, and a two-tier cache that served 22,000 reads with 50.
• Observability & Tracing — loop lag, queue wait against service time, OpenTelemetry through contextvars, and logging that never blocks the loop.
• Memory & Resource Leaks — four counters, four tools, and a watchdog that names the leaking line by itself.
• AnyIO & Trio Interop — one pipeline running unchanged on asyncio and trio, with cancel scopes and memory object streams.

Suggested reading paths

• Event loop path: Asyncio Fundamentals & Event Loop Architecture → Event Loop Configuration → Task Scheduling & Lifecycle → Event Loop Debugging & Instrumentation
• Worker topology path: Concurrent Execution & Worker Patterns → Threading vs Multiprocessing vs Asyncio → Worker Pool Implementations → Rate Limiting & Throttling
• Network path: Network I/O & Protocol Handling → Async HTTP Clients & Servers → Connection Pooling & Keep-Alive → Streams, Transports & Protocols
• Event-driven path: Message Brokers & Event Streams → ASGI Servers & Frameworks → Background Jobs & Task Queues → Async Caching & Deduplication
• Operations path: Resilience, Cancellation & Error Handling → Timeouts & Deadlines → Graceful Shutdown & Signal Handling → Circuit Breakers & Bulkheads → Observability & Tracing → Memory & Resource Leaks → Testing Async Code