Production Python · Async & Concurrency
Concurrent Python that survives real traffic
This knowledge base focuses on the decisions and failure modes that surface after systems reach real traffic: event loop saturation, queue backpressure, upstream rate limits, cancellation bugs, shutdown races, and cross-runtime coordination between async tasks, threads and processes. Eighty-plus pages across four topic areas, each one built from a specific production symptom rather than an API tour.
Explore the library¶
Asyncio Fundamentals & Event Loop Architecture
Event loop internals, coroutines, tasks, futures, scheduling, synchronization — and the instrumentation that makes a stall explainable.
Concurrent Execution & Worker Patterns
Threads vs processes vs asyncio, worker pools, queues, CPU-bound offloading, and rate limiting that keeps you inside someone else's quota.
Network I/O & Protocol Handling
HTTP clients and servers, WebSockets, connection pooling, async database drivers, and building your own protocol on asyncio streams.
Resilience, Cancellation & Error Handling
Timeouts and deadlines, cooperative cancellation, retries with backoff, exception groups, graceful shutdown, circuit breakers and bulkheads.
Start here for a specific symptom¶
- Latency spikes with low CPU — a blocking call is freezing the loop: finding blocking calls with asyncio debug mode, then measuring event loop lag in production so it never surprises you again.
- Memory grows until the container dies — an unbounded queue or send buffer: bounded asyncio.Queue with backpressure and drain and write backpressure in asyncio streams.
- The upstream keeps returning 429 — your limiter is above their real quota: token bucket rate limiter and handling 429 and Retry-After.
- Every deploy drops requests — the shutdown sequence is out of order: draining in-flight requests before shutdown and handling SIGTERM in asyncio services.
- One slow dependency takes everything down — nothing is contained: bulkhead isolation with per-dependency semaphores and implementing an async circuit breaker.
- Background work silently disappears — the loop only holds a weak reference: preventing task garbage collection with strong references.
What you will get¶
- Practical patterns for timeout, retry, cancellation, throttling and graceful shutdown behaviour.
- Trade-off guidance for selecting
asyncio, threads, processes, or hybrid models, with the arithmetic behind each choice. - Diagnostics-first examples for tracing starvation, deadlocks, contention, pool exhaustion and leaked resources.
- Production-oriented references for I/O scaling, protocol design, connection reuse and throughput tuning.
Audience¶
- Python engineers running web services, gateways, streaming systems and data pipelines.
- Teams modernizing legacy concurrency stacks with minimal operational risk.
- Developers who want architecture-level context and implementation-level examples in one place.
How to navigate the content¶
- Use the four overview pages for mental models, boundaries, and system trade-offs.
- Use the section overviews beneath them for the patterns and failure modes of one specific area.
- Use the deep-dive articles for step-by-step implementation, verification steps and diagnostic hooks.
- Follow the inline links across topics to connect a design choice to its operational behaviour.
Suggested reading paths¶
- Event loop path: Asyncio Fundamentals & Event Loop Architecture → Event Loop Configuration → Task Scheduling & Lifecycle → Event Loop Debugging & Instrumentation
- Worker topology path: Concurrent Execution & Worker Patterns → Threading vs Multiprocessing vs Asyncio → Worker Pool Implementations → Rate Limiting & Throttling
- Network path: Network I/O & Protocol Handling → Async HTTP Clients & Servers → Connection Pooling & Keep-Alive → Streams, Transports & Protocols
- Operations path: Resilience, Cancellation & Error Handling → Timeouts & Deadlines → Graceful Shutdown & Signal Handling → Circuit Breakers & Bulkheads