Skip to content

Offloading CPU Work to InterpreterPoolExecutor in Python 3.14

An asyncio service validates and transforms uploaded documents, and the transformation is pure Python that takes 100 ms of CPU per document. Running it on the loop freezes every other request. A ThreadPoolExecutor keeps the loop responsive but runs the work one core at a time, because every thread shares the interpreter's GIL. A ProcessPoolExecutor gives real parallelism at the cost of a separate process per worker: its own memory, its own imported modules, and pickled arguments crossing a pipe. Python 3.14 adds a third option in the standard library — concurrent.futures.InterpreterPoolExecutor — which runs each worker in a separate subinterpreter inside the same process, each with its own GIL. This guide wires it into asyncio, measures it against threads and processes on this kind of workload, and covers the cases where it does not deliver.

Prerequisites

Where each executor runs your function 4 stacked layers from Event loop to Process pool. Where each executor runs your function Event loop main interpreter one thread awaits futures Thread pool same interpreter one shared GIL no parallel Python Interpreter pool same process GIL per interpreter parallel Python Process pool separate processes GIL per process parallel Python Interpreters isolate like processes but live inside one process.

1. Run a CPU-bound function through the interpreter pool

The executor implements the same concurrent.futures.Executor interface as the thread and process pools, so loop.run_in_executor() works unchanged. Define the work as a module-level function — each subinterpreter imports your module independently and has no access to the calling interpreter's objects.

import asyncio
from concurrent.futures import InterpreterPoolExecutor


def burn(n: int) -> int:
    """Pure-Python CPU work: holds the GIL of whichever interpreter runs it."""
    total = 0
    for i in range(n):
        total += i * i % 7
    return total


async def main() -> None:
    loop = asyncio.get_running_loop()
    with InterpreterPoolExecutor(max_workers=8) as pool:
        results = await asyncio.gather(
            *(loop.run_in_executor(pool, burn, 3_000_000) for _ in range(8))
        )
    print(results[:2])


if __name__ == "__main__":
    asyncio.run(main())

Keep the if __name__ == "__main__": guard. Subinterpreters do not re-execute your script the way spawned processes do, but the guard keeps the module importable by both executors, which you need for the comparison in step 2 and for switching back if a dependency proves incompatible.

Verify: while the gather runs, top shows one process using roughly eight cores, not eight processes. The loop stays responsive: a concurrent asyncio.sleep(0.01) heartbeat keeps firing on time.

2. Measure it against threads and processes

Parallel speed-up depends on the workload, so measure your function rather than trusting a headline. The benchmark below runs the same eight jobs through each executor after a warm-up batch, so pool start-up is excluded.

import time
from concurrent.futures import (Executor, InterpreterPoolExecutor,
                                ProcessPoolExecutor, ThreadPoolExecutor)


def bench(executor: Executor, label: str, jobs: int = 8, n: int = 3_000_000) -> None:
    with executor:
        list(executor.map(burn, [1_000] * jobs))          # warm-up: start workers
        start = time.perf_counter()
        list(executor.map(burn, [n] * jobs))
        print(f"{label:>12}: {time.perf_counter() - start:.2f}s")


if __name__ == "__main__":
    start = time.perf_counter()
    for _ in range(8):
        burn(3_000_000)
    print(f"{'serial':>12}: {time.perf_counter() - start:.2f}s")
    bench(ThreadPoolExecutor(8), "threads")
    bench(ProcessPoolExecutor(8), "processes")
    bench(InterpreterPoolExecutor(8), "interpreters")
Eight pure-Python jobs, four ways 4 bars comparing serial, one core with the others. Eight pure-Python jobs, four ways serial, one core 0.84 s ThreadPoolExecutor(8) 0.89 s ProcessPoolExecutor(8) 0.14 s InterpreterPoolExecutor(8) 0.13 s Measured with the step 2 benchmark on a 24-core Linux host, Python 3.14, after warm-up. Shorter is faster. Threads share one GIL; processes and interpreters do not.

On a 24-core Linux machine running Python 3.14, eight pure-Python jobs took 0.84 s serially, 0.89 s on threads, 0.14 s on processes and 0.13 s on interpreters. Threads match serial execution because they share one GIL; processes and interpreters both scale across cores, and for this workload they are effectively tied.

Verify: your numbers will differ, but the ordering should hold for pure-Python CPU work: threads close to serial, processes and interpreters well below it. If interpreters look like threads, go straight to step 4.

3. Share one pool for the process lifetime

Starting subinterpreters is not free — each one initialises its own copy of the runtime and imports your modules — so creating a pool per request throws away the benefit. Create one pool at startup, size it to the cores you intend to give CPU work, and put a bound on how many jobs may queue for it.

import asyncio
import os
from concurrent.futures import InterpreterPoolExecutor

CPU_WORKERS = max(1, (os.process_cpu_count() or 2) - 1)   # leave a core for the loop


class CpuOffload:
    def __init__(self, workers: int = CPU_WORKERS, max_waiting: int = 64) -> None:
        self._pool = InterpreterPoolExecutor(max_workers=workers,
                                             thread_name_prefix="cpu-interp")
        self._slots = asyncio.Semaphore(workers + max_waiting)

    async def run(self, fn, *args):
        async with self._slots:                   # backpressure before the executor queue
            loop = asyncio.get_running_loop()
            return await loop.run_in_executor(self._pool, fn, *args)

    def close(self) -> None:
        self._pool.shutdown(wait=True, cancel_futures=True)


async def main() -> None:
    offload = CpuOffload()
    try:
        print(await offload.run(burn, 1_000_000))
    finally:
        offload.close()

The semaphore matters because an executor's internal queue is unbounded: without it, a traffic spike queues thousands of jobs in memory that will finish long after their callers have timed out. It is the same bounded-queue backpressure applied to CPU work. os.process_cpu_count() (3.13+) respects CPU affinity, which matters inside containers pinned to a subset of cores.

Verify: under a burst larger than workers + max_waiting, callers wait on the semaphore instead of growing the executor's queue, and process memory stays flat.

4. Check what your extension modules do

Subinterpreters with their own GIL require extension modules to support multi-phase initialisation and per-interpreter state. Pure Python code always qualifies; compiled extensions vary. Some refuse to import in a subinterpreter, raising ImportError, and some import but do not scale. Test the real function, not a stand-in.

import time
from concurrent.futures import InterpreterPoolExecutor, ProcessPoolExecutor


def hash_chain(n: int) -> str:
    import hashlib                         # C extension on the hot path
    digest = b"seed"
    for _ in range(n):
        digest = hashlib.sha256(digest).digest()
    return digest.hex()[:12]


def timed(executor, label: str) -> None:
    with executor:
        list(executor.map(hash_chain, [10] * 8))
        start = time.perf_counter()
        list(executor.map(hash_chain, [400_000] * 8))
        print(f"{label:>12}: {time.perf_counter() - start:.2f}s")


if __name__ == "__main__":
    timed(InterpreterPoolExecutor(8), "interpreters")
    timed(ProcessPoolExecutor(8), "processes")

On the same machine, this hash-chain workload took 0.70 s serially, 0.11 s on processes, and 0.90 s on interpreters — slower than serial. Short, frequent calls into that extension did not run in parallel across subinterpreters, while separate processes scaled normally. The lesson is not about hashlib specifically; it is that any C extension on your hot path can change the outcome, and only a measurement of the real workload tells you which executor to use.

Verify: run both executors on your real hot path. If interpreters are no faster than serial, or the import fails, keep a ProcessPoolExecutor for that function.

Which executor for this CPU work? A decision on What dominates the hot path with 3 outcomes. Which executor for this CPU work? What dominates the hot path? pure Python, scales in test InterpreterPoolExecutor one process, many GILs extension that does not scale ProcessPoolExecutor full process isolation mostly waiting on I/O threads or asyncio no extra GIL needed Decide from a measurement of the real function, then keep it configurable.

5. Make the executor a configuration choice

Because all three executors share one interface, the choice can be a deployment setting rather than a code change. That lets you switch back to processes if an upgraded dependency stops working in subinterpreters, without a redeploy of different code.

import os
from concurrent.futures import (Executor, InterpreterPoolExecutor,
                                ProcessPoolExecutor, ThreadPoolExecutor)


def make_cpu_executor(workers: int) -> Executor:
    kind = os.environ.get("CPU_EXECUTOR", "interpreters")
    if kind == "interpreters":
        return InterpreterPoolExecutor(max_workers=workers)
    if kind == "processes":
        return ProcessPoolExecutor(max_workers=workers, max_tasks_per_child=1_000)
    if kind == "threads":                       # for free-threaded builds, or debugging
        return ThreadPoolExecutor(max_workers=workers)
    raise ValueError(f"unknown CPU_EXECUTOR={kind!r}")

The threads branch is not only for debugging: on a free-threaded build, threads run Python in parallel without any isolation boundary, which is covered in evaluating free-threaded Python for CPU-bound threads.

Verify: start the service with each value of CPU_EXECUTOR and run the step 2 benchmark through make_cpu_executor; the configured backend is logged at startup and the numbers match the direct measurements.

Verification

The interpreter pool is working as intended when:

  • CPU work runs in parallel: the benchmark shows interpreters well below serial time for your real function, and a single process uses several cores.
  • The loop stays responsive: loop-lag metrics do not move while CPU jobs run; see measuring event loop lag in production.
  • One pool, bounded intake: the executor is created once, and a semaphore caps waiting jobs so memory does not track the arrival rate.
  • Extensions were tested, not assumed: every compiled dependency on the hot path has been exercised inside the pool.
  • A fallback exists: switching to ProcessPoolExecutor is a configuration change.

Pitfalls & edge cases

  • Expecting shared objects. Each subinterpreter has its own modules and objects. Module-level caches, singletons and connection objects are not shared with the main interpreter; pass data in and results out.
  • Large arguments and results. Data still crosses the isolation boundary. Sending a 200 MB structure per job erases the gain exactly as it does with processes; send a reference or a file path instead.
  • Per-request pools. Creating an InterpreterPoolExecutor inside a request handler pays interpreter start-up every time. Build it once at startup and shut it down during graceful shutdown.
  • Blocking I/O in the pool. Workers that mostly wait on sockets or files gain nothing from their own GIL; use asyncio itself or a thread pool for I/O and keep this pool for CPU.
  • Cancellation does not interrupt running work. Cancelling the awaiting coroutine abandons the result, but the job keeps running in its interpreter until it returns. Bound job size so abandoned work finishes quickly.

Frequently Asked Questions

What is InterpreterPoolExecutor in Python 3.14?

It is a concurrent.futures executor added in Python 3.14 that runs each worker in a separate subinterpreter within the same process. Each subinterpreter has its own GIL, so pure-Python CPU-bound functions can run in parallel on multiple cores, while the executor keeps the same interface as ThreadPoolExecutor and ProcessPoolExecutor.

Is InterpreterPoolExecutor faster than ProcessPoolExecutor?

For pure-Python CPU work they scale similarly; in our benchmark eight jobs took 0.13 seconds on interpreters and 0.14 on processes. Interpreters avoid separate processes, but workloads that lean on compiled extension modules can behave very differently, and one hashing workload ran slower than serial in subinterpreters while processes scaled. Measure your real function.

Can I use InterpreterPoolExecutor with asyncio?

Yes. It is a standard Executor, so pass it to loop.run_in_executor and await the result. Create one pool at startup rather than per request, limit how many jobs may wait for it with an asyncio.Semaphore, and shut it down during graceful shutdown.

Why doesn't my code speed up in InterpreterPoolExecutor?

Either the work is not CPU-bound Python, or a compiled extension on the hot path does not run in parallel across subinterpreters, or arguments and results are large enough that transfer dominates. Profile the function, test extensions inside the pool, and fall back to ProcessPoolExecutor where interpreters do not help.