Skip to content

Inspecting Tasks with python -m asyncio ps and pstree

Before Python 3.14, seeing what an asyncio service's tasks were doing meant code inside the process — a signal handler, an admin endpoint — written before the incident. 3.14 adds two commands that read the task graph from outside, like a debugger: python -m asyncio ps <pid> prints a table of every task, its coroutine stack and what it is awaiting, and python -m asyncio pstree <pid> prints the same information as a tree of who awaits whom. Run against a service with three request tasks stuck in a slow query and a consumer waiting on an event, ps listed all five tasks with sleep -> db_query -> handle as the request stacks, and pstree showed the four children hanging off the TaskGroup in main. The first attempt failed with Failed to find the PyRuntime section in process … on Linux platform — which turned out to mean "not allowed to read that process", not anything about the build. This guide covers both commands and the permission they actually need.

Prerequisites

  • Python 3.14 for both the target and the inspecting interpreter, same version.
  • Linux or macOS with permission to read the target's memory (details in step 3).
  • Named tasks, from naming and tracking tasks for observability — the output is far more readable with names.

1. Run ps against a live process

A small service: three request handlers stuck in a slow query and a consumer waiting on an event, all under one TaskGroup.

import asyncio


async def db_query(i):
    await asyncio.sleep(3600)


async def handle(i):
    await db_query(i)


async def consumer():
    await asyncio.Event().wait()


async def main():
    async with asyncio.TaskGroup() as tg:
        for i in range(3):
            tg.create_task(handle(i), name=f"request-{i}")
        tg.create_task(consumer(), name="orders-consumer")


asyncio.run(main())

python3.14 -m asyncio ps 3699861

tid      task id         task name        coroutine stack              awaiter chain                                     awaiter name  awaiter id
-------------------------------------------------------------------------------------------------------------------------------------------------
3699861  0x78d43d345130  Task-1           TaskGroup._aexit -> TaskGroup.__aexit__ -> main                                                0x0
3699861  0x78d43d345310  request-0        sleep -> db_query -> handle  TaskGroup._aexit -> TaskGroup.__aexit__ -> main   Task-1        0x78d43d345130
3699861  0x78d43c9403f0  request-1        sleep -> db_query -> handle  TaskGroup._aexit -> TaskGroup.__aexit__ -> main   Task-1        0x78d43d345130
3699861  0x78d43c940790  request-2        sleep -> db_query -> handle  TaskGroup._aexit -> TaskGroup.__aexit__ -> main   Task-1        0x78d43d345130
3699861  0x78d43d496c10  orders-consumer  Event.wait -> consumer       TaskGroup._aexit -> TaskGroup.__aexit__ -> main   Task-1        0x78d43d345130

Each row is one task: its name, its coroutine stack innermost-first, and the task awaiting it with that task's own stack. Three requests parked in sleep -> db_query is the kind of pattern you are looking for — in a real service, that column would show acquire -> Pool.acquire -> fetch or similar, naming the resource everyone is queued on.

Verify: every task you expect appears, with the names you gave it.

How ps reads tasks from another process A flow of 4 stages. How ps reads tasks from another process inspector process python3.14 -m asyncio ps PID read target memory needs ptrace rights walk tasks + frames no code runs in the target table or tree per task, with awaiters Nothing executes inside the target, which is why this works even when its loop is blocked.

2. Read the tree with pstree

pstree prints the same data shaped as the await graph, which is easier to read when tasks are nested:

└── (T) Task-1
    └──  main svc.py:7
        └──  TaskGroup.__aexit__ asyncio/taskgroups.py:72
            └──  TaskGroup._aexit asyncio/taskgroups.py:121
                ├── (T) request-0
                │   └──  handle svc.py:4
                │       └──  db_query svc.py:3
                │           └──  sleep asyncio/tasks.py:702
                ├── (T) request-1
                │   └── …
                └── (T) orders-consumer
                    └──  consumer svc.py:5
                        └──  Event.wait asyncio/locks.py:213

(T) marks a task; the lines beneath are its coroutine frames with file and line. The tree makes structured concurrency visible: everything hangs off the TaskGroup in main, so cancelling Task-1 would cancel the lot. Tasks created with bare create_task and never awaited appear as separate roots — a quick way to spot fire-and-forget work, the subject of handling exceptions in fire-and-forget tasks.

Because the tool reads memory rather than running code in the target, it works even when the target's loop is blocked in a synchronous call — the case where an in-process signal handler cannot run.

Verify: the tree's roots are the tasks you expect to be top-level; unexpected roots are untracked tasks.

3. Grant the ptrace permission it needs

Both commands read the target with the same mechanism a debugger uses. On most Linux distributions the Yama security module restricts that: with /proc/sys/kernel/yama/ptrace_scope set to 1 (the common default), a process may only read its own descendants. Run from another shell, the command fails — and the error is misleading:

Error retrieving tasks: Failed to find the PyRuntime section in process 3700302 on Linux platform

The interpreter build was fine — the section exists in the binary — the inspector simply could not read the target's memory. Options, from most to least contained:

# 1. The target opts in to being read by any process of the same user (Linux prctl)
import ctypes
PR_SET_PTRACER, PR_SET_PTRACER_ANY = 0x59616D61, ctypes.c_ulong(-1)
ctypes.CDLL(None).prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY, 0, 0, 0)

# 2. Run the inspector with CAP_SYS_PTRACE (root, or a debug container with the capability)
sudo python3.14 -m asyncio ps 3700302

# 3. Relax Yama host-wide (affects every process; usually not acceptable in production)
sudo sysctl kernel.yama.ptrace_scope=0

Verified: with option 1 in the target, the same unprivileged ps command that had failed printed the full table. Opting in only from staging or behind a flag keeps production hosts at the stricter default.

Verify: cat /proc/sys/kernel/yama/ptrace_scope on your hosts, then test the command against a staging process before an incident.

Why does ps say it cannot find PyRuntime? A decision on What blocked the read with 3 outcomes. Why does ps say it cannot find PyRuntime? What blocked the read? ptrace_scope=1, not the parent permission opt in, or CAP_SYS_PTRACE PID is a wrapper or shell wrong process check /proc/PID/exe inspector version differs layout mismatch use the same 3.14.x The error names a symptom of the failed read, not its cause; check permission first.

4. Make it work in containers

In Kubernetes and Docker, the inspector usually cannot see the target's PID namespace at all. Two arrangements work:

# Kubernetes: an ephemeral debug container sharing the pod's process namespace
kubectl debug -it pod/api-7d9f --image=python:3.14 --target=app \
  --profile=general -- python -m asyncio pstree 1

# Docker: run the inspector in the target's PID namespace with the capability
docker run --rm -it --pid=container:api --cap-add=SYS_PTRACE python:3.14 \
  python -m asyncio ps 1

The inspector image must have the same Python minor version as the target. PID 1 inside the namespace is usually the app process, but check — a shell entrypoint or an init process may be PID 1, and pointing the tool at the wrapper produces the same misleading error, because a shell has no Python runtime to find. That exact mistake happened while preparing this guide: the first PID tried belonged to a bash wrapper.

Verify: run the command from a debug container in staging and confirm the output names your tasks.

5. Keep an in-process fallback

External inspection is the best tool when it is available and the worst one to discover is unavailable mid-incident. Keep a cheap in-process path that uses the same data, on 3.14's print_call_graph:

import asyncio
import signal
import sys


def dump_tasks() -> None:
    for task in asyncio.all_tasks():
        asyncio.print_call_graph(task, file=sys.stderr)


async def main() -> None:
    asyncio.get_running_loop().add_signal_handler(signal.SIGUSR2, dump_tasks)
    await serve()

The in-process version needs the loop to be responsive, while ps does not; ps needs ptrace rights, while the signal handler does not. Between them every case is covered. The full set of dumping techniques, including thread dumps for a blocked loop, is in dumping stacks of a hung asyncio program.

Verify: both paths produce matching task lists for the same process.

External ps versus an in-process dump A grid of 4 rows by 3 columns. External ps versus an in-process dump property asyncio ps / pstree in-process call graph dump needs code in the target no a signal handler or endpoint needs ptrace rights yes no works with a blocked loop yes no Python version 3.14+, matching 3.14+ (cr_await walk before) Each covers the other's blind spot; have both ready.

Verification

Task inspection is ready when:

  • ps and pstree run successfully against a staging process, with the permission model you will use in production.
  • Tasks are named, so the output identifies handlers, consumers and background jobs.
  • Containers have a documented debug path — an ephemeral container or a sidecar with SYS_PTRACE.
  • An in-process dump exists for environments where ptrace is unavailable.

Diagnostic Hook: during an incident, pipe ps output through awk to count tasks by their coroutine stack column; the most common stack is usually the bottleneck. Capture the output to the incident record — it is a snapshot of every task's position at that moment, which is exactly what is lost when the process is restarted.

Pitfalls & edge cases

  • The misleading PyRuntime error. Check ptrace permission and the PID's executable before suspecting the build.
  • Mismatched inspector version. Use the same 3.14.x as the target; internal layouts can change between releases.
  • Very large task counts. Output for tens of thousands of tasks is large; aggregate rather than read.
  • Assuming the snapshot is atomic. Tasks may change while being read; treat rare inconsistencies as noise.

Frequently Asked Questions

What does python -m asyncio ps do?

Added in Python 3.14, it reads another running Python process's memory and prints one row per asyncio task: task name and id, its coroutine stack, and the task awaiting it. pstree shows the same data as a tree of awaiting relationships.

Why does asyncio ps say 'Failed to find the PyRuntime section'?

Usually because the inspector is not allowed to read the target's memory, most often Linux's Yama ptrace_scope=1 blocking a non-parent process. Pointing it at a shell wrapper instead of the Python process gives the same error. Grant ptrace rights or use the right PID.

Can I use asyncio ps inside Kubernetes?

Yes, from a container that shares the pod's process namespace and has ptrace rights, such as an ephemeral debug container created with kubectl debug --target. The inspector needs the same Python version as the application.

Does asyncio ps work if the event loop is blocked?

Yes. It reads memory from outside and runs no code in the target, so it works even when the loop is stuck in a synchronous call.