Skip to content

Running asyncio Subprocesses with a PTY

Many programs behave differently when their output is a terminal: they flush each line, add colour, show progress bars, or refuse to run at all. Run them from asyncio with pipes and that behaviour disappears. A pseudo-terminal (PTY) restores it, with a few differences from pipes that the reading code must handle. Measured on Python 3.14 on Linux, with a child that printed a line every 0.2 s without flushing: through a pipe, all six lines arrived together at 1.01 s, when the child exited; through a PTY, they arrived at 0.01, 0.21, 0.41, 0.61, 0.81 and 1.01 s. Setting PYTHONUNBUFFERED=1 gave the pipe the same timing — for Python children only. ls --color=auto produced 0 ANSI escape sequences through a pipe and 49 through a PTY. PTY output used \r\n line endings, the child saw a terminal width of 0 until the parent set one with TIOCSWINSZ, after which it saw 132, and reading ended with OSError errno 5, Input/output error, instead of an end of file. This guide wires up a PTY and handles each of these.

Prerequisites

1. See what a pipe changes

A child writing to a pipe sees isatty() return False, and the C library and Python both switch stdout to block buffering. Measured with a Python child printing five lines 0.2 s apart and then its terminal width:

proc = await asyncio.create_subprocess_exec(
    sys.executable, "-c", CHILD, stdout=asyncio.subprocess.PIPE,
)
while line := await proc.stdout.readline():
    print(round(time.perf_counter() - start, 2), line)

All six lines arrived at 1.01 s, as the child exited and flushed its buffer, each reporting tty=False. For a progress display or a log tail that is the difference between live output and none. Where the child is Python, PYTHONUNBUFFERED=1 in its environment fixed the timing — lines at 0.01 to 1.01 s through the pipe — and stdbuf -oL does the same for many C programs. A PTY fixes it for any program, and also restores everything else that depends on a terminal.

Verify: for each program the service runs, check whether its output arrives live through a pipe, and choose environment flags or a PTY accordingly.

The same children through a pipe and a PTY A grid of 7 rows by 3 columns. The same children through a pipe and a PTY aspect pipe PTY arrival of 6 lines, 0.2 s apart all at 1.01 s 0.01, 0.21 ... 1.01 s with PYTHONUNBUFFERED=1 0.01 ... 1.01 s same child's isatty() False True ls --color=auto escape sequences 0 49 line endings \n \r\n terminal width seen by child none 0, or 132 after TIOCSWINSZ end of output EOF OSError errno 5 (EIO) Python 3.14 on Linux.

2. Connect a PTY to the child

Open a PTY pair, give the child the slave side as its standard streams, close the slave in the parent, and read the master through the event loop:

async def spawn_in_pty(*argv) -> tuple[asyncio.subprocess.Process, asyncio.StreamReader, asyncio.BaseTransport]:
    master, slave = pty.openpty()
    proc = await asyncio.create_subprocess_exec(*argv, stdin=slave, stdout=slave, stderr=slave)
    os.close(slave)                                      # the child holds its own copy

    loop = asyncio.get_running_loop()
    reader = asyncio.StreamReader()
    transport, _ = await loop.connect_read_pipe(
        lambda: asyncio.StreamReaderProtocol(reader), os.fdopen(master, "rb", buffering=0),
    )
    return proc, reader, transport

Measured with the same child: lines arrived as they were printed, 0.2 s apart, each reporting tty=True. Closing the parent's copy of the slave matters: while the parent holds it open, the master never reports the end of the child's output. stdout and stderr share the terminal, so they arrive interleaved on one stream, as they would on screen.

Verify: the child reports isatty() as True, and its output reaches the parent as it is written.

3. Handle EIO, CRLF and escape codes

PTY output differs from pipe output in three ways that reading code must expect. When the child exits and the last slave descriptor closes, Linux reports EIO on the master rather than an end of file:

async def read_lines(reader: asyncio.StreamReader):
    try:
        while line := await reader.readline():
            yield line.rstrip(b"\r\n")                 # PTY lines end with \r\n
    except OSError as e:
        if e.errno != errno.EIO:                       # EIO here means "the child closed the terminal"
            raise

ANSI = re.compile(rb"\x1b\[[0-9;?]*[ -/]*[@-~]")

def plain(line: bytes) -> str:
    return ANSI.sub(b"", line).decode(errors="replace")

Measured: reading ended with OSError errno 5, Input/output error, after the last line. Every line ended in \r\n, because the terminal's output processing translates newlines. And ls --color=auto emitted 49 escape sequences through the PTY against none through a pipe — the colour is the point when output goes to a browser terminal, and noise when it goes to a log, where it should be stripped as above or disabled with the program's own flags or NO_COLOR=1.

Verify: the reader ends cleanly at child exit, lines have no trailing \r, and logs contain no escape sequences.

ANSI escape sequences from ls --color=auto / 2 horizontal bars comparing through a pipe with the others. ANSI escape sequences from ls --color=auto / through a pipe 0 through a PTY 49 Programs decide on colour from isatty(); strip it before logging.

4. Set the terminal size

A fresh PTY reports a size of zero, and programs that format to the terminal width — tables, progress bars, pagers — may misbehave. Set rows and columns with TIOCSWINSZ before starting the child:

def set_winsize(fd: int, rows: int, cols: int):
    fcntl.ioctl(fd, termios.TIOCSWINSZ, struct.pack("HHHH", rows, cols, 0, 0))

master, slave = pty.openpty()
set_winsize(slave, 40, 132)

Measured: without it, the child's os.get_terminal_size().columns was 0; with it, 132. When the PTY is backing a terminal in a browser, call set_winsize on the master again when the user resizes, and send SIGWINCH to the child's process group so it re-reads the size, as in sending signals to subprocesses.

Verify: the child reports the width the parent set, and changes when the parent resizes.

5. Write input and stop the child

The master is also the child's keyboard. Writing to it is how the parent types, including control characters:

os.write(master_fd, b"yes\n")           # answer a prompt
os.write(master_fd, b"\x03")            # Ctrl-C: SIGINT, if the PTY is the controlling terminal

The terminal turns \x03 into SIGINT only for processes whose controlling terminal it is, and a child started with create_subprocess_exec does not get the PTY as its controlling terminal just because its standard streams point to it. Measured with a child sleeping for 5 seconds: writing \x03 echoed ^C and the child slept on, printing no signal, both with default settings and with start_new_session=True. Making the PTY the child's controlling terminal fixed it, and the child raised KeyboardInterrupt:

def take_terminal():
    os.setsid()                                    # new session, no controlling terminal yet
    fcntl.ioctl(0, termios.TIOCSCTTY, 0)           # stdin (the PTY slave) becomes it

proc = await asyncio.create_subprocess_exec(*argv, stdin=slave, stdout=slave, stderr=slave,
                                            preexec_fn=take_terminal)

preexec_fn runs in the child between fork and exec and is not safe in a process with threads; where that matters, pty.fork() in a helper process does the same setup. Writes to the master can block if the child is not reading and the terminal's input buffer is full, so for large inputs write through loop.connect_write_pipe or from a thread. When the session ends, stop the child with the escalation in sending signals to subprocesses, wait for it, and close the read transport, which closes the master.

Verify: sending \x03 interrupts a running child, and after shutdown no PTY file descriptors remain open in the parent, checked in /proc/self/fd.

Running a child behind a PTY A flow of 5 stages. Running a child behind a PTY openpty + TIOCSWINSZ e.g. 40 x 132 Start child slave as stdin, stdout, stderr Parent close the slave copy Read master strip \r and escapes EIO end of output; wait(), close master A PTY makes a program behave as it would in a terminal.

Verification

A PTY-backed child is wired up correctly when:

  • Output arrives as it is written, and the child reports a terminal.
  • The parent closes its slave copy, so end of output is detected.
  • EIO is treated as end of output, and \r\n and escape codes are handled.
  • The window size is set and updated on resize.

Diagnostic Hook: when a subprocess's output arrives in one burst at exit, check whether the child sees a terminal. Through a pipe, six lines printed over 1 second all arrived at 1.01 s; through a PTY, or with PYTHONUNBUFFERED=1 for a Python child, they arrived 0.2 s apart.

Pitfalls & edge cases

  • Expecting an end of file. The PTY master raised EIO at child exit.
  • Keeping the slave open in the parent. End of output is never detected.
  • Logging raw PTY output. Measured: 49 escape sequences from one ls.
  • Expecting Ctrl-C to work without a controlling terminal. The child ignored \x03 until TIOCSCTTY.

Frequently Asked Questions

Why does subprocess output arrive all at once in asyncio?

The child block-buffers stdout when it is a pipe. Six lines printed over 1 s arrived together at 1.01 s; a PTY or PYTHONUNBUFFERED=1 delivered them 0.2 s apart.

How do I run an asyncio subprocess with a pseudo-terminal?

Use pty.openpty(), pass the slave as stdin, stdout and stderr to create_subprocess_exec, close the slave in the parent, and read the master with connect_read_pipe.

Why do I get OSError errno 5 reading from a PTY?

On Linux, the master reports EIO once the child closes the terminal. Treat it as end of output.

Why does my PTY output contain \r\n and escape codes?

The terminal translates newlines to \r\n, and programs add colour when they see a terminal: ls added 49 escape sequences. Strip both before logging.