Skip to content

Upgrading Connections with STARTTLS in asyncio

Some protocols start in plaintext and switch to TLS on request: SMTP, IMAP and POP3 with STARTTLS, PostgreSQL's SSLRequest, LDAP's StartTLS, XMPP. Since Python 3.11, StreamWriter.start_tls upgrades an existing asyncio stream in place, on both client and server sides. The subtle part is the boundary: anything the peer sent before the handshake is plaintext, and a server that treats leftover plaintext as if it arrived over TLS is vulnerable to command injection — the class of bug behind several STARTTLS CVEs in mail servers. Measured on Python 3.14 with a minimal SMTP-style server: an honest client's upgrade took 1.08–1.23 ms, after which commands arrived over TLS as expected. A malicious client that sent STARTTLS\r\nRCPT TO:<attacker@evil.example>\r\n in one write left the injected line in the server's StreamReader buffer; when the server called start_tls, the handshake failed with SSL: RECORD_LAYER_FAILURE and the client saw ConnectionResetError — the injected command was never executed, but the failure was accidental rather than designed. With an explicit check that rejected pipelined bytes before upgrading, the server answered 501 and closed the connection cleanly. This guide upgrades connections with the boundary enforced on purpose.

Prerequisites

1. Upgrade a server-side stream

The server reads the upgrade command in plaintext, acknowledges it, and calls start_tls on its writer; the same reader and writer then carry TLS:

import asyncio
import ssl

SERVER_CTX = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
SERVER_CTX.load_cert_chain("cert.pem", "key.pem")


async def handle(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
    writer.write(b"220 ready\r\n")
    await writer.drain()
    line = await reader.readline()
    if line.strip().upper() == b"STARTTLS":
        writer.write(b"220 go ahead\r\n")
        await writer.drain()
        await writer.start_tls(SERVER_CTX)            # same reader/writer, now encrypted
        while line := await reader.readline():
            writer.write(b"250 ok\r\n")
            await writer.drain()
    writer.close()

start_tls replaces the stream's transport with a TLS transport and returns when the handshake completes; there is no new reader or writer to pass around. Measured: the client side of the upgrade took 1.08–1.23 ms on loopback, comparable to a fresh TLS connection, because it is the same handshake. Before 3.11, the same upgrade required loop.start_tls and rewiring the protocol by hand.

Verify: after the upgrade, writer.get_extra_info("ssl_object") is not None on both sides.

A STARTTLS upgrade on one connection A sequence of 6 messages between 2 participants. A STARTTLS upgrade on one connection client server 220 ready (plaintext) STARTTLS (plaintext) 220 go ahead (plaintext) TLS ClientHello ... handshake (1.1-1.2 ms) MAIL FROM:<...> (encrypted) 250 ok (encrypted) The trust boundary is the handshake; nothing before it is authenticated.

2. Upgrade a client-side stream

The client mirrors the server: send the command, wait for the acknowledgement, then upgrade with a context that verifies the server:

CLIENT_CTX = ssl.create_default_context()            # verifies against the trust store


async def send_mail(host: str, port: int = 587) -> None:
    reader, writer = await asyncio.open_connection(host, port)
    await reader.readline()                           # 220 greeting
    writer.write(b"STARTTLS\r\n")
    await writer.drain()
    reply = await reader.readline()
    if not reply.startswith(b"220"):
        raise ConnectionError(f"server refused STARTTLS: {reply!r}")
    await writer.start_tls(CLIENT_CTX, server_hostname=host)
    writer.write(b"EHLO client.example\r\n")
    await writer.drain()

server_hostname is required for certificate verification: without it the client cannot check that the certificate matches the server it meant to reach. A client should also refuse to continue in plaintext when the server does not offer or does not accept STARTTLS — silently proceeding unencrypted is the downgrade attack STARTTLS is notorious for. The shared-context advice in reusing SSL contexts in async clients applies to upgrades too.

Verify: a test server that answers STARTTLS with an error makes the client raise rather than send credentials in plaintext.

3. Reject bytes pipelined before the handshake

A client — or an attacker on the path — can send the upgrade command and more data in the same packet. The server's readline() returns the command, and the rest sits in the reader's buffer:

async def upgrade(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> bool:
    line = await reader.readline()                    # b"STARTTLS\r\n"
    # reader may still hold b"RCPT TO:<attacker@evil.example>\r\n": plaintext, unauthenticated
    if line.strip().upper() != b"STARTTLS":
        return False
    if reader._buffer:                                # private attribute; see below
        writer.write(b"501 no commands may follow STARTTLS before the handshake\r\n")
        await writer.drain()
        writer.close()
        return False
    writer.write(b"220 go ahead\r\n")
    await writer.drain()
    await writer.start_tls(SERVER_CTX)
    return True

Measured without the check: the reader buffer held the injected RCPT TO line; start_tls then failed with SSL: RECORD_LAYER_FAILURE and the attacking client saw ConnectionResetError. On Python 3.14 the leftover plaintext reached the TLS layer instead of being handed back as a post-TLS command, so the injection did not execute — but that is an implementation detail of this version, and earlier designs and other frameworks have handed such bytes to the application as if they were encrypted. With the check: a clean 501, a closed connection, and the honest client on the same server unaffected. StreamReader has no public way to inspect its buffer, so the check uses the private _buffer; pin it with a test like this one, or implement the protocol on an asyncio.Protocol that owns its own buffer and can assert it is empty before upgrading.

Verify: a test that pipelines a command after STARTTLS gets a protocol error, not a handshake failure and not a successful command.

STARTTLS with a command pipelined in the same packet A grid of 2 rows by 3 columns. STARTTLS with a command pipelined in the same packet server pipelined client honest client no buffer check handshake failed: RECORD_LAYER_FAILURE; client reset upgraded, commands over TLS rejects buffered bytes before start_tls 501, connection closed upgraded in 1.08-1.23 ms Python 3.14; the injected command never executed, by accident without the check and by design with it.

4. Bound the handshake with a timeout

A peer that sends STARTTLS and then never completes the handshake holds a connection and a task indefinitely. start_tls accepts a handshake timeout:

async def upgrade_with_timeout(writer: asyncio.StreamWriter) -> bool:
    try:
        await writer.start_tls(SERVER_CTX, ssl_handshake_timeout=10.0)
    except (ssl.SSLError, TimeoutError, ConnectionError) as exc:
        log.info("STARTTLS failed from %s: %r", writer.get_extra_info("peername"), exc)
        writer.close()
        return False
    return True

The default handshake timeout is 60 seconds, which is long for a server accepting many connections from untrusted peers. Handshake failures are routine on the internet — scanners, old clients, aborted connections — so log them at a low level and count them rather than letting the exception escape the handler, where it would be reported as an unhandled error, as with the ConnectionResetError measured above. The general timeout discipline for connections is in implementing idle timeouts for connections.

Verify: a client that connects, sends STARTTLS and goes silent is disconnected after the handshake timeout.

5. Keep authentication after the upgrade

The reason to upgrade is to protect what follows: credentials, message bodies, queries. Enforce that ordering in the protocol state, not by convention:

class Session:
    def __init__(self) -> None:
        self.tls = False
        self.authenticated = False


async def handle_command(session: Session, cmd: bytes, writer) -> None:
    verb = cmd.split(b" ", 1)[0].upper()
    if verb == b"AUTH" and not session.tls:
        writer.write(b"530 must issue STARTTLS first\r\n")
        return
    ...

After an upgrade, also discard any state learned before it — the client's announced capabilities, a HELO name — because it arrived unauthenticated; SMTP requires a fresh EHLO after STARTTLS for exactly this reason. Tracking tls and authenticated as explicit session state, rather than inferring them, is the same discipline as modelling connection lifecycles as async state machines: the commands allowed depend on the state, and the state changes only through defined transitions.

Verify: AUTH before STARTTLS is refused, and capabilities announced before the upgrade are not trusted after it.

How should this connection get TLS? A decision on What does the protocol offer with 4 outcomes. How should this connection get TLS? What does the protocol offer? implicit TLS on its own port connect with ssl= no plaintext phase at all STARTTLS only, server side reject buffered bytes, then start_tls 501, not injection STARTTLS only, client side refuse plaintext fallback no downgrade either side handshake timeout, reset pre-TLS state 10 s, fresh EHLO The safest STARTTLS is the one you do not need; when you do, enforce the boundary.

Verification

STARTTLS upgrades are safe when:

  • start_tls is used on the existing stream, with server_hostname on the client side.
  • Servers reject any bytes buffered before the handshake, and a test pins that behaviour.
  • Clients refuse to continue unencrypted when the upgrade is refused or fails.
  • Handshakes have a timeout, and state learned before TLS is discarded after it.

Diagnostic Hook: count STARTTLS outcomes by type — completed, rejected for pipelining, handshake failed, timed out. Rejections for pipelining are rare from legitimate clients; a steady stream of them is either a buggy client or an injection attempt worth investigating.

Pitfalls & edge cases

  • Leftover plaintext in the buffer. Measured: the injected line was buffered; reject it explicitly.
  • Client fallback to plaintext. It is the downgrade attack STARTTLS is known for.
  • No handshake timeout. The default is 60 seconds per stalled peer.
  • Trusting pre-TLS state. Re-negotiate capabilities after the upgrade.

Frequently Asked Questions

How do I do STARTTLS with asyncio streams?

After the plaintext negotiation, call await writer.start_tls(ssl_context) on the server or await writer.start_tls(ssl_context, server_hostname=host) on the client (Python 3.11+); the same reader and writer then carry TLS. The upgrade took about 1.1–1.2 ms in testing.

What is STARTTLS command injection?

A client or attacker sends commands in the same packet as STARTTLS; if the server treats those buffered plaintext bytes as arriving after the handshake, it executes unauthenticated commands. Reject any buffered data before upgrading.

What happens in Python 3.14 if data follows STARTTLS in the same packet?

In testing, the buffered plaintext reached the TLS layer, the handshake failed with RECORD_LAYER_FAILURE and the client got ConnectionResetError. An explicit buffer check gave a clean protocol error instead.

Should I use STARTTLS or implicit TLS?

Implicit TLS on a dedicated port, where the connection starts encrypted, avoids the plaintext phase and its downgrade and injection risks; use STARTTLS only where the protocol requires it.