Skip to content

Adding TLS to asyncio Streams with SSL Contexts

Adding TLS to an asyncio stream is one keyword argument — ssl=context on open_connection() or start_server() — and the entire difficulty lies in what goes into that context. An SSLContext decides whose certificates you trust, what identity you present, whether the hostname is checked, which protocol versions are allowed and which application protocol is negotiated. Get it right and TLS is invisible: the same reader and writer, the same drain(), roughly 3–5 ms of extra handshake. Get it wrong and the failure modes range from a clear SSLCertVerificationError to a first read that returns b'' for no apparent reason. This guide builds both ends against a private certificate authority, and reproduces each failure so you can recognise it.

Prerequisites

  • Python 3.11+, standard library only. StreamWriter.start_tls() needs 3.11; earlier versions have only loop.start_tls().
  • Stream basics from Streams, Transports & Protocols.
  • A test certificate authority. Everything below was run against a locally generated CA, a localhost server certificate with a subject alternative name, and a worker-1 client certificate.
What happens inside open_connection(ssl=...) 5 ordered steps. What happens inside open_connection(ssl=...) TCP connect plain socket, no crypto yet ClientHello carries SNI and ALPN offers chain verified against load_verify_locations hostname checked against server_hostname application data the first write is encrypted A failure at any step raises from open_connection, before you have written a byte.

1. Build the two contexts

The asymmetry between client and server is expressed by Purpose. A client context trusts certificate authorities; a server context loads a key and certificate.

import ssl


def client_context(ca_file: str) -> ssl.SSLContext:
    ctx = ssl.create_default_context(cafile=ca_file)   # check_hostname=True, CERT_REQUIRED
    ctx.set_alpn_protocols(["myproto/1"])
    return ctx


def server_context(cert: str, key: str) -> ssl.SSLContext:
    ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    ctx.load_cert_chain(cert, key)                     # the identity this server presents
    ctx.set_alpn_protocols(["myproto/1"])
    return ctx

Always start from create_default_context(). A bare ssl.SSLContext(PROTOCOL_TLS_CLIENT) has no trust store, no cipher restrictions and none of the hardening the default carries, and every one of those has to be re-added by hand. Passing cafile here replaces the system trust store rather than adding to it, which is what you want for a private CA: an internal service should not accept a certificate signed by a public authority.

Wiring them into the stream APIs is unremarkable:

server = await asyncio.start_server(handle, "127.0.0.1", 8443, ssl=server_context(cert, key))
reader, writer = await asyncio.open_connection(
    "127.0.0.1", 8443, ssl=client_context(ca), server_hostname="localhost")

server_hostname matters whenever you connect by IP address or through a proxy: it is the name sent in SNI and the name checked against the certificate. Connecting to 127.0.0.1 without it fails, because the certificate names localhost.

Verify: writer.get_extra_info("ssl_object").version() returns TLSv1.3 and .cipher()[0] a modern suite — on this machine, TLS_AES_256_GCM_SHA384, after a 3.4 ms handshake.

2. Read the verification failures

Three misconfigurations produce three distinct messages, and knowing them saves an afternoon each:

# wrong name for the certificate
ssl.SSLCertVerificationError: Hostname mismatch, certificate is not valid for 'wrong.example'.

# private CA, but the client used the system trust store
ssl.SSLCertVerificationError: self-signed certificate in certificate chain

# CA certificate generated without the right extensions
ssl.SSLCertVerificationError: CA cert does not include key usage extension

All three are raised by open_connection() itself, before any application data. The third catches people generating test certificates: a CA certificate needs basicConstraints=critical,CA:TRUE and keyUsage=critical,keyCertSign, or OpenSSL 3 refuses the chain no matter how correctly you signed with it.

The temptation when any of these appear is ctx.check_hostname = False; ctx.verify_mode = ssl.CERT_NONE. That does not fix a certificate problem; it removes authentication, leaving encryption against an unverified peer — which any intermediary can supply. Fix the name or the trust store instead.

Verify: each failure raises at open_connection(), not at the first write().

The SSLContext settings that decide security A grid of 5 rows by 2 columns. The SSLContext settings that decide security setting controls the common mistake create_default_context safe defaults for the role using SSLContext() directly load_verify_locations whose certificates you trust trusting the system store only load_cert_chain the identity you present leaving out the intermediate check_hostname name matches certificate disabling it to silence an error minimum_version the oldest protocol allowed leaving 1.0 and 1.1 enabled create_default_context gets four of these right; you only add trust and identity.

3. Require client certificates for mutual TLS

Service-to-service traffic often authenticates both ends. The server adds two lines, and reads the client's identity from the negotiated ssl_object:

    ctx.verify_mode = ssl.CERT_REQUIRED
    ctx.load_verify_locations(ca_file)                 # who may present client certificates


async def handle(reader, writer):
    ssl_object = writer.get_extra_info("ssl_object")
    peer = ssl_object.getpeercert()
    common_name = dict(x[0] for x in peer["subject"])["commonName"]
    print("client:", common_name)                      # 'worker-1'

The client adds ctx.load_cert_chain(client_cert, client_key). With both sides configured, the handshake reports client=worker-1 — an identity your authorisation code can use, checked cryptographically rather than taken from a header.

Now the failure mode worth memorising. When a client with no certificate connects to this server:

  • Under TLS 1.2, client authentication is part of the handshake, so open_connection() raises ConnectionResetError immediately.
  • Under TLS 1.3 — the default — the client certificate is requested after the handshake completes. open_connection() succeeds, version() reports TLSv1.3, and then the first read returns b''.

An empty read is indistinguishable from a peer that closed the connection normally, which is why "mutual TLS works in staging and the client sees an empty response in production" is such a common report. If a TLS client gets b'' on its first read, check whether it is presenting a certificate before you look anywhere else.

Verify: run the same client against the same server with maximum_version = ssl.TLSVersion.TLSv1_2 and watch the error move from the read to the connect.

Where a missing client certificate shows up 2 columns contrasting TLS 1.2, TLS 1.3. Where a missing client certificate shows up TLS 1.2 fails during the handshake client auth is part of the handshake open_connection raises ConnectionResetError easy to attribute TLS 1.3 fails after the handshake certificate is sent post-handshake open_connection succeeds first read returns b"" looks like the peer hung up Measured on this machine: identical server, identical missing certificate, different symptom.

4. Upgrade a live connection with start_tls

Protocols like SMTP, IMAP and PostgreSQL start in plaintext and negotiate an upgrade. StreamWriter.start_tls() performs the handshake in place, keeping the same reader and writer:

# client, after the protocol-level negotiation
writer.write(b"EHLO\r\n")
await writer.drain()
assert (await reader.readline()).startswith(b"250 STARTTLS")
await writer.start_tls(client_context(ca), server_hostname="localhost")
writer.write(b"secret\n")                              # this one is encrypted
await writer.drain()

# server side, inside the handler
writer.write(b"250 STARTTLS\r\n")
await writer.drain()
await writer.start_tls(server_context(cert, key))

Measured on loopback, the upgrade took 3.9 ms on the client and 5.1 ms on the server, after which both report TLSv1.3. Two rules make it safe. Nothing sensitive may be sent before the upgrade — that traffic went out in the clear, and an attacker who can modify the stream can also strip the STARTTLS offer, which is why security-sensitive deployments require implicit TLS instead. And the client must still pass server_hostname, because there is no URL for asyncio to infer it from.

Verify: a packet capture shows the pre-upgrade lines in plaintext and everything after the upgrade as TLS records.

Upgrading a live connection with start_tls 5 stages from plaintext connect to same streams. Upgrading a live connection with start_tls plaintext connect no ssl argument client asks protocol-level server agrees last plaintext line start_tls on both handshake now same streams traffic encrypted Anything buffered before the upgrade was sent in the clear; treat it as public.

5. Bound the handshake

A TLS handshake is a network round trip, and a peer that accepts the connection but never responds will hold a task forever. Both open_connection and start_server take ssl_handshake_timeout (default 60 s):

reader, writer = await asyncio.open_connection(
    host, port, ssl=ctx, server_hostname=host, ssl_handshake_timeout=5)

Against a server that accepts and stays silent, a 0.5 s setting produced:

ConnectionAbortedError: SSL handshake is taking longer than 0.5 seconds: aborting the connection

Note the exception type: it is not TimeoutError, so a handler catching only TimeoutError will miss it. On the server side the same argument protects against clients that connect and stall — a cheap resource-exhaustion attack otherwise. There is also ssl_shutdown_timeout for the close handshake, which matters during graceful shutdown when many connections close at once.

Verify: connecting to a silent listener raises ConnectionAbortedError after the configured timeout rather than hanging.

Verification

A TLS setup is correct when:

  • Contexts come from create_default_context(), with trust and identity added rather than defaults removed.
  • Hostname checking is on: check_hostname is True and server_hostname is passed whenever connecting by IP.
  • Negotiation is as expected: ssl_object.version(), .cipher() and .selected_alpn_protocol() report what you configured.
  • Client identity is enforced where required: getpeercert() returns a subject, and a client without a certificate fails.
  • Handshakes are bounded: ssl_handshake_timeout is set on both ends and the resulting ConnectionAbortedError is handled.

Pitfalls & edge cases

  • Creating a context per connection. Building an SSLContext parses the trust store; build one at startup and share it across every connection.
  • Missing intermediate certificates. load_cert_chain wants the full chain in the certificate file. It works against a client that already has the intermediate cached, and fails against one that does not.
  • check_hostname with an IP address. The certificate must carry an IP subject alternative name; a DNS: entry will not match.
  • Blocking the loop on a handshake. The handshake itself is event-driven, but loading keys and building contexts is blocking work — do it before the loop starts, or in a thread.
  • Certificate expiry. Nothing in asyncio warns you; expiry arrives as a verification failure on every connection at once. Monitor getpeercert()["notAfter"] from a health check.
  • ssl=True on open_connection. It uses the system trust store with default settings. Fine for public HTTPS, wrong for a private CA.

Frequently Asked Questions

How do I use TLS with asyncio streams?

Build an ssl.SSLContext and pass it as the ssl argument: asyncio.open_connection(host, port, ssl=ctx, server_hostname=host) for a client, asyncio.start_server(handler, host, port, ssl=ctx) for a server. The reader and writer behave exactly as they do without TLS; only the handshake cost and the failure modes are new.

Why does my asyncio TLS client get an empty read instead of an error?

Under TLS 1.3 the server requests the client certificate after the handshake completes, so a client with no certificate connects successfully and then sees b'' on its first read. Check whether the server sets verify_mode = CERT_REQUIRED and whether your client calls load_cert_chain.

What does server_hostname do in asyncio.open_connection?

It is the name sent in the TLS SNI extension and the name the server's certificate is checked against. asyncio infers it from the host argument when you connect by name, but you must pass it explicitly when connecting by IP address or through a proxy — otherwise verification fails against a certificate that names the service.

How do I add mutual TLS to an asyncio server?

Set ctx.verify_mode = ssl.CERT_REQUIRED and ctx.load_verify_locations(ca_file) on the server context, and have clients call load_cert_chain with their certificate and key. In the handler, writer.get_extra_info("ssl_object").getpeercert() gives you the verified client subject to authorise against.

How do I upgrade a plaintext asyncio connection to TLS?

Call await writer.start_tls(context) on both ends after the protocol-level negotiation, passing server_hostname on the client. The same reader and writer continue to work, now encrypted. Remember that anything sent before the upgrade travelled in plaintext.