Skip to content

Broadcasting to Thousands of WebSocket Clients

The naive broadcast is three lines and works perfectly with fifty connections:

for ws in connections:
    await ws.send(message)

At five thousand connections it is a different program. The loop is serial, so the last client receives the message after every earlier send() has completed — and if any one of them is on a mobile link that has stopped draining, the whole fan-out stops there. Message rate collapses, memory grows, and the symptom reported is "the feed is laggy for everyone", which is technically true and diagnostically useless.

Fixing it means changing three things: serialise the payload once instead of per connection, decouple each connection with its own bounded queue and writer task, and enforce a per-connection policy so a slow client is dropped rather than allowed to set the pace. This guide builds that, measures it, and covers when a single process is no longer the right answer.

Prerequisites

  • Python 3.11+ for TaskGroup and asyncio.timeout():
pip install websockets
Serial await versus queue per connection Two columns contrasting serial broadcast with decoupled per-connection queues. Serial await versus queue per connection await in a loop 5,000 sends in series Slowest client sets the pace One frozen peer stops the fan-out Latency grows with subscriber count Memory hides in transport buffers queue + writer task 5,000 independent writers Broadcaster only enqueues A frozen peer fills its own queue Latency flat as subscribers grow Memory bounded by depth x size The broadcaster must never await any individual subscriber.

1. Serialise once, not per connection

JSON encoding the same payload 5,000 times is pure waste, and it happens on the loop thread.

import json


async def broadcast(connections: set, payload: dict) -> None:
    frame = json.dumps(payload, separators=(",", ":"))     # encode ONCE
    for ws in connections:
        ws.send_nowait(frame)                              # str, reused by all

At 5,000 connections and 200 messages per second, per-connection encoding is a million json.dumps() calls a second; encoding once makes it two hundred. Passing the same str (or bytes) to every send also lets the library skip re-encoding to UTF-8 per frame. Verify: profile a broadcast burst before and after; serialisation should disappear from the top of the profile.

2. Give every connection its own bounded queue

The fan-out must not await any single client. One queue and one writer task per connection decouples them.

import asyncio
import logging

log = logging.getLogger("broadcast")


class Subscriber:
    def __init__(self, ws, depth: int = 64) -> None:
        self.ws = ws
        self.queue: asyncio.Queue[str] = asyncio.Queue(maxsize=depth)
        self.dropped = 0
        self.task = asyncio.create_task(self._writer(), name="ws.writer")

    def offer(self, frame: str) -> bool:
        """Never awaits: the broadcaster must not be slowed by any subscriber."""
        try:
            self.queue.put_nowait(frame)
            return True
        except asyncio.QueueFull:
            self.dropped += 1
            return False

    async def _writer(self) -> None:
        try:
            while True:
                frame = await self.queue.get()
                await self.ws.send(frame)          # backpressure lands HERE only
        except Exception:                          # closed, reset, or too slow
            pass
        finally:
            await self.close()

    async def close(self) -> None:
        self.task.cancel()
        await self.ws.close()

The queue depth is now the explicit per-connection memory bound: 64 frames × 5,000 connections × 400 bytes is roughly 128 MB worst case, which you can decide to accept. Without it, the same backlog lives in the transport's send buffer where it is unbounded and invisible. Verify: stall one client and confirm its queue fills to exactly depth while broadcast throughput to everyone else is unchanged.

3. Choose the drop policy for your stream

A full queue is a decision. Three answers are defensible; one is not.

# (a) drop newest — chat, notifications: never reorder, lose the tail
if not sub.offer(frame):
    METRICS["dropped"] += 1

# (b) drop oldest — prices, positions, telemetry: the newest supersedes
if sub.queue.full():
    sub.queue.get_nowait()
sub.queue.put_nowait(frame)

# (c) disconnect — state-sync protocols where a gap is unrecoverable
if not sub.offer(frame):
    log.warning("subscriber too slow, closing")
    await sub.close()

# (d) unbounded buffering — never: converts one slow client into an OOM

For a live market feed, dropping the oldest is obviously right — nobody wants a five-second-old price. For a chat room, dropping anything is visible to users, so disconnecting and letting the client re-sync from history is usually better. Decide per stream and make it explicit. Verify: with an artificially frozen client, the chosen policy fires and is counted, and process memory stays flat.

A subscriber queue is full A decision diamond on stream recoverability with three policy cards. A subscriber queue is full Is a gap in this stream recoverable? yes, values supersede drop oldest prices, positions, telemetry yes, but order matters drop newest chat, notifications no, state would diverge disconnect and re-sync collaborative state, gameplay Unbounded buffering is the only answer that is always wrong.

4. Flush concurrently when you must await

Some designs genuinely need to know the broadcast reached everyone. Do it concurrently with per-connection deadlines, never serially.

import asyncio


async def broadcast_awaited(subs: list, frame: str, budget: float = 2.0) -> int:
    async def send_one(sub) -> bool:
        try:
            async with asyncio.timeout(budget):
                await sub.ws.send(frame)
            return True
        except (TimeoutError, Exception):
            await sub.close()                    # this one cannot keep up
            return False

    results = await asyncio.gather(*(send_one(s) for s in subs),
                                   return_exceptions=True)
    return sum(1 for r in results if r is True)

gather rather than TaskGroup matters here: a TaskGroup cancels its siblings when one child fails, so a single dead connection would abort the entire broadcast. With gather plus per-send timeouts, the slowest client costs its own budget and nothing more. Verify: with one frozen client among 500, the call still returns in about budget seconds having delivered to 499.

5. Know when one process is not enough

The single-process ceiling is real and arrives around the point where per-connection bookkeeping dominates the loop.

# Rough per-connection cost in a single Python process:
#   ~10-20 KB      transport + protocol + task overhead
#   +  queue depth x average frame size
#   + one loop callback per frame delivered
#
# 10,000 connections x 200 msg/s = 2,000,000 send callbacks/second: not feasible.
# 10,000 connections x   1 msg/s =    10,000 send callbacks/second: comfortable.

The binding constraint is messages × connections per second, not connections alone. When that product exceeds what one loop can dispatch, shard: partition connections across processes (one per core) behind a load balancer with sticky routing, and fan the broadcast out through Redis pub/sub or a message bus so every shard receives it once and delivers locally. Verify: measure loop lag during peak broadcast; if p99 lag exceeds a tenth of your acceptable delivery latency, you are at the ceiling for that process.

Send callbacks per second by shape Three bars comparing broadcast dispatch rates against a single loop ceiling. Send callbacks per second by shape 10k idle connections, 1 msg/s 10k/s — comfortable 5k connections, 20 msg/s 100k/s — at the edge 10k connections, 200 msg/s 2M/s — shard instead One loop dispatches on the order of 100k callbacks/s before lag dominates. The ceiling is messages x connections, not connections alone.

Verification

  • One slow client is contained. Freezing a client leaves broadcast latency for everyone else unchanged, and its policy (drop or disconnect) is counted.
  • Memory is bounded and predictable. Peak RSS matches connections × queue depth × frame size, not the message rate.
  • Latency scales with the product. Delivery p99 stays flat as connections grow until messages × connections approaches the loop's dispatch ceiling.

Pitfalls and edge cases

  • Awaiting each send in a loop. The slowest client sets the delivery rate for every client after it in the iteration order.
  • TaskGroup for fan-out. One failed send cancels the rest of the broadcast.
  • Unbounded per-connection queues. The memory moves from the transport to your queue and still grows without limit.
  • Serialising per connection. At thousands of connections this dominates the loop; encode once.
  • Mutating the connection set during iteration. Disconnects arriving mid-broadcast raise RuntimeError; iterate over a snapshot (list(subs)).
  • Ignoring ConnectionClosed from the writer task. Every writer needs a finally that removes the subscriber, or the set grows forever.
  • Sharding without sticky routing. A client reconnecting to a different shard loses its subscription state unless the state is external.

Frequently Asked Questions

Why does one slow WebSocket client slow down the whole broadcast?

Because the fan-out awaits each send in turn. Every client after the slow one waits for its send to complete, so delivery latency for the whole room tracks the worst connection. Decouple with a bounded per-connection queue and a dedicated writer task, so the broadcaster only enqueues and never awaits any individual client.

What queue depth should each WebSocket subscriber have?

Deep enough to absorb a normal burst, shallow enough that connections times depth times frame size fits your memory budget — 32 to 128 frames is typical. Treat the product as the real number: 5,000 connections at 64 frames of 400 bytes is about 128 MB worst case, which is a decision you make rather than a surprise you discover.

Should a slow WebSocket client be dropped or should messages be dropped?

It depends on whether a gap is recoverable. For superseding data such as prices or positions, drop the oldest queued frame and keep the client. For chat or notifications, dropping is user-visible, so disconnecting and letting the client re-sync from history is often better. For state-sync protocols where a missing message corrupts the client's view, always disconnect.

How many WebSocket connections can one Python process broadcast to?

The limit is messages times connections per second, not connections alone. A single loop comfortably handles 10,000 mostly-idle connections, but the same 10,000 at 200 messages per second means two million send callbacks per second, which no single loop will dispatch. Shard across processes and distribute the broadcast through a bus once the product approaches your loop's ceiling.