Skip to content

Sending Binary and MessagePack over WebSockets

Switching a WebSocket feed from JSON text frames to MessagePack binary frames is often proposed to make messages smaller and faster. Measured, the two claims came apart. On Python 3.14 with websockets 17.1 and msgpack 1.2.3, streaming 20,000 order-book messages — a few header fields and 40 price levels of [price, size] floats — from a server to a client over loopback: without compression, JSON ran at 25,001 messages per second at 803 bytes each on the wire, and msgpack at 72,691 per second at 827 bytes — almost three times faster, and 3% larger, because msgpack stores every float as 9 bytes while short decimals in JSON take fewer. With permessage-deflate enabled, JSON shrank to 268 bytes and msgpack only to 431, and both slowed to about 18,500–19,400 per second. Packing the price levels into a float32 array behind a msgpack header gave 371 bytes uncompressed at 77,907 per second. This guide measures the choice for your own messages.

Prerequisites

1. Measure size and codec cost on real messages

Encoding choice depends on what the messages contain. Measure with a representative sample before changing protocols:

def tick(i):
    return {"type": "book", "symbol": "BTC-USD", "seq": 1_000_000 + i, "ts": 1791000000.123 + i / 1000,
            "bids": [[round(60000 - k * 0.5 + r.random(), 2), round(r.random() * 3, 4)] for k in range(20)],
            "asks": [[round(60000 + k * 0.5 + r.random(), 2), round(r.random() * 3, 4)] for k in range(20)]}

as_json = json.dumps(msg, separators=(",", ":"))       # compact separators matter
as_msgpack = msgpack.packb(msg)

Measured over 20,000 messages: compact JSON averaged 799 bytes and msgpack 823. Encoding took 28–30 µs per message with json.dumps and 5.5–6.2 µs with msgpack.packb; decoding took about 40 µs with json.loads and 15–32 µs with msgpack.unpackb across runs. Prices like 60000.37 are 8 characters of JSON but always 9 bytes as a msgpack double; messages dominated by integers and booleans favour msgpack, messages dominated by short decimals do not.

Verify: average encoded size and per-message encode and decode time are measured for a sample of production messages in each format.

20,000 order-book messages, server to client over loopback A grid of 6 rows by 5 columns. 20,000 order-book messages, server to client over loopback format deflate msg/s bytes on wire CPU per message JSON text frames off 25,001 803 40.0 us JSON text frames on 18,468 268 54.1 us msgpack binary frames off 72,691 827 13.8 us msgpack binary frames on 19,362 431 51.6 us msgpack header + float32 arrays off 77,907 371 12.8 us msgpack header + float32 arrays on 30,585 286 32.6 us websockets 17.1, msgpack 1.2.3; CPU for server and client together in one process.

2. Send binary frames for msgpack

WebSocket frames are either text, which must be valid UTF-8, or binary. In websockets, sending bytes produces a binary frame and str a text frame, and received messages come back as the same type:

async def feed(ws):
    async for update in updates():
        await ws.send(msgpack.packb(update))           # bytes -> binary frame

async def consume(uri):
    async with connect(uri) as ws:
        async for raw in ws:                            # bytes for binary frames
            handle(msgpack.unpackb(raw))

Measured over loopback without compression: msgpack binary frames ran at 72,691 messages per second against 25,001 for JSON text frames, with 13.8 against 40.0 µs of CPU per message for both ends together. The speed came from encoding and decoding, not from size — the msgpack messages were slightly larger. Browser clients receive binary frames as Blob or ArrayBuffer and need a msgpack library in JavaScript, which is a dependency JSON does not need. Version the format — a first byte or a field — so the two ends can change it later.

Verify: binary-frame clients decode every message, and a format version is checked on connect.

3. Check what compression does to each format

websockets negotiates permessage-deflate by default, with a compression context kept across messages, so repeated keys and similar values compress well. Measured: JSON shrank from 803 to 268 bytes per message, msgpack from 827 to 431. Text compresses better than binary doubles, and the shared context removed JSON's repeated keys almost entirely. Compression also cost CPU: throughput fell to 18,468 for JSON and 19,362 for msgpack, erasing msgpack's speed advantage.

server_off = serve(handler, "0.0.0.0", 8765, compression=None)          # off
server_on = serve(handler, "0.0.0.0", 8765, compression="deflate")      # websockets' default

So the choice is a pair: JSON with deflate gave the smallest messages; msgpack without deflate gave the highest throughput. Which matters depends on whether bandwidth to clients — mobile, metered — or server CPU is the scarcer resource.

Verify: bytes on the wire are measured with the compression setting actually negotiated with real clients, not just with the encoder's output.

Bytes on the wire per message 6 horizontal bars comparing msgpack, no deflate with the others. Bytes on the wire per message msgpack, no deflate 827 B JSON, no deflate 803 B msgpack, deflate 431 B msgpack + float32, no deflate 371 B msgpack + float32, deflate 286 B JSON, deflate 268 B Text with shared-context deflate was the smallest.

4. Pack numeric arrays when size and speed both matter

For numeric payloads, a format chosen for the data beats a general one. Keep a small msgpack header for the fields and send the price levels as a fixed-layout float32 array:

def pack_book(m: dict) -> bytes:
    header = msgpack.packb({k: v for k, v in m.items() if k not in ("bids", "asks")})
    levels = [x for level in m["bids"] + m["asks"] for x in level]
    return struct.pack("<H", len(header)) + header + struct.pack(f"<{len(levels)}f", *levels)

def unpack_book(b: bytes) -> dict:
    n = struct.unpack_from("<H", b)[0]
    m = msgpack.unpackb(b[2:2 + n])
    flat = struct.unpack_from(f"<{(len(b) - 2 - n) // 4}f", b, 2 + n)
    pairs = list(zip(flat[0::2], flat[1::2]))
    m["bids"], m["asks"] = pairs[:len(pairs) // 2], pairs[len(pairs) // 2:]
    return m

Measured: 371 bytes per message uncompressed and 77,907 messages per second, both better than plain msgpack; 286 bytes with deflate, at 30,585 per second. float32 has about seven significant digits: a round trip turned 60000.13 into 60000.12890625, within 0.002 of every original value in the message — enough to round back to cents, and no finer. Check the precision the data needs, or use float64 at twice the size, or integers in the smallest price unit.

Verify: a round trip through pack_book and unpack_book reproduces every field within the precision the application requires.

5. Keep the format negotiable

Whatever format wins today, clients will lag behind servers. Let the client ask for a format, and default to the most widely supported:

FORMATS = {"json": lambda m: json.dumps(m, separators=(",", ":")),
           "msgpack": msgpack.packb}

async def handler(ws):
    fmt = ws.request.headers.get("X-Feed-Format", "json")
    encode = FORMATS.get(fmt, FORMATS["json"])
    async for update in updates():
        await ws.send(encode(update))

A WebSocket subprotocol — Sec-WebSocket-Protocol, set with subprotocols=["feed.v2.msgpack", "feed.v1.json"] in websockets — is the standard way to negotiate this during the handshake, and browsers support it. Measure per format in production, since the right answer depends on the message mix and the clients' links. For broadcasting the same update to many clients, encode once per format rather than once per client, as in broadcasting to thousands of WebSocket clients.

Verify: old clients that request JSON keep working after a server adds a binary format, and each update is encoded once per format.

Picking a WebSocket encoding A decision on What is scarce with 3 outcomes. Picking a WebSocket encoding What is scarce? client bandwidth JSON + permessage-deflate 268 B/msg server CPU msgpack, deflate off 72,691 msg/s both, numeric data header + typed arrays 371 B, 77,907 msg/s Measured on order-book messages; measure yours.

Verification

The encoding choice is sound when:

  • Size and CPU are measured on production-like messages in each candidate format.
  • Compression is part of the comparison, with the setting real clients negotiate.
  • Binary formats are versioned or negotiated, so clients can migrate gradually.
  • Packed numeric formats are checked for precision.

Diagnostic Hook: when a move from JSON to msgpack does not reduce bandwidth, look at the message contents and the compression setting. Float-heavy order-book messages were 827 bytes in msgpack against 803 in JSON, and with deflate JSON fell to 268 against 431.

Pitfalls & edge cases

  • Assuming binary is smaller. Measured: msgpack 827 B against JSON 803 B.
  • Comparing without compression. Deflate reversed the size ranking in JSON's favour.
  • Expecting deflate to be free. It cut throughput from 72,691 to 19,362 for msgpack.
  • float32 for values needing more precision. About seven significant digits survive.

Frequently Asked Questions

Is msgpack smaller than JSON over WebSockets?

Not always. For float-heavy order-book messages it was 827 bytes against 803 for compact JSON, and with permessage-deflate 431 against 268.

Is msgpack faster than JSON in Python?

Here, yes: encoding took 5.5-6.2 µs against 28-30 µs, and the feed ran at 72,691 against 25,001 messages per second without compression.

How do I send binary WebSocket frames in Python?

With websockets, pass bytes to ws.send(); str sends a text frame. Received binary frames arrive as bytes.

Should I enable permessage-deflate with msgpack?

Only if bandwidth matters more than CPU: it reduced msgpack from 827 to 431 bytes but throughput from 72,691 to 19,362 messages per second.