WebSockets

Pradeep P
3 days ago
3 min read

Series: Modern System Design · Layer 2 — Communication

Layer 2 · Post 8 of 10

← Previous: Event-Driven Architecture → Next: Server-Sent Events

Layer 2 — Communication · Post 23 of 88

WebSockets keep a two-way connection open so the server can push updates instantly instead of waiting for the client to ask again.

What you'll learn

Why polling hurts, and what a WebSocket actually is
How to scale a fleet of socket servers (sticky connections, pub/sub fan-out)
When WebSockets are overkill

The idea in one minute

HTTP is request/response. The server cannot talk until the client asks.

WebSocket starts as HTTP (Upgrade: websocket) and becomes a long-lived, bidirectional TCP connection. Either side can send frames whenever.

That is how chat, live cursors, multiplayer, and "the dashboard just updated" feel instant.

Why it matters

Polling every 2 seconds from 100,000 tabs is 50,000 empty requests per second. Load balancers and origin hate you. Latency is up to 2 seconds.

A socket per tab is not free either (memory, file descriptors, idle timeouts), but for true realtime it is the standard browser tool.

How it works

Client connects to wss://api.example.com/ws (TLS).
Handshake: HTTP 101 Switching Protocols.
Messages: text or binary frames. You define the JSON protocol (type: "message", roomId, ...).
Heartbeats/ping so load balancers do not kill idle connections.
Reconnect with backoff when the phone goes through a tunnel.

Scaling beyond one box

A connection is stuck to one server. User A on server 1, user B on server 2, A messages B:

Server 1 must publish the message to Redis/NATS/Kafka.
Server 2 subscribes and writes to B's socket.

Without that bus, you have a chat app that only works when both users hit the same pod.

Sticky load balancing (cookie or IP) keeps reconnects on the same node when you can. It is not a substitute for the bus.

Auth

Pass a token on the handshake (query string is ugly but common; first-message auth is cleaner). Authorize per subscription (join room X) so a stolen socket cannot listen to everything.

A simple example

Support chat:

Browser opens wss://.
Sends { "type": "join", "ticketId": "99" }.
Agent types; server publishes to ticket:99; all sockets in that room receive { "type": "msg", "text": "..." }.
History still lives in a DB. The socket is the live tail, not the system of record.

Common mistakes

No reconnect / no resume. Mobile networks drop. Clients must reconnect and fetch missed messages by seq or timestamp.

Business logic only in memory on the socket box. Deploy = amnesia. Persist.

One process, all users. You will need the fan-out bus sooner than you think.

Using WebSockets for occasional updates. SSE (next post) or short polling may be simpler. WebSockets for a daily notification is a lot of connection state.

How this shows up in real systems

Slack, Discord, Figma, trading UIs: sockets or socket-like (sometimes QUIC / custom).
Socket.IO, ws, SignalR: application libraries on top.
API Gateway WebSocket APIs, Cloudflare Durable Objects: managed connection routing.

Layer 6 "design a chat system" is this post plus fan-out, presence, and storage.