1. WebSockets vs. Long Polling vs. SSE
Three genuinely different ways to get server-initiated updates to a client, each with different tradeoffs:
- Long polling — the client sends a request that the server holds open until there's an update (or a timeout), then the client immediately reopens a new request. Works over plain HTTP, no special infrastructure, but each update round-trip pays full HTTP request overhead.
- Server-Sent Events (SSE) — one long-lived HTTP connection over which the server streams events to the client. Simple, works over HTTP, but is one-directional (server → client only); the client still needs a normal request to send anything back.
- WebSockets — a persistent, full-duplex connection: either side can send a message at any time, with minimal per-message overhead after the initial handshake. The right choice when the client also needs to push frequent updates (typing indicators, live cursor positions), not just receive them.
A live sports score ticker only needs server → client — SSE is a simpler, sufficient choice, and simpler infrastructure is a real advantage. A chat app needs both directions frequently — WebSockets fit better. Reaching for WebSockets by default, even when SSE would fully cover the requirement, adds unneeded complexity (Section 3's stateful-connection problem applies to both, but WebSockets add bidirectional handling on top).
2. Deep Dive: Online/Offline Presence
"Is this user currently online?" sounds trivial but has a real failure mode: a client's connection can drop silently (a phone loses signal, a laptop sleeps) without ever sending a clean "I'm disconnecting" message — the server needs a way to detect that on its own, not just react to an explicit signal.
Client, every 15 seconds: sends a small "heartbeat" message
over its open WebSocket connection
Server: on each heartbeat, updates
last_seen[user_id] = current_time
Server, background job every 30 seconds:
for each user, if current_time - last_seen[user_id]
> 45 seconds (3 missed heartbeats):
mark user OFFLINE, notify their contacts
The 45-second threshold (three missed 15-second heartbeats, not one) is a deliberate tolerance for normal network jitter — marking someone offline after a single missed heartbeat would produce false "went offline" flickers from perfectly healthy connections that had one slow network moment. This is the same kind of tuning-knob thinking as Week 14's celebrity-fanout threshold: a number chosen against a measured tradeoff (false-positive offline flickers vs. detection speed), not an arbitrary constant.
Where last_seen actually lives matters at scale: a single in-memory map
works for one server, but Section 3 shows why real systems run many connection
servers — presence state needs to live somewhere all of them can read and write,
typically a fast shared store like Redis, keyed by user_id.
3. The Stateful Connection Scaling Problem
Every system so far in this course has assumed stateless services (Week 3) — any instance can handle any request, and a load balancer distributes freely. A WebSocket connection breaks that assumption: once a client connects to a specific server, that connection lives on that specific server for as long as it's open, often minutes or hours.
User A opens a WebSocket --> connects to [Connection Server 2]
(stays connected to Server 2 for
the entire session)
User B, on Server 5, sends User A a message.
Server 5 cannot deliver it directly -- it has no open
connection to User A. It needs to know User A is specifically
on Server 2, and get the message there. This is the problem
Section 4 solves.
This is a real architectural fork from everything earlier in the course: connection servers need a way to route a message to "wherever this specific user's connection currently lives," which requires both a lookup (which server holds User A's connection right now?) and a delivery mechanism between servers that don't share any direct connection to each other.
4. Pub/Sub Fan-out Across Connection Servers
The standard solution layers Week 8's pub/sub pattern between the connection servers: every connection server subscribes to a shared message bus, and publishing a message to a user's channel reaches whichever server actually holds that user's connection.
[Server 2] --subscribes to channel "user:A"--> [Pub/Sub (Redis/Kafka)]
[Server 5] --subscribes to channel "user:B"-->
User B (on Server 5) sends a message to User A:
Server 5 --> publish to channel "user:A" --> [Pub/Sub]
|
delivered to whichever
server subscribed to
"user:A" -- Server 2
|
Server 2 --> pushes over
its open WebSocket to
User A's actual device
Each connection server subscribes to a channel per connected user (or per conversation/room, for group chat), and the pub/sub layer handles the "which server actually needs this message" routing — no connection server needs to know which other server holds any given user's connection directly. This decouples "which server sent the message" from "which server delivers it," the same decoupling principle Week 8 introduced for producer/consumer decoupling, now applied to routing across a fleet of stateful servers instead of just absorbing bursty load.
A client that disconnects and reconnects (a phone switching from WiFi to cellular) will very likely land on a different connection server than before — the new server needs to freshly subscribe to that user's channel on connect. Any messages sent during the brief gap between disconnect and reconnect need a separate mechanism (a short-term buffer, or falling back to Week 8's durable queue) or they're simply lost, which is worth naming explicitly as a real gap most naive designs miss.
5. Hands-on Exercise
Design real-time typing indicators for a group chat
Add a "so-and-so is typing…" indicator to a group chat feature, visible live to every other member of the conversation.
Requirements:
- Choose between WebSockets, long polling and SSE for this feature (Section 1), and justify the choice against whether it needs bidirectional communication.
- Design the pub/sub channel structure for a group conversation with 20 members spread across multiple connection servers (Section 4) — what's the channel, and who publishes/subscribes to it.
- Decide what should happen if a user starts typing but their connection drops mid-message without sending an explicit "stopped typing" signal — propose a timeout-based fix, referencing Section 2's heartbeat tuning logic.
- Explain why "stopped typing" doesn't need Week 8's durable, at-least-once queue guarantees the way Week 14's fan-out pipeline did — what's different about this data that makes losing an occasional update acceptable.
- State one thing that changes in your design if the conversation has 50,000 members instead of 20.
For requirement 4: a typing indicator is ephemeral, current-moment state — if one "typing" event is lost, the very next one (a few hundred milliseconds later, as the user keeps typing) supersedes it anyway. That's a meaningfully different reliability requirement from a chat message itself, which must never be silently dropped.
6. Knowledge Check
Four quick questions. Expand each to check your answer.
Q1
Why would Server-Sent Events be a poor fit for a chat application, even though they're simpler than WebSockets?
Why would Server-Sent Events be a poor fit for a chat application, even though they're simpler than WebSockets?
SSE is one-directional, server to client only — a chat app needs the client to frequently send messages too (the actual chat content, typing indicators), which SSE alone can't carry. WebSockets' full-duplex connection is a better fit whenever the client needs to push updates as often as it receives them.
Q2
Why does a presence system wait for three missed heartbeats before marking a user offline, instead of acting on the very first missed one?
Why does a presence system wait for three missed heartbeats before marking a user offline, instead of acting on the very first missed one?
A single missed heartbeat is often just normal network jitter on a connection that's actually fine, not a genuine disconnection. Requiring multiple consecutive missed heartbeats trades a small amount of detection delay for avoiding false "went offline" flickers on healthy connections — a deliberately tuned threshold, not an arbitrary one.
Q3
Why can't a load balancer just route WebSocket connections the same stateless way it routes normal HTTP requests?
Why can't a load balancer just route WebSocket connections the same stateless way it routes normal HTTP requests?
A WebSocket connection is long-lived and tied to a specific server instance for the entire session, unlike a stateless HTTP request that any instance can handle independently. Once a user connects to a specific server, a message meant for that user has to reach that exact server, which is a routing problem stateless load balancing was never designed to solve.
Q4
What problem does subscribing every connection server to a shared pub/sub layer actually solve?
What problem does subscribing every connection server to a shared pub/sub layer actually solve?
It lets any connection server deliver a message to any user without needing to know, in advance, which specific server currently holds that user's WebSocket connection — the server sending the message just publishes to that user's channel, and the pub/sub layer routes it to whichever server is actually subscribed for that user, decoupling senders from the specific connection holder.