Back to System design
Design WhatsApp / Chat System
hardA chat app has to deliver a message in under a second, to a phone that may be asleep, behind a patchy network, or switched off.
Scale100B msgs/day → ~1.2M msg/s peak globally
Storage~300 B/msg × 100B = 30 TB/day (encrypted)
Key Concepts
1
The messages themselves are small and simple. The hard part is keeping hundreds of millions of phones
connected at once, and knowing which server each one is attached to.
2
Why connections are the whole problem. The phone cannot ask "any messages for me?" every second —
that would flood your servers. So the phone opens one connection and keeps it open, and the server pushes
messages down it.
Why connections are the whole problem.
3
An open connection costs memory even when nothing is happening. A well-tuned server handles about
100,000 of them. For 500 million people online at once, that is 5,000 servers before you have
written a line of business logic. Say this number early; it shows you know where the cost is.
100,000 of them5,000 servers
4
Finding the right server. You are connected to server A. Your friend is connected to server B. When
you send a message, server A has to find out where your friend is.
Finding the right server.
5
So you keep a directory: user ID to server ID, held in memory and updated when someone connects or
disconnects. Server A looks your friend up, forwards the message to server B, and server B pushes it down
the open connection.
6
The directory can be slightly out of date. If it is, the message goes to the wrong server, which says "not
mine", and the sender tries again. That is cheaper than keeping a perfectly accurate directory.
7
Counting the real traffic. Every message produces more messages. There is the message, then "sent",
then "delivered", then "read". Ten visible messages in a conversation move thirty or more items across the
network. Budget for that.
Counting the real traffic.
8
Offline is normal, not an edge case. Most phones are unreachable most of the time. Undelivered
messages wait in a queue for that user, and drain when the phone reconnects. A push notification wakes the
app.
Offline is normal, not an edge case.
9
The design decision is what happens after delivery. Delete the message, and a new phone has no history.
Keep it, and you are now storing everyone's conversations. That is a privacy decision before it is a
storage one.
10
Groups multiply everything. A message to a group of 256 people becomes 255 deliveries, plus up to 255
read receipts. Do that fan-out on the server, not on the sender's phone, or one person on a weak
connection slows the whole group. Above a few hundred members, read receipts have to be summarised or
dropped.
Groups multiply everything.
11
Ordering is per conversation. Two phones have two clocks that disagree, so timestamps cannot decide
order. The server gives each message in a conversation a sequence number, and everyone uses that. There is
no need for a global order across all conversations, and no cheap way to get one.
Ordering is per conversation.
12
What the interviewer is probing.
What the interviewer is probing.
13
1. "Design WhatsApp." Probing: whether you spot the real cost. Stalls: starting with the message
table. Moves up: 500 million open connections, 100,000 per server, 5,000 servers — in the first two
minutes.
1. "Design WhatsApp."
14
2. "How does a message reach someone on another server?" Probing: whether you invent the directory.
Stalls: "The load balancer routes it." Moves up: a user-to-server directory, read on every message,
allowed to be slightly stale.
2. "How does a message reach someone on another server?"
15
3. "The recipient is offline." Probing: when you confirm the message. Stalls: "Save it in the
database." Moves up: store it before telling the sender it is sent, queue it, wake the phone with a push
notification.
3. "The recipient is offline."
16
4. "It is a group of 256." Probing: where fan-out happens. Stalls: the sender's phone sends 255
copies. Moves up: the server fans out, and receipts get summarised past a certain size.
4. "It is a group of 256."
17
5. "Two messages arrive out of order." Probing: how you order things. Stalls: "Use timestamps."
Moves up: whose clock? A server-assigned sequence number per conversation.
5. "Two messages arrive out of order."
High-level design
Connection: long-lived WebSocket/XMPP to gateway; sticky by user_id.
Send: client → gateway → message service → persist (sender outbox + recipient inbox) → presence → online? push to gateway : queue push notification.
Receive: client opens socket → server delivers buffered messages → client acks → server marks delivered.
Groups: fanout to per-recipient inboxes (bounded by group size cap).
E2EE: client-side encryption with Signal Protocol; server stores ciphertext only.
Media: encrypted blobs in S3-class storage.
Components
- WebSocket gateway (stateful; user → connection routing). Erlang/Elixir famous fit; Go and Rust also common.
- Presence service (Redis with TTL heartbeat).
- Message service + per-user inbox (Cassandra wide row by user_id, clustered by timestamp).
- Push gateway (APNs, FCM with batching and retry).
- Media service (encrypted blobs in S3 + CDN for distribution).
- Group metadata service (members, admins, settings).
- E2EE key server (prekey bundles, identity keys, signed prekeys for X3DH).
- Anti-abuse service (spam, content moderation if not E2EE).
Capacity, worked
Monthly users 2,000,000,000 Concurrent fraction at peak 25% Messages per user per day 40 ----------------------------------------------------------- Concurrent connections 2B x 0.25 = 500,000,000 Connections per tuned box 100,000 Connection servers 500M / 100k = 5,000
Messages per day 2B x 40 = 80,000,000,000 Messages per second 80B / 86,400 = 926,000 With sent/delivered/read (x3) = 2,800,000 At 3x peak = 8,300,000
Per-connection memory
Socket buffers + app state ~10-40 KB
100,000 x 25 KB = 2.5 GB/box Session registry
Entry: user id 8 B + server id 4 B + timestamp 8 B = 20 B
500M x 20 B = 10 GBConsequences:
- 5,000 servers exist before any feature does; connection handling is the
budget.
- 10 GB of registry fits in memory, so routing is a hash lookup.
- 8.3M frames/s at peak is why receipts are aggregated in large groups.
Connection and routing path
Connect
Device -> load balancer (TCP/TLS passthrough) -> connection server.
Server writes {user -> self} into the session registry with a TTL.
Heartbeats refresh both the socket and the registry entry. Send A -> B
1. A's server validates and assigns a per-conversation sequence number.
2. Persists to the recipient's queue (durability before acknowledgement:
acknowledge only once it is stored, or a crash loses the message).
3. Looks up B in the registry.
4. Online: forward to B's server, which writes to the socket.
Offline: leave it queued; a push notification wakes the device.
5. B's client acknowledges; a delivery receipt flows back to A. Disconnect
Registry entry expires by TTL rather than requiring a clean close --
phones lose signal without saying goodbye. Why the registry tolerates staleness
A stale entry costs one forward to the wrong server and a retry. Making
it strongly consistent would put a consensus round trip on every
message. The cheap failure is the right trade.Failure modes in production
Thundering herd on reconnect
Symptom: a regional network blip drops 10M sockets; reconnects arrive
together and take down the connection tier.
Cause: clients retry immediately and in lockstep.
Fix: exponential backoff with jitter on the client, and connection
admission limits per server.
Registry entry outlives the server
Symptom: messages forwarded into a black hole after a node dies.
Cause: entries removed only on clean disconnect.
Fix: short TTL refreshed by heartbeat; forwarding failure triggers a
re-lookup and, failing that, the offline queue. Acknowledgement before durability
Symptom: senders see one tick, the message never arrives after a crash.
Cause: acknowledging on receipt rather than after persistence.
Fix: acknowledge after the write; it costs a few milliseconds and
removes a whole class of "it said delivered" support tickets. Receipt storm in large groups
Symptom: receipt traffic exceeds message traffic; battery complaints.
Cause: per-member read receipts fanned out to every member.
Fix: aggregate ("read by 47"), or drop per-member receipts above a
group size and say so in the product.