Back to System design

Back-of-Envelope: Twitter Timeline Capacity

medium
Scale: Drives shard count, cache size, fanout worker count Storage: Drives total disk and replication factor Meta, Twitter, LinkedIn
MathCapacity PlanningFeed

Capacity estimation turns vague requirements ('design Twitter') into concrete numbers that constrain the rest of the design. The muscle to build is doing this in 3-5 minutes without a calculator. Twitter timeline is canonical because it covers all the levers.

ScaleDrives shard count, cache size, fanout worker count
StorageDrives total disk and replication factor

Key Concepts

1
1. Step 1: Users → activity → writes. State assumptions explicitly. MAU 300M, DAU 150M (50%), tweets/user/day = 2. Total writes: 300M × 2 = 600M tweets/day. Per second avg: 600M / 86,400 ≈ 7,000/s. Peak factor 5-10x: 35K-70K/s peak.
1. Step 1: Users → activity → writes.
2
2. Step 2: Reads. Each DAU opens app ~10x/day, sees ~100 tweets per session. Total reads: 150M × 10 × 100 = 150B/day. Per second avg: ~1.7M/s. Peak: ~17M/s. Read:write ratio ~250:1 — confirms 'read-heavy' and justifies fanout-on-write.
2. Step 2: Reads.
3
3. Step 3: Storage. Per tweet: text + metadata ~300 B. 600M tweets/day × 300 B = 180 GB/day raw. 10 years: 180 GB × 365 × 10 ≈ 660 TB raw, ~2 PB with 3x replication. Media: assume 20% of tweets have photos averaging 200 KB → 24 TB/day media. Media dominates text by 100x; store on object store + CDN.
3. Step 3: Storage.
4
4. Step 4: Fanout and cache. Avg 200 followers per user. 600M tweets × 200 followers = 120B timeline writes/day → ~1.4M/s avg, ~10M/s peak. Distribute across many fanout workers; celebrities (>1M followers) skip fanout. Cache size: 150M DAU × 800 entries × 16 B reference ≈ 2 TB. Easily holds in sharded Redis.
4. Step 4: Fanout and cache.
5
5. Step 5: Sanity check and translate to infrastructure. 17M reads/s ≫ any single node → CDN/cache mandatory. 120B fanout writes/day → Kafka partitions sized accordingly. 2 PB / 100 TB per shard ≈ 20 shards minimum; in practice 40-80 for headroom. Compare with reality: Twitter publicly mentioned ~500M tweets/day and 200M DAU in 2022 — our estimate is within 20%. End with the dominant cost: 'Read-side caching + CDN egress is 80% of cost.'
5. Step 5: Sanity check and translate to infrastructure.

Related Problems

Evaluate Reverse Polish NotationmediumBasic CalculatorhardPower of TwomediumMissing NumbermediumDesign Twitter / News Feedhard

Approach

Step 1 — Users:
MAU = 300M, DAU = 150M, tweets/day = 2 ⇒ writes = 600M/day.
Per second avg ≈ 600M / 86,400 ≈ 7,000/s. Peak ≈ 35-70K/s.
Step 2 — Reads:
Each DAU opens 10× × sees 100 tweets ⇒ reads = 150B/day.
Per second ≈ 1.7M/s avg, 17M/s peak. Read:write 250:1.
Step 3 — Storage:
Tweet ~300 B ⇒ 180 GB/day raw text. 10 yr ≈ 657 TB ≈ 2 PB with 3x replica.
Media (20% of tweets, 200KB avg) ⇒ 24 TB/day media. Object store + CDN.
Step 4 — Fanout:
Avg 200 followers × 600M tweets = 120B timeline writes/day.
Per second ≈ 1.4M/s avg, 10M/s peak. Distribute across many fanout workers.
Step 5 — Cache:
Active users × 800 entries × 16 B ≈ 150M × 13KB ≈ 2 TB cache.

Sanity checks

  • 17M/s reads ≫ any single node → CDN/cache layer mandatory.
  • 120B fanout writes/day → workers, Kafka partitions sized accordingly.
  • 2 PB / 100 TB per shard ≈ 20 shards minimum; in practice run 40-80 for headroom.
  • Sanity: Twitter's published numbers (~500M tweets/day, 200M DAU) — our estimate within 20%.
  • Media dominates storage by 100x; CDN cost will dominate text DB cost.

Common interview pattern

1. State assumptions explicitly: MAU, DAU%, activity, peak factor.
2. Compute writes/s and reads/s with arithmetic.
3. Compute storage/day → /year → with replication.
4. Derive: cache size, shard count, Kafka partitions, fleet size.
5. Verify: is any number absurd? Adjust and explain.
6. State the dominant cost: 'Read caching + CDN for media is 80% of cost.'
Keep numbers round; don't pretend precision.

Capacity → infrastructure

Writes/s ÷ writes/s/node = shard count.
Reads/s ÷ reads/s/node = read replicas needed.
Storage / per-shard cap = additional shards for storage.
Fanout writes/s ÷ fanout-worker rate = worker count.
Cache size / per-node RAM = cache nodes.
Network egress (media) → CDN PoP count + per-PoP capacity.

Adjusting for hot users

Power-law follower distribution.
Average 200 followers, median ~25, top users >10M.
Fanout cost: dominated by top 0.01%.
Mitigate: hybrid push (normal) + pull (celebrity) — see ID 170.
Capacity model must account: 80% of users contribute 20% of fanout writes; top 0.01% contribute 50%.