Back to System design
Caching Strategies: Cache-aside, Write-through, Write-back
easyA cache is a small, fast copy of data you would otherwise fetch from somewhere slow. Reading from memory takes about a millisecond; reading from a database might take fifty.
ScaleRedis ~100K-1M ops/s/node; <1ms p99
StorageWorking set, not full DB — typically <10% of DB size
Key Concepts
1
Caching questions are rarely about the cache itself. They are about three things: what happens when the
data is not there, what happens when someone writes, and what a user sees when the copy is out of date.
2
Hit rate decides everything, and the arithmetic is worth memorising. If the cache answers 90% of
requests at 1 ms and the other 10% take 50 ms, the average is 5.9 ms. Push the hit rate to 99% and it
drops to 1.5 ms.
Hit rate decides everything, and the arithmetic is worth memorising.
3
That looks like a small change and is not. Look at what reaches the database instead:
4
10,000 requests/second, 90% hit rate -> 1,000 reach the database
10,000 requests/second, 99% hit rate -> 100 reach the database
5
Nine percentage points of hit rate is ten times less load. That is why teams keep chasing it long after it
looks good enough.
6
Cache-aside is the usual pattern. The application looks in the cache. If it is not there, it reads the
database, puts a copy in the cache, and returns it.
Cache-aside is the usual pattern.
7
Only data someone actually asked for takes up space. If the cache goes down, everything still works, just
slower. The cost is that a miss takes three steps instead of one.
8
Write-through writes to the cache and the database together, so a read straight after a write always
sees the new value. Writes are slower because they pay for both.
Write-through
9
Write-back writes to the cache and confirms immediately, then writes to the database later. It is the
fastest, and it is the only one that can lose data: anything not yet written out is gone if the server
dies. Fine for a view counter. Not fine for a payment.
Write-back
10
Deleting beats updating. When data changes, delete the cached copy rather than updating it. Two
updates racing can reach the cache in the opposite order from the database, leaving the cache holding the
older value permanently. Deleting cannot go wrong that way — the next read simply fetches fresh.
Deleting beats updating.
11
What happens when a popular key expires. Every request that wanted it misses at the same moment, and
they all hit the database together. A key used 5,000 times a second produces 5,000 identical queries.
What happens when a popular key expires.
12
Three fixes, each cheap. Let the first request rebuild while the others wait for its answer. Serve the old
value while refreshing in the background. And add a random amount to each expiry time, so keys written
together do not expire together.
13
What the interviewer is probing.
What the interviewer is probing.
14
1. "You added a cache. What is the hit rate?" Probing: whether a number drives the design. Stalls:
"It should be high." Moves up: the average-latency sum, and that 90% to 99% is ten times less database
load.
1. "You added a cache. What is the hit rate?"
15
2. "Cache-aside or write-through?" Probing: matching the pattern to the use. Stalls: naming one as
better. Moves up: cache-aside by default because it fails gracefully; write-through when a stale read is
unacceptable.
2. "Cache-aside or write-through?"
16
3. "The cached value is wrong. Why?" Probing: update versus delete. Stalls: "We update the cache
when we update the database." Moves up: two writers can arrive in the opposite order; delete instead.
3. "The cached value is wrong. Why?"
17
4. "A popular key expires. What happens?" Probing: the stampede. Stalls: "The next request
reloads it." Moves up: every concurrent request misses at once; let one rebuild, serve stale meanwhile,
and jitter the expiry times.
4. "A popular key expires. What happens?"
18
5. "The cache server dies." Probing: what degrades. Stalls: "The site goes down." Moves up:
with cache-aside everything still works at database speed — whether the database survives that is a
capacity question you should have an answer to.
5. "The cache server dies."
Components
- Cache cluster — Redis (rich data types, persistence, pub/sub) or Memcached (simpler, faster, eviction-only).
- Sharding — consistent hashing with virtual nodes, client-side or via proxy (twemproxy, mcrouter, KeyDB).
- Eviction — LRU (default), LFU, TinyLFU (better hit rate at small sizes), allkeys-LRU vs volatile-LRU.
- Invalidation channel — Redis pub/sub or Kafka topic for cross-region purges.
- Stampede control — single-flight mutex, probabilistic refresh, or queue-based fill.
- Hot-key handling — Top-K detector, replicated copies, client-side micro-cache.
- Metrics — hit rate, p99 latency, evictions, memory, hot-key alarms, slow-log.
Hit rate, worked
Cache latency 1 ms Database latency 50 ms ----------------------------------------------------------- Effective latency = h x 1 + (1 - h) x 50
h = 0.50 = 25.5 ms
h = 0.80 = 10.8 ms
h = 0.90 = 5.9 ms
h = 0.95 = 3.5 ms
h = 0.99 = 1.49 ms
h = 0.999 = 1.05 ms Database load behind the cache, at 10,000 req/s
h = 0.90 misses = 1,000 req/s
h = 0.99 misses = 100 req/s
h = 0.999 misses = 10 req/sThat second table is the one that matters in an incident: the difference between 90% and 99% is not 9% of latency, it is 10x the load your database survives.
Sizing the cache
Working set, not dataset. Measure, do not guess:
Total dataset 500 GB Keys accessed at least once a day (measured) 12% Working set 500 GB x 0.12 = 60 GB Overhead (Redis ~50-80 B/key + fragmentation) ~15% Provision ~70 GB
If you cannot measure yet, the shape to state:
- Zipfian traffic means the head is tiny: the top 1% of keys often
serve 50%+ of requests.
- So the first 10 GB buys most of the hit rate, and the last 10 GB
buys the tail. Diminishing returns are steep. Eviction policy
allkeys-lru general purpose; recent wins
allkeys-lfu better when popularity is stable and bursts are noise
volatile-ttl only when every key carries a meaningful TTL
noeviction writes fail when full -- correct for a cache that is
acting as a store, dangerous otherwise.Stampede control
The failure
One hot key expires. 5,000 concurrent requests miss simultaneously and
all of them query the database for the same row. Single-flight (request coalescing)
First miss acquires a short lock on the key; others wait on it and read
the result. One database query instead of 5,000. Serve stale while revalidating
Keep the expired value for a grace period. On a miss, return the stale
value immediately and refresh in the background. Users never wait for a
rebuild; the cost is bounded staleness. Jittered TTL
Writing TTL = base + random(0, base x 0.1) stops keys written together
from expiring together. Cheap, and it removes the synchronised cliff. Early recomputation
Refresh a key when it is close to expiry and currently popular, so the
rebuild happens before anyone misses.Failure modes in production
Cache and database disagree after a write
Symptom: a user updates a profile and sees the old value on refresh.
Cause: update-in-place racing with another writer.
Fix: delete on write; let the next read repopulate. If updating,
version the payload and reject stale versions.
Synchronised expiry cliff
Symptom: latency spikes at a regular interval matching the TTL.
Cause: keys written in a batch expire in a batch.
Fix: jitter. Hot key saturating one shard
Symptom: one Redis node at 100% CPU, the rest idle.
Cause: a single key cannot be split by sharding.
Fix: replicate the key across shards with a suffix, or cache it in
process memory for a few seconds. Cache as a store by accident
Symptom: a restart loses data nobody expected to lose.
Cause: write-back, or writes that only ever landed in the cache.
Fix: be explicit about which data is reconstructible. If it is not,
it does not belong in a cache alone.