Back to System design

Caching Strategies: Cache-aside, Write-through, Write-back

easy
Scale: Redis ~100K-1M ops/s/node; <1ms p99 Storage: Working set, not full DB — typically <10% of DB size Meta, Twitter, Amazon
FundamentalsCachingPerformance

A cache trades correctness guarantees for speed by keeping a hot subset of data in fast storage close to the consumer. Caches live at every layer (CPU, OS page, application, distributed, CDN); the interview cache is usually a distributed cache (Redis, Memcached) between a service and its database.

ScaleRedis ~100K-1M ops/s/node; <1ms p99
StorageWorking set, not full DB — typically <10% of DB size

Key Concepts

1
1. Why caches earn their place. Redis is ~100 µs vs Postgres ~1-10 ms. Caches absorb a 10-100x read multiplier on hot data, protect the DB during spikes, and let you scale reads independently of writes. Caching is essentially mandatory once read QPS exceeds the primary store's capacity.
1. Why caches earn their place.
2
2. Five strategies. Cache-aside (default): app checks cache, misses fall through to DB and populate. Read-through: cache library transparently fetches on miss. Write-through: write cache and DB synchronously — strong consistency, slower writes. Write-back: write cache, flush async — fastest writes, data loss risk. Refresh-ahead: reload hot keys before TTL expiry to avoid cold-miss tails.
2. Five strategies.
3
3. The hard part is invalidation. TTLs are the simplest tool — short TTL = fresh + DB hot; long TTL = hits but stale. Jitter TTLs (base ± 10-20%) to prevent synchronized expiry stampedes. Explicit invalidation on write is precise but needs reliable pub/sub. Versioned keys (cache key includes record hash) eliminate invalidation entirely.
3. The hard part is invalidation.
4
4. Production failure modes. Thundering herd: popular key expires, N requests stampede the DB. Fix with single-flight mutex (SET NX), probabilistic early refresh, or queued fill. Hot key: one key saturates one node — replicate hot key across N nodes, add client-side micro-cache, or shard the hot counter. Cache penetration: requests for keys that don't exist — add negative caching or a Bloom filter front. Cold cache after restart hammers the DB — warm via shadow traffic or persistent Redis.
4. Production failure modes.SET NX
5
5. In production. Meta TAO: write-through over MySQL serving billions of social-graph reads/s. Netflix EVCache: cross-AZ Memcached cluster. Pinterest: multi-tier (Memcached L1 + Redis L2). Stripe: aggressive caching of idempotency keys and rate-limit counters.
5. In production.

Related Problems

CAP Theorem and PACELCeasyLoad Balancing: L4 vs L7, Algorithms, Health CheckseasySQL vs NoSQL: When to Pick WhicheasySharding Strategies: Range, Hash, Directory, GeomediumConsistent Hashingmedium

Approach

  1. Place the cache. Application sidecar (e.g. local Caffeine) gives sub-µs latency for tiny hot sets. Distributed cache (Redis cluster, Memcached) for shared state.
  2. Choose the strategy. Default: cache-aside with TTL. Use write-through for safety on critical data; write-back for high-throughput counters; refresh-ahead for predictable hot keys.
  3. Set TTLs deliberately. Mix base TTL (1-15 min for most data) + random jitter (10-20%) to spread expiry.
  4. Solve stampede. Single-flight via mutex (Redis SET NX) so only one filler hits the DB. Or probabilistic early refresh: refresh with probability that increases as TTL approaches.
  5. Solve hot keys. Detect via Top-K (Redis-cell module, sketches). Mitigate via replication across cache nodes, in-process client cache, or sharded counters.
  6. Decide invalidation. TTL-only is simplest and resilient. Add pub/sub purge for tight freshness windows. Versioned keys (cache key includes hash of underlying record) eliminate explicit invalidation.
  7. Plan for cache loss. Restart cold-cache fills overwhelm the DB. Use rolling restart, shadow traffic to warm, or capacity buffer.

Strategies in depth

Cache-aside: app reads cache → miss → read DB → populate. Write: write DB → invalidate. Simplest, most flexible. Two writes race possible (TOCTOU).

Read-through: cache library transparently fetches on miss. Cleaner code; same semantics as cache-aside.

Write-through: write to cache and DB synchronously. Cache never stale relative to DB. Doubles write latency.

Write-back: write to cache → flush to DB async. Fast writes but data loss if cache crashes before flush. Used for high-throughput counters with WAL backup.

Refresh-ahead: predictively reload hot entries before TTL expires. Good for predictable hot keys; wasteful for long-tail.

Components

  • Cache cluster — Redis (rich data types, persistence, pub/sub) or Memcached (simpler, faster, eviction-only).
  • Sharding — consistent hashing with virtual nodes, client-side or via proxy (twemproxy, mcrouter, KeyDB).
  • Eviction — LRU (default), LFU, TinyLFU (better hit rate at small sizes), allkeys-LRU vs volatile-LRU.
  • Invalidation channel — Redis pub/sub or Kafka topic for cross-region purges.
  • Stampede control — single-flight mutex, probabilistic refresh, or queue-based fill.
  • Hot-key handling — Top-K detector, replicated copies, client-side micro-cache.
  • Metrics — hit rate, p99 latency, evictions, memory, hot-key alarms, slow-log.

Trade-offs

Cache-aside: flexible, but two-write race means brief inconsistency.

Write-through: strong consistency, slower writes.

Write-back: fastest writes, can lose data on cache failure.

TTL: short = fresh + DB hot; long = stale risk + cache effective. Jitter mandatory at scale.

Explicit invalidation: precise but requires reliable pub/sub. Pubsub messages can drop — combine with TTL as backstop.

Local + distributed cache: 2-layer cache (sidecar L1 + Redis L2) cuts latency further but increases invalidation complexity.

Failure modes and fixes

Thundering herd: N requests miss simultaneously, all hit DB. Fix: single-flight mutex (SET NX), probabilistic early refresh, or queued fill.

Hot key: one key gets all traffic, saturates one node. Fix: replicate key across N nodes, client-side micro-cache, or split hot counter into shards summed periodically.

Cache penetration: requests for keys that don't exist. Fix: negative caching (cache the 'not found' result with short TTL), Bloom filter front of cache.

Cache avalanche: many keys expire at once due to synchronized TTL. Fix: TTL jitter.

Cold cache after restart: DB hammered as cache refills. Fix: warm via shadow traffic, persistent Redis (AOF), or staged restart.

Real-world patterns

  • Meta TAO: write-through cache layered on MySQL; serves billions of social-graph reads/s.
  • Twitter Twemcache: heavily tuned Memcached fork; segmented LRU, fine eviction control.
  • Netflix EVCache: Memcached-compatible cluster across AZs with cross-region replication; CDN-warmth for cold-start avoidance.
  • Stripe: aggressive caching of idempotency keys and rate-limit counters in Redis Cluster.
  • Pinterest: multi-tier cache; Memcached L1 + sharded Redis L2 for pinboards.