Back to System design

CDN and Edge Caching

easy
Scale: Sub-50ms global p99 with good PoP coverage Storage: Edge cache size per PoP << origin dataset Cloudflare, Fastly, Akamai
FundamentalsCDNPerformance

A CDN is a globally distributed cache that serves content from the PoP nearest each user. Originally for static assets, modern CDNs also do TLS termination, edge compute, bot management, and DDoS scrubbing.

ScaleSub-50ms global p99 with good PoP coverage
StorageEdge cache size per PoP << origin dataset

Key Concepts

1
1. The hierarchy. PoPs (Points of Presence) terminate user TLS and serve cached content. A PoP has many edge servers behind anycast IPs — same IP advertised from every PoP; BGP routes the user to the closest. On miss, request goes to a regional shield, then origin shield, then origin. The shield layers consolidate cache-miss traffic so the origin sees a few requests per cold key instead of N.
1. The hierarchy.
2
2. Cacheability via headers. Cache-Control: max-age=N for browser; s-maxage=N overrides for shared caches. immutable says content never changes. stale-while-revalidate lets the CDN serve stale while refreshing in background. private blocks shared caching. The cache key is URL by default; including too many headers/cookies explodes key space and tanks hit rate.
2. Cacheability via headers.Cache-Control: max-age=Ns-maxage=Nimmutablestale-while-revalidate
3
3. Versioned URLs are the gold standard. Use app.a1b2c3.js instead of app.js. The build pipeline emits hashed filenames; the URL changes when content changes. Old URLs can have infinite TTL — they'll never serve wrong content because they are the old content. Cache-bust is automatic.
3. Versioned URLs are the gold standard.app.a1b2c3.jsapp.js
4
4. Beyond static caching. TLS termination at edge cuts 100+ ms vs origin termination. HTTP/2 and HTTP/3 enabled at the CDN even if origin only speaks HTTP/1.1. Edge compute (Cloudflare Workers, Lambda@Edge, Compute@Edge) runs small functions at the PoP for personalization, A/B bucketing, simple auth — avoids origin RTT entirely. WAF and DDoS scrubbing before reaching origin.
4. Beyond static caching.
5
5. In production. Netflix Open Connect: in-house caches embedded inside ISPs — zero transit cost. Cloudflare: anycast + V8-isolated Workers. Fastly: VCL-programmable edge + Wasm Compute@Edge. Shopify: Cloudflare in front of Rails with cache tags per shop / product. Watch: cache hit ratio, origin offload, per-PoP p99 latency. CDN config as code (Terraform, Pulumi) — clicking dashboards doesn't scale.
5. In production.

Related Problems

CAP Theorem and PACELCeasyLoad Balancing: L4 vs L7, Algorithms, Health CheckseasyCaching Strategies: Cache-aside, Write-through, Write-backeasySQL vs NoSQL: When to Pick WhicheasySharding Strategies: Range, Hash, Directory, Geomedium

Approach

  1. Pick a CDN — Cloudflare, Fastly, Akamai, AWS CloudFront, GCP Cloud CDN. Compare PoP coverage in your user geographies.
  2. Identify content tiers: immutable (long TTL, versioned URL), semi-dynamic (short TTL, stale-while-revalidate), dynamic (no cache, edge compute).
  3. Use versioned URLs for immutable assets. Build pipeline generates hashed filenames.
  4. Set Cache-Control + s-maxage + stale-while-revalidate appropriately.
  5. Enable origin shield to consolidate cache-miss traffic.
  6. Use edge compute for personalization, geo routing, simple auth, A/B test bucketing.
  7. Terminate TLS at edge; enable HTTP/2 and HTTP/3.
  8. Configure cache key carefully — include only headers that meaningfully vary content.
  9. Monitor cache hit ratio, origin offload, p50/p99 latency per region.
  10. Configuration as code (Terraform, Pulumi, vendor SDK).

Components

  • Edge PoPs with cache + TLS terminator + HTTP/2/3 parsers.
  • Cache hierarchy: edge → regional shield → origin shield → origin.
  • DNS-based routing (GeoDNS) or anycast (most modern CDNs).
  • Purge API for explicit invalidation; tag-based purges if vendor supports.
  • Edge compute runtime (Workers, Lambda@Edge, Compute@Edge).
  • WAF rules and bot management.
  • Real-time analytics — per-PoP hit ratio, origin offload, cache miss reasons.

Cache invalidation strategies

Versioned URLs (best): change the URL when content changes. Old URLs keep working with long TTL. Build pipeline does the rename.

Tag-based purge: tag content with logical groups; purge by tag (e.g., purge all 'product-123' content). Fast, scoped.

Path-based purge: purge specific paths or prefixes.

Soft purge: mark stale, serve stale-while-revalidate. Smooth invalidation under load.

Full purge: nuclear option. Slow, expensive, only for emergencies.

Avoid relying on TTLs alone for freshness-critical content; combine with one of the above.

Trade-offs

Long TTL = high hit rate, stale risk. Short TTL = freshness, origin pressure.

Versioned URLs solve both: immutable + new URL on change. Strongly preferred.

Edge compute reduces latency but limits runtime (CPU budget, memory cap, no native libs).

Push CDN warms popular assets but requires pre-knowledge of hot content. Pull CDN is more common and self-tuning.

Multiple cache keys (per-cookie, per-header) tank hit rate. Normalize aggressively.

Cache hierarchy adds RTT but absorbs cache misses; net win at scale.

Real-world patterns

  • Netflix Open Connect: in-house CDN with caches embedded inside ISPs.
  • Cloudflare: anycast edge with Workers (V8-isolated edge compute).
  • Fastly: VCL-programmable edge with Compute@Edge (Wasm).
  • Spotify: aggressive use of versioned URLs + multi-tier cache.
  • Twitch: hybrid CDN + P2P (peer-assisted delivery) for very high concurrency events.
  • Shopify: Cloudflare in front of Rails origin with cache tags per shop/product.