Back to System design
CDN and Edge Caching
easyA CDN is a network of servers spread around the world that keep copies of your files close to your users.
ScaleSub-50ms global p99 with good PoP coverage
StorageEdge cache size per PoP << origin dataset
Key Concepts
1
Without one, someone in Chennai loading your site waits for every file to travel to your server in Virginia
and back — about 200 milliseconds each way. With one, the files come from a machine in Chennai, a few
milliseconds away.
2
Static files are the easy part. The interesting work is the cache key.
Static files are the easy part. The interesting work is the cache key.
3
The cache key is what the CDN uses to decide whether it already has a copy. By default it is the URL.
Anything not in the key is invisible to the cache.
4
This causes the worst CDN incident there is. Suppose a page shows the logged-in user's name, and the CDN
caches it under the URL alone. The first user's page is now served to everyone else.
5
So: either the response must not be cached at all, or whatever makes it different — the user, the
language, the currency — must be part of the key. There is no safe third option.
6
Too much in the key is the opposite failure. Include the full browser identification string and every
browser version gets its own copy, so almost nothing is ever reused. Include tracking parameters and
/product?id=5&src=facebook is stored separately from /product?id=5.Too much in the key is the opposite failure./product?id=5&src=facebook/product?id=5
7
Strip the parameters that do not change the response, and keep the ones that do.
8
Hit rate is what you are designing for. At 100,000 requests a second:
Hit rate is what you are designing for.
9
90% cached -> 10,000 reach your server
99% cached -> 1,000 reach your server
10
The last few percent come from rare files, and that is where an extra layer helps. Put one cache between
all the edge servers and your origin. Without it, 200 edges each fetch the same file separately; with it,
they all get it from one place and your server sees a single request.
11
Three ways to update cached content, with very different costs.
Three ways to update cached content, with very different costs.
12
Expiry time. Free. The copy is used until it expires, so you accept being out of date for that long.
13
Purge. You tell the CDN to delete something. Immediate, but rate-limited by the provider and slow to
reach every location, so it is for mistakes and takedowns rather than routine updates.
14
Change the filename.
app.a1b2c3.js instead of app.js. A new build produces a new name, so there is
nothing to invalidate and you can cache it for a year. This is why build tools put a hash in filenames.app.a1b2c3.jsapp.js
15
Serve stale while fetching fresh. Tell the CDN it may keep serving the old copy while it fetches a new
one in the background. Users never wait for the refresh, and your server gets one request instead of a
flood when something expires.
Serve stale while fetching fresh.
16
What the interviewer is probing.
What the interviewer is probing.
17
1. "Why use a CDN?" Probing: whether you say more than "it is faster". Stalls: "It caches static
files." Moves up: the round-trip saving with real numbers, and taking load off your own servers.
1. "Why use a CDN?"
18
2. "A user saw someone else's data." Probing: the cache key. Stalls: "A caching bug." Moves up:
a personalised page cached under a key that did not include the user.
2. "A user saw someone else's data."
19
3. "How do you update cached content?" Probing: the three tools. Stalls: "Purge the cache." *Moves
up:* expiry for most things, filenames with hashes for build files, purge kept for emergencies.
3. "How do you update cached content?"
20
4. "Your hit rate is 70%." Probing: diagnosis. Stalls: "Increase the expiry time." Moves up:
look at what is in the key first — query parameters usually split one file into many.
4. "Your hit rate is 70%."
21
5. "Can you cache an API response?" Probing: beyond static files. Stalls: "No, it is dynamic."
Moves up: yes, briefly, for public read-heavy endpoints — and only per-user when the user is in the
key.
5. "Can you cache an API response?"
Components
- Edge PoPs with cache + TLS terminator + HTTP/2/3 parsers.
- Cache hierarchy: edge → regional shield → origin shield → origin.
- DNS-based routing (GeoDNS) or anycast (most modern CDNs).
- Purge API for explicit invalidation; tag-based purges if vendor supports.
- Edge compute runtime (Workers, Lambda@Edge, Compute@Edge).
- WAF rules and bot management.
- Real-time analytics — per-PoP hit ratio, origin offload, cache miss reasons.
Latency and offload, worked
Round trip, Chennai user
to origin in Virginia ~200 ms
to edge in Chennai ~5-10 ms
Saving per request ~190 ms Page with 40 assets
HTTP/2 multiplexed over one connection: the saving is not
40 x 190 ms, but the connection setup and each miss still
pays the full path.
First-byte improvement alone typically 150-200 ms
-- which is the difference between "instant" and "loading". Origin offload at 100,000 req/s
Hit rate 70% origin sees 30,000 req/s
Hit rate 90% origin sees 10,000 req/s
Hit rate 95% origin sees 5,000 req/s
Hit rate 99% origin sees 1,000 req/s Origin shield effect
Edges 200
Object expires; every edge misses independently
without shield: origin receives 200 requests
with shield: origin receives 1 request
-- the shield turns a fan-in of 200 into 1 for every
cacheable object, every TTL.Cache key discipline
Default key
scheme + host + path + query string Common mistakes
Query string included wholesale
/product?id=5&utm_source=fb&fbclid=xyz
-> every tracking parameter creates a separate object.
Fix: strip known tracking parameters; allow-list the
parameters that actually change the response. Vary: User-Agent
-> thousands of variants, near-zero hit rate.
Fix: vary on a derived class (mobile/desktop) if you must. Vary: Cookie on a page that is mostly public
-> effectively uncacheable, and dangerous if mis-set.
Fix: split the page -- cache the shell publicly, fetch the
personalised fragment separately. Deliberate additions to the key
Accept-Encoding yes, almost always
Accept-Language when you serve translated content
Currency/region when pricing differs
Device class when markup differs materially The safety rule
If the response contains anything identifying a user, it must
either be uncacheable (private, no-store) or keyed on that
user. There is no third option that is safe.Invalidation compared
TTL expiry
Cost: free.
Latency to effect: up to the TTL.
Use for: everything you can tolerate being briefly stale. stale-while-revalidate
Cache-Control: max-age=60, stale-while-revalidate=600
Effect: after 60 s the edge serves the stale copy INSTANTLY and
refreshes in the background; users never wait for a refetch.
Use for: feeds, listings, anything where 60 s of staleness is
invisible but a slow refresh is not. Explicit purge
Cost: rate-limited by the provider; can take seconds to
propagate globally.
Use for: mistakes, takedowns, legal removals -- not routine
updates. Versioned (fingerprinted) URLs
/static/app.a1b2c3.js with max-age=31536000, immutable
Effect: a new build emits a new URL, so there is nothing to
invalidate. The old object ages out on its own.
Use for: every build artefact. This is why bundlers hash
filenames. Soft purge / tag-based
Attach surrogate keys (e.g. product:123) to responses; purge by
key so one product update clears every page featuring it,
without listing URLs.