Back to System design
Design a News Feed Ranking Pipeline
hardA feed in time order is easy: take everything from people you follow, newest first. Ranking means choosing a different order, so that what you are most likely to care about appears first.
Scale<150ms feed render; 100K candidate scorings/s/serving-node
StorageFeature store TB; model artifacts GB
Key Concepts
1
Why not just score everything? Because there might be a million posts you could be shown, and you have
about a fifth of a second before the app feels slow.
Why not just score everything?
2
So it happens in stages, each one smaller than the last.
3
Everything possible ~1,000,000 posts
Stage 1: gather candidates -> 2,000
Stage 2: quick scoring -> 200
Stage 3: careful scoring -> 50
Stage 4: tidy the order -> 30 shown
4
Stage 1 is about not missing things. Pull from several cheap sources: people you follow, posts doing
well near you, things liked by people similar to you. Each contributes a few hundred. The goal is coverage,
not precision.
Stage 1 is about not missing things.
5
Stage 2 and 3 are about being right. A small fast model scores all 2,000 and keeps the best 200. A
larger, slower model then scores only those 200. That is the trick: the expensive model never sees more
than a couple of hundred items, so its cost is fixed no matter how much content exists.
Stage 2 and 3 are about being right.
6
Stage 4 fixes what scoring alone gets wrong. A pure score might give you five posts from the same
person in a row. Rules at the end enforce variety, limit how much any one author fills the screen, and
leave room for something outside your usual pattern.
Stage 4 fixes what scoring alone gets wrong.
7
Where the time goes.
Where the time goes.
8
gather candidates 50 ms
fetch the information about them 40 ms
quick scoring of 2,000 30 ms
careful scoring of 200 40 ms
variety rules 10 ms
------------------------------------------------
total 170 ms
9
Fetch the information in one batch, not per post. Two thousand separate lookups at one millisecond each is
two seconds, and the whole budget is a fifth of that.
10
The bug that ruins ranking systems. The model is trained on data prepared one way and then runs on data
prepared another way.
The bug that ruins ranking systems.
11
For example, training uses "posts liked in the last 7 days" computed from a complete table. Live, the same
number comes from a cache that is ten minutes out of date. Same name, different values. The model has
learned on numbers it will never actually see, and it performs beautifully in testing and badly in
production.
12
The fix is one definition used by both, and logging exactly what the live system used so you can train on
that.
13
The loop that slowly ruins the product. The model learns from what people click. People can only click
what the model showed them. Left alone, it narrows towards whatever worked last week, and the measurements
keep improving while the feed gets worse.
The loop that slowly ruins the product.
14
Counters: keep a few slots for things the model is unsure about, enforce variety, and measure more than
clicks — how long people stay, whether they come back, what they hide.
15
Deciding whether a change is good. Offline scores tell you the model learned something. They do not
tell you the product improved. Run a live test with guardrails agreed before you start: a change that
raises clicks 3% and doubles the "hide this" rate is a loss, and you only see that if you wrote it down
first.
Deciding whether a change is good.before
16
What the interviewer is probing.
What the interviewer is probing.
17
1. "How do you rank a feed?" Probing: the staged shape. Stalls: "Score every post and sort."
Moves up: the funnel, with rough numbers and why each stage exists.
1. "How do you rank a feed?"
18
2. "You have 10,000 candidates and 50 milliseconds." Probing: the budget. Stalls: "Use a bigger
model." Moves up: cheap model over everything, expensive model over the survivors, information fetched
in one batch.
2. "You have 10,000 candidates and 50 milliseconds."
19
3. "It performs well in testing and badly live." Probing: the training/serving gap. Stalls:
"Overfitting." Moves up: the same feature computed differently in the two places; one definition, and
train on what was actually served.
3. "It performs well in testing and badly live."
20
4. "Users say the feed is repetitive." Probing: the feedback loop. Stalls: "Retrain the model."
Moves up: it learns from its own output, so add exploration slots and variety rules, and treat hides as
a real signal.
4. "Users say the feed is repetitive."
21
5. "How do you know a change is good?" Probing: evaluation. Stalls: "The offline score went up."
Moves up: a live test with guardrails decided in advance.
5. "How do you know a change is good?"
High-level design
Stage 1 — Candidate generation: union of timeline + trending + recommendations + ads, deduped.
Stage 2 — Feature fetch: online feature store batch read.
Stage 3 — Score: ML model (GBDT or DNN), often cascaded.
Stage 4 — Rerank: diversity rules, freshness, ad insertion, business rules.
Stage 5 — Log: served features + scores + impressions to training data lake.
Offline: nightly retraining on logs; A/B framework for new models.
Components
- Candidate generators (timeline service, trending detector, recommendation engine, ad server).
- Feature store (Feast, Tecton, custom) with online (Redis/Cassandra) + offline (Parquet/Hive).
- Model server (TF Serving, Triton, ONNX Runtime).
- Re-ranker / business rules engine.
- Impression logger (Kafka → training data lake).
- Training pipeline (Spark + ML framework).
- A/B test framework.
- Counterfactual / off-policy evaluator.
The funnel, worked
Stage Items Budget Per-item cost
------------------------------------------------------------------
Eligible universe ~1,000,000 -- --
Candidate generation -> 2,000 50 ms ~25 us
(follow graph, trending, similar-user signals,
each source contributing a few hundred)
Feature fetch 2,000 40 ms ~20 us
(batched from the feature store, never per item)
Light scoring model -> 200 30 ms ~15 us
Heavy re-rank model -> 50 40 ms ~800 us
Diversity / rules -> 30 10 ms --
------------------------------------------------------------------
Total ~170 ms within a 200 ms
budget Why the funnel narrows this way
- The expensive model only ever sees 200 items, so its cost is
bounded regardless of how many posts exist.
- Feature fetching is batched: 2,000 individual lookups at 1 ms
each would be 2 seconds; one batched call is ~40 ms.
- If the budget tightens, you cut candidates first and model
size last -- recall degrades more gracefully than precision.Features and skew
Feature families
User history, declared interests, session context,
time of day, device
Item age, author, media type, topic, early engagement
velocity
Cross affinity between user and author, past engagement
with this topic, similarity of embeddings
Context position in feed, what was just shown, network
quality The skew trap, concretely
Training job computes "posts liked in the last 7 days" over a
batch table with complete data.
Serving path computes it from a cache that is 10 minutes stale
and misses the last few events.
Same feature name, different values. The model's weights were
learned on the first and applied to the second. Defences
- One definition, both paths: the feature store computes and
serves the same value.
- Log the exact feature vector used at serving time, and train
on THOSE logs rather than recomputing from raw tables.
- Monitor feature distributions in production against training
distributions; drift shows up here before it shows up in
metrics. Label leakage, the sibling bug
A feature that encodes the outcome (e.g. "total likes" including
the like you are predicting) gives a spectacular offline score
and no production value.Feedback loops and guardrails
The loop
Model ranks -> user engages with what was ranked -> those
engagements become training labels -> model reinforces.
Left alone this converges on a narrow slice of content and
authors, and the metric keeps improving while the product gets
worse. Interventions
Exploration slots reserve 5-10% of positions for items the
model is uncertain about
Diversity constraints cap items per author/topic in a window
Position debiasing train with the position an item was
shown in, so clicks at the top are not
mistaken for quality
Negative signals "see less of this", hides, reports --
weighted meaningfully, not as a footnote Guardrail metrics for every experiment
Session length and return rate (not just clicks)
Complaint / hide rate
Creator diversity (how many distinct authors get distribution)
Time to first meaningful interaction The discipline that matters
Decide the guardrails BEFORE the experiment. A ranking change
that raises clicks 3% and hides 20% is a loss, and that is only
obvious if the hide rate was a stated guardrail in advance.