Design a News Feed Ranking Pipeline

hard

Scale: <150ms feed render; 100K candidate scorings/s/serving-node Storage: Feature store TB; model artifacts GB Meta, Twitter, LinkedIn

Case StudyFeedML

Feed ranking decides what each user sees in their home feed. Twitter, Facebook, Instagram, TikTok, LinkedIn all use ML-ranked feeds — chronological is dead at scale because engagement demands it. The ranker sits on top of the feed fanout (ID 170) with a tight 150 ms budget on app open.

Scale<150ms feed render; 100K candidate scorings/s/serving-node

StorageFeature store TB; model artifacts GB

Key Concepts

1. Three phases: recall → score → rerank. Candidate generation: union sources — precomputed timeline, trending, recommendations, ads — dedup → a few hundred to a few thousand items. Recall-stage, doesn't care about exact ranking, cares about not missing things. Scoring: ML model predicts engagement per (user, candidate). Reranking: business rules + diversity + freshness + ad insertion.

1. Three phases: recall → score → rerank.

2. Feature store powers scoring. Online store (Redis, Cassandra, DynamoDB) — 1-10 ms per fetch. User features (recent engagement, interests), item features (creator, freshness, embeddings), interaction features (clicked similar items). Offline store (Parquet, ORC, Hive) for training. Parity is critical: training-serving skew silently degrades models. Log serving-time features for the next training cycle.

2. Feature store powers scoring.

3. Model architecture. GBDT (XGBoost, LightGBM): fast inference, interpretable, handles tabular features. Default for many production rankers. DNN: richer with sparse/embedded features, higher quality at higher cost. Two-tower (user tower + item tower → embedding similarity): candidate generation. Distillation: smaller fast model trained on a bigger model's labels — cuts latency.

3. Model architecture.

4. Rerank stage matters more than people think. Diversity rules: no two consecutive items from same source — pure relevance over-clusters. Freshness boost. Ad insertion at fixed slots or via auction. NSFW filter. Recency cap. Strict business rules executed last to guarantee they always apply.

4. Rerank stage matters more than people think.

5. A/B and counterfactual. Every model change is a comparative experiment with engagement metrics. Counterfactual evaluation (what would have happened with the new model?) is hard because logs only contain what was shown — use importance-weighted sampling and exploration (epsilon-greedy, Thompson sampling). Watch: filter bubble, training-serving skew, latency drift.

5. A/B and counterfactual.

High-level design

Stage 1 — Candidate generation: union of timeline + trending + recommendations + ads, deduped.
Stage 2 — Feature fetch: online feature store batch read.
Stage 3 — Score: ML model (GBDT or DNN), often cascaded.
Stage 4 — Rerank: diversity rules, freshness, ad insertion, business rules.
Stage 5 — Log: served features + scores + impressions to training data lake.
Offline: nightly retraining on logs; A/B framework for new models.

Components

Candidate generators (timeline service, trending detector, recommendation engine, ad server).
Feature store (Feast, Tecton, custom) with online (Redis/Cassandra) + offline (Parquet/Hive).
Model server (TF Serving, Triton, ONNX Runtime).
Re-ranker / business rules engine.
Impression logger (Kafka → training data lake).
Training pipeline (Spark + ML framework).
A/B test framework.
Counterfactual / off-policy evaluator.

Model architecture

GBDT (XGBoost, LightGBM): fast inference, interpretable, handles tabular features well. Default for many production rankers.

DNN (TF, PyTorch): better with sparse / embedded features, higher quality at higher cost.

Two-tower models: user tower + item tower → embedding similarity. Used for candidate generation.

Multi-task: predict click + like + share + dwell-time jointly; weighted sum gives engagement score.

Distillation: train a smaller fast model on the labels of a bigger model. Common for latency.

Feature store

Online: low-latency key-value (Redis, Cassandra, DynamoDB). 1-10 ms per fetch.

Offline: data lake (Parquet, ORC) for batch training.

Parity: same code path computes features; or log online features to use for training. Prevents training-serving skew.

Streaming features: real-time updates (last hour engagement) computed via Flink / Kafka Streams.

Trade-offs

Latency budget caps model size — distill or cascade.

Offline training: cheap but drifts; combine with streaming features.

More candidates: better recall, higher cost.

Pure relevance: filter bubble; need diversity and exploration.

Logging bias: only see outcomes for served items. Off-policy evaluation hard.

Multi-task: balances objectives; weights are political (the team owning each metric fights for theirs).

Pitfalls

Training-serving skew silently degrades model quality. Catch with serving-side feature logging.

Filter bubble: pure relevance starves diverse content. Add diversity constraints + exploration.

Counterfactual evaluation hard. Live A/B is the gold standard but slow.

Cold start: new users / items have no features. Fallback to content-based or popular content.

Latency drift: features cost grows silently as new features added; budget enforcement matters.