reliability
Rate Limiting & Retries
Protect the service from runaway clients via rate limits, and help well-behaved clients back off via standard headers.
An API with no limits is at the mercy of its busiest client: a buggy loop, a scraper, or a thundering herd of retries can saturate the service and degrade it for everyone. Rate limiting caps how many requests a client may make in a window, protecting capacity and ensuring fair sharing, while a cooperative retry protocol lets well-behaved clients recover from transient failures without making things worse.
A movie theater that lets in N people per minute. Tell the rest "come back in 5 minutes" instead of letting the crowd crush the doors.
Key Concepts
1
On the server side, the common algorithms each shape traffic differently. A fixed-window counter is simple but allows bursts at window boundaries; a sliding window smooths that out; and the token bucket — the most popular — refills tokens at a steady rate and lets each request spend one, permitting short bursts up to the bucket size while bounding the long-run rate. When a client exceeds its limit the server returns 429 Too Many Requests, ideally with a Retry-After header and RateLimit-* headers telling the client its limit, remaining quota, and reset time, so it can self-regulate rather than hammering blindly. On the client side, the correct response to a 429 or a 503 is exponential backoff with jitter: wait, then double the wait on each subsequent failure, with a random component so that many clients failing simultaneously do not all retry in lockstep and create a synchronized spike.
429 Too Many RequestsRetry-AfterRateLimit-*429503
2
The points that distinguish a strong answer are pairing the two sides — the server signals limits via standard headers, the client honours them with backoff — and only retrying what is safe to retry. Idempotent operations (or those guarded by an idempotency key) can be retried freely; a non-idempotent POST without such a guard should not be blindly retried, or you risk duplicate side effects. Distinguishing retryable failures (429, 503, network timeouts) from non-retryable ones (400, 401, 404) is part of getting this right.
POST