Apache Kafka
Producers & Partitioning
Control which partition a record lands in — the lever that governs ordering and load balance.
When a producer sends a record, Kafka must decide which partition of the topic it goes to. That choice matters because ordering is guaranteed only within a partition, and because uneven partition assignment creates hot spots.
A post office sorting letters into numbered bins by postcode: every letter for the same postcode goes in the same bin and stays in arrival order, while different postcodes fill different bins in parallel.
Key Concepts
1
If the record has a key, Kafka hashes the key (murmur2 by default) and maps it to a partition, so all records with the same key always land in the same partition and are therefore strictly ordered relative to each other. If there is no key, the producer spreads records across partitions (sticky batching in modern clients) for even load.
2
Producers also batch records, compress them, and send asynchronously for throughput. The acks setting controls the durability/latency trade-off, and enabling idempotence prevents duplicates from retries.
3
What the interviewer is probing.
What the interviewer is probing.
4
1. "How does a producer choose a partition?" *Probing:* the mechanism. *Stalls:* "Randomly."
*Moves up:* by hash of the key if one is set, otherwise sticky batching across partitions — which is
why the key choice decides ordering and skew.
1. "How does a producer choose a partition?"
5
2. "What does acks=all actually guarantee?" *Probing:* the durability detail. *Stalls:* "It is
saved." *Moves up:* the leader and all in-sync replicas have it — but only min.insync.replicas makes
that meaningful, otherwise one surviving replica satisfies it.
2. "What does acks=all actually guarantee?"
6
3. "What causes partition skew?" *Probing:* the practical failure. *Stalls:* "Bad luck." *Moves
up:* a low-cardinality or uneven key, so one partition takes most traffic and one consumer becomes
the bottleneck.
3. "What causes partition skew?"
7
4. "What does enable.idempotence do?" *Probing:* the correctness feature. *Stalls:* "Nothing."
*Moves up:* sequence numbers per producer so a retry does not append a duplicate, which is the
foundation of exactly-once.
4. "What does enable.idempotence do?"