Apache Kafka
Producers & Partitioning
Control which partition a record lands in — the lever that governs ordering and load balance.
When a producer sends a record, Kafka must decide which partition of the topic it goes to. That choice matters because ordering is guaranteed only within a partition, and because uneven partition assignment creates hot spots.
A post office sorting letters into numbered bins by postcode: every letter for the same postcode goes in the same bin and stays in arrival order, while different postcodes fill different bins in parallel.
Key Concepts
1
If the record has a key, Kafka hashes the key (murmur2 by default) and maps it to a partition, so all records with the same key always land in the same partition and are therefore strictly ordered relative to each other. If there is no key, the producer spreads records across partitions (sticky batching in modern clients) for even load.
2
Producers also batch records, compress them, and send asynchronously for throughput. The acks setting controls the durability/latency trade-off, and enabling idempotence prevents duplicates from retries.