Apache Kafka
Replication, ISR & Durability
Survive broker failures without losing committed data.
Each partition is replicated across several brokers, set by the replication factor. One replica is the leader (serves all reads and writes); the rest are followers that continuously fetch from the leader. The set of replicas that are fully caught up is the In-Sync Replica (ISR) set.
A document signed in triplicate kept in three different offices. A change is only "official" once at least two offices hold the updated copy, so losing one office never loses the record.
Key Concepts
1
A write is considered committed once all members of the ISR have it (when acks=all). If the leader fails, the controller promotes an in-sync follower, so no acknowledged data is lost. The min.insync.replicas setting defines how many replicas must acknowledge before a write succeeds; combined with acks=all it is the core durability guarantee.
2
There is a deliberate trade-off: requiring more in-sync replicas increases durability but reduces availability for writes if replicas fall behind or go offline.