Apache Kafka
Delivery Semantics
Choose between at-most-once, at-least-once, and exactly-once — and know what each really costs.
Every messaging system delivers under one of three guarantees. At-most-once means messages may be lost but never duplicated (commit offset before processing). At-least-once means messages are never lost but may be redelivered (process, then commit) — the most common default. Exactly-once means each message affects the result once, with no loss and no duplicate.
At-least-once is mailing a letter with delivery confirmation — it always arrives, but if confirmation is lost you might send it again. Exactly-once is a bank transfer with a transaction id the bank uses to ignore any duplicate submission.
Key Concepts
1
Kafka achieves effective exactly-once within Kafka using the idempotent producer (dedupes retries via a producer id and sequence number) plus transactions, which atomically write to multiple partitions and commit consumer offsets in the same transaction (the read-process-write loop). The consumer must set isolation.level=read_committed to skip aborted records.
2
For systems writing to an external database, true end-to-end exactly-once usually still relies on idempotent writes or the outbox pattern, because Kafka's transactions don't extend into your database.