Apache Kafka
When to Use Kafka
Recognise the workloads Kafka is built for — and the ones it is wrong for.
Kafka shines as a high-throughput, durable event-streaming platform where data is read by many independent consumers, where replay of history matters, and where you want to retain events as a source of truth. Throughput in the millions of messages per second, horizontal scale via partitions, and long retention are its signature strengths.
Kafka is a freight rail network — enormous sustained throughput along fixed lines, unbeatable at volume, but overkill and clumsy for delivering a single parcel across town, where a courier (a queue) is the right tool.
Key Concepts
1
It is a poor fit when you need classic broker features: complex routing, per-message priority, per-message TTL, or per-message acknowledge/redelivery with delay. It is heavier to operate than a managed queue, and for simple point-to-point work with modest volume a queue like SQS or RabbitMQ is simpler and cheaper.
2
The decision usually comes down to: do you have a stream consumed many ways with replay needs (Kafka), or discrete tasks/commands handed to workers (a queue)?