RabbitMQ
Prefetch / QoS & Competing Consumers
Balance load fairly across workers and prevent one consumer from hoarding messages.
The competing consumers pattern attaches multiple consumers to one queue; RabbitMQ round-robins messages among them, scaling throughput by adding workers. But naive round-robin can be unfair: a consumer that gets a slow message still receives its share of new ones, building a backlog while others sit idle.
A kitchen pass where each chef is handed only as many tickets as they can cook at once. A chef stuck on a complex dish does not keep getting new tickets piled up — the expeditor hands the next ticket to whoever is free.
Key Concepts
1
Prefetch (basic.qos with prefetch_count) fixes this. It caps how many unacknowledged messages a consumer may hold at once. With prefetch_count=1, a worker gets one message and receives the next only after acking — so fast workers naturally pull more and slow ones are not overloaded. Higher prefetch improves throughput by pipelining but risks uneven distribution and larger loss windows.
2
Choosing prefetch is a tuning exercise: low values for long, uneven tasks; higher values for many small, fast messages where round-trip latency dominates.
3
What the interviewer is probing.
What the interviewer is probing.
4
1. "What does prefetch control?" *Probing:* the mechanism. *Stalls:* "Speed." *Moves up:* how
many unacknowledged messages a consumer may hold — which is the main lever for fair distribution.
1. "What does prefetch control?"
5
2. "What happens with an unlimited prefetch?" *Probing:* the failure. *Stalls:* "Nothing."
*Moves up:* one consumer buffers everything while the others idle, and a crash redelivers that whole
buffer.
2. "What happens with an unlimited prefetch?"
6
3. "What prefetch would you start with?" *Probing:* the practical answer. *Stalls:* "One
hundred." *Moves up:* low — often one for slow uneven tasks, higher for fast uniform ones; it is
tuned from observed distribution.
3. "What prefetch would you start with?"
7
4. "How does the competing consumers pattern scale?" *Probing:* the model. *Stalls:* "Add
queues." *Moves up:* add consumers to one queue — unlike Kafka there is no partition ceiling on
consumer count.
4. "How does the competing consumers pattern scale?"