RabbitMQ
Prefetch / QoS & Competing Consumers
Balance load fairly across workers and prevent one consumer from hoarding messages.
The competing consumers pattern attaches multiple consumers to one queue; RabbitMQ round-robins messages among them, scaling throughput by adding workers. But naive round-robin can be unfair: a consumer that gets a slow message still receives its share of new ones, building a backlog while others sit idle.
A kitchen pass where each chef is handed only as many tickets as they can cook at once. A chef stuck on a complex dish does not keep getting new tickets piled up — the expeditor hands the next ticket to whoever is free.
Key Concepts
1
Prefetch (basic.qos with prefetch_count) fixes this. It caps how many unacknowledged messages a consumer may hold at once. With prefetch_count=1, a worker gets one message and receives the next only after acking — so fast workers naturally pull more and slow ones are not overloaded. Higher prefetch improves throughput by pipelining but risks uneven distribution and larger loss windows.
2
Choosing prefetch is a tuning exercise: low values for long, uneven tasks; higher values for many small, fast messages where round-trip latency dominates.