Don't want to read this article? The short version is, "If you didn't create a queue, don't pull messages from it." Removing messages from a queue is a destructive action.
This post is really a summary of the RabbitMQ docs. It's a quick highlight of the two big messaging patterns commonly seen on the broker.
How does Rabbit work?
RabbitMQ is, at its heart, a message broker. It has one focus. That focus is to deliver messages sent it to consumers. That's all it does, and it does it very well. It does not offer more complex patterns like sagas.
Key to understanding how Rabbit works is knowing that there are two parts to the broker: exchanges, and queues. Exchanges receive messages from publishers. Queues hold messages for subscribers. Inside the broker, messages move from exchanges to queues based on different rules. Once a message is taken from the queue, Rabbit forgets about it. Exchanges can send copies of a message to different queues, but they do not send multiple copies to the same queue.
Above is pictured the basic message flow. This can be expanded in two ways. Multiple consumers can get messages from the same queue. Multiple queues can be wired to the same exchange. Multiple consumers pulling messages from the same queue create the competing consumer pattern. Multiple queues wired to the same exchange create the Publish/Subscribe pattern. These patterns are not exclusive; it is possible to have competing consumers on one of the pub/sub queues.
What are Competing Consumers?
The Rabbit docs describe a work queue as a queue with multiple consumers connected. The intent is to distribute tasks to multiple consumers. This lets the tasks be distributed across all the consumers connected to a queue. Want to scale horizontally? Just add more consumers. Rabbit will deal the message out, one per consumer, until all messages have been dealt. It is important to know that each message will be sent only once. Once a message has been passed to a consumer, it will be removed from the queue. That means it will be unavailable to any other consumers of that queue. Below is an illustration of how the messages flow.
What is Publish/Subscribe?
Publish/subscribe is another common use of brokers. This is the pattern to use if you want multiple copies of the same message to be sent to different consumers. In this pattern, exchanges are used to send copies of messages to multiple queues. This pattern is especially useful for logging, audits, or passing messages to consumers with very different purposes. It is important to remember that multiple consumers on a queue will form competing consumers on that queue. The illustration below shows publish/subscribe, along with a competing consumer on one of the queues.
In closing...
Remember that when you want only one copy of a message to be distributed across consumers, the competing consumer pattern is the way to go. Using publish/subscribe will send copies of messages to different consumers. If you want the same message to go to different consumers, publish/subscribe is your friend.
