Getting Started with RabbitMQ Federations and EasyNetQ

Preface

One of the uses we needed out of RabbitMQ was passing messages between two separate departments. These departments each had their own Rabbit cluster. They each had their own LAN. This led us down the path of using a federated exchange to transport messages between the clusters.

The source code is available on GitHub.

Stuff Used

1. Two RabbitMQ clusters.
2. Management plugin.
3. Federation plugin.
4. EasyNetQ client library.
5. Similarly named virtual hosts on each cluster (FederationDemo is used in this example).

Disclaimer

RabbitMQ is very flexible. There are always about a dozen ways to solve the same problem. This is an example of how to use topic-based routing across a federated exchange with EasyNetQ as a client library. This is not meant to be the one true way of doing it.

About Federations

A federation allows messages to flow from one RabbitMQ cluster to one or more downstream clusters. The simplest RabbitMQ federation exists between two clusters: an upstream and a downstream. The downstream cluster can be thought of as subscribing to messages from the upstream cluster.

Getting Started

Downstream (or Beta)

The downstream cluster will be receiving messages from the upstream server. There are a few things which have to be set up on the downstream cluster:

1. An upstream address.
2. A policy to identify the federated exchange.
3. An exchange to be federated in the virtual host.
4. Exchanges for the messages bound to the subscriber.

Define the Policy

A good place to start is by defining the policy. Policies are managed in the Admin->Policies screen. It’s important that the correct virtual host is selected. You’ll also want to make note of the pattern you use.

Define the Upstream

Next up is defining the address of the upstream cluster. This can be done in the Admini->Federation Upstreams screen. Again, be sure to have the correct virtual host selected.

Message Queues and Exchanges

The easiest way to create the queues and exchanges is to let EasyNetQ do it for us. Subscribing to a message with EasyNetQ causes it to create the appropriate Queues and Exchanges. It also binds the Exchanges to the Queues. After running the subscriber, create an excahnge named ‘fed.exchange’. The policy should be applied if the exchange name matches the pattern.

The subscriber code would be similar to the following:

        public void Run()
        {
            bus.Subscribe<VisaTransaction>("Beta", PrintTransaction, configuration => configuration.WithTopic(typeof(VisaTransaction).Name));
            bus.Subscribe<MasterCardTransaction>("Beta", PrintTransaction,
                                                 configuration => configuration.WithTopic(typeof(MasterCardTransaction).Name));

            SpinWait.SpinUntil(() => Console.ReadKey().Key == ConsoleKey.Escape);
        }

Once the federated exchange has been created, it will need to be bound to the message exchanges. Since we’re using topics to route the various messages, we’ll want to ensure that the topics are passed through. For now, we’ll just use a ‘#’ to indicate that all topics should be passed through. The bindings should then look like the following image:

Upstream (or Alpha)

Setting up the upstream is pretty simple. The virtual host needs to be created. It needs the appropriate users assigned to it. We’ll be using the default guest account for this demo. If the guest account isn’t granted access to the virtual host, then the the downstream cluster will not be able to establish a connection.

As with the subscriber, EasyNetQ can create the exchanges for us. Messages will be discarded, until the exchanges are bound to something. Fortunately, the outbound exchange will have been created when the downstream cluster connects.

The publishing code looks like this:

        private void PublishMessage<T>() where T : Transaction
        {
            var message = Builder<T>.CreateNew().Build();
            using (var channel = bus.OpenPublishChannel())
                channel.Publish(message, configuration => configuration.WithTopic(typeof(T).Name));
        }

Here, we’re going to bind the exchanges to the outbound federated exchange. We’ll want to bind the upstream clusters exchanges to the federated exchange. It’s important to remember the ‘#’ as a routing key. These bindings should be similar to the following picture.

Running the Apps

When the apps are run, you can see that the messages are transported from the publisher to the client. The publisher sends them to the upstream cluster. The upstream cluster then dispatches them to the downstream cluster. The downstream cluster then passes them to the subscriber.

Note that the highlighted bits show the messages were translated into the correct types:

Wrapping It Up

That’s the basics of getting a federation up and running between two clusters. The federated exchange is capable of transporting two different types of messages. Those messages are routed to the correct subscriber, based on the topic of the message.

One final note… There are a lot of other considerations when federating clusters and using topic-based routing. This blog represents the very basics. You’ll want to make sure you’ve familiarized yourself with the subject.