RabbitMQ connection_closed_abruptly error

Preface

One of the hiccups we encountered when starting with RabbitMQ occurred when we put a proxy in front of our cluster. The proxy began probing the broker's nodes to ensure they were alive. This resulted in a lot of spam in the logs:

=INFO REPORT==== 21-Aug-2013::11:14:32 ===
accepting AMQP connection <0.12744.35> (<proxy ip>:55613 -> <rabbit ip>:5672)

=WARNING REPORT==== 21-Aug-2013::11:14:32 ===
closing AMQP connection <0.12744.35> (<proxy ip>:55613 -> <rabbit ip>:5672):
connection_closed_abruptly

A Typical(?) Cluster

I'm not certain there is a typical setup for a RabbitMQ cluster. In our case, we went with what we've been calling a binary star. We took this phrase from the 0mq docs. The idea is pretty simple: it's an active/active or active/passive pair with mirrored queues.

A proxy is placed between the clients and the binary star. This separates the clients from the cluster implementation. The biggest benefit for is is the consistent IP for clients. This simplifies the client configurations. It also allows us to perform maintenance on the cluster w/o interrupting service.

RabbitMQ.config

The default Windows location for the config file, rabbitmq.config, is in the %appdata%/RabbitMQ directory. It's possible to change this location, but we'll assume the default location for now. A clean install will often not have this config file.

The first step is to create the rabbitmq.config file. You can create this in any text editor. Just remember that it must be named rabbitmq.config. It may not have a .txt or any other extension. Add the following to the file, and save it:

[
	{rabbit, [
		{log_levels,[{connection, error}]},
	]}
].

This will configure the connection logging to only log errors. The other types of logging, like start up messages, will be unaffected by this.

Making It Take Affect

The RabbitMQ documents indicate that the service should be restarted for the new config to take affect. We have found it necessary to re-install the broker. Fortunately, re-installation of the broker did not wipe the setup. The steps are as follows:

1. Stop the service in the services management screen.
2. Uninstall RabbitMQ.
3. Install RabbiMQ.
4. Verify the service is running.

More RabbitMQ.config

It's also possible to enable TCP keepalive support, by adding the appropriate line (#11):

[
	{rabbit, [
		{log_levels,[{connection, error}]},
		{tcp_listen_options,
		   [binary,
			 {packet,raw},
			 {reuseaddr,true},
			 {backlog,128},
			 {nodelay,true},
			 {exit_on_close,false},
			 {keepalive,true}]}
	]}
].

Wrap Up

The big gotcha was having to re-install the RabbitMQ service to get the .config file changes to stick. Otherwise, this was a basic config change to ease up on the logging.  Hopefully, this will help someone else...