Message Queue Shootout!

I’ve spent an interesting week evaluating various Message Queue products. The motivation behind this is a client that has somewhat high performance requirements. They have bursts of over a million simultaneous messages. Currently they’re using a SQL server based solution, but it’s not ideal, and I’m suggesting they look at Message Queuing products as an alternative.

In order to get a completely unscientific feel for the performance of some likely contenders, I put together a little test. Each queue would be asked to send one million 1K messages and receive them again. The test was done in somewhat of a hurry, so I haven’t tweaked any settings, just installed each MQ and done the simplest send and receive I could manage after a quick glance at the docs. So this is true out-of-the-box performance. I fully accept that this is going to penalise MQ products that have conservative default configurations.

The candidates are:

• MSMQ. The default choice for where only the products of Redmond are considered worthy. For my clients, if MSMQ can rise to the challenge, then they should use it. The main points against it are its lack of sophistication; nothing but send and receive; and it’s arbitrary hard limits, such as the 4MB maximum message size. However, with something like MassTransit or NServiceBus layered on top, it’s entirely possible to do serious work with it.
• ActiveMQ. The stalwart of the Java world. It has long service and ubiquity going for it. It’s also cross platform and would provide a natural integration point for non-Microsoft platform products. However, it would have to perform better than MSMQ to have a look-in.
• RabbitMQ. I’ve been hearing excellent things about this message broker written in Erlang. It supports the open AMQP (Advanced Message Queuing Protocol) with the potential to avoid vendor lock-in and benefiting from a wide range of clients in almost every language. AMQP provides some pretty sophisticated messaging patterns, so there’s less need for a MassTransit or NServiceBus. It also has ‘enterprise’ resilience and durability. That’s something my client is very interested in.
• ZeroMQ. I only discovered this MQ product when I was researching AMQP. The company that created it were part of the AMQP group and had a product called OpenAMQ. However, they parted company with AMQP quite dramatically, complaining that it had lost its way and was becoming over complicated. You can read their Dear John letter here. ZeroMQ has a unique broker-less model which means that unlike all the other products under test, you don’t need to install and run a message queuing server, or broker. Simply reference the ZeroMQ library, it’s on NuGet, and you can happily send messages between your applications. Interestingly, they are also placing it as a way of creating Erlang style actors in any language by leveraging ZeroMQ’s blazingly fast in-process messaging.

Getting all four MQ products up and running was fun. There’s a definite overhead when you have to install a product based on a non-Windows platform. ActiveMQ required Java on the target machine and RabbitMQ needed Erlang. Both installed without a hitch, but I’m concerned that it’s another layer that can go wrong in production. I would be asking the infrastructure people to understand and maintain unfamiliar runtimes if we chose either of these. ActiveMQ, RabbitMQ and MSMQ all have server processes that need to be monitored and configured, another support concern.

ZeroMQ, with its brokerless architecture doesn’t require any server process or runtime. In effect, your application endpoints play the server role. This makes deployment simpler, but the worry is that there’s no obvious place to go looking when things go wrong. ZeroMQ, as far as I can tell, only provides non-durable queues. You are expected to provide your own auditing and recovery where you need it. To be honest, I’m not even sure that ZeroMQ should be in this test, it’s such a different concept than the other MQ products.

So without further chit-chat, here are the results. This is messages per second, for send and receive. During the transmission of one million 1K messages. The tests were executed a machine running Windows Vista.

As you can see, there’s ZeroMQ and the others. Its performance is staggering. To be fair, ZeroMQ is quite a different beast from the others, but even so, the results are clear: if you want one application to send messages to another as quickly as possible, you need ZeroMQ. This is especially true if you don’t particularly mind loosing the occasional message.

To be honest, I was hoping for more from Rabbit. However much one tries to be fair in these things, you inevitably have a favourite, and everything I’d read and heard about Rabbit made me think it was probably the best choice. But with these results, I’m going to be hard pressed to sell it over MSMQ.

If you’d like to run the tests for yourself, my test code is on GitHub here. I’d be very (no very very) interested in how the tests can be tweaked, so if you can get substantially better figures, please let me know.

https://github.com/mikehadlow/Mike.MQShootout