When building larger scale applications, message queues are often very helpful for both distributing and aggregating workloads. Distributed workloads are a natural fit for message queues, simply having multiple readers attached to the same queue is often enough. Aggregation, usually implemented with one reader and multiple writers, is used to bundle lots of small updates into a large block. This facilitates the use of advanced database techniques such as the use bulk inserts instead of individual insert/update statements.
In the .NET ecosystem there are several options available for message queuing. This article highlights some of the more popular and unique offerings as well as the basic terminology needed to evaluate them.
Terminology
Queues: Queues are quite easy to understand. One or more writers add messages to the queue and one or more readers remove them in the same order. Queue are designed so that each message is delivered to one and only one reader, though the strength of this guarantee varies by product and the settings used.
Topics: The concept of topics may be new to .NET developers who have only used MSMQ. Unlike a queue, this uses a publish/subscribe mechanism where in each message is available to be read by every subscriber to the topic. The use of topics can be is very convenient for sending control messages, but care must be used to prevent a slow reader from fouling the topic for all other readers. Again, the exact effect of a slow reader varies by product and the settings used.
Transactions: Just like databases, message queues often support transactions. The usual semantic is messages are hidden, but not removed, between the time the message is read and the final commit is received by the server.
Durable Messages: A durable message is one that can survive a system reboot. For some types of messages this may be absolutely essential, while for others the performance cost for durable messages may exceed the benefits.
Service Bus: The term service bus is rather nebulous and it is often used to cover a wide variety of products and features. In the context of message queuing we are referring to products that extend the capabilities of a message queuing library.
Products
Microsoft Message Queuing or MSMQ is a venerable product dating back to the Windows NT 4 era. It offers a wide variety of bindings including C++, COM, straight .NET, and WCF. MSMQ doesn’t support topics, which severely limits the design patterns one can use without a lot of development effort. For this reason developers often consider layering a service bus such as MassTransit or NServiceBus on top of MSMQ. For those who need the upmost in reliability, this product has the ability to support Windows-style distributed transactions.
ActiveMQ from Apache is a popular choice for those looking cross platform support. The primary means of communicating with ActiveMQ is Apache’s OpenWire protocol. This is advertised as their high-performance client with bindings for Java, C, C++, and .NET. Another product called Stomp is recommend for accessing ActiveMQ from Ruby, Perl, Python, PHP, ActionScript/Flash, and Smalltalk. ActiveMQ is fully compliant with JMS 1.1 and J2EE 1.4 and offers both queues and topics. This product supports JMS and XA style transactions.
ZeroMQ is a very lightweight message queuing product. Unlike the other products it doesn’t have a stand-alone server; messages are sent directly from application to application. It is designed to offer very high performance, Mike Hadlow was able to show ZeroMQ runs at least 10 times faster than MSMQ or ActiveMQ using 1K messages. It does this by sacrificing the reliability features built into the other products including durable messages and distributed transactions. Like ActiveMQ this is available for most operating systems and languages.
WebSphere MQ: This is actually a family of messaging products from IBM that include support for “HTTP, REST, File Transfers and JMS”. Formally known as MQSeries, it is the only product on our list that requires a separate commercial license. (MSMQ is part of Windows, the rest are open source.) Dating back to 1992, this product is as old as MSMQ but has been seeing far more active development. It can be found on virtually every server-class operating system still in use and has bindings for everything from COBOL through .NET WCF.