Message Queuing Options for .NET

by Jonathan Allen on Apr 22, 2011 |

When building larger scale applications, message queues are often very helpful for both distributing and aggregating workloads. Distributed workloads are a natural fit for message queues, simply having multiple readers attached to the same queue is often enough. Aggregation, usually implemented with one reader and multiple writers, is used to bundle lots of small updates into a large block. This facilitates the use of advanced database techniques such as the use bulk inserts instead of individual insert/update statements.

In the .NET ecosystem there are several options available for message queuing. This article highlights some of the more popular and unique offerings as well as the basic terminology needed to evaluate them.

Terminology

Queues: Queues are quite easy to understand. One or more writers add messages to the queue and one or more readers remove them in the same order. Queue are designed so that each message is delivered to one and only one reader, though the strength of this guarantee varies by product and the settings used.

Topics: The concept of topics may be new to .NET developers who have only used MSMQ. Unlike a queue, this uses a publish/subscribe mechanism where in each message is available to be read by every subscriber to the topic. The use of topics can be is very convenient for sending control messages, but care must be used to prevent a slow reader from fouling the topic for all other readers. Again, the exact effect of a slow reader varies by product and the settings used.

Transactions: Just like databases, message queues often support transactions. The usual semantic is messages are hidden, but not removed, between the time the message is read and the final commit is received by the server.

Durable Messages: A durable message is one that can survive a system reboot. For some types of messages this may be absolutely essential, while for others the performance cost for durable messages may exceed the benefits.

Service Bus: The term service bus is rather nebulous and it is often used to cover a wide variety of products and features. In the context of message queuing we are referring to products that extend the capabilities of a message queuing library.

Products

Microsoft Message Queuing or MSMQ is a venerable product dating back to the Windows NT 4 era. It offers a wide variety of bindings including C++, COM, straight .NET, and WCF. MSMQ doesn’t support topics, which severely limits the design patterns one can use without a lot of development effort. For this reason developers often consider layering a service bus such as MassTransit or NServiceBus on top of MSMQ. For those who need the upmost in reliability, this product has the ability to support Windows-style distributed transactions.

ActiveMQ from Apache is a popular choice for those looking cross platform support. The primary means of communicating with ActiveMQ is Apache’s OpenWire protocol. This is advertised as their high-performance client with bindings for Java, C, C++, and .NET. Another product called Stomp is recommend for accessing ActiveMQ from Ruby, Perl, Python, PHP, ActionScript/Flash, and Smalltalk. ActiveMQ is fully compliant with JMS 1.1 and J2EE 1.4 and offers both queues and topics. This product supports JMS and XA style transactions.

ZeroMQ is a very lightweight message queuing product. Unlike the other products it doesn’t have a stand-alone server; messages are sent directly from application to application. It is designed to offer very high performance, Mike Hadlow was able to show ZeroMQ runs at least 10 times faster than MSMQ or ActiveMQ using 1K messages. It does this by sacrificing the reliability features built into the other products including durable messages and distributed transactions. Like ActiveMQ this is available for most operating systems and languages.

WebSphere MQ: This is actually a family of messaging products from IBM that include support for “HTTP, REST, File Transfers and JMS”. Formally known as MQSeries, it is the only product on our list that requires a separate commercial license. (MSMQ is part of Windows, the rest are open source.) Dating back to 1992, this product is as old as MSMQ but has been seeing far more active development. It can be found on virtually every server-class operating system still in use and has bindings for everything from COBOL through .NET WCF.

Related Editorial

Beyond HTML5 and Native Development: Build Richer, More Functional Mobile Applications

From Killer Apps to App Killers: The Devastating Effect of Architecturally Complex Defects

Free Book: Graph Databases (by O'Reilly) - Download Now

Impact of Java EE Frameworks on the Structural Quality of Applications

APM Survival Guide from a Veteran Performance Geek

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Community comments

RabbitMQ by John Downey Posted

Re: RabbitMQ by Robert Sullivan Posted

Re: RabbitMQ by Jonathan Allen Posted

Re: RabbitMQ by Robert Sullivan Posted

Re: RabbitMQ by Richard Clayton Posted

Re: RabbitMQ by Jonathan Allen Posted

Apache Qpid and WSO2 MB by Paul Fremantle Posted

Really? by Mark N Posted

Seems to be missing a few by Ken Wiebke Posted

Re: Seems to be missing a few by Richard Clayton Posted

Re: Seems to be missing a few by Bryan Ross Posted

Microsoft MQ? Really? by Harry Lord Posted

Re: Microsoft MQ? Really? by Robert Sullivan Posted

.NET applications should be using the NMS API for messaging. by Hiram Chirino Posted

Microsoft BizTalk Server by Leo Gan Posted

And the verdict is... by Morgoth Melkor Posted

ZeroMQ is lightweight because it is low-level by Jared Smith Posted

Re: ZeroMQ is lightweight because it is low-level by rajendar reddy Posted

RabbitMQ by John Downey

I was surprised to see that RabbitMQ was left out of the products section as they have a pretty solid .NET client including some WCF integration. ZeroMQ on the other hand is really nothing more then a bunch of useful abstractions around raw sockets.

Re: RabbitMQ by Robert Sullivan

Yes, it wasn't a very comprehensive list. MSMQ "venerable"? Not so much. That title belongs to MQSeries.

Really? by Mark N

As others said, no mention of RabbitMQ nor MQSeries? And ZeroMQ does not really seem to be any more of message queuing than WCF or Web Services. It is just messaging.

To me, this shows the sad state of affairs in the .NET world when it comes to Message Queuing.

Seems to be missing a few by Ken Wiebke

TIBCO EMS, IBM MQ, Oracle Adv Queues, Weblogic, Redis, ...

Re: RabbitMQ by Richard Clayton

Agreed. RabbitMQ is a pretty big player in this space.

Re: Seems to be missing a few by Richard Clayton

The author was talking about MQ's targeting .NET. I wouldn't call Redis a Message Queue, though it could be used to "back" a Message Queue implementation (RestMQ [www.restmq.com/] being one such implementation). Does Weblogic and IBM MQ support a .NET client?

Re: Seems to be missing a few by Bryan Ross

Both Weblogic and WebSphere MQ support .NET clients.

Re: RabbitMQ by Jonathan Allen

Thank you for that link. I was going to include MQSeries but I couldn't find any information on it. I swear it is like IBM doesn't want us to learn about their products.

As for venerable, I was just trying to be polite. What would call a product that is over a decade old and yet has nearly all of the same limitations as its original version?

Re: RabbitMQ by Jonathan Allen

If you are interested in writing a case study on how you use RabbitMQ (or another message queuing product) in your professional work then please contact me at jonathan@infoq.com . We are always looking for good pieces on how people actually use this stuff.

Apache Qpid and WSO2 MB by Paul Fremantle

Both Apache Qpid and the (currently beta) WSO2 Message Broker support native .NET clients.

Paul Fremantle, CTO, WSO2 and Apache Qpid contributor

Microsoft MQ? Really? by Harry Lord

I think the information in this post is terribly out of date. MSMQ was not included in SQL Server 2008 and was only added back when a small loud group complained. Reporting Services and StreamInsight have replaced MQ. Please be sure to have updated information before suggesting a product.

Re: Microsoft MQ? Really? by Robert Sullivan

This article is about Message Queues, not about Reporting Services or event managers, especially as they are tied to SQL Server. The products in this article are standalone queue engines. You're comparing add-on services with standalone products. And note the author doesn't suggest MSMQ, noting that it is a bit dated and doesn't support modern features such as topics.

Re: RabbitMQ by Robert Sullivan

What would call a product that is over a decade old and yet has nearly all of the same limitations as its original version?

Um, I would call it Visual SourceSafe ;)

Ok, good point, and thanks for updating the article. Part of the fault lies with IBM marketing. They decided to latch on to the "websphere" name, and renamed all their MQ products to use the word "WebSphere". But it's an excellent product.

.NET applications should be using the NMS API for messaging. by Hiram Chirino

If your an application developer that needs to use a messaging API, why lock yourself into a single vendor's API? Instead, use the NMS API. It's an abstract messaging API which has several vendor implementations so that you can easily switch your application between vendors as needed. It currently support ActiveMQ, STOMP, MSMQ, Tibco EMS, and even WCF messaging servers.

It's maintained by the Apache Software Foundation so it's open source and actively maintained. If your favorite messaging solution does not have a NMS implementation, and you want to use it from .NET, push that vendor to implement a NMS API binding! Avoid him locking you in. :)

Microsoft BizTalk Server by Leo Gan

Microsoft BizTalk Server is a great implementation of the message server.
If you mentioned IBM WebSphere, the BizTalk must be in this list.

Then maybe it worth to tell about Queues in AppFabric (Windows and Azure).

And the verdict is... by Morgoth Melkor

Unless you spend your life making something like RabbitMQ work with .Net, there is nothing really useful. MSMQ.. You must be joking. Why not include Biztalk then?

ZeroMQ is lightweight because it is low-level by Jared Smith

Your mention of ZeroMQ caught my attention. I investigated that site and was delighted to find something for which I have been searching a long time. I devoured The Guide and will be using ZeroMQ going forward in all of my experimental programming, until I know it well enough to introduce it to my team.

I write here because you said "It [is fast] by sacrificing the reliability features built into the other products including durable messages and distributed transactions." I believe this unfairly compares ZeroMQ's intentional sparseness with heavier protocols. It is quite simple to build distributed transactions in a lightweight manner on top of ZeroMQ, and it does have durable messages.

Thank you for the article, it opened a door for me to understand how to integrate true messaging into my C# and PHP skillset.

-Jared

Re: ZeroMQ is lightweight because it is low-level by rajendar reddy

www.4microsoftsolutions.com/post/Message-Queue-...