Is AMQP on the way to providing real business interoperability?

Posted by Steven Robbins on Aug 07, 2008 11:24 PM

Community

Architecture

Topics

Open Source ,

Specifications ,

Interop ,

Messaging

Tags

Business Architecture ,

AMQP ,

IONA ,

Middleware management ,

CohesiveFT ,

XMPP

Advanced Message Queuing Protocol (AMQP) came from inside of JPMorgan, thanks to John O'Hara. But his vision was bigger than just a new way to do things internally. The standard and open source technologies around it have been gaining momentum. Jeff Gould and others shed some light on where AMQP came from, who is driving it, and where it might be going.

IONA Technologies, a contributer to the AMQP specification, published a short description of what AMQP is and why to pursue it. AMQP is

• A defined set of messaging capabilities called the "Advanced Message Queuing Protocol Model" (AMQP Model). The AMQP Model consists of a set of well-specified components that route and store messages within a broker service, plus a simple set of rules for wiring these components together
• A network wire-level protocol, AMQP, that lets client applications talk to a broker and interact with the AMQP Model it implements.

In a three part series, Jeff Gould talked about the history of AMQP and John O'Hara's decision to go open source with it. After hiring iMatix to implement an AMQP project for JPMorganChase in 2003, the beta went live in 2006 and processed around 300 million messages per day.

O’Hara’s plan for AMQP was far more ambitious. From the start he wanted the new protocol to match the functionality of the high-end proprietary MOMs. It had to handle all the major use cases, including queue-style store-and-forward messaging, Tibco-style publish and subscribe, and reliable file transfer. The protocol had to be capable of carrying any kind of message, but the focus was on more efficient binary formats rather than text, because in app-to-app messaging human readability is not a paramount concern.

Gould went on to point out that it was O'Hara's quest to have an open standard that really pushed AMQP out of the bank and into the wild. The open standard was what brought players like Red Hat, Apache, WSO2, IONA, and Cisco to the table. Red Hat's MRG Messaging inside of its MRG initiative is an implementation of, and the core contributor to, the Apache Qpid project. LShift and CohesiveFT are jointly developing RabbitMQ, a "complete and highly reliable Enterprise Messaging system" that can be used to build an AMQP network or to enhance an established network. OpenAMQ from iMatix is another AMQP implementation product available. OpenAMQ was described as a product that "provides you with a framework on which to build distributed business applications which communicate using messages."

Even with this momentum, there are still detractors of AMQP. Lustratus Research, headed by somw long time IBM foks, wasn't ready to climb on the train with AMQP in 2007 when they said

guaranteed messaging is almost by definition the backbone of a company's mission-critical operations. If they weren't critical, then guaranteed delivery wouldn't be so important. So, where is the AMQP 'throat to choke'? Who do you run to when it fails, knocking out your key online systems? And going back to the testing question, who will invest in the huge expenses involved in stress testing and performance tuning to ensure it can meet the demands that will be placed on it?

Jean-Louis Seguineau talked about how AMQP is a late arrival to the game XMPP pubsub has already been playing:

XMPP pubsub extensions already address many of these features. Persistent nodes play the role of AMQP store-and-forward queues, instant nodes can be used for on demand AMQP queues. But AMQP also makes way for content-based routing, explicitly using message attributes. XMPP does not provide a similar explicit way to define content-based routing, as the pubsub extension leaves this decision to the implementation.

Alexis Richardson, of CohesiveFT, and Carl Trieloff, of RedHat, think differently about AMQP. In interviews with Gould both men talked about where they see the future of AMQP. Richardson noted it could become an internet protocol that does handles "everything that HTTP and SMTP do badly at present". In his view, AMQP takes care of what business needs:

In a business network you need trust, you need transactional integrity. What goes in has to be the same thing that comes out. You need smart routing and multiple topologies, you need to be able to manage traffic flows and do server-to-server federation. And you need complete interoperability between vendors and implementations. All this is what it takes to do transactional business messaging, and this is what AMQP delivers.

Trieloff talked about the progress of AMQP interoperability and said, "our experience has been that interop between the implementations is continually getting better, because the spec is getting better and less ambiguous, and because the developers are working very hard to track it. I think you need to have a very pragmatic approach to interop."

AMQP may not be quite ready to replace your current message oriented middleware, but it is getting closer and deserves a look if you are building or enhancing yours.

1. Back to top

   Why all this enthusiasm?

   Aug 8, 2008 4:57 AM by Guy Crets

   AMQP only focuses on the client-server part, where it allows an AMQP client to connect to different AMQP servers. Avoiding the hassle of configuring different (JMS) libraries. But AMQP doesn't focus on broker-2-broker communication. And AMQP isn't meant for use between business partners, only within the corporate firewall. I doubt whether AMQP is "gaining momentum". Why all of a sudden this enthusiasm around AMQP? See also http://guycrets.blogspot.com/2008/08/amqp-enthusiasm.html.

   Reply

2. Back to top

   Mainstream adoption?

   Aug 8, 2008 9:51 AM by murali kosaraju

   Wire level interoperability is good, especially when we are dealing with multiple MOM vendors. An AMQP implementation does not necessarily beat a non-AMQP based vendors like TIBCO or IBM MQ, in terms of performance, as they already have optimized versions of their proprietary wire level formats. I am very much enthused about the adoption, but I guess it takes at least 4 years to go mainstream. Murali Kosaraju

   Reply

3. Back to top

   Good question

   Aug 11, 2008 8:36 AM by alexis richardson

   Guy, thanks for your question. I have responded via your blog. In a nutshell, like with other internet protocols, there are many use cases within enterprise boundaries, and many new RabbitMQ use cases that we did not expect, keep popping up. And federation is in the works... Cheers, alexis

   Reply

4. Back to top

   Re: Why all this enthusiasm?

   Aug 12, 2008 7:59 AM by John O'Hara

   Check out the criteria for AMQP1.0's release: http://jira.amqp.org/confluence/display/AMQP/AMQP+1-0+Business+Requirements
   AMQP is certainly meant for use between business partners as well as within firms.
   I *really* like XMPP. The design centre for XMPP (and where the buzz is) is in pleasing end-users with great ways of interacting. The design centre for AMQP is all about meeting the demands of lights-out processing systems in a utilitarian way.
   The general communication patterns of XMPP and AMQP are superficially similar (aren’t all network protocols?) but the qualities of service are different. We could all be using SMTP+IMAP for corporate middleware, but those different design objectives matter (just for fun, read Sun's own definitions of Java Mail and JMS and try to spot the differences).
   It is a good thing that AMQP meets the needs of systems and XMPP meets the needs of people.
   Clouding these objectives harms the result; jack of all trades and master of none.
   Vive la difference! John

   Reply

• Older >