BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Apollo Next Generation Message Queuing Posts Some Impressive Benchmarks

by Rick Hightower on Dec 21, 2011 |

Apollo is a next generation message queuing solution. Hiram Chirino, Apollo project lead and fellow at FuseSource, recently posted some impressive Apollo benchmarks against RabbitMQ, HornetQ, and ActiveMQ. The benchmarks indicate that Apollo will be on a lot of developer's roadmaps for messaging.

Apollo is a rewrite of ActiveMQ. Apollo's goal is to be faster, more reliable, and easier to maintain. The current benchmarks seem to indicate that this goal has been reached.

Apollo uses a significantly different threading and message dispatching architecture. Currently Apollo only supports the STOMP protocol. The STOMP protocol is a wire protocol for messaging that is supported by 20 different clients and 11 servers (clients for C, C#, C++, Objective C, Ruby, Python, Perl, and more). Future versions will support OpenWire protocol. OpenWire is the wire protocol supported by ActiveMQ which has clients written in C, C++, and C#. Apollo was designed to be compatible with the possibility of adding AMQP later.

Apollo uses a Reactor Based Thread Model and it uses the HawtDispatch project. HawtDispatch is modeled after the libdispatch API. The libdispatch API powers the Grand Central Dispatch (GCD) technology in OS X. GCD was developed by Apple Inc. to optimize support for multi-core processors. GCD allows routing which means parallel running tasks are queued up for execution. When processing resources become available the tasks are scheduling to execute on available processor cores.

The improved thread model allows Apollo to perform really well. But the new thread model means all tasks must execute non-blocking, lock-free and wait free. This forced ActiveMQ broker to be drastically rewritten to support the new thread model. What started out as a fork of ActiveMQ ended up being a major rewrite.

The new broker code is written in Scala 2.9. Scala was chosen because it was deemed a more concise way to implement async callbacks. Apollo uses Scala's support for partial functions and closures.

Hiram explains the benchmarks and put the code for the benchmarks in GitHub. He ran the tests in several environments including Amazon Linux: EC2 High-CPU Extra Large Instance, Ubuntu 11.10: Quad-Core Intel 2600K (3.4 GHz), and OS X: 2 x Quad-Core Intel Xeon (3 GHz).

InfoQ got a chance to catch up with Hiram and get his thoughts on Apollo, STOMP, AMQP and the benchmarks.

InfoQ: Why does ActiveMQ need a new version? and what is Apollo?

I am always looking at how we can improve ActiveMQ in future versions. I thought that using a Grand Central Dispatch (GCD) style threading API could provide significant improvements in ActiveMQ's performance on multicore machines with large numbers of active connections and destinations. GCD style threading allows ActiveMQ to run with a constant number of threads and with little contention even when a single resource like a queue is being highly contended by multiple clients.

By starting a subproject, we can work on a future generation of ActiveMQ without disrupting our current user base. So far, the results have been very impressive, and we have a lot of interest in the subproject. Sometimes a subproject will be rolled into the core project as a capability, and other times like with Apache Camel, a subproject grows to become a complementary project.

InfoQ: Are you satisfied with Apollo progress? Can you briefly describe how it did in the benchmarks?

I am extremely satisfied. The STOMP benchmarks I've run show that Apollo is competitive with all the major STOMP servers on the market and in messaging scenarios where there is contention on a single queue or topic, it does significantly better than the competition. This is a great validation of the decision to use a GCD style threading architecture.

InfoQ: Why does Apollo only support STOMP?

STOMP is a messaging protocol which shares many of the same design principles as HTTP. Since it's text based and simple to parse, a large organic community of cross language clients and servers have sprung around STOMP which makes it an ideal protocol to use when you need to do asynchronous messaging across many languages and platforms. ActiveMQ was one of the first severs to support STOMP, so it was natural that STOMP would be the first protocol which Apollo implemented since it is simple yet still exercises all the core messaging concepts of a messaging engine.

Protocols are pluggable in Apollo. We have already started working on an implementation for main protocol which ActiveMQ uses which is Openwire. Once the Openwire protocol is fully implement and a couple of more features are added in, we will be able use Apollo to power the next major release of ActiveMQ.

InfoQ: What is your relationship to the Apollo project?

I'm the founder of the Apollo subproject and PMC chair of the ActiveMQ TLP.

InfoQ: Why doesn't ActiveMQ and/or Apollo support AMQP?

Members of the ActiveMQ team have been following the AMQP specification, but is has continued to change dramatically while under development. Now that AMQP 1.0 has finally been released and backwards compatibility it is possible, we will be looking at adding support for AMQP. Apollo has been architected in such a way that it can support AMQP, which as a specification puts a lot of requirements on the server architecture.

InfoQ: What advantages does OpenWire have over AMQP? Do any other projects besides ActiveMQ support OpenWire?

OpenWire allows ActiveMQ to implement the full JMS specification including distributed transactions using XA. It has native Java, C++, and .NET clients with automatic connection failure handling/recovery so that applications don't have to implement failure recovery logic. The downside is that it's only been adopted by the ActiveMQ messaging server.

InfoQ: What advantages does STOMP have over AMQP?

STOMP is simpler to parse and understand than AMQP. Like HTTP, a TCP stream of a STOMP client and server are easy for a human to visually inspect and understand. That makes implementing interoperable clients and servers simple. Stomp has 10 servers, and over 20 client implementations avail for a variety of clients and servers. STOMP also gives you a large amount of flexibility. It does not force servers to implement particular messaging semantics. While most severs implement queue and topic style semantics, they are free to implement additional delivery and routing semantics that may be useful to their users.

 

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

clustering by Matt Giacomini

It will be interesting to see how benchmarks turn out at some point down the road when clustering is brought into the picture. Clustering is where most messaging server start showing their dark side (in some cases very dark side).

Re: clustering by Richard Hightower

Good point.

Re: clustering by Hiram Chirino

Hi Matt,

Doing clustering in a highly scaleable way while retaining all the features a single message server can provide can be challenging but we have a couple of ideas I have be experimenting with for the next version of Apollo. They basically boil down to using ZooKeeper to partition destinations across the cluster and then hide the partitioning by automatically rerouting messaging operations to the correct partitions.

Hiram
FuseSource

Re: clustering by Matt Giacomini

Hi Hiram,

I didn't mean to say insinuate that it was an easy problem to solve. I like your concept for adding ZK to the mix. I am a big ZK fan, we have had very good luck with using it to power our internal point-to-point message queues.

~Matt

Atrollo by Tim Fox

These benchmarks are somewhat disingenuous.

Apollo has been built from the ground up to be a STOMP messaging system, it doesn't implement any other protocol. RabbitMQ is primarily an AMQP messaging system and HornetQ is primarily a JMS messaging system. The STOMP plugins for both RabbitMQ and HornetQ are not really part of their core functionality.

Having worked on both the HornetQ and RabbitMQ projects I know that the vast majority of the work there is around their primary protocols/apis (AMQP and JMS respectively), and not on their STOMP implementations. I suspect if you were to compare, say, the performance of HornetQ JMS with Apollo the figures would speak a different story.

Lies, damned lies, and benchmarks ;)

Re: Atrollo by Matt Giacomini

Hi Tim,

Are you saying that in HornetQ and RabbitMQ (btw: I like both of these products) the STOMP protocol implementation are the limiting factor on those platforms?

If so, it would be interesting to have somebody benchmark HornetQ and RabbitMQ not against each other, but against themselves using different protocols.

Thanks,
~Matt

Re: Atrollo by Tim Fox

Hi Tim,

Are you saying that in HornetQ and RabbitMQ (btw: I like both of these products) the STOMP protocol implementation are the limiting factor on those platforms?


I think it's safe to say that if you're interested in raw performance with either HornetQ or RabbitMQ, you don't use STOMP.

Now, if STOMP becomes more popular (and personally I think STOMP is a great protocol that deserves to be more popular) then I'm sure the respective development teams would spend more time in optimising their STOMP implementations.

But until then it's only natural that they spend most of their resources on their main user groups: JMS and AMQP users.

Tim Fox as in JBoss HornetQ, Tim Fox... by Richard Hightower

Tim thanks for commenting on this thread. I value your opinion and background.




In this context.... I'd like to properly introduce you as the person who wrote HornetQ. You are or were the HornetQ lead. Correct? Unless you are another Tim Fox. Not that your opinion is biased, but you do get paid to work on HornetQ.



Tim Fox...
I think it's safe to say that if you're interested in raw performance with either HornetQ or RabbitMQ, you don't use STOMP.



From what I read, it does not seem like Apollo is STOMP centric. Rather it seems it is nascent and they decided to start somewhere and decided that somewhere was STOMP. Like HornetQ (according to the website anyway), Apollo was designed to be protocol agnostic as much as possible. I would not give too much weight to your argument unless you can show via some profiling how much time is spent doing STOMP protocol handling versus the other parts of the messaging. I imagine for some of the benchmarks STOMP versus OpenWire is dust on the scale.



Tim Fox...
Now, if STOMP becomes more popular (and personally I think STOMP is a great protocol that deserves to be more popular) then I'm sure the respective development teams would spend more time in optimising their STOMP implementations.



"if STOMP becomes more popular"

How do you gauge popularity? It would seem that STOMP has more servers and more clients than AMQP. Also since ActiveMQ uses STOMP to support all other clients and ActiveMQ is much bigger than HornetQ or RabbitMQ mind share (currently). I would say not only is STOMP likely as popular or more popular than AMQP but that it is going to be a clear winner going forward. The world loves simple wire protocols. Memcached. HTTP. :)



RabbitMQ versus ActiveMQ versus HornetQ
www.indeed.com/jobtrends?q=RabbitMQ%2C+ActiveMQ...


I would say AMQP is not popular. Rather RabbitMQ is popular. It seems the other AMQP providers seem to be a mere pimple.



RabbitMQ versus other AMQP providers....
www.indeed.com/trendgraph/jobgraph.png?q=Rabbit...


And RabbitMQ popularity has a lot more to do with it being backed and supported by SpringSource and VMWare then it does on its technical merits (which might be great). We all know how well EJB 1.0 was marketed and we see where that got us. RabbitMQ might be great. This is not my point per se.




AMQP is not that hot...I base this on the fact that the people that came up with AMQP, abandon it, and said it was bloated and dumb.




Tim Fox....

These benchmarks are somewhat disingenuous.


Apollo has been built from the ground up to be a STOMP messaging system, it doesn't implement any other protocol. RabbitMQ is primarily an AMQP messaging system and HornetQ is primarily a JMS messaging system. The STOMP plugins for both RabbitMQ and HornetQ are not really part of their core functionality.


Having worked on both the HornetQ and RabbitMQ projects I know that the vast majority of the work there is around their primary protocols/apis (AMQP and JMS respectively), and not on their STOMP implementations. I suspect if you were to compare, say, the performance of HornetQ JMS with Apollo the figures would speak a different story.

Lies, damned lies, and benchmarks ;)




I don't buy that AMQP is an unqualified success. I have used STOMP to send messages that were consumed and produced for Python, C and Java using ActiveMQ after failing to get anything to work with any AMQP provider (granted AMQP was young at the time). In the AMQP space, I assert that only RabbitMQ is doing well. That is it. STOMP was brain dead simple to setup and use. It just worked.




I'd like to compare Tim Fox's assertion of being disingenuous with Hiram's thoughts on the benchmark.

Hiram...
I’ve tried to make the benchmark as fair as possible to all the contenders, all the source code to the benchmark is available on github. Please open an issue or send me pull request if you think of ways to improve it!


He seems fairly honest and straightforward to me.

RabbitMQ had some issues. He did not gloat rather.


Note: RabbitMQ 2.7.0 sometimes dies midway through the benchmark. It seems RabbitMQ does not enforce very strict flow control and you can get into situations where it runs out of memory if you place too much load on it. It seems that crash becomes more likely as you increase the core speed of the cpu or reduce the amount of physical memory on the box. Luckily, the RabbitMQ folks are aware of the issue and hopefully will fix it by the next release.


Now quit commenting and go optimize your STOMP support. :) Kick some Apollo butt. :)

Re: clustering by Richard Hightower

Thanks for commenting. I appreciate your time. How soon will you support OpenWire and JMS?
Any plans to do a specjms benchmark? So you can do a head to head against HornetQ?

Rick Hightower
Mammatus Cloud Computing Experts

Re: clustering by Talip Ozturk

Check out Hazelcast distributed queue. It is a distributed/transactional implementation of java.util.concurrent.BlockingQueue.


Of course it is not a complete messaging solution but the point here is that clustering is a first-class feature in Hazelcast Queue.

-talip
@hazelcast

Re: clustering by Hiram Chirino

Thanks for commenting. I appreciate your time. How soon will you support OpenWire and JMS?
Any plans to do a specjms benchmark? So you can do a head to head against HornetQ?

Rick Hightower
Mammatus Cloud Computing Experts


You can already use a JMS API against Apollo by using the stompjms client library. I've created fork of the benchmark which operates against the JMS api instead of using the STOMP protocol. This benchmark test's against's ActiveMQ Openwire protocol, Apollo's STOMP, and HornetQ's Core protocol. I'll be posting the results soon.

Regards,
Hiram
FuseSource

Re: Tim Fox as in JBoss HornetQ, Tim Fox... by Tim Fox

Sigh... infoq lost my reply. I'll try again:

In this context.... I'd like to properly introduce you as the person who wrote HornetQ. You are or were the HornetQ lead. Correct? Unless you are another Tim Fox. Not that your opinion is biased, but you do get paid to work on HornetQ.


No, I haven't worked at Red Hat for a year or so. I now work at SpringSource/VMware. I worked with the RabbitMQ team for a while and now I am working on a non messaging project.

Don't get me wrong. I really like STOMP. I have *never* been a fan of AMQP. In fact, I have publicly and vocally criticised it on several occasions.

Re: clustering by Tim Fox

[Again... my post last post was lost..]

These results might save you some time. community.jboss.org/wiki/HornetQ-ThePerformance...

JMS Benchmark Results by Hiram Chirino

Hi Tim,

I finally got around to posting JMS benchmark results of Apollo vs other JMS implementations. See the ec2-c1.xlarge results.
Like the STOMP benchmark, the benchmark tool is open source and available at github. I've tried to make the benchmarks as fair as possible, but if you anyone sees something, please open and issue or send in a pull request.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

14 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT