Ruby Performance: Great Shootout Results And A Discovery About Binary MRI vs Source Compiled MRI

| by Werner Schuster Follow 9 Followers on Dec 11, 2008. Estimated reading time: 2 minutes |

There is competition between the Ruby implementations, particularly in matters of performance. A year ago, Antonio Cangiano ran the Great Ruby Shootout, which compared 1.8.6, 1.9.0, JRuby, Rubinius as well as the Ruby.NET implementation, which has now been discontinued.

Now, Antonio published the Great Ruby Shootout 2008, which compares 1.8.6, 1.9.1, JRuby, Rubinius, Ruby Enterprise Edition, IronRuby as well as the latest addition, MagLev.

It's important to remember to always take benchmarks with a grain of salt, which was shown again when Antonio found problems with his initial run and published an updated version, which changed the result a bit. One of the problems he found was with the Ruby version available through Debian's package management system:

The really big, flashing warning though is what happens when you install Ruby through apt-get. Compiling from source gives you double the speed, according to these tests. I expected a 10/20% increase, not 100%. The gist of it is that prepackaged Ruby is compiled using the option --enable-pthreads and there is the whole issue of shared vs static libraries.

Prashant Srinivasan has previously mentioned the --enable-ptread issue and explains the reasons why it slows down the system.
The benchmark also shows that Ruby 1.8.x on Windows runs at about half the speed of a 1.8.7 (compiled from source) on Linux.

As for the the fastest, publically available Ruby implementation is now Ruby 1.9.1, followed closely by JRuby 1.1.6RC1. According to these benchmarks, JRuby seems to be the fastest way to execute Ruby 1.8.x code at the moment.

The other Ruby implementations, Rubinius and IronRuby still are mostly slower than MRI. MacRuby 0.3, which is based on Ruby 1.9, was slightly slower too - however, it's not deemed to be production ready yet, which should happen with one of the next releases (MacRuby 0.4 is expected to be out later this year).

The results of MagLev show promise, with many benchmarks running significantly faster then MRI, and a few running slower. To put this in perspective: MagLev is still a very young project, only started earlier this year.

Finally, an important point. The benchmark code used for the Great Ruby Shootout consists of many microbenchmarks, which tests individual features of Ruby and the Ruby runtimes. As Antonio explains:

Many people are interested in the kind of improvements that the tested VMs can bring to a Ruby on Rails deployment stack. Please do not assume that if VM A is three times faster than VM B, that Rails will serve three times the amount of requests per minute. It won't. That said, a faster VM is good news and can definitely affect Rails applications positively in production;

In an InfoQ interview Antonio explains the need for more real world benchmarks, and points to the Ruby Benchmark Suite project he set up for this purpose.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

windows can be faster by Roger Pack

If you use the mingw version for windows, it's much faster.
Also note that 1.9 can run most 1.8 code. See the NEWS file for changes.

Re: windows can be faster by Daniel Berger

It's faster still if you compile with VC++ 9. And run on 2008 Server instead of an OS designed solely for desktop use.

Re: windows can be faster by Roger Pack

I'd be interested in testing your VC++ 9 sometime :)
Also it is faster still if you tweak the GC -- google "ruby gc patch"

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

3 Discuss