Ruby Performance: Great Shootout Results And A Discovery About Binary MRI vs Source Compiled MRI

Posted by Werner Schuster on Dec 11, 2008

Sections

Development,

Architecture & Design

Topics

Performance & Scalability ,

Java ,

.NET ,

Runtimes ,

JRuby ,

Dynamic Languages ,

Ruby

Tags

IronRuby ,

Virtual Machines ,

MagLev ,

Rubinius ,

Ruby1.9 ,

JRuby

There is competition between the Ruby implementations, particularly in matters of performance. A year ago, Antonio Cangiano ran the Great Ruby Shootout, which compared 1.8.6, 1.9.0, JRuby, Rubinius as well as the Ruby.NET implementation, which has now been discontinued.

Now, Antonio published the Great Ruby Shootout 2008, which compares 1.8.6, 1.9.1, JRuby, Rubinius, Ruby Enterprise Edition, IronRuby as well as the latest addition, MagLev.

It's important to remember to always take benchmarks with a grain of salt, which was shown again when Antonio found problems with his initial run and published an updated version, which changed the result a bit. One of the problems he found was with the Ruby version available through Debian's package management system:

The really big, flashing warning though is what happens when you install Ruby through apt-get. Compiling from source gives you double the speed, according to these tests. I expected a 10/20% increase, not 100%. The gist of it is that prepackaged Ruby is compiled using the option --enable-pthreads and there is the whole issue of shared vs static libraries.

Prashant Srinivasan has previously mentioned the --enable-ptread issue and explains the reasons why it slows down the system.
The benchmark also shows that Ruby 1.8.x on Windows runs at about half the speed of a 1.8.7 (compiled from source) on Linux.

As for the the fastest, publically available Ruby implementation is now Ruby 1.9.1, followed closely by JRuby 1.1.6RC1. According to these benchmarks, JRuby seems to be the fastest way to execute Ruby 1.8.x code at the moment.

The other Ruby implementations, Rubinius and IronRuby still are mostly slower than MRI. MacRuby 0.3, which is based on Ruby 1.9, was slightly slower too - however, it's not deemed to be production ready yet, which should happen with one of the next releases (MacRuby 0.4 is expected to be out later this year).

The results of MagLev show promise, with many benchmarks running significantly faster then MRI, and a few running slower. To put this in perspective: MagLev is still a very young project, only started earlier this year.

Finally, an important point. The benchmark code used for the Great Ruby Shootout consists of many microbenchmarks, which tests individual features of Ruby and the Ruby runtimes. As Antonio explains:

Many people are interested in the kind of improvements that the tested VMs can bring to a Ruby on Rails deployment stack. Please do not assume that if VM A is three times faster than VM B, that Rails will serve three times the amount of requests per minute. It won't. That said, a faster VM is good news and can definitely affect Rails applications positively in production;

In an InfoQ interview Antonio explains the need for more real world benchmarks, and points to the Ruby Benchmark Suite project he set up for this purpose.

1. Back to top

   windows can be faster

   by Roger Pack

   If you use the mingw version for windows, it's much faster.
   www.akitaonrails.com/2008/7/26/still-playing-wi...
   Also note that 1.9 can run most 1.8 code. See the NEWS file for changes.
   -=R

   Reply

2. Back to top

   Re: windows can be faster

   by Daniel Berger

   It's faster still if you compile with VC++ 9. And run on 2008 Server instead of an OS designed solely for desktop use.

   Reply

3. Back to top

   Re: windows can be faster

   by Roger Pack

   I'd be interested in testing your VC++ 9 sometime :)
   Also it is faster still if you tweak the GC -- google "ruby gc patch"
   -=r

   Reply

• Older >