InfoQ

InfoQ

News

My Bookmarks

Login or Register to enable bookmarks for unlimited time.

The content has been bookmarked!

There was an error bookmarking this content! Please retry.

Considering Grails vs Rails Benchmarks

Posted by Scott Delap on Mar 23, 2007

Sections
Development
Topics
Web Frameworks ,
Ruby ,
Java
Tags
Mongrel ,
Grails ,
Rails
John Wells recently asked out loud to the development community "Where are all the Grails benchmarks?" while evaluating the technology:

...I can't find a SINGLE benchmark comparing Groovy to competing technologies on the JVM (JRuby, Beanshell, Jython) or dynamic languages in general (Ruby, Python, Perl, and so on). Nor can I find any benchmarks concerning how Grails stacks up to JRuby on Rails, Ruby on Rails, vanilla Hibernate/Spring (something Grails uses under the covers), etc...

In response to this the grails community posted some rough unscientific numbers on Grails versus Rails. Among the benchmarks results:

[In respect to read performance]...Grails is able to handle 40 requests per second to Rails 32. Significantly you can notice Grails has a much poorer worst case response time, but a superior mean response time (The mean time is the average time taken per request over all 1000 requests). This can be put down the Java threading model and the Thread pool in Tomcat at some point running out of available threads and blocking for a moment. It just reflects the differences in architecture more than anything else...

[In respect to creating records] ... Grails is able to cope with 140 requests per second and has a mean average of 345ms per request compared to Rails' 35 (and a half) and 1.3 seconds per request. I believe, without having done any real profiling, the reasons for this can largely put down to Hibernate's session-based model. Each call to save() in ActiveRecord immediately persists the data whilst Hibernate batches updates up until the end...

[In respect to querying] ...the results here are much more equal as it is really reliant on database performance. Notably we can see Grails' worst case is poor down to blocking in the thread pool, but overall the mean time of 5.387ms is superior to Rails' 6.386, which illustrates why Grails' is able to deal with 8.92 requests per second as oppose to Rails' 7.65...

The article finishes by clearly stating that the authors are not Rails performance tuning wizards and are welcome to suggestions in optimizing Rails. The tests conducted were solely to inform Grails users asking for figures not as a definitive benchmark.

The Rails community has not gone quiet in regards to the benchmark data. Noted Rails developer and Mongrel user Jared Richardson commented in response:

...When your application creates a new instance or thread for every new request, it's easy for a server to get overwhelmed. Rails takes another approach. The Rails developers want you to make a conscious decision about how many resources you want an application to have ... So when you deploy a Rails application on Mongrel, you need to decide how many instances are available. This is completely different from the traditional Tomcat model, which looks something more like "spin up threads until the machine stops responding." ... If I've read the page properly, this benchmark is running against a single Mongrel instance, which is the equivalent of creating a Java singleton servlet and testing against it. Of course it's going to be quite slow ... For a real comparison, I'd try using the Apache load balancer plugin or my personal favorite Pound.

10 comments

Watch Thread Reply

Benchmarks by Martin Gilday Posted
Better App Server by Matt Giacomini Posted
Thread model vs. Process model by steven Vetzal Posted
Re: Thread model vs. Process model by jared richardson Posted
Re: Thread model vs. Process model by steven Vetzal Posted
Not a language comparison by Thom Nichols Posted
No mention of just-in-time compilation by James Tikalsky Posted
New results with 10 mongrels... by John Wells Posted
Re: New results with 10 mongrels... by Graeme Rocher Posted
Re: New results with 10 mongrels... by Mohammad Ali Posted
  1. Back to top

    Benchmarks

    by Martin Gilday

    There has been plenty of talk recently on the Groovy mailing list about performance of the language in general, including comparisons with JRuby and c Ruby.

  2. Back to top

    Better App Server

    by Matt Giacomini

    Be interesting to see how grails performs under Jetty.

  3. Back to top

    Thread model vs. Process model

    by steven Vetzal

    You know, as long as you understand each model, it's not hard to work with either.

    Jared Richardson's comment doesn't show much knowledge of Tomcat. It hardly hands out new instances each request, you can configure it just as nicely as you can set up new mongrel instances - except you don't have the 20-50MB of process overhead with just an extra thread :)

  4. Back to top

    Not a language comparison

    by Thom Nichols

    This is a great comparison for the frameworks, but keep in mind this does not equate to relative language performance. Large portions of Grails are written in Java, while Rails is (AFAIK) all Ruby.

    All of this will likely change when YARV is complete and as JRuby improves as well.

    Nonetheless, from the POV of a user of either framework, the comparison is valid. Personally, as a Java user I would like to see Grails' performance compared to similar Java stacks... Say, Struts2 + Hibernate. Then you could see the trade-off of Grails' productivity for performance.

  5. Back to top

    Re: Thread model vs. Process model

    by jared richardson


    You know, as long as you understand each model, it's not hard to work with either.


    I agree completely. My intention wasn't to attack either point of view but to point out that the multi-Mongrel approach is required for a fair benchmark.


    Jared Richardson's comment doesn't show much knowledge of Tomcat. It hardly hands out new instances each request,


    My intended point was that Tomcat creates a new thread with each request, not a new instance. My blog post makes that a bit clearer, but I think the quoted bits make that pretty clear. Is my understanding incorrect?

  6. Back to top

    No mention of just-in-time compilation

    by James Tikalsky

    The article doesn't mention Just-in-Time compilation provided by the JVM they're running Grails on. For all we know, simply "warming up" the application with a few thousand requests before they took their measurements may have increased the speed significantly.

    Also, I believe that Groovy may support dynamic compilation of Groovy code (I may be wrong), which would subject Groovy code to JIT optimizations as well.

  7. Back to top

    Re: Thread model vs. Process model

    by steven Vetzal

    Nope, you don't get a thread unless there's one in the pool to accept the request. The pool has a min and max bounds, default max bounds I personally think is a little high (something like 150 threads). Depending upon your workload this may or may not be appropriate for you - if your workload involves a lot of waiting around for things, lots of threads isn't bad (helps balance wait time and keep the processor best utilized), but if they're all processing, you lose a lot in context switching.

    If memory serves, the connection hangs until there is a thread available or until a socket timeout occurs.

    You can always set the max threads down to 3-4 if you do a lot of processing in your requests.

    IMHO the big disadvantage of the thread model is synchronization across servers. If your app depends on internal state, replicating that state across a server farm is a different problem.

  8. Back to top

    New results with 10 mongrels...

    by John Wells

    The original test was, quite innocently I believe, invalid, as it only tested Tomcat against one mongrel process. I've updated my blog with the same tests run against 10 mongrels + nginx.

    thoughtmining.blogspot.com/2007/03/grails-versu...

  9. Back to top

    Re: New results with 10 mongrels...

    by Graeme Rocher

    I updated the benchmark and configured Rails with 10 Mongrels and Pound as a load balancer as per Jared's blog. I also re-did the Grails tests with a tomcat maxThreads of 10 and the results were the Grails performed better and Rails performed worse:

    docs.codehaus.org/display/GRAILS/Grails+vs+Rail...

    Again, I appeal to the Rails community to suggest optimisation and improvements to the config of Rails. I will try out John's (above) suggestion of using nginx instead of Pound as a load balancer

    Cheers
    Graeme

  10. Back to top

    Re: New results with 10 mongrels...

    by Mohammad Ali

    Sorry for replying 8 months late,

    Redo the test on the same machine with 3 mongrels only + pound (or nginx which is better)

    since you have 2 cores, the best count would be 3 or 4, but since you have only 1 gig of ram then 3 should be your sweet spot.

    As for your previous test with 10 Mongrels, you were SWAPPING memory!!

    I believe that limiting threads for tomcat would cause them to queue more requests rather than spawning threads, which is a heavy process with a high cost of context switching (you only have 2 cores)

Educational Content

New-age Transactional Systems - Not Your Grandpa's OLTP

John Hugg discusses high volume transaction processing applications with high and low frequency profiles, and how VoltDB can be used for that purpose.

Cool Code

Kevlin Henney examines code samples to see what can be learned from them starting from the premise that one won’t write great code unless he knows how to read it.

Collaboration: At the Extremities of Extreme

Jason Ayers share the observations he made watching a team of developers collaborating in real time on the same code base, pushing XP, pair programming and continuous integration to their extremes.

Yesod Web Framework

Michael Snoyman presents Yesod, a web framework written in Haskell and containing a web server, templating, ORM, libraries (templating, gravatar, etc.).

Transactions without Transactions

Richard Kreuter and Kyle Banker on how to avoid classical RDBMS transactional systems by using compensation mechanisms, transactional messaging or transactional procedures.

Attila Szegedi on JVM and GC Performance Tuning at Twitter

Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.

10 tips on how to prevent business value risk

One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.

Interview: Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives

InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.