BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles A Look at Common Performance Problems in Rails

A Look at Common Performance Problems in Rails

Bookmarks

Over the last few months I have analyzed a number of Rails applications w.r.t. performance problems (some of these involved my consulting business, some were open source). The applications targeted a variety of domains, which resulted in enough differences to make each performance improvement task challenging. However, there were enough commonalities that made it possible to extract a number of areas where each of these applications fell short of achieving good performance. These were:

  • choosing a slow session container
  • doing things on a per request basis, which could have been done once at startup
  • repeating identical computations during request processing
  • reading too often and too much from the database (especially in conjunction with associations)
  • relying too much on inefficient helper methods

On top of that, there are still some problem areas within the Rails framework itself, where I'd like to see improved performance in the future. Some of these can be worked around at the application level, some can't. My favorites on this list are:

  • route recognition and route generation
  • ActiveRecord object construction
  • SQL query construction

I'll provide some tips on coding around these problem areas below.

Some Words of Caution

Following my suggestions in this article, may or may not improve the performance of your application. Achieving good performance is a tricky business, especially if the performance characteristics of implementation language constructs are somewhat underspecified (as is the case with Ruby).

I strongly suggest to measure your application's performance before you change anything and then again after each change. A good tool for performance regression tests is my package railsbench. It's really easy to install and benchmarks are defined within a few minutes. Unfortunately it won't tell you where the time is spent in your application.

If you have a windows machine around (or a dual boot Intel Mac), I suggest to evaluate Ruby Performance Validator (RPVL) by Software Verification Ltd. I have found it to be of immense value for my Rails performance work that went into the core of Rails, especially after SVL implemented a call graph feature that I suggested on top of the already existing hot spot view. As far as I know, it's the only tool for Ruby application performance analysis on the market right now. Railsbench has built in support for RPVL, which makes it a snap to run benchmarks defined for railsbench under RPVL.

Choosing a Session Container

Rails comes with several built in session containers. All applications I have analyzed used either PStore, which stores session information in a separate file on your file system, or ActiveRecordStore, which stores it in the database. Both choices are less than ideal, especially slowing down action cached pages. Two much better alternatives are available: SQLSessionStore and MemCacheStore.

SQLSessionStore avoids the overhead associated with ActiveRecordStore by

  • not using transactions (they are not required for correct operation of SQLSessionStore)
  • offloading the work of updating "created_at" and "updated_at" to the database

If you use Mysql, you should make sure to use a MyISAM table for sessions. It is faster than InnoDB, and transactions are not required. I have recently added Postgres support to SQLSessionStore, but Postgres seems to be a lot slower for session storage than Mysql with MyISAM tables, so I suggest to install Mysql just for the session table (I can't think of a good use case were you'd need to join based on session id) if you want DB based session storage.

MemCacheStore is even faster than SQLSessionStore. My measurements show a 30% speed improvement for action cached pages. You need need to install Eric Hodel's memcache client library and do some configuration in environment.rb to be able to use it. Warning: do not attempt to use Ruby-Memcache (it's really, really slow).

For my own projects I tend to use database based session storage, as it enables simple administration through either the Rails command line or administrative tools provided by the database package. You'd need to write your own scripts for MemCacheStore. On the other hand, memchached presumably scales better for very high traffic web sites and comes with Rails supported automated session expiry.

Caching Computations During Request Processing

If you need the same data over and over again, during processing a single request, and can't use class level caching because your data depends in some way on the request parameters, cache the data to avoid repeated calculations.

The pattern is easily employed:

module M
def get_data_for(request)
@cached_data_for_request ||=
begin
expensive computation depending on request returning data
end
end
end

Your code could be as simple as  "A".."Z".to_a  or it could be a database query, retrieving a specific user, for example.

Perform Request Independent Computations at Startup or on First Access

This advice is so simple that I hesitated somewhat if I should include it in this article. But I find that many applications I have analyzed failed to employ this useful optimization.

The technique is actually very simple: if you have data that doesn't change over your application's lifetime, or changes so seldom that a server restart could be employed if it changes, cache this data in an appropriate class variable on some class of your application. The general pattern goes like this:

class C
@@cached_data = nil
def self.cached_data
@@cached_data ||= expensive computation returning data
end
...
end

Some examples:

  • application configuration data (if your application is designed to be installed by others)
  • never changing (static) data in your database (otherwise use caching)
  • detecting installed Ruby classes/modules using ObjectSpace.each

Optimizing Queries

Rails comes with a powerful domain specific language for defining associations between model classes which reflect table relationships. Alas, the current implementation hasn't been optimized for performance, yet. Relying on the built in generated accessors can severely hurt performance.

The first part of the problem is usually described as the "1+N" query problem: if you load a N objects from class Article (table "articles"), which has a n-1 relationship to class Author (table "authors"), accessing the author of a given article using the generated accessor methods will cause N additional queries to the database. This, of course, puts some additional load on the database, but more importantly for Rails application server performance, the SQL query statements to be issued will be reconstructed for object accessed.

You can get around this overhead by adding an :include => :author to your query parameters like so:

Articles.find(:all, :conditions => ..., :include => :author)

This will avoid all of the above mentioned overhead by issuing a single SQL statement and constructing the author objects immediately. This technique is commonly called "find with eager associations" and can also be used with other relationship types (such as 1-1, 1-n or n-m).

However, n-1 relationships can be optimized further by using a technique called "piggy backing": ActiveRecord objects involving joins carry the attributes from the join table(s) along the attributes from the original table. Thus, a single query with a join can be used to fetch all required information from the database. You could replace the query above with

Articles.find(:all, :conditions => ...,
:joins => "LEFT JOIN authors ON articles.author_id=authors.id",
:select => "articles.*, authors.name AS author_name")

assuming that your view will only display the author's name attached to the article information. If, in addition, your view only displays a subset of the available article columns, say "title", "author_id" and "created_at", you should modify the above to

Articles.find(:all, :conditions => ...,
:joins => "LEFT JOIN authors ON articles.author_id=authors.id",
:select => "articles.id, articles.title, articles.created_at, articles.author_id, authors.name AS author_name")

In general, loading only partial objects can be used to speed up queries quite a bit, especially if you have a large number of columns on your model objects. In order to get the full speedup from the technique, you also need to define a method on the model class to access any attributes piggy backed on the query:

class Articles
...
def author_name
@attributes['author_name'] ||= author.name
end
end

Using this pattern relieves you from knowing whether the original query has a join or not, when writing your view code.

If your database supports views, you could define a view containing just the required information and you would get around writing complicated queries manually. This would also get you the correct data conversion for fields retrieved from the join table. As of now, you don't get these from Rails, but need to code them manually.

Note for "living on the edge guys": I got tired of repeating the same patterns over and over again. So I coded a small extension which does most of the work automatically. A preliminary release can be found on my blog.

Avoiding Slow Helpers

A number of helpers in Rails core will run rather slowly. In general, all helpers that take a URL hash will invoke the routing module to generate the shortest URL referencing the underlying controller action. This implies that several routes in the route file need to be examined, which is a costly process, most of the time. Even with a route file as simple as

ActionController::Routing::Routes.draw do |map|
map.connect '', :controller => "welcome"
map.connect ':controller/service.wsdl', :action => 'wsdl'
map.connect ':controller/:action/:id'
end

you will see a big performance difference between writing

link_to "Look here for joke #{h @joke.title}",
{ :controller => "jokes", :action => "show", :id => @joke },
{ :class => "joke_link" }

and coding out the tiny piece of HTML directly:

<a href="/jokes/show/<%= @joke.id %>"
class="joke_link">Look here for joke <%= h @joke.title %></a>

For pages displaying a large number of links, I have measured speed improvements up to 200% (given everything else has been optimized).

In order to make the template code more readable and avoid needless repetition, I usually add helper methods for link generation to application.rb :

def fast_link(text, link, html_options='')
%(<a href="#{request.relative_url_root}/#{link}"> hmtl_options>#{text})
end

def joke_link(text, link)
fast_link(text, "joke/#{link}", 'class="joke_link"')
end

writing the example above as:

joke_link "Look here for joke #{h @joke.title}", "show/<%= @joke.id %>"

Of course, coding around the slowness of route generation in this way is cumbersome and should only be used on performance critical pages. If you don't need to improve your performance today (uh, this sounds like some of the spam mails I get everyday), you could wait for the first release of my upcoming template optimizer, which will do all of this automatically for you, most of the time.

Topics for Future Rails Performance Improvement

As mentioned above, performance of route recognition and route generation leaves something to be desired. The route generation problem will be addressed by my template optimizer. Last week a new route implementation was incorporated into the Rails development branch. I performed some measurements which seem to indicate performance impovements for route recognition. For routes generation, I got mixed results. It remains to be seen whether the new implementation can be improved to a point where it's consistently faster than the previous one.

Retrieving a large number of ActiveRecord objects from the database is relatively slow. Not so much because the actual wire transfer is slow, but the construction of the Ruby objects inside Rails is rather expensive, due to representing row data as hashes indexed by string keys. Moving to an array based row class should rectify this problem. However, doing this properly would involve changing substantial parts of the ActiveRecord implementation, so it should not be expected to arrive before Rails 2.0.

Finally, the way SQL queries are constructed currently, makes the computation of the queries more expensive than the retrieval of the actual data from the database. I think we could improve this a lot by changing it so that most SQL queries can be created once and simply be augmented by actual parameter values. The only way to work around this problem at the moment is coding your queries manually.

Note: these are my own opinions about possible avenues to pursue for better performance and do not represent any official "core team" opinion towards these issues.

Conclusion

The list of problems cited above should not fool you into thinking that Rails might be too slow (or even that I think it's too slow). To the contrary, I'm convinced that Rails is an excellent web application development framework, usable for developing robust and also fast web applications, at increased productivity. Like all frameworks, it offers convenience methods, which can greatly improve your development speed and which are appropriate most of the time for most of your needs. But sometimes, when it's necessary to squeeze out some extra requests per second, or when you are restricted to limited hardware resources, it's good to know how performance can be improved. Hopefully this article has helped outlining some areas which can be used as points of attack, should you experience performance problems.

About the author

Stefan Kaes writes RailsExpress, the definitive blog about Rails performance and is the author of the forthcoming book, "Performance Rails", scheduled to publish in early 2007. Stefan's book will be among the first in the new Addison-Wesley Professional Ruby Series, set to launch in late 2006 with the second edition of Hal Fulton's "The Ruby Way" and the flagship "Professional Ruby on Rails", authored by Series Editor Obie Fernandez. The Series will consist of a robust library of learning tools for how to make the most of Ruby and Rails in the professional settings.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Real World Experiences?

    by Obie Fernandez,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    We'd love to hear from any folks putting Stefan's suggestions into practice.

  • Updated article

    by Obie Fernandez,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    The initial publication of this article and related feed snippets incorrectly identified Stefan as a member of the Rails core team. This was my editorial mistake and I apologize for any confusion caused.

  • good article

    by Michael Kovacs,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Thanks Stefan for this article. Lots of good information here that will come in handy once I'm ready to tune for perf.

  • Very Useful!

    by Geoffrey Grosenbach,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Very practical and useful. Thanks!

  • Great Practicle Information

    by Ben Askins,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    For me this article is a lamp in the dark pointing out little obstacles I wouldn't otherwise have noticed. Thanks Stefan.

  • memcache-client author

    by Eric Hodel,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Bob Cottrell wrote memcache-client, I just packaged it up and released it.

  • Getting Memcache-Client to work with Sessions

    by Adam Michaels,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Does anyone have a link to a tutorial or have an idea how to get Memcache-Client to work with Rails sessions seeing as its the prefered ruby client.

  • 32 percent...

    by chris hulbert,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I just spent today putting into practice I just spent today putting into practice most of the ideas suggested here, especially caching using class variables, memcached sessions, and not using slow helpers.
    With extensive testing, my production server now serves up pages 31.8% faster!
    63.8 seconds versus 93.5 to complete a test suite i created.
    Very happy :)
    Chris - splinter.com.au

  • Re: Getting Memcache-Client to work with Sessions

    by chris hulbert,

    Your message is awaiting moderation. Thank you for participating in the discussion.

  • Re: Getting Memcache-Client to work with Sessions

    by Geoffrey Grosenbach,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I did a straight copy and tweak of the Rails MemcacheStore to work with memcache-client, here:

    topfunky.net/svn/plugins/db_memcache_store/lib/...

  • optimizing queries...

    by Boing Boinger,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    consider cache-fu. little in the way of docs at this point, but it's a good activerecord cache...

    errtheblog.com/static/pdfs/memcached.pdf

  • 'Include' can improve performance?

    by Zhenbo Hu,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I tried, but it seems no performance improvement at all.

  • tell ActiveRecord what you don not want to select from database

    by rafael magana,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    some days ago I made a patch to add a :noselect option to use with the ActiveRecord::find method:

    let's say you have a table with 5 columns, and you want to select 4 of them:

    User.first(:select => "col_1, col_2, col_3, col_4")

    with my patch you can do this:

    User.first(:noselect => "col_5")

    maybe it's not a big improvement in speed but it can help in tables with a bunch of columns. well that's it.

    here's the post where I explain a little bit more how it works and the link to the patch: bit.ly/cIt4pa

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT