Bio Yehuda Katz is currently employed by Engine Yard, and works full time as a Core Team Member on the Rails project. He is the co-author of jQuery in Action and the upcoming Rails 3 in Action, and is a contributor to Ruby in Practice. He spends most of his time hacking on Rails, but also on other Ruby community projects, like Rubinius and Datamapper. Other projects by Katz are Thor and DO.rb.
Strange Loop is a developer-run software conference. Innovation, creativity, and the future happen in the magical nexus "between" established areas. Strange Loop eagerly promotes a mix of languages and technologies in this nexus, bringing together the worlds of bleeding edge technology, enterprise systems, and academic research. Of particular interest are new directions in data storage, alternative languages, concurrent and distributed systems, front-end web, semantic web, and mobile apps.
I’ve actually been really happy with the level of uptake that we’ve gotten. Rails 3 is definitely a huge release we’ve been working on for a while and a lot of the concern that we have wasn’t really so much about compatibility with the existing apps, but compatibility with the existing plug-ins. One of the things that we did right, I feel, is we had a pretty long Beta period leading into an RC period, leading into the final that led a lot of the most common plug-ins get up-to-date for Rails 3. Almost anything that you would tend to use either is updated with Rails 3 or there is an equivalent plug-in now that has been created for Rails 3.
Maybe not every single app that exists on Rails 2.3.* is upgraded to Rails 3. I would say most people are starting new apps on Rails 3 now. I haven’t actually really heard of people who are starting new apps on Rails 2.3.* anymore, so, that’s really exciting. We also did a lot of work in the 2.3.* series that we continue to do in the 2.3.* series of just making it so that 2.3.* apps, as you upgrade you get more and more warnings about things that are changed in Rails 3. Some of the most useful things are the XSS plug-in, which prevents cross-site scripting attacks, which works for Rails 2.3.* in the same way it works for Rails 3, then Bundler support in Rails 2.3.*.
Those are probably the biggest changes in applications before Rails 3. If you move along the golden path in the 2.3.* series, upgrading to Rails 3 is really just a matter of getting your plug-ins in order. Like I said, the community has really stepped up to the plate on that. In terms of most commonly used features, probably the thing people talk about the most is the actor records syntax. We’ve modified actor record to be a lot more flexible in terms of where you put your code. It used to be that if you put your code in a model class, you put your code in a model method; if you put your code in a controller, those were different syntaxes that required a knowledge of all these things, like name, scope and so on.
In Rails 3 the syntax for those things is consistent across all those contexts, so it becomes really easy to say "Oh, I’m using a lot of this stuff in a controller. The same kind of things I’m going to move into a class method or into a scope and the code looks the same." I think a lot of people really like that. Something I use a lot in my application that I’m working on now is the way that Rails 3 builds up the controllers from this mental context all the way up to action controller base. This is mostly useful for performance. If you are building a very performance-critical application, you can more easily opt out of specific features that are slow in a controller and still get all the benefits of the tight integration, features that you need.
Commonly, like you wouldn’t necessarily need rendering in an API controller, you can remove rendering out of deal with an overwrite of that, but still get the benefit of a normal Rails router and the controller things that you need like conditional get. On the top you got the router, a lot of people are talking about trying to back port to 2.3 which would be difficult, but the router is extremely powerful and is also standalone. You can use it with a Sinatra app or something and I’m pretty happy with the way that API evolved and basically there is nothing that you can’t do with the Rails 3 router anymore. It supports any kind of routing, any kind of rules that you want to set on a router where the Rails 2.2 router was really very focused on common Rails usage patterns. So that’s exciting to me and I think a lot of people are talking about that as well.
2. Since Rails started to become very popular, other languages have emulated the ideas behind Rails, for instance Groovy and Grails. There are similar things in other languages. Do you see these other Rails-inspired frameworks adopting some of the features of Rails 3 and some of these more advanced features that have been coming down the pipeline?
It’s actually sort of hard to say. There are some frameworks like Symphony that are like Rails, but they’re not using a language that’s as flexible as Ruby, so they are not really exposing the same feature set they are exposing as the hard-coded feature set into PHP. Symphony - I just looked at it recently and basically there are still Rails 1.x features, and maybe they don’t know or maybe they know (I can’t really speak about that) but the way they’ve emulated the feature set doesn’t make it easy to evolve them. It sort of hard-codes them to whatever exact features that they copy and they can then go and copy another feature set, but it’s really tricky. While Rails, because it uses Ruby, is more easy to smoothly evolve the feature set.
I think Grails is more up-to-date with the Rails feature set. I’ve noticed that Grails’ router is kind of ugly for my tastes. It has some features that weren’t even in the Rails 2.3 but I think that Rails 3’s router, in terms of both expressibility and features, is probably more powerful or as powerful as EngineX or Apache Rewrite, except its syntax it’s actually usable and support for arbitrary Ruby code. You can write any kind of rewrite that you want using a pretty simple and easy to understand API. That’s actually something about the Rails community and that’s how I really want to answer this question.
The Rails community is extremely focused on developer usability and that means that the Rails community doesn’t accept subpar APIs. An example of this is Rails 3 added bundler support, which I worked on for a long time. Bundler is really usable, but the Rails community pushed back pretty hard until all these corners were polished. If you compare that to something like Maven, which is a similar API, pretty much everyone who used Maven knows that it’s sort of a nightmare to work with. That’s not just so much a matter of copying features. Of course, Grails can copy Rails and Symphony can copy Rails, K PHP can copy Rails, but those communities aren’t as extreme about wanting to build feature sets and APIs that are usable.
So they get some of the usability out of directly trans-coding Rails into their language of choice, but there are a lot of things about the APIs that just feel worse. The thing you’ll notice right away is that Grails is sort of shoehorning a lot of the features, like if you look at how hasMany works in models in Grails, it looks like you are defining a property that has many array or something. Similarly, Symphony has "var hasMany = array of things" and that’s less friendly than how Rails does it. If you look at the all-around experience you’ll see that it starts to add up. In some cases it’s really extreme, like in Maven, as a really extreme example, but even in regular features.
I have one last thing to say about this topic, which is that there are some areas of Rails specifically in the way that the REST style support works throughout the entire framework. Back in the day this thing was called simply helpful, but basically the REST API is not just you can post things or you can put things or you can delete things. The REST API means that everywhere throughout the framework, objects coming from ORMs, can make URLs, you can redirect to an object. There is basically all this support for things that are baked in, that other frameworks don’t look at at all.
When you look at Grail’s REST support, it is essentially we are going to allow the router to route to a Put - that’s what the REST support is. But there is nothing I would like to link to this object or I would like to redirect to this object or I would like to make a form for this object. There are some hard-coded cases, but Rails is generic. The entire framework is based around some of these conventions that really improve the usability of the code.
I would say that Rails is constantly looking at ways to make abstractions that are higher level, so there is basically no case where you’ll ever have to write a URL in Rails, and that’s not because we’re doing some crazy magic behind the scenes, it’s that when you write something that is an ORM object and in Rails 3 that includes MongoDB or DataMapper or the SQL ORM - anything that’s an ORM. There is an API that you can implement that allows a URL to be generated from you. Then you just say "The users are resources" and then everywhere throughout the framework you can redirect to or link to or render an object.
And there are all these conventions that sort of say "You want to render a user? – Here is where we should look for the template." It’s not things that as you grow it starts to be great on you. The question of where exactly you should put your templates for users is an obscure question that most people don’t want to bother answering. Why should you have to think about that? I think you can talk about conventional reconfiguration as a general concept and you could say "Well, that doesn’t work for big apps." When you start to drill down and look at the specific things which are what you should name certain methods, where you should stick templates of certain types, what you should do in the router for controllers that are resources.
All these things are cases where it’s not really so much that we’re pushing you down a certain conventional path, it’s that we’re answering questions for you that are annoying. You shouldn’t have to sit down with your team when you start writing an application and decide what the post method in your controller should be. There is basically no application where that’s a useful thing to use. And it doesn’t really matter if your application is big or small, complex or simple, in every application, how your director should be laid out and what you should call your method names for very common cases is something that you shouldn’t have to discuss.
That sort of Rails takes that a lot further. This is like the 30 second clip version, but Rails takes us a lot further than most other frameworks and most of the frameworks don’t even think about it. When you look at a lot of other frameworks where you notice that they’ve copied enough to do a 50 minute blog screen cast and to have scaffolding generators. Then when it comes time to say "Can I link to an object?" suddenly nobody even thought of it. And that’s because they didn’t really looked that carefully at what Rails was doing.
I first want to say that the Rubinius team and the JRuby team have been very aggressive at putting together this project called RubySpec. The Rubinius team started it and the JRuby team came onboard very quickly and that’s a series of executable specifications that you run in order to prove that you are a Ruby. It’s not whether or not you comply with some formal spec, it’s whether or not you can actually execute these tests. Of course, the CRuby version is compliant by definition. If you run the tests against the CRuby and they fail, the tests are the things that have the bug, not Ruby.
Really exciting is that the Ruby team for Ruby 1.9 actually did a lot of the work to make RubySpec make sense for Ruby 1.9 so that you can run the tests against a compatible Ruby 1.9 implementation and get a yes or no and add a bunch of tests for features like encodings that are new to Ruby 1.9. I think before we even talk about Ruby implementations, we definitely have to thank the people who worked on RubySpec and they are continuing involvement in RubySpec for the fact that JRuby and Rubinius can run Rails at all. Rails is the end of the tunnel. When you are developing a Ruby implementation you should expect to run Rails for a while. MacRuby runs a lot of Ruby but it doesn’t run Rails today.
There needs to be more granular steps towards getting to perform where you run Rails and of course, you can do things like see if you can run the standard library and all that sort of things. But being able to run a series of tests and then confirm whether or not you can run Ruby is very important. I would say that JRuby’s level of compatibility, which is basically close to 100% in terms of things that people really do is due in large part to RubySpec and bunch of other tests that they’ve written themselves. I think we have probably the most compliant implementations of people who build multiple implementations for the language. I think that are JVMs that are pretty good, but Ruby’s implementations, at least the Rubinius, JRuby and CRuby are extremely compatible with each other and that’s very good.
In terms of adoption, JRuby has definitely helped. A lot of people don’t know this but JRuby is pretty much a first class implementation. It’s not like Jython, which you could use but it’s not really up-to-date, it’s the running Python 2.5 right there, there are some features it can’t support. JRuby has been really aggressive about being a compatible Ruby implementation to the point where in order to opt into the new Java integration features you have to require Java in your scripts. You can say "require "Java"" then you get a bunch of features, but if you don’t require Java it’s pretty much a vanilla Ruby implementation that’s compatible with CRuby. That is a lot of the reason why Rails pretty much just works for JRuby.
There are other things in the community like the Ruby Gem system supports platforms like Java and JRuby’s continuous work on C extensions sort of pushes even further down the road. But JRuby’s compatibility with CRuby means that we don’t have to do a whole lot inside of Rails to make it work. The JRuby community pretty much all they’ve really done for Rails compatibility has been to submit patches to brittle tests. Maybe they relied on the test being run in a particular order and that’s not a JRuby bug, it is a bug in our tests.
There are some cases like that, they’ve submitted some patches for it, but by and large Rails just works in JRuby an that's been really important for adoption in the more enterprise community, I don't like using that word, but people who have existing Java infrastructure have really been able to adapt JRuby rather aggressively in ways that I think you don't see as much with Python people. I would also say that JRuby is becoming a really good solution, people are just deploying Ruby application, people have a high concurrency need.
They are really seeing a lot of benefit from JRuby and from the ecosystem that's build up around JRuby. In terms of Rubinius, their 1.0 release was actually both compatible with MRI and also sort of close in performance to Ruby 1.8 an they've been continuing to work on performance. In my opinion, Rubinius eventually will be the fastest Ruby implementation but they have a lot of work to go. Their VM is really advanced for Ruby, it's probably the most advanced pure Ruby VM that we have. So I'm not comparing Rubinius to JVM but in terms of the "smarts" that go in to the Ruby part of the VM, Rubinious is probably the smartest so they can do things like inlining blocks, inlining methods that other implementations have hard time with and CRuby doesn't do.
My expectations is that overtime Rubinius will have smarter and smarter VM which will push it into first place. Until thenyou would want to use it because it provides more information, more introspection capabilities and I think Rubinius is going to keep going down that path as well. But it's definitely sort of a struggle for which implementation is the best. I think probably today JRuby is the best Ruby implementation but everyone sort of tangling for a spot at the table.
Sure! So we've talked a little bit about this but it's sort of crystallizing what's going to make it in, what's not going to make it in now. For the past couple of years of pushing into the client there's been a need for smarter and smarter asset compilation so everything from using SCSS or Coffee Script to things like concatenation, magnification and other compilation tricks has become a lot more common and so until now a lot of that stuff exists as plugins but with Rails 3.1 the idea of assets as first class objects that have their own compilation pipeline, that have their own notion of templates enabling to run on the server without needing a separate compilation step so they can be compiled at runtime, cached.
On the same topic is we are pushing further into HTTP caching so we've always had features that enables HTTP caching but a lot of people haven't used them. In part because the only user of the HTTP caching was individual clients, web browsers and that doesn't help if you have a thousand users, each one of them is still going to knock in a lot of HTTP caching so people have used other techniques like fragment caching or page caching instead. And what we are doing with Rails 3.1 is shipping with rack-cache which is a caching layer that' built in.
That will enable you to use HTTP caching more intelligently and have it be equivalent to sort of page caching, action caching, or fragment caching in some cases it means thinking about your application a little bit differently but it also means that we will enable using HTTP caching which should work with other caching servers like SQUID or Akamai without having to sort of rewrite your caching layer, so if you go buy Akamai you have already built your application to take advantage of the features that Akamai provides, because we already have it built-in. And something I should add about rack-cache is that when people hear about it for the first time they often think that rack-cache is gonna do all the caching in the heap, in the Ruby heap. And rack-cache is actually smarter than that.
It uses memcache or the file system or redis. And what we're doing for Rails3.1 is we're just going to have rack-cache wherever your Rails cache is, so if your Rails cache is memcache, or redis or file system, we’ll just use that. And that will mean that while there is certain amount of logic of course that happened in Ruby to validate the cache. The vast majority of the thing that's expensive which is just having these huge files will end up being in more sane places. So that's really exciting!
We aren't going to get flushing into Rails 3.1, I don't think. So there is a auto flashing mainly because while we can get it out there , there's a lot of caveat, so we want to really think through and we probably just take the Rails 3.2 release to think through those things carefully. And maybe have a plugin that turns it on and let people play with it a little bit. Active Record has got a lot of performance worked on against it. We 're doing performance work. There's going to be an identity map added in to Rails 3.1 through Ruby summer code project. Engines have got a lot of improvements so that you can more easily build a Rails app and then making an engine later.
There's a sort of a whole bunch of Ruby summer code projects an community contributed features that will make their way in Rails 3.1 and really I think that I would say about Rails 3.1 is that we did a lot of work in Rails 3.0's infrastructure. There are a lot of cool features in 3.0 that you would want to use like Active Record, like Action Mailer, like the Router, but really Rails 3.0 is about improving internals.
And what Rails 3.1 does is really take advantage of those improved internals with a huge number of new features that maybe we would have wanted to get into 3.0 but we had to ship. So a bunch of Ruby summer code projects, a bunch of community contributed stuff, better acts at pipelining and I think. the HTTP caching stuff is something that will probably matter more later than now. So HTTP caching is sort of a similar bet that we made on REST backing Rails 1.2 where Rails 1.2 said "REST makes sense for requests" and Rails 3.1 says "REST makes sense for the sponsors" and while the initial work might be raw, we’re just adding caching support and cleaning up a few of the existing http caching stuff, I think over time there will be additional conventions that emerge, that will make it even better.
I’m actually really excited in general about leveraging http caching better and shipping with the caching layer and I’m excited to see what people do with it. I think just shipping with it is a bet that will be useful, similar bet tool in 3.0 and 1.1 and 1.2.
6. With a lot of frameworks that have been around for a while, the first one that springs to mind is Java EE and second one Spring, what often happens if that you have this thing that starts of relatively small at 1.0 and then as it builds up you can sometimes get features that make sense of time but later on don’t make sense. People start to complain about code bloat or feature bloat or whatever. How does Rails 3 address this? How does the development plan address and prevent things like this?
Actually we did a lot of things to address that. One large change in Rails 3.0 was that we started to make more use of community dependencies. So instead of writing our own gem handling we spun off the Bundler project. Instead of having the Router be something that was Rails specific, we spun off the rack-mount project which allows any rack framework to use our router. Instead of continuing to maintain the team mail project which is our mail framework, we helped Michael work on the mail gem which is now sort of a full-featured gem. We’ve done a lot of work to sort of take things out, that used to be hard-coded into Rails, so that other people can maintain those, people who are more experts in that topic so that the Rails project can get smaller.
We spent a lot of time thinking about how the Rails framework itself doesn’t need to be as coupled. For instance, there used to be references of Active Record all over Action Pack. In Rails 3.0 we removed all those references and we created a new layer in every piece of Rails called Rail Ties that is the part of Active Record that is needed for Rails. There’s Active Record and there’s Active Record as part of Rails, and we’ve separated out the things that are needed for Active Record as part of Rails, Action Mailer as part of Rails, Active Resource as part of Rails, Active Models as part of Rails, and we moved that into a separate place. So if somebody wants to use Active Model on its own they don’t need to get the whole Rails. That was sort of one blow issue. Before it was that people would have to pull in a lot more of Rails if there were just trying to use Active Record or if they were just trying to use Action Mailer, they would find themselves pulling in many MB extra of RAM, much more code than they anticipated.
Similarly in Active Support, which should be a static library, it used to be that you had to pull out everything to get anything. In Rails 3 we made it so that you can pull in specific files just like in a standard library and you only have to pull in those files; you don’t have to pull in everything. There’s a lot of work on modularity in general.
We spent a lot of time in 3.0 really thinking through the features that are part of Rails and we sort of made a bet on REST and Rails 1 & 2 and until 3.0 we never really cleaned up all the pieces that were there before REST. We spent a lot of time saying "If you don’t want to use REST you probably shouldn’t use Rails". So we’re going to clean up a bunch of that stuff. I think it’s inevitable that as projects live for longer, more use-cases arise and they get bigger. Obviously those features are there for a reason, but if you want to build an application that’s smaller how do you do that? If you don’t need all these features how does that work?
I am happy to say that I myself am using a lot of that functionally for my personal projects [my company’s projects right now]; we’re doing a lot of work with ActionController::Metal which was designed to strip that down, we’re doing a lot of work with removing entire frameworks that we don’t want, and we’ve been able to be really aggressive in ways that were not even possible before Rails 3. At least for me it is working and I hope there will be more blog posts about how to do that. I’ve written a lot, but I hope to see more people talking through "If you don’t want the whole Rails, how does that work?"
Rails 3.1 will be hopefully released by the end of the year, I can’t promise that but that’s the rough time we’re looking at. It may slip a little bit, but we’re locking down the feature set like I told you before, and pretty much all the features that I talked about before are either already in or are being actively worked on right now. I definitely foresee RCs pretty soon and the final are after, for 3.1.
I’m actually happy to say that Rails 3.0.x, the branch of 3.0.x, has had a lot of active work there, I think over one hundred commits since we released 3.0 or even more than that and also a lot of very aggressive work on Active Record. People might know that when we announced Rails 3.0 we knew there were some performance concerns with it. It mostly works. It think it’s not as bad as some people saw when they ran Microbenchmarks , but there’s definitely performance work that could be done and Aaron Patterson who works for AT&T has put in a lot of time on AT&T’s dime actually, to improve the ActiveRelation library that underlies Active Record. He’s finally gotten to that point where most queries will be faster in Rails 3 than in Rails 2.3 by actually a noticeable margin.
We’re looking forward to getting that out there in 3.0.2. We don’t need to wait for 3.1 for that, this is sort of something that had we had more time we would have gotten out in 3.0 but we had to release. I’m actually happy to see first that we’ve gotten some big issues out of the way for 3.0.2 and secondly the amount of community involvement.
One thing that really gratifies me about the Rails 3 project is that we significantly increased the number of regular contributions to the Rails project. Like I said we had a hundred or hundreds of commits since 3.0. Actually I didn’t think that was true; I was like "Oh we should just release 3.0.1, I bet you It’s like a small row up"; then I looked at the commits and there’s a lot. Of course the ones that are more aggressive are in the 3.1 branch, but even in the less aggressive stuff there’s a lot of work in the 3.0 branch and a lot of that comes from the amount of community participation. I’m happy about AT&T, but there’re some other people that are putting in some of their company’s time on this stuff and that’s really exciting to me.