Yehuda Katz explains Merb

Recorded at:

1. We are here at RubyFringe, with Yehuda Katz. How about you introduce yourself?

My name is Yehuda Katz, I work for Engine Yard. Most of my time I work on Merb and a significant amount of time also on our internal control panel application, which will eventually be the public portal control panel for Engine Yard users.

2. What do you do with Merb? Do you design it? Do you implement it?

I'm pretty much the current maintainer of Merb. I handle most packages and most of the bug fixes. Merb has a good community, so there is a significant number of patches and such things coming in, but I'm somewhat of a gate keeper in terms of what patches make it in – Ezra's still somewhat involved, obviously.

3. So, what's the elevator pitch for Merb, if I'm using Rails why Merb?

Rails is really good to get started if you're coming and joining, if you are getting involved in the Ruby community or not been involved with Ruby before and if you want a book or a tutorial to get started, Rails is awesome for that. Almost anybody who's ever made a substantial Rails application, who spends millions of dollars on a Rails application or even just a complex Rails application, eventually find themselves fighting with Rails or dealing with ways in which Rails was not designed to do hard things or ways in which Rails has syntactic vinegar designed to stop you from doing things the way is necessary for your application. Merb tries to get out of your way, give you head room for performance, but also does not try to make too many opinions that will get in the way of doing hard problems with Ruby and web applications.

4. What are the design principles behind Merb that allow you to be more performant or simpler?

It's just a bunch of things in Merb that we don't do. For instance, is like the classic one of Symbol#to_proc, which Rails does. Actually, Rails recently took out a bunch of use of it, because is 10 times slower on Ruby 1.86, so it's just the things that we don't do in Merb that give us a lot of performance right out the box. We also try to keep our code relatively small and modular, so a few just install Merb core, which is the core of framework - there are only 6.000 lines of code in there. Just starting up Merb and having it run it's going automatically takes up less RAM, the critical path will be smaller, we don't do alias_method_chain, which means that the number of method calls for a regular request is going to be smaller out of Merb. These are obviously all performance, enhancements that are not going to you very much good is your application is really slow, but if you application is very performance critical, the fact that we have our core smaller and our requests take much less time will help you.

5. Talking about Symbol#to_proc, what do you think about open classes, or using open classes in the Merb core?

Ruby has open classes, that is a good thing, this is why people use Ruby. In fact, I don't even think of open classes as being open classes - I think of them as in Ruby, when you open a class, what you're doing is you're defining some things about a class and I don't really think of the end as closing the class, that's not really what's happening. You can go too far, Rails does a lot of alias_method_chain and, in my view, alias_method_chain is evidence of bad API or problematic API and it's bizarre to me that anyone will ever do alias_method_chain on their own code. I would sort of vaguely understand why, if you have someone else's code and it doesn't do quite what you want, you might do something like alias_method_chain. I can't figure why you would ever do alias_method_chain on your own code, it should be a code smell, it should be like a siren that something about the way you design the original code is not adequately extensible.

6. Could you explain what method _alias_chain is?

Alias_method_chain, sure. Ruby lets you take a method and rename it to something else, and you can give a new method the original name and call the old method. So Rails has a simple trick code that the alias_method_chain that let's you say let's have a method that prints hello, it's called "print_hello". In Rails what you say is alias_method_chain :print_hello :good_bye" and then you get a method called "print_hello_with_good_bye" and you'll be able to call the old "print_hello_without_good bye" on it. You say "print hello without good bye" an then "print good bye", something like that. That was fairly technical, but the point is that it lets you take a method that didn't quite do everything that you needed and than add some additional things that you'd want it to do.

It turns out that Merb never does the alias_ method_chain - it's not included with Merb - that any time anyone ever comes into Merb room and says: "Guys why you hate alias_method_chain so much?" Here is a really good example where I need it - it turns out that every time is either they misunderstood something and there is a good API that already does it without alias_method_chain or we made a mistake and there is something inadequately extensible and we fix it. If someone proves to me that they need alias_method_ chain, as far as I'm concerned that is a bug in Merb. We didn't adequately exposed an API that needs to be exposed, then we fix it - we've done that several times.

7. Continuing talking about open classes, what classes do you open in Merb? Do you open any standard library classes? What's your policy?

Yes, Merb has core extension library, we do some of the similar things that Rails does, we have for instance camel_case and underscore. This is just the life of a Ruby DSL, web DSL that you need to be able to take things that are coming in as parameters and convert them into the appropriate class, names and things like that. That's fine. It's kind of tricky because DataMapper, Merb and Rails, actually ActiveSupport, will have different ways of doing this and it's going to be crucial moving forward that we fix this. I've been talking to the Rails team and DataMapper team about standardizing a lot of these and that's crucial. We need to have one camel_case.

For the most part, our open classes are really disambiguated, we have a bunch of the standard Rails, things around classes like attr_accessor, we have our own version which works better. It's fairly dangerous to open classes if your user might also open a class. There is not much I could say other than "We do it, we like Ruby for these feature, so we're not going to try away from it, but we do it with care" - I think people should do that in general.

8. You talked about Merbs modularity of design for modularity, what would be an example for that?

My prototypical example: I joined Merb in the 0.37 or earlier era and I joined Merb primarily because I was working on a really hard Rails application that had a lot of mail and mail and Rails is miserable - it's sort of a fork of an early action controller, it never really got updated and I always said "Why is the mailer it's own separate thing, why is it not just another controller?" In Rails the reason why there's not another controller is because the controller knows a lot of things about the request and the response and what status is and headers and all the stuff, and it's actually part of the way controllers work.

You can't just subclass controllers, and make it a mailer, because it's too entangled with all that stuff. When I joined Merb I said "OK, here is how it's going to have to be: we're going to have to make an abstract controller class, or something" - I think Ezra came up with the name abstract controller, but a class that is the super class of controller, that other things can inherit from. It will handle things like rendering, figuring what templates to load, helpers, layouts - filters all that stuff, but it will not know anything about the request and response.

3 days later, Ezra handed it off to me and said "Here, I have now done this, and I built the mailer as a subclass of abstract controller", which means that every time we modify the way controllers work in Merb, the mailer automatically gets updated with those new features. In fact, when we went to Merb 0.9, the mail controller - which is like 100 lines of code - almost didn't change at all, but it now works exactly like the way regular controllers work in 0.9, even though there are significantly different features than 0.5. So, that's one example.

9. You also use gems as an extension method, as compared to Rails plug-ins?

Rails is actually coming on board with gem plug-ins. What we decided really early - in the 0.3 or 0.4 era of Merb -, was that Ruby already has a great packaging system, called gems. It already has dependencies, and there is all this features that people want in Rails plug-ins and they are there, in Ruby gem. We had to figure out some things, like "People want to be able to freeze gems", so we use another feature of Ruby gems, which allows you to make a local package repository. You can say "Here is my local gem" and in Merb you can say "gem install my-plug-in -I gems", and it will stick it in like a frozen plug in. We just use another feature of RubyGems - is like one line of code. Yes, it's kind of a No-Brainer, I think Rails is even coming on board, so there's no reason to reinvent the wheel in this.

10. In the Merb source code I saw some special documentation or some markup used to document your APIs. Could you talk about that?

Sure. First of all, Merb has a very strict notion of public API. One of the things that we feel went wrong with Rails is that Rails said "Ruby is your API. Use alias_method_chain." Who cares if it's private? You send whatever." And everyone did that, every plug-in does it, which means that every method that exists in Rails is a public API, which means that if they want to change something it's guaranteed to break a plug-in. If they would try to modify an internal, it could break a plug-in, which sucks. Our philosophy is that there is a public API, it's the one that regular people should be using.

There is probably also a small extension on that, which is the plug-in API, just like the plug-in user should be using. The vast majority of the code in Merb is actually internal code and we're telling people that "This is not code you should be using in your application." Again, it's like the alias_method_chain thing – if you find yourself having to use it, it's a bug! We will expose whatever public functionality is required for you to get your job done, for you to use Merb the way you want, for plug-ins to work. When Merb Slices came out and we had to shuffle what was public a little bit, but it is not OK to use the private API in Merb, it's not OK to use Ruby as an API – we have an API that's defined.

The first part of the answer to your question is that the markup defines what parts of the API are public and private. We also have API that's more structured - I talked about Yard yesterday in the talk at RubyFringe. We came up with the same idea in parallel, which is "Wouldn't it be great if there was a structured way to specify parameters and returned values and what things yield to, what things take a block, all this structured information that users really want to know?" Users want to know what options are available for an options hash. In Rails you have to go dig the source, you have to try to figure out, unless they happen to grace you with information, but usually you end up having to dig through all the methods that can call a particular option, trying to figure out what's called.

In Merb, all that stuff is required to be documented and I said at Rails Conference in my Birds of a Feather talk that "It's a bug if it's not there." If there is a method that doesn't have a documentation you can file a bug and a bug tracker - it's actually mandatory. Moving forward, we will be switching to the Yard format, which is almost identical, but is even more structured and easier to parse. Right now, our documentation is really structured, but there is no parser for it and rather than write our own parser, which is crazy, we are just going to use Yard, which is basically the same thing but already has a parser.

11. Where does Yard come from? Was that inspired by the Merb documentation? Does it happen in parallel?

It's actually a totally parallel development and I think almost certainly Yard started before Merb, I just didn't know about it. When I moved over to Merb 0.9 I made some documentation standards and then someone said "Hey, there is this thing, Yard. Check it out!" And I said "Wow, that looks almost exactly like what we came up with!" Ours is more trying to make it look right in RDoc so we do things that are RDocy to make it print correctly and then this throws away RDoc all together. It's a weird parallel development that is interesting.

12. Does Yard work with RDoc or is it a replacement of r-doc?

Yard is a replacement for RDoc. It has it's own parser of Ruby structures, it doesn't necessarily parse the insides of methods, but it parses things like attrs and class variables, so it's able to know what the structure of a class looks like, what inherits from what modules it includes. It has it's own template generator, which is substantially simpler than the RDoc template generators.

13. What's the format of this Yard documentation like? Is it like a contract that's defined or is it simpler? What level is it at?

I would say that if it says something that isn't true it's a bug – that's something that I say a lot, but I'm serious. If it says something that's wrong it's a bug and it's designed to be as a way to specify contract. The documentation on top of this code represents the code, it provides useful metadata for code completers and other tools that could exist in the future. Basically, it will make it easier to make an IDE for Merb, but it will also make it easy for someone who is just using API documentation to know things like returned values and what types of parameters they are expecting.

14. Do you have any poster child for Merb, a big applications that you would like to mention that use Merb?

Sure, probably the biggest application that uses Merb is Apple's SproutCore, it's not actually by Apple. Apple uses SproutCore in MobileMe – it's a weird story because SproutCore itself is sort of a client library, so it's like a jQuery thing, but in order to install SproutCore and use it you make a Merb application and do regular Merb things and it's a pretty heavy Java Script thing.

15. What's SproutCore?

SproutCore is the Java Script framework that Apple uses for MobileMe. SproutCore doesn't use the Merb server, but it uses Merb generated HTML, Java scripts that is then statically served by a server. It's like caching all your pages upfront, but it uses Merb and that means that Apple engineers have Merb on their machines.