We are here at RubyConf 2007, we are here with Jim Weirich. Jim why don't we start out with maybe can you introduce yourself, of course you are very popular but still for some watchers who don't know you.

That will be fine. My name is Jim Weirich I am from Cincinnati Ohio and I am the Chief Scientist for EdgeCase LLC a Columbus based Rails Development Company. My background, oh, I have been programming for more years than I am willing to admit. Lots of languages in my background including things like Fortrean and Lisp of course Java and C++ and those, and I discovered Ruby about eight years ago. I remember it was the year 2000 and Dave Thomas made an off hand comment about this little language that he had discovered, and I just got on reading the "Pragmatic Programmer" book and so Dave Thomas likes this Ruby thing, sounds interesting, so let's go and see what it is like. So I downloaded it and loved it immediately. It was exactly what I was looking for, to kind of replace all the tool building stuff I was doing in Perl, I wanted something that would grow a little more naturally, had a little better data structure support and abstraction capability. And Ruby was just exactly what I wanted at the time. So I fell in love with it and have been doing Ruby programming since then for fun and for tool support just recently now for a profit.

That's good. And you have added a lot to the Ruby community with your Rake tool. Could you thoroughly explain what Rake does?

Rake is a version of Make in Ruby. If you are familiar with the Unix Make tools you tell it to build things and it goes out and figures out what it needs and puts them together and builds up exactly what needs to be done to build a program or to do a project, or to do anything actually. And I was really frustrated with Make one day because I needed to do something just a little more dynamic than it was really needed, or that it really support it and I turned to the guy that was working with it and said: "Wouldn't it be great if Make were written in Ruby" and my friend says "I guess, but I have no idea what you are talking about". So I turned around to the white board and I quickly sketched out what a task would look like in Ruby, you would say "task" and then you would give it a name, and then you give it the dependencies then you have a lambda, a block body where you specify what the actions were.

He said "That's brilliant, we agree that it would be wonderful to have but that no one would really take the time to build something like that, because we have to re-implement Make and there is lots of stuff in Make, and just to have it in Ruby wouldn't be worth doing it at all". So he went back to his desk and I sat there for a few minutes and started thinking what would it really take to implement something like Make in Ruby. And I sketched something out and in about half an hour or so I came back to my friend's desk and said "Pull up your terminal I sent you an email, you got to see this" and he pulled out the first beginnings of Rake right there, in just one page of code was the whole Rake engine right there and that's what started it, just a very simple beginning and it grew very naturally from that.

Rake has become pretty standard in the Ruby community now.

Thanks to Rails.

Was it the first big use?

Well Rake was out there probably a year before Rails was around but David Heinemeier Hansson decided to use Rake as the builder for Rails itself. So I always thank David that every time someone downloads Rails they also have to get a copy of Rake to go along with it. So that's kept my download counts really high on RubyForge. And it also really exposed Rake to a lot of people who wouldn't have seen it otherwise.

So it has become quite important, so important that it is actually considered for inclusion in Ruby 1.9. Can you describe what kind of process you are going through with that, if you have to do any changes for it?

There has been a lot of discussion on exactly how we are going to include Rake into 1.9. We have not really decided on a concrete thing yet, but the general feeling is that however it gets in and it is going to be bundled with 1.9 so when you get 1.9 and all its goodies Rake will just come with it automatically. But we wanted to make sure that however it is bundled in, when new versions of Rake become available it will be easy to upgrade them, you won't be stuck with the, we won't tie the version of Rake to a version of Ruby, you will be able to update Rake independently. Whatever the mechanisms are, we are going to make sure that happens.

Which version is Rake now at?

Oh, your question is funny because I don't remember version numbers very well. It's last than one 1.0, I think it's 0.7 maybe, seven something, I always just go and look because I never remember version numbers.

So is Rake 1.0 going to coincide with the inclusion in Ruby 1.9?

No, there's a couple of features I want to get into Rake before I am ready to call it 1.0, and we are not going to get them in before 1.9 is ready. Whatever goes within 1.9 is essentially where Rake stands right now. We have to do a little bit of work in Rake. My understanding is that it doesn't work in 1.9 right now, and some of my homework I have to do after the conference here is go back and figure out exactly what those incompatibilities are and see if we can get those ironed out.

What can we look forward in Rake 1.0, what new features?

Near term, very near term before 1.0, you will get command line arguments in Rake and that's something people have been asking for a long time I kind of resisted mainly because the typical way of doing command line arguments is just to list parameters out on the command line and want those arguments to be passed in, but there's a question since you can have multiple tasks in the command line as well, how would the arguments match up with the tasks? And I didn't see a really easy way of specifying that. And then there's an additional question: how do you pass arguments around from one task to another?

How do you do that? So there are two issues that were really needed to be resolved. Someone suggested to me a couple of months ago which kind of led to this whole, actually putting in argument passing, maybe you could just pass them like functional parameters, put a parenthesis or a comma to the arguments and tie them to the task and then you could say say Rake, the name of your task, for example "test", you have a test target, and then you in parenthesis paste the name of the test you want to run. Or for db:migrate in Rails, instead of saying version equals what version you want to migrate to, you put that in some kind of parenthesis, on that argument and then you know which parameters go with which task name, I thought that was a great idea and we are doing that with the only difference that instead parenthesis, since parenthesis start sub shells on a uni shell, that wasn't a real good idea, but square brackets worked great. It is kind of Ruby-ish too because that's how you call lambdas with square brackets. The new version of Rake will allow you to put a square bracket on your task names and pass on parameters to specific tasks just like that.

Any other big features in 1.0?

The next big feature is that after that is done I really want to work on a plug-in system for Rake. What I really want to enable is for folks to share their Rake tasks very easily and I am not quite sure how exactly that is going to be working. I have been thinking about this for actually a number of years, and haven't really come down with strong decisions but we are going to hammer the details here. There are some things I want to see here: number one I think that plug-ins need to be distributed as gems. So we solved all the distribution problems, we just tie on to the gem mechanism, if you want a plug in it's a simple gem installing , then you're done, you've got it, and then we'll make that plug-in available to all your Rake tasks anywhere in your system.

That being said, there needs to be some kind of protocol that exists between Rake and the plug-in so that we can get it initialized properly and we get it parameterized properly. And I am thinking something very simple here, I know Rails plug-ins, all you have to do is create a init.rb in the top level directory and then it gets run. Something similar to that, but there might me a little more involved, because we want to make sure that the plug-in itself is initialized and then the plug-ins are generally used, for example in Rake right now, we have a test task module that we use to easily create a set of test driven targets for Rake. Other than that the name test task is a really bad choice for a name, and I take full blame for that, what that does it's actually not a task but it's a builder of tasks.

And it will build up the Rake tasks there and with whatever names you give them wire up all the dependencies between those tasks, plug-ins are going to want to do that kind of thing as well. So we are going to have a way of having those builders run from your plug-ins, so you need to initialize your plug-in, you have to be able to run the builders from the plug-in in some ways, so we are going to come up with some syntax to cover that up as well. Those are the two big things that we want to get together for the plug-ins, but the key is to make it easy for people to share Rake tasks.

Rake is one of the big examples for one of the strengths for Ruby, building Domain Specific Languages. It has been quoted and used by Martin Fowler as a big example for embedded Domain Specific Languages. Domain Specific Languages hype has been going on for some years. What's your point, what's your view on that?

It's really funny because I argued with a friend just last night on exactly what is a Domain Specific Language and I feel the term has probably been overused. I like to consider a Domain Specific Language to be something that talks about a domain, a problem domain, such as configuring workflow to solve some business problems or talking about, now I am dry for examples, but something that is business related in the problem space. And I see a lot of us using Domain Specific Languages to solve programmer problems, which are really problems in the solution space.

And Rake is kind of in that solution space problem, I guess you could say it's solving the domain of "how to run tasks" but that's so wrapped up in programmers speak, I'm not really sure Rake is a strong example of a domain specific language. And to me it's a whole continuum that you are dealing with, you have got everything from nice well designed libraries to something that is a DSL, a Domain Specific Language and Rake falls somewhere in the middle of that, somewhere. It's more than just a library, because it provides a little bit of framework, and it makes it a little declarative, but then it's really addressing a domain that only programmers care about, not a domain that you want to solve business problems in, so that's kind of where I fall on that whole situation.

So basically you are thinking that the domain is something that a user or business user should see?

Yes, because what I want to use a DSL for is I want to be able to take a problem that a user comes to me and he says: "Look, I am a lawyer and I have all these legal constructs that I need to deal with. Can we come up with a way of constructing contracts for me that we use terminology, in creating a DSL that might do that?" Or I might be talking to an insurance firm that has "We have this kind of policy and that kind of policy and we need to be able to put a policy together quickly and come up with a price structure for it".

And create a DSL that describes it. I don't think that the business person needs to be able to write the DSL, but I should be able to write it in the DSL, show it to a business person and they can say: ‘"Yes, that's what I want" and it is something that is immediately obvious to them that it solves their problem domain. It's Domain Specific Language and if it is not addressing a domain, the term DSL seams kind of loose for me.

It's just a specific language.

Yes, a specific language, exactly. Maybe we got Domain Specific Languages and maybe a Problem Specific Languages, maybe Rake is a PSL, forget I said that.

This is the first occurrence of this term? PSL?

Yes, I deny I said that.

So what would be a cut of point where, you said you had to have a range?

A range, yes. So if you have a DSL at one end it's something that you can show a business person and someone who is knowledgeable about the domain we are talking about, they can read it and understand it, I think a DSL falls at that end of the spectrum. I think you have nice libraries down here at this other hand of the spectrum and a lot of things fall into that. I think Flexmockock, for example, is a great fluid language for specifying how to mock out things, but that's very programmer specific, it's a library, it's not a DSL in my terms, but it's a really nice interface for doing that kinds of things. So I think you can have libraries with good interfaces but that doesn't necessarily make them DSLs, certainly dropping parenthesis off a library call doesn't make it a DSL.

You also provided the Builder library, first of all would you consider the Builder library to be a DSL or?

I could see that coming. So what domain are we talking about? Writing XML. Is XML a problem domain, a business domain? I don't think so. So I would say that Builder is an excellent library with a really cool interface, a really good interface, but I don't think it's a DSL in that sense so my use of DSL I like to be a little more strict with it. But let me emphasize that doing stuff like Builder and Rake and Flexmock that's good Ruby programming, and whether they are DSLs or not, or whether you call them DSLs or not, is not really that important. Being able to write that kind of code where the interface is clean, and obvious to the person who is using it, I mean that's a really great feature of Ruby.

Would it be better to call it meta-programming, use the term meta-programming for that? Or is that specific in another way?

Meta-programming is another specific term that I think sometimes gets misused or overused. Meta-programming is programming about programming, or writing programs that manipulate or write other programs, and that's certainly something you do in Ruby all the time. But I wouldn't think that those things we just listed are instances of meta-programming.

Talking about Builder, for Builder you faced some problems with using method, method missing. What was the solution to that?

The basic problem in Builder is that the vocabulary that Builder deals with is wide open. Builder basically allows you to generate XML, and you have a Builder object and you call a method to it. Whatever method you call the name of the method becomes a tag in your output XML. Since the vocabulary of XML is wide open, and tags can be any valid string, any XML ID, the number of methods that you call, you can't pre-calculate what those are. So Builder has to allow you to send in any method and respond to it.

And that works great using the method missing technique in Ruby. Except where the method isn't actually missing. If you want to create an XML tag that has the same name as a built-in method in Ruby that's on Object, than the Builder object will call in the build in method rather than generating the tag that you want. And that's the problem that I was running into, I wanted to have Builder be an object but without all the garbage in the Object name space. So blank slate was a technique we came up with, that essentially blank slate goes and removes all the methods from itself, it iterates through its instance methods, and here we are talking about meta-programming, this is meta-programming; walking through the list of methods and removing them so that by time it's loaded blank slate has no methods on it at all, except a handful of very system specific messages which you really shouldn't redefine anyways.

So, if I inherit from blank slate, which Builder does, then I got an object that has only the very minimal number of system specific messages supported on it, and everything else is undefined. Which makes it, I don't know if you have ever worked with Builder, in IRB but it makes it really strange to work with it like that, because you create a Builder object and if you let it be the last thing in the command line of IRB, IRB takes it and wants to print it out. So sends it inspect so all of a sudden you are out there inspect tags appearing in XML because I abused interacting with your object as well. Makes Builder kind of a hard object to manipulate, because it doesn't act like a regular Ruby object. But for the purpose that it was designed that's exactly what you want, you don't want it to be a regular Ruby object.

So which objects, which methods still remain in the blank slate objects?

Well there are two for sure: there is under __id__, which is the build in method that returns the object id for an object, and then there is __send__, which sends a message, those are build in for every object really shouldn't be messed with, the system level methods. And I think there's a third one, I think we leave instance_eval in there, because having that available turns out to be very handy in blank slate, and it's probably unlikely that you are going to have an XML tag called instance_eval so I figured that one was safe as well.

We also have some mechanisms in there that you can dynamically reinstate methods, so if you really want to have to_s available on a blank state there's ways of going in and reinstating those kind of things.

Talking about your projects, you mentioned Flexmock. What's Flexmock?

Flexmock is a mocking framework that we use for testing, and this is actually a very old library, it's been around a long time and like I say I learnt test driven design and Ruby about the same time, so the whole test driven approach and Ruby went very well together for me. When I was doing this a lot of my testing I was looking around for a mocking framework and I remember it was just after OOPSLA 2001 I saw a Java marking framework called EasyMock. It was really the first time I have seen a mock framework and was really impressed with how easy at least in Java it was to create these mock, I thought there should be something in Ruby.

And I looked around and I found a mocking framework for Ruby, and I started using it, and I was a little disappointed with it, it was called rmock and I don't think it's around anymore, but it was very specific in how it accepted methods being called on it. If you defined a method, mocked method A and then mocked method B, when you used that you had to call method A and method B in exactly that order, and it was very strict about how the methods were resolved and I found it really inflexible. So I wanted a marking framework that was more flexible than that, so hence Flexmock and the interface from Flexmock is really evolved over time, the original interface wasn't very good, but as we used it and learnt by it we stole ideas from JMock and the Java framework, when the Mocha and Stubba came along we stole some ideas from that and crossed pollinated at each other.

The interface changed over time and we constantly finds ways of improving that and putting little things into it, what I really want Flexmock to be is a mocking framework that solves your problems, I don't want to restrict you into a particular kind of mold, or doing it exactly my way or exactly that way. So I wanted it to be flexible so a lot of different people can use it to meet their needs.

What would be a quick explanation of the mocking concept? What does Flexmock provide?

When you test an object, a good object oriented program has objects that collaborate with each other so one object will talk to that object will talk to this object over here. But when you test an object, you don't want to have to create the entire infrastructure of a program to deal with that. So, you want to just test this object without everything else involved, so you use a mock to stand in for the object set, objects and the testers communicated with, and that allows you to cut it of at one level of interaction. And that is essentially the whole point of that. So you create an object create a couple of mock and then you can test the code in this very thoroughly.

The mocks allow you things that wouldn't be available otherwise if you are actually using real life code. For example, if I am testing a web framework that goes across the web and grabs something, brings something back, first of all it will be slow, using a mock would be much faster than that, your tests would run much faster, second of all I can control what errors are returned by your mock, so if I have exception handling in my object under test, I can return bad data from my web service and make sure that my exception here and my code is tested, and is much easier to get good code coverage using mocked frameworks.

So mocking is one of the concepts that is used in Test Driven Development or maybe Behavior Driven Development, is that it?

Yes.

Can you say anything about BDD, Behavior Driven Development, another acronym? What's the crucial difference?

The crucial difference is that there really isn't a difference. I see it's a change in terminology that changes your focus. Behavior Driven Design concentrates on writing specifications and avoids the words: test or assert. So it's really used and it changes it to words like context and specification, and should, and those kinds of things. But essentially, there are very similar, and if you are doing good Test Driven Design it's almost identical to doing Behavior Driven Design, at least in my point of view. The difference is in the tools really; because the approach is the same the tools are what's different and how they focus the efforts.

A BDD type framework like RSpec which is really great and I really love it, and actually I love it for different reasons that other people, that's interesting, it lets you focus on behavior and specifying without using the word test. I think if you are using Test Unit right now in Ruby and you have not looked at RSpec you should look at RSpec and use it for a while, get used to it and understand it and then if you don't like it, go back to Test Unitthat's fine but what I have found is that when I looked at RSpec it really changed the way I approached my Test Unitwork as well. My test methods became shorter and more focused on exactly on what they were testing or specifying and my method names became longer. So my Test Unit stuff started to look a lot like my BDD stuff as well.

Did you use TDD and BDD simultaneously? Do you use Test Unit and RSpec or you just use Rspec?

I would say the tool is not the important thing is how you use the tool, and I tend to use both tools in the same way. But using both of them helps shape the better way to deal with. Does that make sense?

You used both of them at the same project?

No, I mean being able to switch and getting experience in both of them, are really shaped the proper way to do it. And it helped me become more focused when I was using test unit.

So BDD removes the word test so people don't confuse it with functional testing?

Right and people who are first coming into the test driven thing, by using RSpec you lose the focus of writing the testing portion of it and you get down to writing specifications, which is what we do in Test Unit anyway. A good test driven design is actually writing specifications and then writing code, it's an important piece of that is the tight red green refractor loop. You write a test, you write a little bit of code, you write a test that fails, you write a little bit of code that makes it pass, you clean up your code then you write another test or specification, depends what the tool calls it, but the cycle is the same whether you are using Test Unit or RSpec so to me it's not the tool that is important is that red green refactoring cycle. And that's critical, if you get that then you are getting it whatever tool you are using.

So to wrap this up, you also published a library for the X10 system?

Not recently. Some time ago.

What is X10 just to explain?

X10 is a protocol for home automation, essentially you plug X10 units into the wall sockets of your house, all over your house, and they communicate with each other over your electrical wire, it's like network in your power system. And it's used for really exciting things like turning lamps on and of, or running security cameras, or sending various simple commands over the wire like that, it's very slow, it's not a high speed network by any means, but you do simple things, like controlling house hold appliances and lamps and things like that.

And your library did what exactly?

There is a small X10 interface called the CM17A firecracker module that plugs in the serial port of a computer, and allows the computer to talk to the little module, it's really about that big, and that contains a radio transmitter in it that transmits to a receiver unit that plugs into the wall, so your computer is not plugged into any electrical outlet through your serial port which I think is a good thing, but just uses a radio transmitter to transmit to that which then rebroadcasts the X10 protocol information over your house wire to communicate with everything else, so from your computer you can the turn on the lamp over there, turn your TV off when you go to sleep or use it to control anything you want to.

All my library did was take the C code that would drive the firecrack unit, which had a bizarre protocol, it's a serial device but you don't talk to it using the serial in and out lines, you actually toggle the two hand shake flow control signals and you toggle them in such a way that the baud rate is real slow, it's bizarre. And so it's very timing oriented, so that code is actually written in C and all my library did was wrap that C code in some Ruby wrappers so that you could call that and send on and off commands from your Ruby code.

And I understand that, as a good TDD person, you used that for automation?

Continuous integration and notification. So we could set up a continuous integration process that when it failed it would turn on a red lamp or turn an alarm or a ringing signal or embed a compile in test passed they will turn up a green lamp.

It's very useful to annoy developers to fix their code.

I've also used it with great effect to annoy my children.

Really?

Yes, "Dad, turn that light off".