Reginald Braithwaite on Rewrite

Recorded at:

1. We are here at Ruby Fringe, so who are you?

I am Reginald Braithwaite. I am a software developer and, at the moment, I am doing a lot of work with the Ruby programming language and especially we are trying to reprogram the Ruby programming language.

2. So what are some of applications that you have worked on over time? For instance you worked with Borland Pascal.

Yes, that was actually a really good experience, the company was called Bonnar Associates and the founder of that company Justin Bonnar ended up working for Intuit in California. We were selling classified advertising systems to companies that do penny savers. So the things like the buy and sell newspapers that you find here in Toronto or other penny savers. I have a feeling that industry may have disappeared in the last couple of years, but at one time you could get these big magazines usually printed on very cheap paper, with a lot of car for sale, stereo for sale, garage sale on Saturday, and at the time we were doing this, people were physically laying out the ads.

So they take ads in a big call center and they take the advertisements and so on and they would physically compose the ads. And some of the really, really, big chains had mainframes that would sort of make the paper and give instructions to these incredibly expensive at the time type setting systems, that could be controlled only by mainframes, don't even think about PCs in those days.

And Justin had seen Macintoshes but they were too expensive, but he liked the laser writer, so his innovation was we ought to be able to make a system that could print these things on Apple laser writers, and to give you an idea of our sophistication we wrote our application on a mini computer and it would then talk post script directly to the Apple laser writers so we were out putting post scripts which would then do all the ads and then they would just tale the ads which were done on four or five columns on eight and a half by eleven pieces of paper, and they would physically cut them and the use wax, or glue or whatever it is they did, and paste them into a magazine. So there is no PageMaker, none of that stuff, they just worked with these but it was a wonderful step. And I am glad you asked about that because as a first experience it was very good.

I got to actually travel to customers and see how they worked, and learned a little bit about the business of selling to organizations, my sales experience was very helpful at that time. And also I got a really good exposure to what I would call the end to end programmer that is an idea from talking to a customer and that looks really interesting but our business doesn't work that way, we need this feature, that feature. If we were to build this feature, that feature, we would be interested in buying, we'll be interested in seeing a demo, sort of going through the whole thing from the customer back to doing the technology back to delivering the product. And that experience has been invaluable with me it's not like I was in the corner just writing printer drivers, or just doing database tuning or something else really specialized, I got a real general overview of a software company as an employee. And that is I believe served me well.

I should make it clear, as it is actually practiced not as you read about it in textbooks. Whereas in textbooks we talk about having the requirements and following all these steps, what I actually saw is what happens in most real software companies, I am sorry to say, which is the sales person says "Yes, we can do that for you, you need it by Tuesday, not a problem", and then he comes back and tells the programmers "By next Tuesday I have got this thing and it is scribbled on the back of my business card, make it so". And I am kind of cracking a joke about it, but I am sure many people listening to that who work in our industry will cringe and say "Yes, that happened to me last Wednesday".

3. How did you get into the Ruby space? What was the language before that?

So, what led up to Ruby is a long story. I was interested in Scheme and then I ended up doing some work for a mutual funds company, and we were just sort of getting into the web and so on and we took a look at some of the technologies that were available by doing CGIs with Perl, and some other things for creating web applications. This company wanted more than websites, right from the beginning they had a very simple requirement: they managed mutual funds for people and people liked to be able to find out what their portfolio is worth, rather than getting it four times a year as required by law, they wanted to be able to tell them and up to that time the way it worked is you called them up on the phone and said "Do you know what it is worth?"

So right from the beginning they had a vision that once they start a website there ought to be a page on their website where you could go and see the various funds, what the funds net asset value is, NAV, in other words what it is worth today. And from there, they sort of knew in the back of their head that one day customers would see their whole portfolio there, but the first step they wanted the challenge they gave us as software developers was can there be a web page which is automatically updated once a day so that it shows what the current, yesterday's, net asset values were, not live like you see today and so on. But at the time that was a huge challenge.

So we looked at a bunch of technologies and one of them was Java. Now, I think many people will not remember this but when Java first came out there were no Java server pages, there was a servlet specification, that came out a little bit after Java itself came out, an we had this servlet specification we saw this was not a good way to produce web pages, so we wanted to create our own thing like a Java server page where we could do a template type thing, because we worked with this technology before. And so we set up to create that, which means we had to create our own language, and I had some background with Scheme, so I said "Our language should be like this".

And my partner said "Why do we want to use this Lisp like thing?" and I said "Because it is very easy to parse". And that's how I sold my partner on writing Scheme, so we wanted Scheme to run inside of a servlet so I wrote a Scheme interpreter in the Java servlet. So you are asking me about Ruby, how does that lead forward? Well to learn about Scheme and Lisp I started reading some books and one of them was by a fellow named Paul Graham who wrote on Lisp, and that got me kind of interested and never did anything with common Lisp object system or common Lisp. I did start getting on some mailing list and Paul was organizing something called the "Lightweight Languages Conference", and there had been two. There had been one and I was going to LLC two.

I went to this thing, it was one or two days affair in Cambridge, and this was November 9th 2001 or 2002. But I went to this conference and there were two key note speakers, this was in 2001 perhaps. And the two key note speakers, one first thing in the morning and one at lunch. The fist thing in the morning key note speaker was a fellow named Joe Armstrong, and the second key note speaker was Matz, so seven years ago Paul Graham had the foresight to realize we were doing a conference about programming languages that the two most important programming languages to learn about would be Ruby and Erlang.

4. So in 2001 you heard about Erlang and Ruby.

On the same day. It is actually, I have to be honest with you, though it sounds great as a story, it's actually overload because it's not a way you can go away and research both of those languages. And for whatever reason, I didn't get curious about Erlang, but I did get curious about Ruby and I started playing with it from that conference. And I also played with Groovy, and some other things, but Ruby just stuck. I really believe there is many rough edges that need to be sanded of, but fundamentally Matz' approach to try to make programming a joy, or trying to work the way the programmer works, I think he was on to something.

Two maybe reasons if you look at this feature or that feature and say "That's not what we would like" and so on, but the fundamental thing he tried to do, which was to design the language the way that one might design an Ipod, or one might design a piece of furniture to be comfortable and useful and practical was spot on, and I just found it a joy to work with. And that got me excited and I have been excited ever since.

5. What's the bigger influence in Ruby? Do you see both influences in Ruby, do you see both influences of Lisp and Smalltalk equally?

He's caught on record saying that Ruby is actually MatzLisp. However, quite honestly, I see more Smalltalk than Lisp in Ruby in a couple of ways. First, unlike Scheme, Ruby doesn't come down to a couple, like Scheme has five special forms, and everything is actually built out of those five things. Ruby isn't like that, also Ruby doesn't have a meta-object protocol the way common Lisp does. I find that Ruby is more like start with Smalltalk and then make some functional like things, the way Smalltalk has its own version of blocks, but make lambdas and prox, but build them out of Smalltalk objects which is the way lambdas and prox actually work in Ruby, they are really objects, they are not first class functions on their own right. It's not for me to argue with the language's designer, but I personally feel there is more Smalltalk than Lisp in Ruby today.

6. ou are currently interested in bringing more functional concepts to Ruby with your work on the Rewrite Gem. So where did that come from? Did that come from macros? Is there something else in Scheme that inspired that?

Well, obviously, I have had some experience both with the common Lisp style of what is called the unhygienic macro and with the Scheme hygienic macros. And I read about Haskell rewriting and on top of that I actually done some work with some XXLTs that wrote some other XXLTs, so I was a little familiar with the idea of transforming trees and all these things kind of came together when I actually run into a problem completely unrelated to Rewrite. I had been doing some work that involved transforming large chunks of XML and JSON into a different format, and I really wanted a programming language that would be really good for that. And XXLT is certainly capable of doing anything but it is not a joy to work with.

And at the time I had been inspired by Symbol#to_proc and String#to_proc, and I thought really what we need is Array#to_proc, a way to specify with an array what kind of transformation you want by example, showing what you want and showing an example of what you were going to receive and an example of what you want to create, and finding an easy way to specify the transformation. Anybody knows XXLT would say "That's what it does" it just doesn't do it in a way that is really obvious and a joy to work with. So I was really enthusiastic about this. And then I got involved in writing some other Gems, starting with andand, and a few other things, and I really was disturbed by some of the issues to do with meta programming in the global environment and Ruby, and I would put the plug-in, Matz has talked about, adding selector name spaces to the language 2.0 time frame.

I would really like to see some solution to this problem. Anyways I have the problem today, not in the Ruby 2.0 time frame, and somehow it clicked with me that my experience with macros as you pointed out, my fooling around with Array to Array#to_proc, and transforming trees, because Ruby programs like Lisp programs, behind the scenes are trees of symbols, that's the abstract symbol tree or S expressions. And really what you want to do is transform one kind of tree into another kind of tree. The trees just happen to represent programs instead of representing the data that one bank might send to another, or something else in the more business like context.

This previous experience with XXLTs this previous experience with Lisp macros and this itch that I wanted to scratch, ho do I do things without constantly opening up the object class, or opening up the nill class, or the array class, or the symbol class, or the string class. It just came to me: let's just transform Ruby programs from one form into another and not monkeying around there is an expression out there I will not repeat with the core classes like objects.

7. So what approach do you use? How do you do that? How is it implemented the Rewrite?

So, the Seattle Ruby crew are some phenomenally smart people, and speak of which if I may digress, I am not a smart person. What I am is a person who is interested in some of the same things that smart people are interested in. So I will say that the Seattle Ruby people are very smart, and they have written some gems, one of them is ParseTree, and another is called RubyToRuby, which actually reach into the interpreter and extract the symbol tree that Ruby is working with and then they cleaned it up, and I take these gems and what they allow you to do is write something like a block or a proc, and they give you the symbol tree back. I mean if you give it the symbol tree, they can produce the text of the Ruby code and then Ruby already has a Eval that can take a string.

The painful process that it goes through is you take some code that includes some macros or some transformations you want, you say "I want to apply these transformations to the code" it turns it into a symbol tree, it uses some of the things, very much like XXLT, more like XXLT than macros to be honest with you, in the implementation it simply transforms the tree by applying these rules. It says "Look for this, turn it into this, look for this, turn it into this". And then when it's finished complying no more transformations it then hands it back to one of these gems RubyToRuby, it says "Give me the source code for the new tree" and then it evaluates the new source code. So behind the scenes it's really just taking one tree of symbols and turning it into another tree of symbol using patterns that re-write the tree. And under the covers or behind the scenes depending on your metaphor, it is very much like the regular expressions, only instead of working on a one dimensional string of symbols, as a regular expression does, these work on trees of symbols.

8. Have you written a general pattern matcher for S expressions or is this sort of specialized for the Rewrite gem?

t's what I am working on is very important, so again I apologize for digressing, but I found this to be an important point in my own programming, and I don't know if that's a general point but quite often we see something and we are very tempted to say "Oh, this is a special case of a more general thing", and then we solve the general thing. However that is an infinite recursion, it's always the special case of something more general. And if you are always climbing up the tree into the more general thing you will eventually wind up with a PhD in computer science and no code.

However on the other hand, if we just scrabble around the Earth, and we never sort of pock our head around to see the more general thing. We are constantly resolving the same problems, we are not even recognizing that two different things we solved are both aspects of the same thing. So what I try to do is I always try to recognize what the general case is and then discipline myself not to solve the general case until it's really an imperative, but not to be ignorant of the general case. So coming back to your question, at the moment I am aware that it is a general case, and I am fighting on an almost daily basis not to make it too general because for example one of the things that really excites me when you look at the world of JavaScript frameworks, the ability to do things like say you see CSS like selectors to modify JavaScript, is very interesting and I can almost smell it in my nostrils, how exciting it would be if we could specify re-writing roles if we could use things like CSS selectors.

When you see a variable inside of a lambda do the following, when you see a variable that shadows another variable, this is for example something that the Java compiler will warn you about, if we had a more general purpose pattern matching agenda that I currently have, and in Rewrite you can write these things using an actual pattern specification language. At the moment, Rewrite is a little more crude than that. It looks more like a regular regular expression but you can have some blanks in there, you can have some spots template, so you can say something like Receiver.andand.method, where receiver and method are both sort of wild cards, so you can put anything in there, so it can be foo(bar) or it can be an expression and so on, transforming into this.

But much more general things which are really exciting to me that I would like to work on like when you see any variable that is nested inside of an expression such that it shadows another variable like do this with it, like rename it or something. I haven't implemented those but I am aware of it and painfully aware of it. But I need to get code written so I am not working on that yet.

9. Probably I shouldn't mention this but the Smalltalk community has their refactory support is using a pattern matching approach to match the code, but I wasn't going to mention it because you will be tempted.

Well, it's ok to mention it and one of the things that I have noticed is a lot of people of our industry are very ignorant of its history. They are not aware of what has been accomplished by pioneers working with Lisp, they are not aware of amazing things that have been accomplished by people like Ralph Griswold, who invented the SNOBOL programming language, they are not aware of people talking about the fact that the block syntax and yield was inspired by the CLU programming language, they are not aware of the work that has been done with Smalltalk, and many of these people use Java and always brag about how wonderful the Eclipse refactoring support is and they are completely ignorant to the fact that Eclipse is a Java port of a product originally written in Smalltalk, for Smalltalk using the same technologies, but it just like whoosh right over their head.

And I don't mean that as a beat, because I believe they are not aware of these things because they have never been made aware of it as opposed they have looked at it and decide it is unimportant. Hopefully some of the great work that people like InfoQ are doing to get the word out about some of these things will help to spread that seeds that people be curious and say "What's going on with Lisp, what's going on with some of these languages that aren't popular today, what's going on with Smalltalk" and then they will bring that forward, not necessarily they will drop off Java an pick up Ruby or something, but they will say "Hey some of these things apply, I can use them in the programming language I am using today, I can use them in the environment I am using today, there are practical things I can learn from studying these other things that aren't mainstream that I could use with the stuff that pays the rent".

10. Do you think it would be interesting to have a history lesson in the computer science curriculum?

If there isn't one now, the people who put the curriculums together should be beaten. Yes, the history is phenomenally important, we have a challenge, probably an industry challenge that is very hard to see the source code for a lot of things that have been written because of the way our laws and proprietary and so on work. I am not Richard Stallman or anything, but I would certainly love to see a change to copyright law such that software automatically becomes public domain after ten years or something, you know the old stuff somehow just gets out there so people can read. I can't understand this source code to some of the early Unix systems, but it is gold to try to piece through it and figure it out.

I have seen disassembled I remember I first got into Mackintosh hacking, and looked at dissembled bits of Mackintosh ROM, and just was amazed of the work of people like Eddie ..., but even that is kind of bootleg work. I would love not only that we teach the history of what we know in academia, but also that we can get to the history of great products that have been written in the commercial world that are out there today, which are inaccessible, which we only get to through anecdotes. When you meet a person and they give a talk and they open the ... and say a little bit and talk about some commercial product, but we can never download the source code, to say Windows, and see all the wonderful things that must be inside there.

11. If you could choose one language, other than Ruby, that you could spend time with, a lot of time with, what would that be? You can choose maybe two languages.

One is fine, gosh this is like taking me to the ice cream store and say "Oh don't get any weight". If there is one other language that I could spend a huge amount of time with? I know this is unfashionable but I am going to say Haskell. I am going to say the problem that I had with Haskell is that I looked at it, I played with it, I read a book about it and I am really in a blub space about it.

The problem is that right now all I do when I look at Haskell is try to figure out how to make Haskell do the things that I currently do with Ruby whereas if I really knew the language, I would probably try to do different things entirely, and that is really a sign, I am not going to say master it, but the sign that you are one step forward is that you are trying to solve different problems really understand your tool, you look at problems in a different way, you see them in a different light. I don't see problems in a different light with Haskell, and I believe that if I had the opportunity to spend a lot of time with Haskell, that I would start to see problems in a different light with it and I think that would be really exciting.