BT

John Lam on IronRuby, Microsoft and Open Source
Recorded at:

Interview with John Lam by Rob Bazinet on Oct 30, 2008 |
29:14

Bio John Lam is Program Manager on the Dynamic Language Runtime team at Microsoft. He created RubyCLR, and is now working on the team that is creating IronRuby, Microsoft's implementation of Ruby.

   

1. Hey I am Rob Bazinet from InfoQ and I am here with John Lam. John can you tell me who you are?

I am John Lam I work for the man, Microsoft, and I was one of the founders of the IronRuby project of the company.

   

2. What is IronRuby?

IronRuby is Microsoft's implementation of the Ruby programming language. We are one of the seven implementations that are currently actively being developed today. What makes us different is that we run on top of the .Net platform. So we run anywhere where the common language runtime runs. So today that certainly is on our desktop and server operating systems. It's also in Silverlight which is our cross platform version of the CLR that runs on Mac and Windows and runs in all of the major browsers on those platforms as well so that is in Firefox and Explorer and Safari.

What's also interesting about the stuff that we do is we also run anywhere where a compliant CLR will run as well. So the Mono project certainly runs on Linux and runs on Mac and also runs on Windows. So we can run anywhere where Mono runs as well. And Mono is also a Moonlight project which will bring Silverlight to Linux based desktops as well. We pretty well cover all of the major platforms today in terms of IronRuby and where are our reach is and where we can run.

   

3. I mean Microsoft has C#, VB.Net, interesting languages they have standardize on. Why are they interested in Ruby?

C# and VB are statically type dlanguages. And statically typed languages have certainly served the company quite well in the .Net era, but statically typed languages and the things you get from static typing don't come for free. So because they don't come for free the opportunity cost of using these things is something that in a lot of cases is somehow hidden from software developers, there is the cost of creating the code, of maintaining the code, there is a certain amount of Agility, that winds up being lost by using these things.

Not to say that there isn't a place for statically type languages but there certainly is a place for dynamically typed languages in addition to statically typed languages. And IronRuby is a language since it is implemented and runs on top of the .Net platform we get the best of both worlds. You can use dynamically typed languages like IronRuby or IronPython, in places where it makes sense to, where you want to optimize along with axis of Agility or time to market and you can also rewrite parts of your application in statically typed languages, in cases where you want to get things like performance or you want to get things like verifiability of the code that you have written as well.

And since there is an awful lot of investments these days both for Microsoft and other people but certainly our research division around static type verifiers, which offer a lot more guarantees that you would otherwise get simply by being another compiler in your statically typed language.

   

4. Now is Microsoft planning on, I noticed you have IronRuby and IronPython. Are you in the same group? Is it a dynamic language effort that Microsoft is putting forward?

Yes, so we have whatever random internal organization is which is not pretty probably interesting to people outside. We have separate teams that build Python and Ruby, we have members of our team that sometimes work on both, we have a separate team now that we have matured that works on the dynamic language runtime so those guys working exclusively on just the DLR pieces.

Now given the fact that all of the DLR developers for the most part came from IronPython as well so some of the original IronPython devs go off to work on the DLR effort which is where most of the attention is by those guys anyways, so we are working on those things. They are customers in a sense so they are building the base platform and we are building the languages.

And what's interesting at the company is that we build all the stuff in parallel. So the DLR is being developed at the same time as the languages, the CLR is also being developed at the same time as the DLR so there is an awful lot of interesting moving pieces in the puzzle as we go off and build this integrated thing.

   

5. You mentioned DLR. What's a DLR?

So let's step back a little bit and look at the history of this thing. Originally there is a thing called IronPython. IronPython was a project that Jim Hugunin worked on initially before he joined the company and then worked on after he joined the company as well. IronPython is our implementation of the Python programming language that runs on top of the .Net platform and IronPython has the distinction of being the fastest Python implementation today in terms of throughput.

So while they were building the IronPython compiler it was initially done as a way to show whether or not, or to demonstrate whether or not it was possible to build a high performance implementation of a dynamic language on top of a statically typed runtime. But because there have been some successes and some failures in the past in different projects and Jim really wanted to find out for sure whether or not it was possible to do this kind of thing. Because Jim is a very well regarded compiler implementer, he implemented Jython on the JVM before he came to the company, he also implemented the AspectJ compilers.

So IronPython was yet another compiler that he had built and he was really interested in seeing how why would building Python on top of the CLR would be so different than building it on top of the JVM. He didn't really understand why, and as it turns out you can build it, it turned out to be harder than it probably should have been to build on top of the CLR there were lots and lots of little decisions that had to be made, that had to be made the right way, by the right people and what he realized was that it would be interesting to take all of that learning, that they got from building IronPython and build it into a set of libraries which could be used by other language implementers and that's really what became the DLR, was the pieces of the IronPython compiler and runtime that they thought could be reused across multiple languages.

So the best way of doing stuff like that is to extract the DLR to IronPython, so the first version of the DLR really was the Python language runtime. And then to make sure that we can call it truly a dynamic language runtime that runs across multiple languages. We implemented two other languages on top of it. So Managed JavaScript is one and the second one is our IronRuby effort. So those are three fairly different dynamic languages. We also had a prototype of a Visual Basic compiler that was built on top of it as well called VBX. So those four languages were really able to drive a lot of design changes and decisions into the DLR and also remove the Pythonisms that were specific only to Python. So that's what the DLR is, it's a set of libraries that makes it easier for language implementers to get the language up and running on the CLR.

   

6. How does Silverlight tie in with the DLR into these dynamic languages? I mean what's the story that Microsoft is trying to tell us about Silverlight DLR? What's important?

Silverlight to me, different people have different opinions about what Silverlight is but Silverlight to me is an interesting programming platform. A lot of people think about Silverlight as being this thing: "Oh well it's this thing for building Rich Internet Applications, that run inside the browser and across platform" and it certainly can do that kind of stuff. But I think that a more interesting application of Silverlight is the fact that it's a ubiquitous high performance computing platform.

If we do our jobs right and Silverlight gets distributed out to folks that are interested in video, much like how Flash was distributed, it was done entirely through the vector, people wanted to watch youtube, Silverlight is going to get a big boost from the Olympics this year, from a lot of folks in the US from NBC they are going to have an opportunity to watch the Olympics via Silverlight. That should allow us to distribute this programming platform very widely and Silverlight is about 4.5 MB today as a download for Silverlight 2.

In that 4.5 MB you have a full implementation of the CLR you have a fairly full implementation of the base class libraries as well as these rich media libraries. I think the first two are the really interesting parts because now you have this high performance computing platform that allows you to write code that runs cross platform. And more importantly it also allows you, since it is a CLR and since the DLR targets Silverlight as well as targeting the desktop, server versions of the CLR we have this ability to run IronRuby code inside of the browser.

And back to that idea of when should you write code in a statically typed language versus when you should write code in a dynamically typed language, you get to do the same thing in the browser. You really have never had the opportunity to be able to partition things like that in the browser before. Here is some static stuff here is some dynamic stuff. The best you have was dynamic stuff with JavaScript in the Ajax world. And I think that we had a really interesting opportunity to continue building this very interesting browser based programming platform via Silverlight.

So our stuff just runs on top of Silverlight and it runs on top of anything that's compatible with Silverlight so certainly Moonlight and it also allows us to go run dynamic code on Linux based desktops as well as inside of the browser.

   

7. We are at RubyFringe and RubyFringe is made up of many Ruby developers running on Macs, Mac is everywhere here. Why should these people here be interested in IronRuby?

Sure, that's a fair question, I would argue that, so Kathy Sierra has an awesome blog post where she has got this kind of axis and on one side is "we love you" and on the other side it's "we hate you" and in the middle it's this broad swath that she calls the zone of mediocrity. For the most part what's kind of interesting about our efforts when I go out and I talk to folks and communities that are other than our own, it's that for the longest time we kind of lived in this zone of mediocrity.

Nobody cared, it's liked "Oh Microsoft is doing Ruby? That's nice, doesn't affect my life, it's not big deal". What was fascinating was back at RailsConf, starting with Daryl Taft and I coined this term called ARAX. Asynchronous Ruby in XML and because I was talking about how it's interesting that we can now run Ruby on both sides of the wire, we can run Ruby in the browser, we can run Ruby on the server, being able to use the same programming language on both sides of the wire, it's an interesting thing.

And that got us slashdotted, lots of hate and vile were directed our way and I thought that was awesome because it was moving us out of the zone of mediocrity. It's getting into people's attention like obviously we would prefer that people loved us but it's also better that people hate us as well. Because now we have this thing were people have stopped not caring because when people are indifferent that's far worse than people loving or hating you.

   

8. How do you feel personally about tying IronRuby to this open source world that we are all part of as Ruby developers?

At OSCON we are going to do a bunch of stuff where we talk about Microsoft participating in the open source community. That's really important that notion, that word, participating, because we don't drive it, for the longest time this is the way Microsoft has worked with our customer base. We invent some technology, we drive that technology, people adopt our technology, and they are happy for the most part, and life goes on and it's awesome.

In the open source community - driving is still a very important idea, in the sense that people will still need to exert effort to make things happen but you don't own the thing, whatever the thing is that you are trying to drive. Instead you are participating at it, anyone can participate and drive something. And I think that this is something that we realized the developer division which is this idea that we really want to be a great platform for running open source software. Why can't Windows be the best platform for running open source software?

Everyone likes to use the Mac for this kind of stuff, why can't Windows be an awesome platform for using, running, building open source software? And in order for us to have any credibility in that space whatsoever, we need to be able to show that we are a good participant and a good citizen inside that community. So we have to do that by our actions, words are fairly meaningless in this kid of stuff, there is plenty of rethoric out there, and IronRuby is a project that really allows us to demonstrate what it means for Microsoft to be a participant inside of the open source community and I think that is a really important thing. It allows me to use that as the vector to really try and encourage more openness.

There are all sorts of teams at the company that want to do open source stuff. I get called upon by all sorts of teams across the company because I get this reputation of being an expert in doing this stuff. These teams approach us, approach me personally and say "We have this thing, should this be an open source thing, should not be an open source thing? How do we think about this?" And so there is an awful lot of discussions happening as well. Because clearly we still are a company that makes money by selling software to people and I don't see that stopping any time soon, but certainly there is an awful lot of ways where open source software can certainly increase the value of the thing that we turn around and sell, being a server operating system, or desktop operating system, or a database or anything else, our core products.

So given all that stuff I think yes it has been really awesome being able to use IronRuby as a mechanism to try and effect change around the organization: around be more open towards open source, understanding what this thing is really all about, and this goal of being a good citizen and a participant inside of the open source community.

   

9. How has it been? I mean has this been an easy task? A hard task? You are one of the first people to bring open source into Microsoft, IronPython, IronRuby, how has this experience been so far?

Here is the way to kind of look at this problem: a lot of other companies tend to adopt open source as a strategy when they don't have a choice. There are some good existing proofs out there. I will leave it up to you to figure out who they are. In some senses the desperation has driven up to senior management "Oh my God, we need to do this open source thing, because we can't continue to fund our existing non open source development anymore. We need some additional help from the community". That's really how at an executive level they look at these kinds of decisions.

Things at Microsoft there is nothing wrong. If you look at the company, the company has made more money this year that it has ever made in the past. For the most part people are happy with that stuff. Managers are awarded based on how much business did you bring in? What do your earnings look like? And it's more challenging when you have to run around and say, it's a much different argument when you have to go and say to the successful guy who got a nice bonus for doing a great work in the year to bring in more business like Microsoft's business.

And say "Well you know this open source thing" and the guy is going to turn around and say "What exactly am I doing wrong right now about the thing that I am doing?" And so it's a more challenging conversation that you have, it's not like those folks are not willing to have that conversation, it's just that the impact of it and the immediacy of that conversation is very different at Microsoft and it would be at another company which would be under more financial pressure for us in order to make those calls earlier. So I think that it is happening, the conversations are almost always about how does this thing add value to some other line platform that we sell?

And as long as that line is clear, these things get funded, we do things, IronRuby exists for that reason, DLR exist for that reason, other projects around the company, the Ajax control toolkit, as another example the Silverlight control toolkit, or other examples of open source projects that add value to some component or platform that we care about. So those conversations are easy, there is going to be other things over time, we will cross those bridges when we get there, but I think that folks are very receptive to at least hearing the ideas, that's always credit to the people that you talk to. I won't call it easy, but it's not impossible either at the same time, it's challenging.

   

10. How difficult has it been to open Microsoft up as a project manager, to open up some of the policies of Microsoft as to way they do source control, things like that, how hard is that? How hard has that been from your side to change the way Microsoft may do things internally for projects, accepting contributions from the outside?

That's a big part, the contribution thing was an interesting challenge because there is a lot of institutional culture, I don't know how you want to call it, around how we treat contributions from the outside in terms of code that we ship inside of our products. And IronRuby crossed that bridge a long time ago in the sense that IronRuby was always going to be a component of something else but it won't necessarily be a mandatory component of that other thing, whatever that thing is that IronRuby might ship with.

So you can imagine in the future it would be possible to turn around and write office macros in Ruby there's no reason why you shouldn't be able to do something like that. But what you are going to find is that we are not necessarily ship IronRuby on the DVD that Office ships, but instead it would be some downloadable component of Office. Those are some kind of mitigations that we put in place right around some of the real things that the lawyers are concerned about.

How you ship code that comes from the outside with a product that you charge money for? I don't think that one is a terrible inconvenience for an adoption blocker for a lot of folks, they might block for some people, but for the most part I think that that's a reasonable compromise given the kind of world that we live in today. I think overall in the future we can certainly push the bar higher on that kind of stuff but again a lot of things of how you affect change in an organization are about you make some small step, you demonstrate the management that this was a reasonable step to make, you make these arguments and after that step has been made you gather some feedback a few months later and look around and see "Did that work? Are people happy with your choice? The company is still around making money?" and then they will be more tolerant to the next change that you want to ask them for. So, it's a lot about that, taking small steps, one step at a time and we will get there.

   

11. Now that Microsoft has opened up for outside contributions how have you seen the community willing to give to Microsoft, willing to be part of an open source project within a large company like Microsoft?

This is a challenging thing in a sense that you take JRuby as the closest comparable that we have as a project. JRuby was started outside of Sun, certainly Sun hired the main drivers around JRuby but it was always viewed as a community project first and that just happens to have a couple of Sun employees contributing to it. IronRuby is different in the sense that it was started by Microsoft, and most of the contributions are by Microsoft employees and that's something that we really want to change and I think it's a perception thing.

We want IronRuby to be a community project, in the sense that the community owns it not Microsoft; we are a participant right back to this community that is IronRuby. I don't think we are there yet, we certainly have got very useful and very valuable contributions from our community and I think that over time as the barrier to entry drops, in the sense that we get more stuff done, it's working, it works well, we're going to see more contributions coming from people because people have to do things in their own interest. If you contribute to a project you do it for a selfish reason, for the most part and the selfish reasons might have to do "I got to get this working in this project that I care about, that my boss wants me to do" and so you go off and you apply fixes and patches to get you there if that's the thing that you want.

That's when open source works really well it's when the interests are aligned both from the people that are driving sort of the core of the project as opposed to people who want to consume and use that project as well. So I think that with time we are going to see an awful lot more of those things, we are definitely getting contributions today and we certainly welcome people to bring more contributions to the play.

   

12. How does has that helped, people like Charles and some of the other people that came along before there was an IronRuby, with JRuby and those things, I mean Ruby.Net how does that information help?

So the good news of having guys like Charlie on the list is the fact that Charlie is more or less blazed the trail in terms of technical issues of running on top of a statically typed virtual machine. He has solved a lot of those problems. JRuby is an awesome product, those guys have done a very very good job, the startup time is awesome, working set is awesome, certainly these are the things comparing right now to our implementation to see what we can certainly do better. So it's very useful to have folks who ask questions. So this feature, how widely is frozen actually used, they actually have data on this kind of stuff.

And he can help us share this stuff, taint level and other wacky things and inside the language. And so it's very useful to be able to kind of lean on folks that have been there and Charlie is this really awesome guy that goes around and actually spends his time trying to help out other implementations. Sometimes he pokes us in the eyes that's all fine and dandy because I think in the end he really does want to see Ruby succeeded as a language. It's not just about Java, it's not about JRuby it's really a lot about Ruby.

   

13. Where are we with IronRuby at this point I mean as far as your goal beginning to an official release where are we with IronRuby?

There's lies, damn lies and schedules and right now our goals on our team is to try and get a 1.0 release by the first quarter next year. Now what does a 1.0 release mean? Because this is a typical open source project, so you can declare a victory at some point in time so I am 1.0 so that's a pretty important signal that more or less tells people "Hey you can use this in real things" that's really what the signal is. In order for IronRuby to be treated as a real Ruby implementation, to me there are two goals, there are two clear exit criteria for the project.

Let's talk about three exit criteria. The first is that it needs to be a compliant Ruby implementation it needs to be true to the language first, and in order to be true to the language one of the awesome projects that came out of the Rubinius effort is the Ruby spec project and the Ruby spec project has certainly been spending a lot of time building up test cases for Ruby that will be used to validate behavior of the various Rubies and we still are now seven implementations on all over different platforms. And it's a very ambitious project, it's one that we are participating in now, and it's one that I think is going to go long ways in terms of I won't call it certification but it's a useful thing to see "Well let's see of it does pass the specs, and if it does pass the specs, it's Ruby and if it doesn't pass the specs it's probably not Ruby". So, that's one thing.

Second thing is it has to be able to run real Ruby programs. And there is no more real of a Ruby program than running Rails. So a second exit criteria for us is we need to be able to run significant real Rails applications. That's something that our team takes very serious we have to be able to make this happening in order for us to be taken serious as a real Ruby implementation.

Third thing is our various quality matrix that we have, we have to not suck at startup time, we have to not consume all the memory on your machine, and we have to have good throughput numbers. And we have exit criteria of how those things look like, in terms of start up time, how much working set we can use and that kind of stuff. And when we hit those quality gates those are the three criteria that determine whether or not we are ready for 1.0.

   

14. What's important to you, what do you feel personally you are with this project? I mean are you happy, things are going along well? Are we at a point that if you look back a year ago looking at who you are today, would you be happy with where everything is?

Yes, I think so. It's always a lot of programmer hubris I still remember my interview "We'll get this done in six months, it's no problem". Ok fine, it's been longer than six months and we are not done yet. I think that when we get to this kind of two year thing which is where we are roughly getting to right now, that's not a horrible timeframe to go from scratch, from zero to a 1.0 implementation. Certainly if you look at all the other implementations they took a lot longer than two years to get to a 1.0 implementation.

Considering that we are tracking on Ruby 1.8.x, whatever x ends up being when we are done, that's the ultimate goal for us to get to that. I am very happy with the progress. I have awesome folks that work with me on the team, that's another thing with coming to Microsoft, I worked with a lot of folks in different places in the past and there is always this mythical "Oh my God there is all those programmers out there that are so much better than other programmers" and having met these guys not so much that I hadn't met them, I had not worked with them.

And Microsoft is certainly one of those places where the developers in our team are just awesome and working with these guys every day is really an awesome experience and there is absolutely no way we would be where we are today if it wasn't for the developers that we have working on this right now.

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT