00:45:16 video length
Bio Ryan Dahl is a programmer at Joyent and creator of Node.js. Even before Node, his focus in the past few years has been on web servers. Ryan grew up in Rochester, New York but now lives in San Francisco.
QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community.QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.
Yes. Thank you. I am Ryan Dahl, I work at a company called Joyent which does hosting services and I made Node.js which I assume is why I am being interviewed.
I was working a lot with Ruby web servers. Originally I am from a math background so I only got into computing a little while ago and I got really interested in Ruby and was fascinated by this problem of uploading files and giving a progress bar to the person uploading the file and it just amazed me that this was such a hard problem because obviously the web browser actually has like the data to display that stuff, but it's just not accessible from the DOM so just what you do is send a request to the web server and ask it how much of this file have you received and it comes back and then you update the DOM with this thing. And there is a nice little module for Mongrel which did this.
4. Since most server-side developers are not used to asynchronous APIs, would you like to describe how they work for Node.js? How are things implemented regarding disk I/O, interaction with a DB or a RESTful services?
Everything is a callback. So where you would traditionally say: "Access the database, write to file, move file over there and do something else" you kind of do these sequential sort of actions one after another. In Node you can't do those sort of things because you might take some amount of time for you to move a file from one place to another because the disk might have to spin, or if you query your database that might take some milliseconds for you to respond and in Node everything is non-blocking and so it doesn't allow you to just sit there and then return the response.
You have to supply a callback and so there are many anonymous functions in Node where you are giving a callback to get a response, which is disconcerting to people who are used to this traditional sort of blocking threaded server. However, I think it's a style sort of thing that you can get used to.
Right. I think that is the main selling point of Node is that we've actually been like training a generation of programmers to do exactly this and they know that when you are making XHR that you have to do it asynchronously. Everybody knows that you don't do an XHR synchronously because it locks up the webpage. But somehow on the server, of course I am going to do an XHR synchronously; I am going to request something from a database. It has to be synchronous and if it's not I don't even know how I could program that.
I was into Ruby for a while, I said, but eventually the VM just drove me insane, because it's just so slow, every time you try to make it faster you realize: "OK, I am just going to write this part in C" and every single line of Ruby that you add to your application, actually just slows down the server noticeably. And so eventually I just kind of ended up with this big C thing that I was kind of happy with, like OK I could write a web server with it and I could do file IO and so I had for a long time the idea of "I would have this kind of library that abstracted some part of the problem away and people can write it in C and they can write their little web servers.
People don't like writing stuff in C. Basically I want to put people in this non-blocking environment because it is the right way to design a server, it just kind of maneuvering around all the other pieces of the computer system and I would really like to do such a thing in Haskell or some sort of declarative language like this, where you could literally be purely functional when you receive events off of the socket, because all your side effects would happen on the event loop and you would just allow that to happen and then you kind of get a function call with some data and you would do whatever with that, make some call which would not have a side effect, you just write something to some buffer that would get flushed to the kernel and then you drop back down to the event loop.
And so I think that is an attractive selling point to the people, but for me it's just kind of small language to use.
The problem that it solves right now that actually doesn't have a good solution are like little web socket servers. Things like a little game, or you have a bunch of people walking around in a room and you kind of have to relay the event that you are walking and kind of sent it out to all the other people and so people are using it a lot for this, or say a chat room or something like that because there is actually no real good ways to do this right now.
I mean of course you can use various technologies to do this, but it seems that Node is kind of a nice simple solution for those kinds of things. That said, I mean, I think Node can be used in a lot of different ways, even for a traditional sort of request-response, response-database query-response sort of websites, but also maybe for like sensor networks or something like that.
Realtime sort of a little device that sits there and measuring temperature, but also getting like some information from this thing and it kind of has to relay information. I mean I think there is a lot of applications where you need a nice real-time system that kind of just sits there, obviously that you can easily develop to. I mean obviously there are problems that require hard-real-time sort of things and you are no going to be able to use this for those situations, but I think there is a large class of problems that this could be use for.
Yahoo likes Node a lot and they are actually interested in using it for several projects I think and kind of experimenting with it, the YUI group is also pretty heavily because Yahoo is really into this idea of "degregation", simple degregation, progressive degregation or something like that, where if you go to Yahoo, without Java script, it still displays with less features.
Yes. I mean it's different. I think one of the big challenges is that you are always loosing your state. If you have threads you kind of build up this call stack and then you switch to some other and do something else and if you hit some exception somewhere, usually have this history of how you got to this thing and more generally in this kind of single threaded environments you are destroying your state, you are always going back down to the event loop and then something else happens.
And so you run into these situations where you hit an exception and you get the line number and you find out where you were from, but you don't know necessarily how you arrived at that situation because the history of that is gone. So I think that is a real problem in event based programming. I think a lot of people have a hard time changing style from the synchronous point of view to the callback sort of view point and they imagine that this is an intractable problem, but it's not, they just need to get used to it.
So I think there are some real problems and there are some imagine problems with this. It's not the best environment for doing large batch operations where you are one thing after another because you kind of do this sort of thing. So there are ways to get around that, of course, but it's better for this sort of situations where something comes in here, goes over there, takes up this one and events kind of occurring spontaneously that you have to handle rather than doing a shell script sort of deal.
Node is by design a very small executable that actually doesn't like spew files across the file system; it's all kind of packaged up in one file. So you have to access it from the command line, but basically you can use any editor that you want to.
I use Vim. People use various things. V8, as I said, is a really great VM and they are doing a lot of work. All the debugging stuff is there in V8 and everything that you can do through kind of the Chrome developer tools which is like stepping to code and profiling your code and doing all that stuff is all available in Node as well. You need a client to kind of talk to that and so there are various projects trying to develop Chrome tools basically, but outside of that. So there is one called node-inspector that is really nice; there is an Eclipse plug-in that does that, which is actually supposed to be used for web site stuff as well.
There is one called NDB (No Debugger). So we've got Editor and Debugger and what's more ...Monitoring deployment, this sort of stuff is not there and what people use is Monit and they kind of have their own custom solutions.
Yes. My chat application is just running in a screen thing actually. It's actually kind of stable, which is surprising. SSH did it in one day and started a little screen thing. So I put zero effort into that at all, but people who are actually concerned about their website being up, because obviously they crash, would run Monit or something and at Joyen we use Solaris, we use SMF, so I think anything that you would use for a traditional web sort of thing.
So I said like doing this kind of serial actions is kind of difficult because you create file-intend->callback, write the file->callback. You kind of tend to indent very far if you are doing a bunch of serial actions. So there are ways around this and so for example Tim Caswell has a library called Step which basically queues these things up. So you kind of say the functions and the arguments and more or less put them in a ray and it queues them up and kind of shuffle them out as necessary and you can kind of insert callbacks where necessary if you want to or just add the end of this action.
So it's kind of interesting. I mean correct me if I am wrong, I think Node is kind of unique in that, it's the single thread environment, but it also saying we are going to be a single execution stack environment, we are actually not going to have green threads, we are not going to have code routines, you will destroy your stack every time you go to the event loop. So the people are required to kind of come up with new abstractions to deal with this problem, how do you deal with this bunch of problems that need to be executed in sequence. So it's kind of an interesting problem, but there is a couple of them.
Yes. Node provides this low level sort of networking infrastructure. It doesn't attempt to solve problems of scaling out over multiple CPUs or multiple data centers or multiple machines. This is left up to the user to kind of decide for themselves and I mean this is a hard problem. If you have a chat server and you are running it in one process and you realize that you are running up to 20000 people and now garbage collection is becoming compute bound and the server is becoming slow, you say: "Ok, I am spending all my time in GC, it would be nice if I could use this other core that's like sitting idle right now."
So now you can start another process, another Node process, which generally the answer is start more processes, let the kernel schedule these things to different. But I mean you have to talk about the specifics of the problem. So if it's a chat application you have to start another core and now you can have people connect there and people connect there, but you also have to connections in between, like an IRC server network, you are going to have to kind of have a fat pipe in between them and send a bunch of data between.
If you just have, say a simple website where you're just request-database-response sort of thing and all the connections are independent of each other and they don't have to talk to each other, then scaling out is very easy. You just start a bunch of them and load balance across it in any way that you want to. You can stick an Nginx server in front of it, you can IP, DNS load balance or whatever, you can do this trick to actually start one server socket and then "fork" multiple do it in quotes. Because it's not actually forking, but you can make a prefork server where you fork your process times and being the number of cores that you have.
And then you kind of get a copy of this file descriptor on each one of them and so all of these cores are kind of looping trying to accept connections on this socket. And when an incoming connection comes whichever one of those guys has scheduled they are kind of racing to do this and so load balance seems kind of done kernel like Nginx workers work that way. So generally it just depends on the problem. I mean scaling out something can either be easy, depending if it's an easy problem or it can be very hard.
Generally these benchmarks on the internet are not so interesting. There is a couple of them that are actually interesting. Here is the performance situation: Node is pretty fast and it's kind of at the same order of magnitude as any other event loop system like this. I mean you kind of get this big jump when you go from a system like Ruby on Rails to Event machine.
Yes, doing anything. But I mean like going from the traditional sort of web framework to event machine twisted you get a big jump. You also get a big jump even in C where you go from a server that handles connections with OS threads to a system that handles connections in an event loop. So you kind of get this big jump, but when you compare these different systems, more or less it's the same. Node basically serves as fast as Event machine and in terms of like a "hello, world" server, sort of situation. So it's in that category of like: Ok, now it's evented and it's kind of fast.
It can handle many connections fairly well, you can load it up with Idle TCP to whatever limit your OS provides and it will sit there idle, which is a good sign, but not so interesting. You can load up several thousand connections, let's say 20000, and have them all writing a small amount of data to the server and getting it echoed back but not to each other and it will handle that sort of situation with 10s of thousands of clients. If you get in more complicated situations, where, you know, one guy is talking to all of the clients the performance degregates very quickly because now you have to write to n sockets.
Anyway that would be the same situation in any system. Anyway performance is fairly good. What is not good is pulling strings out of V8. So writing a string to a socket, small strings are OK, but larger strings, dumping them to the socket is very slow. So say a 10 kilobyte string, if you want to write it to a socket it actually performs very poorly, which is surprising, like nobody has done this test on benchmarks and said: "Look at that, that really sucks", because it really does suck. I think there are some ways around this problem.
The problem is that their systems can go into the heap of VM or whatever they are doing and write that directly to the socket and V8 forces me to copy it out into a separate buffer and then send it out. So that extra copy kills you on large strings. But there is a way to solve that, which is going to the VM and actually copy it out from that and so these things can be solved, I think. We do see some pretty bad situations with V8. Generally I am very happy with V8, but we do see situations where garbage collection takes a second.
I am very ashamed to say it, but there are some bad situations with V8 GC. Maybe it will be improved, but generally I think those are the two major things. I guess the other big concern is the 64 bit V8 is bounded by 1.7 Gigabyte heap, so that is a fairly constraining problem. I don't know if the 32 bit is bounded by anything. So you are on 64 bit, your heap is maxed out at 1.7 Gigs, so that is not so good. And the situation there is less clear to me because I don't actually know what the problem it.
I guess it had something to do with when they do the compact thing they have some sort of offset for where it is and there is only a certain number of bits available for determining where those things are going to be compacted to. So that is a hard problem that is something that V8 is going to have to solve.
I don't know about great gains, but it's certainly going to be improved. I mean they are doing work all the time. It looks bright for its future. I mean there is a patch being discussed on the V8 mailing list that would fork out threads for doing the mark phase of the garbage collections. So you could actually use multiple course like walk the object graph. So on a server system where you might be sitting on a 32 core box that might actually help significantly. There is a lot of ideas of how this stuff can be improved and the V8 people are very smart.
You might have some problems with Free BSD, but generally you can compile it on Linux. And so any VPS things you should be able to up and running without too much problems. I work for a hosting company, Joyent, and we are working on a service called NO.DE which is a Node hosting service and it's going to be one of these "get deployment" things where you write all your code and you manage it with git and then you do git push Joyent and then it kind of manages the restarting of the server and that sort of stuff.
And so that's in Beta right now and should be released in the near future, but at the moment what you are forced to do is set up your own VPS and do it, but it seems like in the near future there will be some options to kind of simplify this and kind of having "AppEnginy" sort of feel.
Basically we are fixing things. Like I talked about this string problem, this kind of requires a bit of a refactor and so we are addressing these performance concerns and kind of generally fixing bugs and making it more stable. And I think that is mostly what my roadmap is about. After that I think we'll be fixing bugs for a long time, but there is not too much I want to add to this thing. It should be small. I mean it's the sort of thing you are not going to use by itself.
You are going to have to add libraries and you are going to have to build stuff on top of it because it's a very kind of foundational sort of thing, but I want that to exist outside of the core project and so I don't see the core project changing enormously. I don't think we are going to be adding any major components to it or changing the API drastically in the future. There are just kind of touch-ups and bug fixing into the foreseeable future is about where it's going.
Yes in 10 different browsers. So this thing, I think it works on everything, but what I define everything to be IE 6 and later. So that is great and a lot of people are using that. There is a library called Connect which is somewhat Rack for Node, with the fact that Node is very asynchronous and kind of does these streaming sort of things and so you have your Node web server and then you can kind of put these blocks, these filters that get passed through your application, like a GZIP sort of filter. So it kind of gives you this nice API to put these blocks or you want a static file server to handle a certain path and you just kind of stick that filter on there and it handles it.
It's a real OS process. It starts a new process and so now you can start sending JSON between these things and you can say: "I need to convolve this image. Please do this gausian blur for me." And you tell it to that other worker via JSON and it does that, it's spinning the CPU super heavily; you go back you start serving your request. And so this is kind of a nice way to distribute tasks over multiple CPUs.
And there is a really nice pcap library for listening to your ethernet interfaces on your computer, so you can actually get callbacks, it's a very asynchronous sort of thing already and so you can kind of get callbacks into Java script saying: "Hey, a new packet arrived on your interface. Oh a new packet is going on your interface. And so Matt Ranney who did this binding to pcap also hooked it up to the HTTP parser and it does all the TCP reordering of the packets and then sends it to the parser and now you can get events of every HTTP request that is occurring on the server.
So I think those are fun libraries that you should get you started. So I should say that if you go to the Github page there is a wiki and on the wiki page there is a module's page which has many modules and you can look through there.
29. Do you think it is realistic to hope also, I mean you talked about being interoperable between different server-side solutions, what about code that was written to be executed in the browser? Is it realistic to say: "I've got that piece of code that I can actually use on my browser" or are there actually cases like that, because most of the browser code is DOM manipulation?
It's not as much as people hope. I always meet this people who are just like: "Oh, great, I can just share my web server on my browser and then everything is going well". I mean your web browser and the web server are doing very different things and usually there is not so much code that can be shared. But there are situations where you might share something like validation sort of libraries where you might want to highlight like a form field and check that they enter the email address correctly, but once they actually submit it, just so you are not believing what they what they entered on the form you can re-check before you send it to the database.
Then pull out the text in that dive that you were interested in which you had no possibility of doing before, because all you could get is the HTML and these days a lot of HTML is being generated on load time by some JSON that is being loaded by an XHR. So I think for screen scraping this is really interesting. But generally, I think this sharing code between the two environments, first situations make a lot of sense, for the typical:" I am going to write a website" there is not so much code that gets shared.