QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community.QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.
So the most interesting thing to me about Node.js was basically real-time, like I have been using different software to basically make a real-time communication possible, including modules for Nnginex that did publishe-subscribe, or Java or Python based solutions but I couldn’t find any APIs that gave me the flexibility and the ease of use that Node.js gave me. Node.js was always - their "Hello world" was always an HTTP server, so that set the foundation in my mind so I can build more complex HTTP servers on top of this that can do real time because of the non blocking evented IO. So I got interested immediately very soon after it was released because of this fact.
Sure, so to establish a bi-directional real time communication between a client and a server there has been many different ways throughout the years, HTML5 is proposing WebSocket as the easiest and most direct way of doing this. It basically sends an HTTP handshake, it utilizes the upgrade header, it says "I’m going to make this connection a WebSocket connection" and then it uses a very lightweight frame in the mechanism for passing messages back and forth in order. And what SocketIO does is it gives you the API that looks like WebSocket and is really easy to use but if the browser or the network don’t support WebSocket, it will use other transports and this is seamless to the developer so SocketIO takes care of picking the best network transport mechanism for you while giving you the ease of use that HTML5 is proposing with the WebSocket.
So, one of the focus of today’s presentation at QCon SF was precisely what the drawbacks of implementing WebSockets have been and some of them are related to existing network infrastructure that doesn’t support web sockets, like proxies, firewalls, even desktop antivirus software other issues have been related to the capabilities of browsers themselves and most of the market share of browsers doesn’t support WebSockets. So obviously we’ve been relying heavily on less optimal transports, not just in terms of latency I mean the speed at which data is sent back and forth, but also the model by which we can scale on the server. The way that we scale long poling or other transports that are not web socket on the server, are less optimal than web sockets. And what it presents is a challenge both in the client and the server we need to solve. And we’ve done that successfully so far.
5. What would you advise a team that is considering to start off on its first Node.js SocketIO project? What are the common pitfalls and problems they will be facing and how can they avoid it with best practices?
So mostly you want to focus on making your client side code really reliable and good like handling what happens if the network disconnects, and what happens in between that disconnection and re-connection, what happens if the user refreshes the browser quickly, those are issues that are not related to SocketIO but your application needs to consider for example if you are building a chat application and the user decides to open a new tab, how do you handle that? How do you make sure you sync state between tabs? And these are different challenges that depend on each application that have been explored now with SocketIO as the fundamental way of passing data back and forth. What I advise usually is look at what your application needs, look at the scaling model of your application and focus on that and not just in pre-made recipes.
So, there is different tools for profiling, there is tools for debugging code, node-inspector is one of them, it’s really useful, personally I also, we deploy code in such a way that it gets reloaded, automatically, I think that’s really useful as well. Whenever you make a change to the server the code reloads, and like I said earlier many practices of development in general are very useful in node like for example Test Driven Development. Test Driven Development for us allows you to not having to deal with testing the serve so much, reloading and making small changes and reloading, because we basically run the tests, if they don’t pass, we don’t bother closing that particular module.
Yes, so today I announced a project called Engine, so Engine is basically we are taking out part of the SocketIO core base and breaking it down into a simple abstraction. The socket API doesn’t change at all but what we are introducing is a more reliable way of establishing real time communication that starts with long poling first, and then it performs not just feature detection, like we have been doing to see if a particular browser has web socket or not, but we are also probing and testing those transports on the side, so we establish a connection that is always reliable, and always fast in the beginning, so if you are relying on like a connect call back, where you are trying to sync a state with a server, that happens immediately, and if the network is not capable of handling web sockets, or the browser, that’s not going to affect that initial connection time.
But if the browser is modern and the network is fine, then it’s going to upgrade to web socket as fast as it can, initially this happens immediately when there is no perception of slowness, there is no long connection time, there is no proxies blocking the traffic, and delaying the fallback, that’s what SocketIO does, so it improves reliability dramatically, it makes for maintanence of the project this way, so we are very excited about this.
So what I see happening in the next year or so is I think all these patterns that we have been doing when we create real-time applications or we create single-page apps, that don’t try to constantly refresh the browser, get all the assets again, or send HTTP requests, try to see if an asset has changed or not, so I think frameworks are going to emerge to solve fundamental problems like lazy dependency loading on the client, server side rendering of code, that also can be rendered progressively on the client, so if you go to a given page, a given URL, the server can produce a response, but if you go to that same page, by clicking somewhere it’s going to be more efficient than doing that.
So I think that’s one of the existing challenges that frameworks are going to start tackling, currently but don’t have any universal solution yet. And it might be the case that there is going to be a solution that is as popular as Rails was for Ruby, however those problems exist, and they need to be solved in a way that users don’t have to worry about the technical difficulties behind them or they don’t have to think about the difference between a website and a web app that sometimes try to be made where people say don’t render any state on the server because that’s a web app and you don’t care about SEO or other factors.
However in my experience that specific difference doesn’t really matter and there needs to be a framework that tackles this problem. Another problem of course is always writing applications that restore state so for example if you are writing a chat application what happens when you refresh the browser and you want to reload existing chat sessions? So all these problems that are tricky small things, that have a very big impact on user experience, need to be tackled by an ubiquitous framework.