Q&A with Dylan Schiemann co-creator of Dojo on AJAX, Comet, Bayeux, RIAs and the Dojo Toolbox
InfoQ: Dylan, could you give us a small overview of how you started developing Dojo, what was your initial vision and what has Dojo grown to become?
InfoQ: Could you give us an architectural overview of the toolkit? What do we get by downloading Dojo version 1.1.1?
Dojo is structured in 4 main projects: Dojo Core, Dijit, DojoX, and Dojo Utilities. Simply including the standard dojo.js gives you:
- A very fast, lean (26KB gzipped) core packed full of APIs for XHR, events, DOM, query, animations, namespacing, and other highly used convenience functions.
- Optional modules for dates, colors, back button/history, currency, and more
In Dojo 1.2, we’ve actually provided an optional facility to get your base build down to 6KB and build up from there. The Dojo core also includes additional code that may be optionally included such as a behavior system, common regular expressions, date and currency APIs, and more.
Everything in Dojo and Dijit are deemed production-ready and API compatibility will be one of our higher priorities as we work towards Dojo 2.0.
DojoX, or Dojo extensions, contain features that are less widely, complex, or not yet production quality. Examples include encryption, additional Dojo data stores, dojo.gfx for native vector graphics support, charting, offline, grids and other widgets, and much more. Features in DojoX are not guaranteed to be backwards compatible across 1.x releases. DojoX contains a significant code base with great new additions arriving on a regular basis, most recently an XMPP client, JSON-Schema, JSON-Referencing, and a secure Ajax module, all of which will ship with Dojo 1.2 and are available in the current nightly builds.
- ShrinkSafe, a compression tool that removes comments, extraneous whitespace, and more, which also makes use of Rhino
- DocTool, a highly advanced system for generating API documentation from source code, that also can resolve complex mixin inheritance structures and more
- DOH, the Dojo Objective Harness, a unit test tool that works both in the browser and on the command-line
Dojo is organized in a very modular manner, making it easy to just get the features you need so you don’t expend system resources for everything else in Dojo.
Dojo is also an open-source foundation, and is home to other projects including Cometd, DWR, Open Record, and the Psych Desktop.
First, I think the books were possible because we released a stable version 1.0, and we were able to find some great authors interested in writing about Dojo.
Feature-wise, companies such as IBM, Google, Sun, Blue Coat Systems, Nexaweb, and many others, along with our great individual contributor base, made significant and focused efforts towards creating a toolkit that is fast, powerful, and feature-rich. I think the best answer to the question is that we just have a great community that works hard and our hard work is starting to pay off after spending several months in 2007 on completely rewriting Dojo to be a faster leaner toolkit.
A lot of this also comes from how we have grown our community. Being very open and transparent, using favorable licensing (BSD or AFL), and also using a CLA process for making sure there are no intellectual property issues with using source code. Dojo scores, by Dion Almaer’s standards, a perfect 100 points on the how open is your project scale.
InfoQ: A few days ago SitePen released Dojo Toolbox, an AIR application for Dojo developers. Would you like to talk to us about that?
Adobe approached SitePen first about refining Dojo’s support for AIR prior to the release of AIR, and then with the proposal to co-sponsor development of an application, which led to Dojo Toolbox.
Dojo has such a large feature-set that creating a printable PDF of the API documentation or an easy-to-view structure is always a challenge, so we decided to bring the 18MB of Dojo documentation offline. We also knew that providing an easy way to create optimized versions of a Dojo build would be handy and AIR was a great solution. All of this was done under the BSD license. In the few short weeks this has been released, we’ve received a number of great feature requests and plan to increase the app’s functionality as a result. We’ve also created something that is modular enough that people can create their own customized toolbox builds or contribute their own modules back to the toolbox for consideration in the official release.
I think too often there’s a tendency to say that you have to choose X over Y, rather than giving developers the flexibility to choose combinations of what they like or what works best. We like giving people choices, and in fact, people are currently working on a project to bring Dojo to GWT.
I urge people to experiment and determine the most productive choice for them, taking into account factors that matter to their business.
Dojo receives wide, practical, “we need this feature now” improvement from the likes of IBM, Sun, AOL, and many other companies including the clients of SitePen. Our open licensing, attention to the details important to enterprise, from IP issues to accessibility and internationalization support, makes us a great open source solution for companies looking to optimize their development power. Financial institutions, militaries, governments, and many other large organizations use Dojo throughout their web development models. Much of this work is Ajax “Dark Matter”, used behind the firewall.
Kris Zyp of SitePen currently participates in the ECMAScript working group. To date, it has had little impact on the Dojo Toolkit itself because we’re pragmatic and wait until something is ready before we use it.
A single browser page, or any web or software application for that matter, has access to a finite amount of system resources. In addition to the issues you’ve mentioned, a common trap is for new developers to not understand the implications of their work, e.g. implementing 100 dynamic charts in a page with a sub-second XHR polling implementation. With Dojo 1.0 and beyond, we’ve tried to be less “magical” so developers can better understand that each and every feature comes at some price, that the fastest toolkit is 0KB, and that most things that are difficult require challenging decisions and trade-offs.
Another common mistake developers make is assuming that browsers are consistent and rational. Long-held opinions on the best way to do something are often incorrect and surprising. Simply “looking at the data” with tools like Firebug reveal performance bottlenecks and tell us more about the true performance behavior of browsers..
I think this is simply a case of the rapid rate of invention over the past four years drastically outpacing the capacity of browser vendors to create better deployment environments. Browser vendors are often slammed for trying a non-standards-based approach, which is unfair because something can’t become a standard if there’s no room for experimentation and innovation.
Flex and Silverlight have done a decent job with providing a good development experience, but don’t count out the Open Web yet. We believe that an evolutionary approach to constantly improving what we have today is a viable option. Take WebKit for example, and its success as a great environment for apps in the browser, inside Adobe AIR, on many mobile devices obviously including the iPhone, or Opera on the desktop, mobile devices, and the Nintendo Wii.
I think they hold up quite well. OpenLaszlo is in many ways more about being a compiler like GWT that can deploy to Flash or the Open Web. What’s interesting is that we’ve started to take things in another direction. For native vector graphics, dojox.gfx has supported SVG, Canvas, and VML for a while. Recently, we added support for Silverlight as a renderer because VML is terribly broken in the IE alpha builds, and we’ve been experimenting with Flash and Flex for rendering vector graphics and charts as well. We also support Flash’s local storage facility in browsers that don’t have native local storage.
Again, I mention that it’s about choice and also pragmatism.
I think it is too soon to tell. The first release felt like two steps forward (6 connections instead of 2, secure cross-domain requests), one step back (severe bugs with VML), so we’re cautiously optimistic for the next release.
InfoQ: After 2005 when AJAX became widely known, the latest buzzword is Comet. There are still people debating on terminology so I would like to ask you to give us your own definition.
Comet is a collection of techniques, transports, protocols, servers, and clients to deliver low-latency data transit between the server and the client.
The XHR is the least common denominator of Comet, and everything else is an attempt to improve upon the XHR to deliver faster, real-time applications. My definition intentionally avoids some points of the debate, as I prefer to be inclusive and broad, much as Ajax has come to represent everything dynamic in a browser to most people.
InfoQ: An important components for enabling Comet applications seems to be the Bayeux protocol which facilitates routing JSON encoded events between clients and servers in a publish subscribe model. Would you like to elaborate on that? It seems that Bayeux is gaining de facto industry acceptance since it is unique, but do you feel it will be heading for a more formal standardization, e.g. through W3C?
To date, I think the informal nature has been highly useful, though if there’s interest, I’m sure a standards-body will eventually pick it up. Bayeux is the standard protocol for servers and clients that are part of the Cometd project. XMPP is of course another popular protocol. I’m also quite interested in the recent draft WebSocket addition to the HTML5 efforts, and the API abstraction of this already implemented within the Orbited project.
InfoQ: In the Java realm there are several ways to develop Comet applications at this moment. With the emergence of the Servlets version 3 spec (JSR 315) which defines a standard way of doing Comet, how do you think this genre of application will grow?
In the Java space, a variety of open-source Comet server options such as Jetty, DWR, and GlassFish, as well as free versions of Lightstreamer and Caplin Liberator are available. The new servlets specification has already experimental support in Jetty and GlassFish, and I’m sure when the final specification is ready, everyone will make use of this.
One interesting thing to emerge is the distinction between on-board and off-board Comet by Joe Walker at JavaOne. The language that your Comet server of choice is written in is less important than making sure you have client access (client being a web-based client, or a web server based client communicating with your Comet server) in your language of choice
We use the Dojo DocTool and the utilities found in Dojo. Some of the less common development tools we use include: Windmill, Tito Web Studio, Charles, and Versions. I can’t say we have a standard across the board, here’s the best way to maintain and refactor a large codebase tool like you would have for Java, but I’m not sure the need is as great either.
InfoQ: How do you see Dojo evolving? What are your next steps and what can we expect in the following months?
In the short term, we’ll continue to improve performance, widgets such as the grid, charting, tree and editor, the overall look and feel of Dijit, form and data validation, the stability of our Comet client, our validation system, and much more. We’re also starting to look towards Dojo 2.0, probably in 2010. It’s really about keeping Dojo useful, promising and efficient for developers who are interested in coding impressive web apps and providing excellent experiences for their users.