BT

David Nolen on ClojureScript, Javascript, Source Maps
Recorded at:

Interview with David Nolen by Werner Schuster on Feb 05, 2013 |
19:18

Bio David Nolen is a JavaScript developer for The New York Times. He also runs a Brooklyn based workshop called Kitchen Table Coders. In his free time he does fun stuff with Clojure and ClojureScript.

Tech Mesh, the alternative programming conference,focuses on promoting useful non-mainstream technologies to the software industry. The underlying theme is "the right tool for the job", as opposed to automatically choosing the tool at hand. By bringing together users and inventors of different languages and technologies new and old), speakers will get the opportunity to inspire, to share experience, and to increase insight.

   

1. We are here at TechMesh in London and I’m sitting here with David Nolen. David, who are you?

Hi, I’m a developer at the New York Times, I do a lot of front development there, but I also do a lot of open source work heavily in the Clojure community. And I’m also a musician, I play in a band, in Brooklyn and I also run this thing called Kitchen Table Coders which we run on weekends in Brooklyn, where we get a very small group of people who are interested in learning some new technology and we sort of sit around the kitchen table and teach ourselves something new.

   

2. Whose Kitchen Table? Any Kitchen Table?

So what’s actually kind of cool about Kitchen Table Coders is that we sort of started off as just a way to help pay for our rent, it was a very modest thing what we were doing but it since has taken off and other people seem to join the idea, I actually want to say that somebody in London wants to start a Kitchen Table Coders here, one of our Kitchen Table Coder guys was in Japan at a conference and I hear that somebody in Japan wants to start one, so we sort of have this notion that it's kind of an open source idea and we happen to do it in Brooklyn but we really are excited about this idea of learning things in an environment that is conducive to learning, I think that requires four or five people and one or two people leading the discussion.

   

3. It’s like a mini barcamp, in a way?

Yes, like ultra mini.

   

5. Ok. So you already mentioned Clojure and ClojureScript. So maybe for our audience that hasn’t heard of ClojureScript yet, what’s the pitch for ClojureScript? Why can’t we just use Clojure?

So Clojure targets the JVM, so the JVM it’s great, it’s fast, it’s efficient, it’s a mature runtime, but there was this fantasy in the early days in the JVM that it would be sort of client oriented and that hasn’t really played out, in fact, clients are mostly browsers these days or some sort of browser based technology and really the only runtime that's available there is JavaScript. So Rich Hickey introduced ClojureScript to be able to leverage the JavaScript runtime that are, again, much more available on clients then the JVM.

   

6. Is ClojureScript a subset of Clojure or does it add something?

So ClojureScript is an implementation of Clojure, people often get confused, it’s not a dialect, when you use it you really feel like you’re using a full-blown Clojure. There are a couple of design decisions there that are different than the Clojure under the JVM but in general we try to make it such that it feels like the full language. Of course some things don’t apply in browsers, like JavaScript is single threaded, so all the concurrency primitives aren’t available but that actually doesn’t really matter because the benefits of immutability go far beyond concurrency, and it’s a lie to say that there’s no concurrency in a browser, because any asynchronicity is inherently concurrent, all the problems can occur.

   

7. Does ClojureScript offer anything to help with the bane of event handling?

It doesn’t yet. So people often talk about “callback hell”, when they talk about writing JavaScript and the community is working on possible solutions, it’s also possible that we may decide to add some compiler support, particular we don’t have a good implementation of what’s called Promises in ClojureScript but there’s some work the compiler can do to make that work.

   

8. I’m sure there’s a monad in there to be used somewhere for everything asynchronous?

Oh sure, well that’s the fun thing about ClojureScript I haven’t heard anybody just doing monads, you could just do monads and you have macros, so sure, you could totally create very sequential looking code via monads.

   

9. We're at a functional programming language conference, I’m contractually obliged to ask about monads. So I seem to remember that ClojureScript didn’t have a parser or a compiler for Clojure, can I write ClojureScript in my Chrome shell, can I just type it in and I have it evaluated?

No, you can’t do that. There’s a lot of work that we need to go into making that work. We’re heading, we’re going there but we haven’t gotten there yet but one thing that’s cool about Chrome is that Chrome is integrating something called CodeMirror, which is a sort of JavaScript-based text editor, which has support for many languages including Clojure. So I haven’t had the chance to actually try it out but supposedly, we have basic source maps, so supposedly you can turn on CodeMirror and then you should be able to edit Clojure code in the browser, thought of course that, again, the work needs to be set up so that like it hooks into the compilation process.

   

10. The compilation process, why can’t ClojureScript compile itself or parse itself, is there a fundamental problem?

The fundamental problem is that we haven’t gotten there yet, so we know we want it, projects like Light Table, which people probably have heard of, would like to deliver the text editor without the JVM. That would require the compiler to be self hosted. There’re a couple of big pieces that are missing, one thing is for example, we don’t have the Clojure Reader, the thing which actually parses the Clojure source code, is not written in Clojure, we need a version of a Reader in Clojure. Somebody started some good work on that, we need to integrate that, we also need to round that currently symbols and keywords, which are a fundamental aspect of Clojure source, are represented as Strings in ClojureScript and that needs to change, but I think once we get a Reader, once we get real symbols, it is a very short matter of time before you could just compile the compiler directly into JavaScript.

   

11. So it’s mainly historical reasons, so I guess the first Reader was written in Java and not in Clojure?

Yes, Clojure is still relatively young programming language and again, it’s just now that people are realizing the benefits of having a self hosted compiler. People haven’t really needed it because they were just writing Clojure apps, like “oh, I can write a little bit of JavaScript, a little bit of ClojureScript for the front then” and as these projects become more sophisticated, the desire for a self hosted compiler is growing.

   

12. An interactive shell, does this work with the Clojure[Script] model, because I seem to remember that ClojureScript was designed to write applications, locked off applications that are then run through the Google Closure optimizer and are shipped as one component, if you want to have an interactive shell, do you have to script the whole ClojureScript library? Is that a problem at all?

You do have to ship the whole thing, but again, if that’s what people want to do, that’s ok, right? It’s not like, I mean, JavaScript engines have come a really long way, people are compiling Quake, which is like hundreds of thousand of lines of JavaScript and browsers are able to handle it fine, so even when you compile the entire standard library, I think when we come out like twenty thousand, like no optimization in the code, I think you generate about twenty thousand lines of JavaScript, modern JavaScript engines can parse that in seconds, it’s not a big deal so if you want the flexibility, it’s not like the code that we generate creates any challenges, if that’s what you want.

   

13. It will be really interesting to have a full interactive Clojure environment in your browser, zero installation, no more evil JVM...?

Yes, there are some people who really don’t like it, again there are many reasons why you wouldn’t want it.

Werner: And particularly, if you’re on your tablet, on your phone and there’s no Java there and it’s only the browser really.

David: Totally.

   

14. You mentioned a curious word: Source Maps. That’s the new thing that will make JavaScript an even better compilation target. What are Source Maps?

HTML5 Source Maps are becoming a standard, it’s sort of designed by Google, so they have those things called Google Closure Compiler. They were probably the company that takes JavaScript more seriously, only because they produce so much JavaScript, their applications rely so heavily on it, so for them it was a really good reason to have an extremely powerful optimizer, which Google Clojure is. The problem with the transformation that Google Closure does to your code is that when you go to production, how do you debug it? We’re not even talking about producing a different language like compiling to JavaScript from a different language. We’re just talking about understanding what the JavaScript maps back to you after you’ve just done so many optimization passes on it where it’s basically obfuscated code. Source Maps were created honestly to solve that problem and where you want to be able to map any symbol from the original source to some mangled thing in the generated source.

There’s a good reason for it and what’s great is that for creators of languages who want to target JavaScript, that problem is solved for us, so if we generate the correct Source Map, we can allow users to set breakpoints, they can actually see the original ClojureScript source in their browser, they can set breakpoints and they can step through code and they’ll see all the locals will be renamed and all that stuff.

   

15. I guess it also works for profiling, for console.log() and all those things?

The one thing that doesn’t quite work it will work for profiling from what I understand. The thing that probably needs work is when you want to eval directly in the browser, say you set a breakpoint and you’re right there and then you have a JavaScript console, that’s not a ClojureScript console and I think, probably the way to work around that is that WebKit has a remote debugging protocol where you can actually connect over a port and connect to the debugger and if you’re at a breakpoint you can actually create a tool that can access that and you can do anything at a terminal if you want. So our idea it would be to, say you want to eval something, at the command line say I want to hook into Chrome right now and I want to evaluate in the stack frame some piece of ClojureScript and then the next step would be once we get that working somebody could write a Google Chrome plug-in, so we would actually just replace that JS console with our own console which just does all that stuff for you.

   

16. So you could potentially also use, I guess with a Google extension or Chrome extension, you can connect an existing JavaScript process, with the debugger and the fiddle around with it essentially?

Yes. That’s further out there, we just want to get Source Mapping, again, there are very large ClojureScript applications in production now, thousands of lines of ClojureScript, and I think, just to be able to step through code and see the call stack, that would solve a large number of problems for a lot of people, even if they can’t eval ClojureScript yet.

   

17. You mentioned Light Table, can you talk about that, are you familiar with it, does it use ClojureScript?

Light Table, when it first came out, it was written, from what I understand, mostly in Clojure, Chris Granger has since then re-written all the JVM parts in ClojureScript, so Light Table is actually all ClojureScript. The only thing that I think Clojure is there for is just to compile the code, again this is why we want self hosting to just eliminate that step, but yes, Light Table is all written in ClojureScript, which is pretty amazing, I think right now it’s like six thousand lines of ClojureScript.

   

18. It’s astonishing, I think it bundles ClojureScript with WebKit and so it starts up as an app?

To me it’s so awesome that we can leverage, WebKit is a truly great open source resource so Light Table I believe is built of Node WebKit, it’s WebKit with bindings into Node, so from the application you can query the file system and do all the sorts of unique things that Node lets you to do.

   

19. Just like a PhoneGap application for your desktop, JavaScript has come a long way?

A very long way and there’s a lot more to go, when I gave my talk yesterday, I really looked forward for the day that ClojureScript isn’t really weird, people should be programming their browsers in whatever language they want.

Werner: And with JavaScript and the powerful JavaScript engines we finally don’t have to have the discussion which language will be officially supported, we just plug it in there.

David: Yes.

   

20. What’s the part of ClojureScript you focus on, are you focused on some part?

For me ClojureScript has been really fun, because I’m a JavaScript developer, so I just have a lot of experience writing in JavaScript, so Rich, who is an incredible programmer, he doesn’t write JavaScript, that’s not his thing. So a lot of what I did when I got involved, three months after it got released in 2011, I started contributing code and Rich and the core team, they are very busy with really Clojure oriented, Datomic oriented projects, and Rich was sort of offering ClojureScript as something for the community to really push and it happened. So we have 51 contributors to the codebase, like Michal Marczyk who was actually here at TechMesh, he ported all of the persistent data structures into ClojureScript, so ClojureScript has a full set of persistent data structures now and we get patches constantly and my role is often just supplying patches, some of the work that I do is I do focus on performance optimizations so even though we have persistent data structures, there are many optimizations to make to the compiler to make those persistent data structures be performant.

   

21. Can those persistent data structures be used from plain JavaScript code?

It’s possible, it’s not that ideal because they’re quite large, so the persistent data structures make about half of the seven thousand lines of the core standard library, just the data structure definitions, so it’s kind of heavy weight and it’s also… if you’re going to use the data structures, you don't really use the data structures directly, there’s a whole standard library of polymorphic functions, so in order for somebody to consume them it really requires wanting to buy in, so I imagine eventually we can come up with some solution, if someone wants persistent vectors we can export just persistent vector and be smart enough to not include anything else but it’s not a high priority and I will say that you have to be in the Clojure mind set to appreciate them and it’s hard to convince JavaScript programmers why that might be useful.

   

22. What’s the integration story between JavaScript and ClojureScript, let’s say you have a team, one team builts the GUI with jQuery or whatever and the other team does the fun stuff with ClojureScript and algorithms, so can they live in the same space, are there problems or is it fine?

There are challenges, it is pretty easy to integrate jQuery specifically because Google, there’s already an extern file that jQuery can integrate with Google Closure style JavaScript, which is the kind of JavaScript we generate but the inter-op story is actually pretty good, because Clojure was designed as a hosted language, it sort of assumes the presence of some host and so similar with ClojureScript, there’s a whole amazing world of HTML5 APIs that you want to be able to access without having to jump through hurdles and so we make it pretty easy to do that. The thing that won’t work for example is that when … When Dart first came out, people often would complain because you write “Hello world” and it generates ten thousand lines of code, but I think that’s sort of narrow thinking because why somebody might adopt Dart is that for n number of reasons, JavaScript is not that great when you want to build a large application if that’s what you’re going to do. I think ClojureScript is optimized towards the building of medium to large applications and maybe it’s not so great if all you want to do is write a very small JavaScript program.

At the same time I often see people say “I don’t want to use ClojureScript…” and then you look at their project and they’re including Backbone and they’re including Underscore and they’re including jQuery and maybe they’re using a tiny part of those things and their payload is bigger than a ClojureScript app because, again, Google Closure is very good about getting rid of JavaScript or the generated code that you don’t use.

Werner: That’s extremely interesting and I think our audience will now look at ClojureScript. Thank you, David!

David: Thanks!

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT