Nada Amin on Scala DSLs with Lightweight Modular Staging and Compiling to JavaScript

Recorded at:

1. We are here at Scala Days 2012 in London and I’m sitting here with Nada Amin, so who are you?

I’m first year PhD student in Martin Odersky's lab at EPFL, previously I worked for two years as a software engineer at Google, and before that I got my bachelor and my master [degrees] in the US and I’m from Switzerland and Bahrain.

2. So you are talking here about an interesting topic of compiling JavaScript and the Scala feature called Lightweight Modular Staging. So that is a lot of words, what is it?

Lightweight Modular Staging is a technique for embedding DSLs in Scala purely as libraries, so that’s why it’s Lightweight and it’s modular because you specify the construct of your DSL in Scala traits in a modular fashion, you just mix in a trait to have a feature, and it’s also modular because the implementation is completely separate from the specification of the DSL. The idea of staging is that the program is split into two parts, and the first part is a code generator which when executed uses the second part, the program, so in our case the DSL would be the program generator and the second stage program would be the generated JavaScript code. So this staging feature actually allows flexibility and abstractions only exist in the DSLs snippets and are not part of the generated code. In our DSL for JavaScript we actually use this feature to implement a nice abstraction over asynchronous programming that avoids callback hell, and we do this by using features that are available in Scala like the CPS plugin, continuation passing style plugin, to implement this and this all gets evaluated away at staging time, which means that the generated code is not complicated by these abstractions, so that is the basic idea of staging and lightweight modular staging.

3. Your project allows users to write JavaScript or JavaScript-like code inside Scala code?

Exactly, at a first approximation the DSL is just a thin layer on top of JavaScript so the idea is that you can really take snippets of JavaScript and with very simple changes just run it in the DSL, and then we added type safety where you have the usual Scala type system in place that just works for the DSL, and we also allow a gradual shift for more dynamic features to more type safe features using the new Scala feature, Dynamic. So the basic idea is that you write these snippets of DSL and then you can generate JavaScript code from them or you can run it as Scala code.

4. What is the idea for using this, so why would I want to write JavaScript inside Scala? I already have Scala.

So one big use case is sharing code between the clients and the server and that is very useful for validation logic for example, so in that case often you need to have the code on the client for interactivity and on the server for security. And this duplicated logic needs to maintained and it’s always bad to have to duplicate things, so what we do here is solve the impedance mismatch between having server and client code differ. So we’ve actually integrated our DSL with the Play framework and what we do there is that Play has already a language to specify validation logic so we’ve extended this with the ability to specify constraints in our DSL and the whole validation logic is checked on both the client and the server.

5. You mentioned interesting points, that you have support for some sort of async/await kind of programming, how do you solve that?

The idea there is to actually use staging to be able to use the continuation plugin to reify the continuation for us so that we can then basically pause and resume execution of things. So the way this works is that in JavaScript you have to write a lot of code using callbacks, and we twist this around by just saying: “Ok, these callbacks will be generated by the CPS plugin”, and you can write your code in a more linear fashion. Even a very simple example where you just want to print, just want to do something in a loop, and in the end if you have callbacks involved it becomes very convoluted, but here you can just write your for loop as if it was a linear operation and it all works magically, and the nice thing is that because of staging all of this gets abstracted away in the generated code, and of course you have a lot of functions in the code, but you don’t have any of the abstractions from the continuation plugin.

6. That is obviously very useful.

Yes, it’s actually from this, for example, we can also use the AJAX API to do, you can just write a for loop and do AJAX calls in parallel for example, so that is an interesting use case which we’ve done a little application which goes and fetches Twitter feeds in parallel and it’s very simple to write because you avoid this callback hell style programming.

7. So this JavaScript code you generate, do you run it through any projects like the Closure project?

At this moment we don’t , the framework, the Lightweight Modular Staging Framework allows us to do further optimizations. We have not explored that too much, we do some simple optimizations to be able to hook it up with the Clojure compiler, we would need to also propagate the type information, but since we start with Scala, this is feasible, we have that information, so this could be a further work to improve the generated code.

8. I’m asking because you have some history with the Google Closure project?

Yes I worked mainly on the type system actually of the Google Clojure Compiler, the type inference in particular.

9. So using the type annotations to infer the correctness of the code?

The idea there was that we do the data flow analysis to allow more precise inference of types within a method, and then we have type checking which actually checks that when you call a method you have the right parameters and the return type is correct and things like this.

10. As a quick question about compiling to JavaScript, do you see any big problems with JavaScript as a compilation target, what are the biggest problems?

I think some of the issues are with the semantics of JavaScript. One issue we had with compiling from Scala was the semantics of this, we actually also have some lightweight structure for reifying classes and simple object literals, and for classes and inheritance, in JavaScript this is very peculiar, you can dynamically bind it, if you have multiple function calls, you can “lose” it if you want by just having a bare function call where you don’t use this.method() , so this is a bit complicated when you are compiling to JavaScript as a target, specifically when you are trying to also have class based inheritance. I guess another thing is that there's lots of quirks with the browser incompatibilities and we actually have kind of glossed over this, we have not thoroughly checked if our code is running in all browsers, so that why it’s still a research project in that sense.

11. What about numbers, are numbers a problem?

We also gloss over this in the sense that we just translate to JavaScript numbers without regarding the issues, like the particularities of, eg. the addition operation mixed with strings and all that, this is probably not preserved exactly as it is in the translation, so there are a lot of quirks, I guess. JavaScript was not meant to be a target of compilation, but now it’s something that is increasingly happening.

12. As a user of Scala, we have to ask you what is the feature that you would like to see most in next Scala version or in a future Scala version? What are you missing most?

I actually like the way Scala is right now, one thing is that the type system seems pretty complicated from the outside, there are a lot of features, a lot of options and one thing that we are trying to do is actually to simplify it, to make it more uniform in the next version of Scala, so this is something that I’m exploring right now with Martin Odersky, the type system foundation of Scala.

13. And to wrap-up we have a special ScalaDays question, so Nada Amin please describe yourself as a Monad?

As a Monad? Well ok, so I guess I’m not really sure what means but I would be a free Monad, I saw that in one of the talk titles and I guess is just appeals to me to be able to just, yes, freely follow my interests.

14. So you are a Continuation Monad?

Yes, but a free Monad, but I guess Continuation would also work with pause and suspend, I have to think about this one more.