Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Vaclav Pech and Alexander Shatalin on MPS

Vaclav Pech and Alexander Shatalin on MPS


1. We are here at the CodeGeneration conference in Cambridge and I am sitting here with some guys from JetBrains, so who are you?

Vaclav Pech: Hello I am Vaclav Pech and I work at JetBrains as a software engineer on the MPS Project.

Alexander Shatalin: Hello my name is Alexander Shatalin, I am team lead of MPS Project actually.


2. So what is this famous MPS Project that I’ve been hearing about so much at this conference?

Vaclav Pech: Well MPS stands for Meta Programming System, so it’s a Language Workbench and that is kind of specific in the way that it approaches code that it gives you a projectional editor, so it’s not a Language Workbench based on a parser but on projectional editing. The way you edit your code, you don’t edit it through a normal editor where you would have free text that you can freely edit, but you get the AST that represents your code projected on the screen, so you edit it in a controlled way. So you can’t create invalid, so to say, AST or invalid code that wouldn't be parseable.


3. So how can I edit this AST? Do I have to fiddle around with a tree or do I get a textual representation?

Vaclav Pech: That is exactly the task for the language designer to give you an editor or multiple editors, that give you way to edit the AST indirectly, so you don’t touch the tree itself, but you have some predefined editors which might look very close to text editors actually and in many cases they do, but they can differ from plain text editors, they can provide constructs like decision tables for example or state machines sort of abstraction to edit it in a more comfortable way.


4. So a kind of graphical editing? So you can edit nodes graphically, is that correct?

Alexander Shatalin: Currently not unfortunately, but we are planning to implement this feature in the future, so it’s kind of on the roadmap of the project.


5. I saw these state tables, or these decision tables - are they somewhat in between text and graphic?

Alexander Shatalin: You know that from the side of MPS all these elements are just a kind of rendering functionality for particular node of the abstract syntax tree, so we have a specific node rendering a specific element in editor definition rendering abstract syntax tree node as a table. So that is how it works basically. And nearly the same approach would be used for diagrams, so we will introduce additional nodes in the language which you can use to define editors which will allow you to create diagrams with nodes and links between them, somehow edit them. We'll use or tightly integrate this and which definitely what we have currently for editing stuff in MPS so you will be able to use Completion and the rest on diagram nodes. That is how it should be actually.

Vaclav Pech: But the tables you can view them as just a different way to arrange things on the screen. We might take a different example, let’s say an If statement, right, so you have If and a condition and typically you have a body and then you have Else and underneath you have the alternative branch. But with Projectional Editing you can decide that you would like to see these two branches not below one another but next one to another and we can project it on the screen any way you like and we can give each branch a different background color or different font, or whatever. It is basically playing with the visual appearance of the code on the screen. But it is all textual, is not that you have to take the mouse and draw a table and then fill in stuff, you do it from the keyboard and you refactor the code as if it was just a normal piece of code, so you can wrap the table, take the table and assign it to a variable or have it come back as a return value from a method.


8. So how is my code actually stored in the IDE in MPS?

Alexander Shatalin: We have a proprietary XML-based format for storing that code, but basically you as a user of MPS should not think about that format at all, because we are hiding it absolutely from end users. We provide you with the possibility to work with AST nodes directly from the editor and we provide all the tooling around that with the possibility for example hide corresponding XML files when you commit or check out sources from the version control. So we do provide kind of similar projectional editing covering all the Version Control interoperability.

Vaclav Pech: So this low level format should never leak out, you should never see it even when you check into Version Control System it’s still hidden and you have the projectional editor and really you work with code and not with the XML representation of it.


9. So you work with an abstract representation of this tree but you don’t see a tree?

Vaclav Pech: Yes, you don’t see the format that is used for persistence, it’s so solely used for persistence on disk, for nothing else.


10. MPS is a Language Workbench, I can come in and build my own language, how do I define it? Do I build a grammar or do I build something procedural, what do I have to do?

Vaclav Pech: There are a couple of stages that you have to go through. So if you want to design your own language, first you define the abstract structure of your language, the abstract syntax, so the nodes that should form the abstract syntax tree. So if are building a version of Java for example, you would create a node that represents an If statement, but that node need a condition so it would have a child element that would represent the condition. It would typically be an expression and perhaps you would somewhere say it should be Boolean. And then you would say an If statement needs an If branch so that is a list of statements and then it has an Else branch which also is a list of statements.

And then you have to define what a statement is, what a list of statements is, so you would define these nodes and how they can be combined into a valid tree. And then you design the editor which we talked about, as the representation on the screen, so how the If statement should look on the screen, so we get an If text here then a place where the user can edit the condition, and then the place where the statements go. Once you have that you need to generate code, so you have to translate your language in to either some other language, so that is model to model transformation, so you say: "My abstract syntax tree needs to be changed into that abstract syntax tree".

For example my loop statement that I have my fancy loop statement needs to be somehow translated into the ordinary For loop in the target language, so you translate the concepts you have into the concepts that are in the target language. So you gradually simplify your language, I mean your construct into simpler and simpler languages and then you generate text. That is what you generate and then you run a compiler against it to create a runnable application. So that is the core that you have to have something runnable and there are extra things that you also need to define.

Alexander Shatalin: We have a lot of IDE specific features around that like you can basically start with Type System Rules for example to define some type system aspects for the language and then specify a lot of other stuff like data flow analysis and do some more IDE specific integration with the environment. So your actual DSL will look like a normal programming language in MPS, so you use all those features during the editing process and in addition to that I have to say that we have definitely some integration with the ANT scripting language where you can create actually an And script and call the MPS code generator from there so you’ll be able to basically get the source code on build servers somehow from the models. So it’s more or less integrated so this lifecycle is covered as well.


11. So your representation is basically an intermediate representation, like that of a compiler, and you write the backend for this compiler basically?

Alexander Shatalin: You mean the models in MPS? Basically you can think of the models like the sources in Java, because sources in Java or in any other programming languages actually intermediate representation of your program as well, so if you finally have to compile that and then you get something executable, so it’s something similar but in MPS instead of thinking of Java sources like as a single source of information, we actually think of Java sources as about some intermediate state like intermediate object models used by compiler.


12. And you offer the tools that compilers need to compile, as you said, data flow or type systems?

Alexander Shatalin: It’s mostly used inside of the IDE itself, so we’re mostly using that for providing IDE functionality for your language, for providing actually a software developer with the necessary assistance from the side of MPS, because once you have the program working your data flow, you control that and it’s ok, I mean you don’t have any Null pointer warning, then it’s most probably going to the final Java sources which will be compiled properly and no problem there. You have to be sure about that.

Vaclav Pech: Data flow is pretty much an analysis tool that you run inside your IDE when you edit the code so it will analyze your code on the fly and tell you "Hey, this piece of code can’t be reached, you never get there or you might get a Null pointer exception from here". But you know once the data flow is happy then you don’t need to run it further down the chain, you just generate Java and provided the generator is correct and then the data flow related problems in the generated code.


13. So what languages do you ship with MPS, what are the biggest languages that were built with MPS? I think here we saw C by Markus Voelter, but what other languages can you talk about ?

Alexander Shatalin: Basically, that sounds funny but we are trying to reduce the number of languages distributed with MPS obviously because we are not going to address all the domains in the world, so the main focus for MPS is developing new domain specific languages. So first we are selling or we are distributing those languages which are used for describing new domains specific languages in MPS. Obviously since we are based on Java platform we have our good and powerful support of Java platform so it’s not a problem for you if you want to create some extensions in MPS going down to Java, the rest is kind of outside of the scope of the project a little bit. And I have to mention that we have a couple of powerful extensions for the Java language, right?

Vaclav Pech: So it's not the only Java that we distribute which is basically a projectional clone of Java so to say, but we also enhance Java with something that we consider to be very useful, so things like working with collections in a functional style or having closures as first class citizens in the language or working with regular expressions in a convenient way and date and time. So these are all composable extensions you may or you may not use them (it depends on you), but we typically use them because we enjoy the power of these extensions. The thing is that even using these extensions you still generate Java 1.6 compatible source code, so you don’t need any runtime dependencies on MPS in your code if you generate it from MPS.


14. So it’s always going back to plain Java?

Vaclav Pech: All the closures are translated into anonymous inner classes and working with collections gets translated into loops.


15. Basically if I want to create my own language I could use your fancy enhanced Java to build it, is that true?

Vaclav Pech: Yes you can build on the Java that we have, so we can built on Java with collections and closures and create whatever you like. Or you can start from the ground up so you might say "I assume there is no language and I want my own language that looks totally different from Java or C", so you can start from that to build your own language and then someone else can create extensions to your own language.

Alexander Shatalin: Or you can pick one of the existing languages which was already built for MPS by somebody else for example by Markus Voelter, who actually created a C language support in MPS and actually you can pick this language because it’s open-source right now and stack your own extensions up on that language, similar to the way you do that with Java in MPS just out of box.


16. I hear that JetBrains has a new language in the pipeline called Kotlin; could I build it with MPS?

Alexander Shatalin: I doubt to make sense to build Kotlin with MPS, because Kotlin is addressing a little bit different use cases as far as I know, but basically what we are going to do is we are going to extend MPS with additional language constructions, more or less reflecting the power of Kotlin but based on Java.

Vaclav Pech: But if you wanted, you could build from the ground up a Kotlin language or any language you like in MPS so you could have the projectional version of any programming language that you’d like to create, but Kotlin is an independent activity of JetBrains so it’s kind of a different group.


17. So MPS, do I have to buy it? What is the distribution model? What is the business model for you, how do you get paid?

Vaclav Pech: About MPS itself it’s an open source project, and it’s under the Apache 2 license, so you can get it to use it for commercial purposes if you like, and actually we have commercial users who use MPS to do business and I’m very happy to see all these people here at the conference, we have pretty large set of customers and users here at this conference. Typically people in Academia use MPS as well to do research because again there are no restrictions in how you use MPS.

Alexander Shatalin: So actually MPS is open source and this is a good platform to start something upon it, so we are open for any requirements and requests from the community and we are willing to make this community grow, so that is basically the state of licensing.

May 30, 2012