BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Article: Developing a Complex External DSL

Article: Developing a Complex External DSL

Bookmarks

In this article Vaughn Vernon explains the difference between internal and external DSLs and shows the steps involved in developing a complex external DSL.

Read: Developing a Complex External DSL

Before starting his example of developing an external DSL, Vernon makes the difference between internal and external DSLs. For an internal DSL, “the language that is developed could be very closely tied to and implemented on top of the primary general purpose programming language in use in your project, such as Java, C#, or Ruby.” Creating an external DSL is:

analogous to creating a general purpose programming language that is either compiled or interpreted. The language has a formal grammar; that is to say, the language is constrained to allow only well-defined keywords and expression types.

The first step in DSL development is defining a syntax, and Vernon offers advice on how to do that:

  1. Study other languages.
  2. Experiment with various syntax ideas using agile techniques.
  3. Identify as many language features as possible.
  4. Present your language before communities of potential users.
  5. Beta test your language.

The next step is defining a metamodel:

Think of source code written in a language's grammar (syntax) as a model of concepts that you are describing. The concepts you are describing could be data, structure, and behavior, which are typical concepts in computerlandia. From the language designer's point of view, the descriptions of these concepts are a model, not just source code. Thus, when you parse a source model and put its representational contents into objects, the objects are called a metamodel.

Next comes the parser, and Vernon gives the reader a basic example how it can be done. The last step is generating the code from the model, and the author deals with multiple approaches, like: generating the code directly from the model, walking the metamodel, eventing the metamodel, using code templates, the last representing a recommended choice.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • DOMAIN SPECIFIC language

    by Rusty Wright,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I've always been irritated with people calling an "internal DSL" a DSL because when I see the words domain specific language the only thing that makes sense to me is an external DSL. If it's really DOMAIN SPECIFIC then I shouldn't be able to just drop down into the hosting language and start using it.

    Someone needs to come up with a better name instead of DSL when they're referring to internal DSL.

  • RE: DOMAIN SPECIFIC language

    by Chris Peterson,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Someone needs to come up with a better name instead of DSL when they're referring to internal DSL.

    Maybe "domain-driven design" or simply "code"? If someone is writing a lot of code or modules that are not related to their application domain, maybe they should reconsider their choice of programming languages.

  • Re: DOMAIN SPECIFIC language

    by Werner Schuster,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    "MultiParadigm Programming"?

  • Re: DOMAIN SPECIFIC language

    by Rusty Wright,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I don't have a problem with the term Domain Specific Language, as long as it's used to describe a language that really is specific to a domain, which means an external DSL.

    For example, when you have syntax errors, its error messages should be specific to the domain language, not the underlying hosting language. And, as I said before, you shouldn't be able to drop down into the hosting language; even though that might be quite useful, my issue is simply with calling such pseudo languages a DSL.

  • Re: DOMAIN SPECIFIC language

    by Vaughn Vernon,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    you shouldn't be able to drop down into the hosting language


    Are you saying your DSL users should not be able to author Foreign Code? I'd agree that non-programmers should not, but it really depends on your DSL target user community. If your DSL is targeted at programmers then allowing Foreign Code is a possibility.

    Vaughn

  • Re: DOMAIN SPECIFIC language

    by Vaughn Vernon,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    As far as naming an "internal DSL" if you don't like that nomenclature, perhaps fluent API says it best. The concept of fluent API, while not necessarily the name, has been around for a long time.

    Vaughn

  • Re: DOMAIN SPECIFIC language

    by Rusty Wright,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Are you saying your DSL users should not be able to author Foreign Code?


    No. What I'm kvetching about is calling something that allows that a DSL. I'm not against doing the things you can do with these so-called DSLs, just calling them a DSL; they're clearly not Domain Specific.

  • Re: DOMAIN SPECIFIC language

    by Vaughn Vernon,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    No. What I'm kvetching about is calling something that allows that a DSL. I'm not against doing the things you can do with these so-called DSLs, just calling them a DSL; they're clearly not Domain Specific.


    That's an interesting perspective. From your definition, then, you'd have to conclude that ANTLR's support for including custom/foreign code snippets to be inserted into the generated browser means that ANTLR's EBNF cannot be classified a DSL. It seems to me that (E)BNF is clearly abstracting away the details of writing a lexer and parser manually and brings us closer to a pure parser domain. That to me renders it a DSL. But by your definition Terence Parr would have to come up with some sort of EBNF extension that manages metamodels and semantic models (which, has nothing to do with parsers), similar to supporting ASTs, and we'd use that instead of Foreign Code. If so it sounds impractical and limits our ability to deliver solutions that can be used by a broader community. Either we'd fail to support unimagined targets or we'd end up creating a much broader and more complex DSL.

    One thing that you are hitting on that makes a lot of sense is like the last statement made in the presentation "DSLs: The Good, the Bad, and the Ugly" posted just a day or two before my article. The fellow states something like 'the narrower our domain the higher the benefit/reward of developing a DSL.' That is very true, but I'd also suggest that we don't all have the luxury of narrowing our domain that finely.

  • Re: DOMAIN SPECIFIC language

    by Rusty Wright,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Again, I'm not saying anything about how flexible or narrowly defined a so-called DSL should be, or how much or how little it's capable of doing. I'm not saying a so-called DSL shouldn't be allowed to drop down into the hosting language. All I'm fussing over is that once you allow that, it's really not correct to call it DOMAIN SPECIFIC. At the moment I can only think of one example of a domain specific language; PostScript. If you make a syntax error in PostScript you'll get a PostScript specific error. PostScript isn't implemented inside or on top some other language that you can drop down into. Therefore I think it's correct to call PostScript a DSL; a page description Domain Specific Language.

    On the other hand, Active Record allows you to drop down into Ruby code. If you make a Ruby syntax error you get an error from the Ruby interpreter. What I'm not saying is that Active Record should not have been implemented on top of Ruby. I'm also not saying that Active Record should have been implemented as an external DSL. I'm not concerned about the capabilities or implementation of Active Record. All I'm saying is that because of those things it's obviously not DOMAIN SPECIFIC, so let's call a spade a spade and call it something else, something closer to the truth.

  • Re: DOMAIN SPECIFIC language

    by Rusty Wright,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    In response to your ANTLR remarks, I haven't used ANTLR but it sounds sort of similar to what I have used, Yacc and Bison, where with those you had EBNF for the parsing rules and after each parser rule was C code that did the underlying grunt work.

    I think the difference is is that back in the days when Yacc and Bison were big, programmers weren't so self deluding and presumptuous to call that parser generator configuration language a Domain Specific Language.

    I often feel like the term DSL is a more of a marketing term; it's much more impressive sounding than calling these things something like a "hosted pseudo language" or a "hosted configuration language". People are making a purse out of a sow's ear and calling it a silk purse when it's obviously not.

  • Re: DOMAIN SPECIFIC language

    by Vaughn Vernon,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    All I'm fussing over is that once you allow that, it's really not correct to call it DOMAIN SPECIFIC.


    I have not misunderstood what you are saying. Your viewpoint reminds me of a summer job I had back when I was in school. I worked for Mr. H and he was there to teach me a thing or two, every day. One of the things he taught me was that I used a pushbroom incorrectly. Whenever I got to a corner of the building he told me to sweep I would pick the broom up and set it against the wall and pull the dirt backward toward me. Mr. H couldn't handle that. He said "you are using that incorrectly because if you were supposed to pull the broom they would have called it a pullbroom, but it's called a pushbroom which means you push it." Mr. H then grabbed the broom out of my hands and demonstrated. Of course I only used the broom the way I did because unlike a spider I couldn't scale a wall and turn to put leverage on the broom to push it forward away from the wall. And of course when Mr. H demonstrated how I should push the broom he never got within three feet of a wall. While I still call it a pushbroom, I still from time to time will get caught pulling the broom toward me. Oh well.

    From Mr. H's viewpoint I'd have to invent something called a push-pull-side-to-side-leaning-or-hanging-broom. The leaning-or-hanging part is because you can't push, pull, or sweep side-to-side constantly. You have to stop at some point and lean the thing against a wall or hang it on a wall.


    If you make a syntax error in PostScript you'll get a PostScript specific error.


    In my DomainMETHOD DSL, if I make a syntax error in my DSL source I get a DomainMETHOD specific error message. But once I am past the code generation part and a user of the domain model tries to set an invalid value into some domain object they will get an error (exception) produced by the language that the model was generated in.

    It appears that by your definition a DSL can only be interpreted or generated into a final machine executable format. While I have no vested interest in what an internal or external DSL is called, or what the pure definition of both is, I disagree with the assertions that none of these things are DSLs.

    I personally have chosen to use the term DSL so others will know what I am talking about. I also believe that the efforts of those who place an emphasis on the possible need to create new languages in a given domain is important to the advancement of innovative software solutions in a lot of places. Since I have been using what are now called DSLs for a long time I could throw my arms up and say that the new industry nomenclature is wrong. I have simply decided to go along with the naming because it makes sense that this important direction have a name.

    I would suggest, however, that if you feel you have a strong point to make I am the wrong person to change industry direction. Like many things in this industry I just end up consuming them and sometimes writing or presenting about them. You probably owe it to yourself to talk to those who can actually change the way things are. For the rest of us, we are much better off staying within a ubiquitous language within our industry so everyone else can relate to what we are saying and doing.

  • Re: DOMAIN SPECIFIC language

    by Knut Forkalsrud,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    ... let's call a spade a spade and call it something else, something closer to the truth.


    Maybe the term jargon is a better fit than "language". In my dictionary "jargon" is described as "a form of language regarded as barbarous, debased, or hybrid."

  • Re: DOMAIN SPECIFIC language

    by Rusty Wright,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I finally figured out my problem; I am taking the term literally given the order of the words. Similar to your push broom analogy, if I were to say that I have a plastic orange bowl I think that most people would assume that I have a bowl that is plastic and orange. Not that I have a bowl that I use for holding plastic oranges. So what I finally realized is that a DSL is a (pseudo) language for a domain specific problem.

    My problem is that when I see the 3 words Domain Specific Language I naturally see the is meaning.

  • Re: DOMAIN SPECIFIC language

    by Vaughn Vernon,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    if I were to say that I have a plastic orange bowl


    I won't go all English professor on you (because I am not one), but I think proper English is "orange plastic bowl." See "Concepts In Composition" p343-344. Clearly, however, the way in which something is named can cause confusion. I think in the case of DSL it's something that we need to accept because it represents something more than the name itself may be able to convey on its own.

    I think as far as the representation goes, it can mean either thing: (1) it is a DSL in the sense that you originally desired it to mean (e.g. PostScript is a page description DSL) if you create the language in the appropriate way for that to be true; (2) it is a language for a domain specific problem, as in my DomainMETHOD tool that brings a designer closer to the domain of domain-driven design and then translates the design into an object model that is about the domain that the designer is targeting. (I could also argue that DomainMETHOD is a domain modeling DSL, and it probably is even by your definition, but it is not important for me to argue that point.)

    One thing I should clear up, which I thought was obvious, but may not be. I don't claim that the domain model code generated from my DomainMETHOD DSL tool is automatically an internal DSL. It may well be if you accept that a fluent API is an internal DSL, and if the modeling techniques used by the designer form a fluent API. But the generated domain model code is not always fluent just as a natural outcome and side effect of using the tool. Also, in general, there is no reason that any external DSL tool needs to automatically generate an internal DSL in order for the source code that tool parses and translates to qualify as an external DSL.

  • Re: DOMAIN SPECIFIC language

    by Rusty Wright,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    if I were to say that I have a plastic orange bowl


    I won't go all English professor on you (because I am not one), but I think proper English is "orange plastic bowl."


    More semantic hair splitting from me: ordinarily yes; taken out of context, yes. But in certain situations I think it could mean a bowl for a plastic orange. Let's suppose we're talking about a precious plastic orange I have and a cut crystal bowl that I only use to put that plastic orange in. If I were to then refer to my "plastic orange bowl" I think most would understand that I mean the bowl for my plastic orange.

    Unless you're talking about whether orange goes in front of plastic, or vice versa. That's splitting the hairs even finer than I would venture.

  • Re: DOMAIN SPECIFIC language

    by Vaughn Vernon,

    Your message is awaiting moderation. Thank you for participating in the discussion.


    More semantic hair splitting from me...

    Since I risk no longer being sure what we are talking about, I'll have to go with the following and call it a day :)

    I think in the case of DSL it's something that we need to accept because it represents something more than the name itself may be able to convey on its own.

    It's been fun. Have a great weekend!

  • Re: DOMAIN SPECIFIC language

    by Steven Kelly,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I read "plastic orange bowl" as meaning "a plastic bowl for holding oranges", so there's another interpretation. Try interpreting the phrase in the following contexts, and see the different meanings you can get:
    - a shop selling plastic fruit
    - a shop staffed by non-native speakers selling bowls of various colours and materials
    - a shop selling specialized fruit bowls
    - discussion of playing surfaces in intercollegiate American football
    - scale models of historical American football stadiums
    ...Enough already!

    That's a feature of a DSL: the context - the domain - gives information without which the statements are ill-defined, even for someone who recognizes the meanings of the individual components (e.g. "plastic", "orange", "bowl"). That's why DSLs can get by with less verbiage than even well-designed GPLs, and more importantly why statements in a DSL are faster and less error-prone to read and to write than in a GPL.

  • Sharing experiences

    by Johan den Haan,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Hi Vaughn,

    Thanks for sharing your experiences! Triggered by your effort I've also shared some of my experiences with DSL development. See: DSL development: 7 recommendations for Domain Specific Language design based on Domain-Driven Design

  • Re: Sharing experiences

    by Vaughn Vernon,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Hi Johan,

    I enjoyed your article. It appears that we have very similar experiences and I liked the way you expressed yours. Also the Mendix product line is an excellent example of putting DSLs to work with graphical editors.

    Thanks!

    Vaughn

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT