BT

Domain Specific Languages: A summary of recent ideas & debates

by Floyd Marinescu on Sep 18, 2006 |
A number of parallel discussions in blogspace have introduced some new distinctions useful for understanding the proper use cases for domain specific languages. Joel Spolsky explained how the use of a DSL avoided large porting costs and simplified deployment/maintenance of their infrastructure into client data centers.  Mark Dominus made the case that design patterns are a sign of language deficiency and that we should seek to improve the language, not build pattern frameworks.  Buko Obele introduces the variable of "change" into the context of discussion, and explains how DSLs are a bad idea because they do not do a good job controlling change over time.  The rest of this entry summarizes some of the key points from these sources.

Idea: Use a DSL to preserve existing code investments and simplify deployment and tech support. Joel Spolsky's company sells FogBugz, a bug tracking and feature scheduling software system built in VBScript over a 6 year period. The software is now written in Wasabi, an internally built language that is backwards compatible with VBScript but supports many higher level features such as "closures, active records, lambdas, embedded SQL a la LINQ." Joel's firm implemented a custom compiler to support their customers target platforms. Joel's customers want to deploy locally instead of trust a hosted model, leading Joels firm to build their own language and compiler instead of program to existing platforms such as PHP, Java, or .NET. This set off  a number of criticisms in blogspace, with the most active discussion over at codinghorror.com where Joel himself defended his decision by presenting the technical requirements that lead to the decision:
  1. Six years of code already written in VBScript
  2. Needs to run on customers' Windows and Unix servers
  3. Minimize tech support costs
  4. Many customers refuse to install new runtimes on their servers, either because of IT policies or out of stubborness
Spolsky concludes:  
We could port the VBScript to PHP/.NET/Java by hand, or we could spend two months writing a compiler. The latter decision took less work (a couple of months) and produced better working code that runs on all our customers' machines without modification.
On the DSL con-side, .NET Guru Dare Obesanjo also responded to Joel's blog with a criticism of DSLs, stating that they they present larger problems when you consider maintaining the language in the face of constant evolution of other major languages, as well as recruitment of developers and attrition of your main compiler gurus. 

Idea: Design Patterns are a sign of language deficiency.   Mark Dominus examined the evolution of software languages since the 70's, and taking a number of common definitions of what a design pattern is, and used them to show that subroutines, Structs, Objects, and Iterators were at one point patterns themselves which later became implemented directly into modern programming languages. Mark argues that:
Instead of seeing the use of design patterns as valuable in itself, it should be widely recognized that each design pattern is an expression of the failure of the source language.
Continuing to program with patterns and re-implementing patterns is inefficient and time wasting. Once a pattern is identified, "the correct place to implement a common solution to a recurring design problem is in the programming language" [not in multiple implementations of pattern frameworks]. Mark was also careful to suggest in his conclusion that improving the language should be a "long term goal."   Update (Sept 19): Ralph Johnson, original member of the GoF (Gang of Four) who wrote the most popular patterns book, responded that "this is wrong":
Patterns might be a sign of weakness, but they might be a sign of simplicity. There is a trade-off between putting something in your programming language and making it be a convention, or perhaps putting it in the library...Many of the patterns will get subsumed by future languages, but probably not all of them. Patterns should be subsumed into languages when doing so makes programs simpler and more reliable.
Idea: DSLs are a bad idea because they do not do a good job controlling change over time. Buko Obele was an early DSL convert who says he has implemented dozens of languages and is now against the idea of writing DSLs. Buko interpretted Mark Dominus' blog as suggesting that DSLs should be built instead of using patterns, but says it can be very time consuming; "everytime I found myself writing some "boilerplate" code...I went back to redesigning the whole language."  Buko introduces the concept of change and explains how DSLs don't respond well to it:
What I discovered during my time of inventing and re-inventing DSLs over and over is that I couldn't handle change well. Often the smallest change would lead to me changing the very language of my solution; essentially a total rewrite. This was the reality of software maitenance...And so I began to question the wisdom of creating DSLs.
Buko goes on to suggest that if you have a "mechanical view" of programming languages (just syntax and semantics; a simplistic tool that was good only for solving a set of problems"), then DSLs make sense, they suggest using the right tool for the right job.  However, the missing variable is change:
Because of change nobody ever really understands the [business] problem in the first place... Programs always seem to require ongoing human guidance or maintenance...because of the nature of change and it's counterpart in the human psyche -- choice, the idea that programming languages merely solve problems is flawed....
Looking further at the impact of change, Buko suggests that programming languages are really a framework for dealing with change, and change is "a structural problem of software development." Making his main point:
a programming language is not simply a means of solving some particular problem. A programming language doesn't define the solution to the problem, it defines how the solution will change over time. This is the categorical mistake that so many make when criticizing languages: the expressive power of a language is not the measure of a language's ability to model a problem domain, it's rather the ability of the language to control changes in the problem domain. This is ultimately why DSLs and their ilk may be more expressive than a general solution, and still possess less expressive power than a general solution.
Buko concludes, based on his own experiences, that DSLs are "pretty much never a good idea except for trivial, static problem domains."

As modern dynamic languages (which make writing DSLs easier) become more popular in the Java and .NET communities (and of course Ruby is it's own), the idea of DSLs will certainly become a topic of greater discussion. Where do you stand?

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Very informative by Don Brown

This is a very informative summary - thanks Floyd. I toyed with the idea of creating a DSL for XML transformations out of XSLT once [1], but never felt the benefit warrented the complexity and overhead. DSL's are one of those tools that I always want to use, but never have found a good situation.

Nice article by Alexandre Poitras

Very nice article. I think Buko got it right. I tend to agree with what he says.

Re: Very informative by Jonathan Allen

I think there are good situations for creating a new DSL, they just happen to be few and far between. VB was essentially a DSL for Windows programming back when C/C++ were just too time consuming to use for small projects. You can say the same for HTML/JavaScript.

I think there are a lot more DSLs out there than people think. A lot of data formats are essentially programming languages for a specific program. I have even written some "data formats" appeared to be turning complete. It wasn't intentional, but as my needs grew so did the format.

fear: increased cost of maintenance if DSLs become popular by Floyd Marinescu

Thinking further on Buko's assessment, I personally think the key consideration for what is and isn't good for a DSL might be whether the problem domain is static.

This reminds me of the use cases for annotations - use annotations for aspects intrinsic to the problem domain - stuff what will not change over time. Do not use it for things that might change over time.

Perhaps that is the correct analysis to apply when considering a DSL. If problem you are modelling is static and unlikely to change - it makes sense to produce higher level constructs - if there is any risk of change, then use a general purpose language + a framework that is built to suit.

One fear I have with the spread of DSLs is maintenance. Think that maintaining an in-house framework is hard? Imagine the problems with maintenance if every company is maintaining multiple languages. :)

VBscript++ =! DSL by Joost de Vries

If 'writing your own fork of VBscript' means 'writing a DSL' to people I'd say that DSL has become a meaningless or worthless term.
Let's see how Fowler defines a DSL:
The basic idea of a domain specific language (DSL) is a computer language that's targeted to a particular kind of problem, rather than a general purpose language that's aimed at any kind of software problem.

VBScript++ sounds like a general purpose language thas has been pimped to please hot shot developers.

Buko Obele's observations are interesting. Good post.

Patterns and language features by Maciek Makowski

Thanks for the summary, it's very interesting. In my opinion Mark's remark is a very accurate observation that applies to general-purpose programming languages rather than DSLs; e.g. delegate pattern is usually used to make up for lack of multiple inheritance/mixins/traits. I guess it will be exciting to see how programming language designers implement other popular patterns as first-class language features. Does anyone know any good examples?

I don't think "pattern" means what you think it means by Jason Yip

"Design Patterns are a sign of language deficiency"

That doesn't make any sense. Different language means different context and therefore different solutions (aka different patterns). Every language will have it's own patterns. Does that mean every language is deficient? Well, perhaps it does.

The point of Patterns is to try to communicate how experts solve problems given a particular context. Better languages still have experts, and novices that can learn from them.

Where do business rules fit in all this? by James Taylor

I liked the summary and I wondered where the audience thinks the use of a business rules management system/business rules engine fits with respect to DSLs? With their ability to express higher level constructs and define patterns as well as their usefulness for business users they seem like they have many of the features of DSLs while remaining robust in the face of change. Thoughts?
Enterprise Decision Management Blog

Re: Very informative by tanguy rademakers

+1

Floyd, thanks for taking the time to write this, a very good synthesis of various positions. This kind of pondered, quasi-editorial content is great for balancing the signal-to-noise ratio.

/t

Re: fear: increased cost of maintenance if DSLs become popular by David Roussel

One fear I have with the spread of DSLs is maintenance.


That all depends on what kind of changes commonly occur in maintence.

I worked on a project that gone live, and released serveral times. Each release they added new features that required 5 months of Java dev, even though what it was doing was similar in the problem domain to the previous work.

I suggested making a DSL, but there are too much percieved risk and I was told to implement as specified.

Anyhow, many months later, whilst waiting for testers to find bugs in code, I decided to make a DSL for the problem domain.

It was basically a wrapper over already developed java classes, that allowed them to be plugged together in a DSL script.

As a developer, the DSL wasn't really at a much higher level than my java code, it just had less syntax. changes could be prototyped in a running system by the business analysts themselves, without the need for compilation.

Since most changes happen at the problem domain layer, if you build a stable platform for expressing and configuring business domain concepts, then most maintence becomes either changing the DSL scripts, on writing code to extend it.

Re: VBscript++ =! DSL by David Roussel

Ruby != DSL, but it can be used to make an internal DSL. And if you can do it in Ruby, then why not in VBScript++ ?

Re: Patterns and language features by Gerald Loeffler

Does anyone know any good examples?


I suppose a fairly obvious and suitable example is the combination of a MethodInterceptor and Java dynamic proxies (a pattern used in Spring et al.) which can be replaced by an around-advice (a first-class language construct in AOP-languages).

Another possible example: a "workflow" is a first-class language construct in languages like jPDL or BPEL but must be implemented as a combination of patterns (e.g. for state handling) in a lower-level language like Java.

So the notion that patterns (can) give rise to new language features actually resonates quite strongly with my personal perception.

cheers,
gerald

www.gerald-loeffler.net

Re: Patterns and language features by Alexandre Poitras

Does anyone know any good examples?


I suppose a fairly obvious and suitable example is the combination of a MethodInterceptor and Java dynamic proxies (a pattern used in Spring et al.) which can be replaced by an around-advice (a first-class language construct in AOP-languages).

Another possible example: a "workflow" is a first-class language construct in languages like jPDL or BPEL but must be implemented as a combination of patterns (e.g. for state handling) in a lower-level language like Java.

So the notion that patterns (can) give rise to new language features actually resonates quite strongly with my personal perception.

cheers,
gerald

www.gerald-loeffler.net


Factories are also a first class citizen in smalltalk and visitors are not needed in ML or any functional language. I totally agree with the premise that a pattern can be made a language feature. But I think the question is more : should it be? There are a lot of patterns out there and you can't obviously make them all fit in a single language. The natural solution would be to combinate a lot of small DSL but as Buko pointed out this leads to a maintenance nightmares in most cases.

I think the solution is DSL should be provided by frameworks in most situation as it is in Java right now. Just make it part of the core language when the pattern is used very often and is a low level abstraction (observer, closures, ...). Those are usually very stable abstractions.

Re: Where do business rules fit in all this? by kevin schwarz

I wrote a DSL for business rules. It allows technical analysts to read and understand what our application does and helps them with future planning and production support. The language is very simple, it controls visibility/requiredness of fields, validation constraints and labeling. This application is a brokerage account opening system and is very rules intensive (consolidates about 100 paper forms).

We implemented a custom execution engine that ties in with these rules (implements the observer pattern over some live forms j2ee/ajax) that is stored in configuration files and generated at compile time. So far this has worked well for us and has been very testable and maintainable. Should I need to use a new runtime target in the future, I can simply extend my code generator.

I wonder what other uses of DSLs have been applied this audience?

Re: VBscript++ =! DSL by Joost de Vries

To me the essential point of a DSL is that it provides an idiom that is constrained to a certain domain. This idiom should be similar to a language in that it is composable: the number of possible expression is not linear to the number of atomic 'keywords' but should be exponential to the number of atomic keywords.
VBScript++ fails the test of being constrained to a domain.
When people say that you can use Ruby to express DSL's to me that means that one has an API, or set of classes and methods, that has an especially rich generative power.

To me these precise points that decide wether it's meaningful to call something a DSL are important because otherwise any configuration file is a DSL.

Spolsky just had the problem of a large codebase in an unattractive programming language and made it more sexy by adding language constructs du jour and calling it DSL.
Unrightfully I think.

Re: Patterns and language features by Corby Page

The natural solution would be to combinate a lot of small DSL but as Buko pointed out this leads to a maintenance nightmares in most cases.


Buko makes some good points, but I think his case is overstated here. My apps have several small DSL's that don't become maintenance nightmares.

My application context xml files for Spring are DSL's that implement the Builder pattern.

My OSWorkflow configuration xml files are DSL's that abstract my workflow processes.

My Webwork xwork.xml files are DSL's that implement Front Controller, and a couple of other patterns.

In all three cases, the DSL's are well documented, and the overhead associated with maintaining these DSL's is well worth the benefits that I am getting out of them.

It should also be noted that all three of these DSL's have extension points. If there is something I want to do that is outside of the capabilities of the DSL, I have the ability to cleanly drop out to a higher-level language to implement the functionality, without having to change the implementation of the DSL itself.

I believe that DSL's that have extremely narrow applications, or that lack clean extension points, are the ones that are most likely to cause the problems that Buke describes.

Re: Patterns and language features by Alexandre Poitras

The natural solution would be to combinate a lot of small DSL but as Buko pointed out this leads to a maintenance nightmares in most cases.


Buko makes some good points, but I think his case is overstated here. My apps have several small DSL's that don't become maintenance nightmares.

My application context xml files for Spring are DSL's that implement the Builder pattern.

My OSWorkflow configuration xml files are DSL's that abstract my workflow processes.

My Webwork xwork.xml files are DSL's that implement Front Controller, and a couple of other patterns.

In all three cases, the DSL's are well documented, and the overhead associated with maintaining these DSL's is well worth the benefits that I am getting out of them.

It should also be noted that all three of these DSL's have extension points. If there is something I want to do that is outside of the capabilities of the DSL, I have the ability to cleanly drop out to a higher-level language to implement the functionality, without having to change the implementation of the DSL itself.

I believe that DSL's that have extremely narrow applications, or that lack clean extension points, are the ones that are most likely to cause the problems that Buke describes.


You have a good point. My statement was kind of vague. What I wanted to say that is DSLs should be provided in most cases by frameworks and not the language itself. This way, there are less chance than you end up in a intangible mess because you cam't invent DSL for every small problems.

Re: I don't think "pattern" means what you think it means by Jonathan Allen

The point is that if you need a design pattern, the language and libraries are not as good as they should be.

Compare this design pattern for reading a file:

1. Get a file handle
2. Open a file using that handle
3. Read the bytes from that file
4. Convert those bytes into UTF-8
5. Close the file
6. Release the file handle

... with this one-liner.

1. Call the ReadFile function, passing UTF-8 as the encoding

Obviously the one-liner is a better solution than the design pattern. Therefore the langauge that requires the design pattern is deficient.

Re: VBscript++ =! DSL by Jonathan Allen

To me the essential point of a DSL is that it provides an idiom that is constrained to a certain domain.


I would argue that "VBScript++" does have a specific domain, specifically "web applications that need to target both VBScript and PHP".



To me these precise points that decide wether it's meaningful to call something a DSL are important because otherwise any configuration file is a DSL.


Is that necessarily wrong? I think there is a lot of grey area between configuration files and DSLs.

Are the make files used by ANT or nANT configuration files or DSLs?

Re: VBscript++ =! DSL by Joost de Vries

"web applications that need to target both VBScript and PHP"
If that's a domain there isn't any programming language that is not a DSL.
is that necessarily wrong?
No, just very uninformative and thus uninteresting.
Maybe everybody agrees with you and then that usage will define what a DSL is but then the concept does not add anything apart from the buzz.

Complex volatile domains by John Harby

'Buko concludes, based on his own experiences, that DSLs are "pretty much never a good idea except for trivial, static problem domains."'

My experience has been the opposite. I worked in a very mature environment where DSLs had been developed for some 15 odd years to support a fairly well known product. This product has been very successful netting billions in revenue.

The argument for using the DSLs was firstly that the domain was very complex. Software engineers could not produce the product as they didn't have the domain experience. However, domain experts weren't proficient in common programming languages so the software engineers needed to produce simplified languages for them to use.

Secondly, this particular domain was very volatile. Extensive changes to the business rules and definitions occured on a regular basis. This fact added support to the argument for using DSLs since the changes could almost always be made in the simplified DSL code rather than the lower-level codebase. Also, developing several different DSLs allowed for decoupling of the problem space thus limiting the ripple effect of changes.

Re: I don't think "pattern" means what you think it means by Tim Vernum

Compare this design pattern for reading a file:

I'll refer back to Jason's original point. "I don't think pattern means what you think it means"

Reading a file in that manner is an algorithm, not a design pattern.
The whole point of patterns is that they are constructs that the implementation language cannot express natively.
Any language that has some form of subroutine/method call has the capability of implementing the second form directly.

If that represented a pattern then the Gang of Four book would have been a library instead of a book. They could have said "here is some code for reading a file", etc. The whole reason they wrote a book and discussed abstract concepts is because the ideas cannot be directly implemented in code (in any current OO language) in a general way.

To quote wikipedia
Algorithms are not thought of as design patterns, since they solve computational problems rather than design problems.

Re: VBscript++ =! DSL by Christian Romney

+1. If you have to build a compiler it ain't a DSL. DSLs are the new acronym for the little languages pattern, the key term there being 'little'. Was Fortran a DSL?

Re: VBscript++ =! DSL by Jonathan Allen


is that necessarily wrong?
No, just very uninformative and thus uninteresting.
Maybe everybody agrees with you and then that usage will define what a DSL is but then the concept does not add anything apart from the buzz.


Perhaps DSLs really are just not interesting. The more I think about it, the more I lean towards believing that maybe they are just another term for what we were doing all along.

Re: I don't think "pattern" means what you think it means by Jonathan Allen

Reading a file in that manner is an algorithm, not a design pattern.


I maintain that it is design pattern in the sense that it is...

a design pattern is a general repeatable solution to a commonly occurring problem in software design.


(Also from wikipedia)


The whole point of patterns is that they are constructs that the implementation language cannot express natively.
Any language that has some form of subroutine/method call has the capability of implementing the second form directly.


I used that as an example because it is simple and there are languages that do not have any form of subroutine/method call. Thankfully we don't use them very much any more, but there was a time when they were quite common.

A more modern, but much longer, example would be making a database call and loading the rows into objects.

As for the Gang of Four, I don't believe it to be either a comprehensive coverage of common design patterns or particularly relevant to modern languages. Many of the patterns have since been replaced by language constructs and many of the problems simply don't exist any more.

I favor design pattern books that either focus on a specific context or address underlying principals.

Re: VBscript++ =! DSL by Jonathan Allen

+1. If you have to build a compiler it ain't a DSL.


Why does it matter if it uses a compiler or interperter?

Re: VBscript++ =! DSL by Joost de Vries

That is an interesting conclusion! :-)

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

27 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT