Facilitating the spread of knowledge and innovation in professional software development

Contribute

### Topics

InfoQ Homepage News Article: An Approach to Internal Domain-Specific Languages in Java

# Article: An Approach to Internal Domain-Specific Languages in Java

In this article, Alex Ruiz and Jeff Bay describe Java's suitability as a DSL-producing language, delve into the creation of internal DSLs in Java, walk through an example of a Java-based internal DSL, and give recommendations on writing DSLs in Java.

From the article's conclusion:

Java can be suited to create internal domain-specific languages that developers can find very intuitive to read and write, and still be quite readable by business users. DSLs created in Java may be more verbose than the ones created with dynamic languages. On the bright side, by using Java we can exploit the compiler to enforce semantics of a DSL. In addition we can count on mature and powerful Java IDEs that can make creation, usage and maintenance of DSLs a lot easier.

Creating DSLs in Java also requires more work from API designers. There is more code and more documentation to create and maintain. The results can be rewarding though. Users of our APIs will see improvements in their code bases. Their code will be more compact and easier to maintain, which can simplify their lives.

There are many different ways to create DSLs in Java, depending on what we are trying to accomplish. Although there is no "one size fits all" approach, we have found that combining method chaining and static factory methods and imports can lead to a clean, compact API that is both easy to write and read.

Style

## Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

• ##### Brilliant article

by Henri Frilund,

• ##### A Pointless Exercise

by Erkki Lindpere,

• ##### Re: A Pointless Exercise

by Michael Hunger,

by Richard L. Burton III,

• ##### Good stuff

by Manuel Palacio,

• ##### Re: SQL-Example

by Michael Hunger,

• ##### Re: Good stuff

by Lukas Eder,

by Michael Hunger,

by Michael Hunger,

• ##### Very Beneficial

by Jacob Northey,

• ##### Re: Very Beneficial

by Michael Hunger,

• ##### Re: Very Beneficial

by Jacob Northey,

• ##### Very nice (but is it a DSL?)

by Ron Kersic,

• ##### A Great Article

by John DeHope,

• ##### Another nice example of an internal DSL for integration

by David Greco,

• ##### A dsl for collection manipulation

by Mario Fusco,

• ##### Fluent builder generator

by Jakub Janczak,

• ##### Nice article - a few nitpicks . . .

by Peter Bell,

by Kevin Wright,

• ##### Brilliant article

Your message is awaiting moderation. Thank you for participating in the discussion.

Very interesting article. I'm right now working on a personal project trying to create a tool for creating web applications with an internal Java-hosted DSL, backed by a component model (Wheel).

Designing the API is indeed quite difficult especially if the DSL is rather large. Worst problems for me so far have been limitations posed by the static type system (which on the other hand helps a lot when using the DSL with a modern IDE), method grouping and not having named attributes.

Thanks to some of the ideas in your article, I think I can get around the missing named attributes.

The grouping-issue...by which I mean the situation where you start to have dozens of methods (DSL expressions) in a class and you need some way to group them functionally, but can't break the class-hierarchy either (in my case the DSL is used by extending a a class containing the DSL so I can't split the class itself). I ended up creating sort of "side-show" classes that hold the methods, but share state with the "main" class. Example: A Configuration-class that holds method for manipulating the "main" classes configuration properties. The "main" class then as a method that returns the Configuration-object.

In code it looks like this:

config().    initialFieldValue("guess", "give a number").    exposeField("guess");

What the config() does is that is returns the Configuration-object with methods that will return this. It's not very OO, but it works and the result looks pleasing.

Not sure if the grouping-issue is common, but thought I'd mention it.

-Henri

• ##### A Pointless Exercise

Your message is awaiting moderation. Thank you for participating in the discussion.

Creating internal DSL-s in Java is a pointless exercise. Scala, Groovy, JRuby or another DSL-friendly language should be used instead. My personal preference is Scala. It has limitations on how nice the syntax can be, but it performs as well as Java, unlike the dynamically typed languages.

Your message is awaiting moderation. Thank you for participating in the discussion.

I know a lot of developers are often reluctant to follow a DSL naming convention like expressed in the examples contained within this article. I've recently become a firm believer in method chaining and following what I like to call "DSL Naming Convention".

Good job guys.

Best Regards,
Richard L. Burton III

• ##### Good stuff

Your message is awaiting moderation. Thank you for participating in the discussion.

Very good article. Looking forward to seeing more of the SQL example.

• ##### Re: SQL-Example

Your message is awaiting moderation. Thank you for participating in the discussion.

Perhaps you'd like to have a look at the embedded Java-SQL DSL named JEQUEL.

See www.jequel.de.If someone is interested to join we also have a codehaus project at
jequel.codehaus.org.

Small example:

public void testSimpleSql() {        final SqlString sql =                select(ARTICLE.OID)                        .from(ARTICLE, ARTICLE_COLOR)                        .where(ARTICLE.OID.eq(ARTICLE_COLOR.ARTICLE_OID)                                .and(ARTICLE.ARTICLE_NO.is_not(NULL)));        assertEquals("select ARTICLE.OID" +                     " from ARTICLE, ARTICLE_COLOR" +                     " where ARTICLE.OID = ARTICLE_COLOR.ARTICLE_OID" +                     " and ARTICLE.ARTICLE_NO is not NULL", sql.toString());    }

For some more material on DSLs have a look at the

DSL-Book (Work in Progress)
and the

thoughtworks podcast

by Martin Fowler.
Have fun

Michael

Your message is awaiting moderation. Thank you for participating in the discussion.

Nat Pryce and Steve Freeman did a lot of this java dsl stuff during the jMock and Hamcrest development. Martin Fowler links to the paper regarding jmock at the end of the
expression builder bliki entry.

The blog of Nat Pryce is an invaluable source regarding java DSLs in the domain of testing (i.e. jmock, test data builders, etc.)

There is an ongoing effort of the Quaere Team led by Anders Norås to build a LINQ like DSL for java.

Michael

Your message is awaiting moderation. Thank you for participating in the discussion.

Sorry, I forgot: Guice binding setup is also an internal DSL in java.

The DSL booking example is totally confusing. You have inconsistent syntax mixed with imperative methods (add) and I doubt that a user of such an DSL would know which methods to call on which objects and which Parameter objects to fill in created by the (he has to know this) statically imported methods. There is also a missing paren at the last line (in both boxes).

Something more interesting would be:

Trip trip = on("10/09/2007").flyWith(airline("united").flight("UA-6886")).to(city("Paris").hotel("Hilton")).andReturnWith(airline("united").flight("UA-6887")).on("10/17/2007")

What I also miss is the discussion of the concrete usage of interfaces as intermediary builder objects. This is much more important as it is java's only way to support multiple inheritance (and you need this when composing grammars from reusable expressions). See Martins Book for details.

Michael

• ##### Very Beneficial

Your message is awaiting moderation. Thank you for participating in the discussion.

Great article, I'm sure we will see some if not all of this content in an upcoming DSL book.

Although I would never use an internal DSL in Java as a customer facing DSL, using the builder pattern and method chaining is extremely beneficial for generating test objects in unit tests. It is much more communicative than a simple ObjectMother which hides a lot of the test values inside static factory methods.

Somebody should create a code generation library for generating builders from POJOs. That would reduce a lot of time spent keeping builder code current with the POJOs.

• ##### Re: Very Beneficial

Your message is awaiting moderation. Thank you for participating in the discussion.

The book by Martin Fowler is as cited already in progress.
Nat Pryce did quite a lot with the TestDataBuilders you mention.

I agree that an internal DSL in a language like Java is nothing for customers (neither reading nor writing) although reading may be improved by changing the color scheme of the IDE to fade parentheses, semicolons, brackets, language identifiers as suggested by Nat Pryce, Anders Noras and others.
(see nat.truemesh.com/archives/000188.html)

The generic pojo builder generation thing would only work if you could specify the relevant attributes that shall be a part of the language and also define the associations between your POJOs. Choosing the right names for the fluent interfaces (which often don't correspond to the names of the attributes set but are more natural language constructs) is another problem with a generic approach.

Michael

• ##### Re: A Pointless Exercise

Your message is awaiting moderation. Thank you for participating in the discussion.

I don't agree with you. Please don't forget that there is not only the top notch of the developer crowd you see here at infoq, the other online resources and the conferences. If I look around my workplace there is a lot of people who just do java and have neither heard nor used Scala, Ruby, Groovy and the like. So a DSL in Java is a first step to take in this direction when introducing DSLs and fluent interfaces in an development team/organisation.

The static typing helps a lot when someone learns a new DSL (if you are fluent in your DSL a dynamic language should pose no problem but I don't want to imagine having a customer staring at the error messages of the ruby or groovy runtime due to misspelled syntax).

So editor support is quite important here. You could argue that a tool like the xText facilities of openArchitectureWare (openarchitectureware.org/) would be a better choice their with their full fledged text editor generation support. But they don't generate DSLs for executable languages per se but parsers for creating/reading models.

Michael

• ##### Very nice (but is it a DSL?)

by Ron Kersic,

Your message is awaiting moderation. Thank you for participating in the discussion.

Nice article!

But I’d argue that it is more on fluent interfaces then on (internal) DSL’s. I know the issue of is-it-a-DSL-or-not is a bit of a grey area but my personal litmus test is on the errors and warnings thrown at me when creating a program with the DSL. I expect a DSL to be supported by errors and warnings at the level of the DSL and not at the level of the language implementing the DSL.

In case of the of the booking example, I’d guess that if I type booking.add(airline(“united”)) the error would be that the booking.add method expects a BookableItem (instead of an Airline). But for a ‘real’ DSL I want the error message to state that I need to specify a flight to go with that airline. Likewise, if I’d type booking.add(flight(“UA-6868”)) I want a warning message stating that this flight is interpreted to be by United Airlines and I want to be offered a quick fix to add the airline(“united”) bit.

I guess it comes down to the fact that you need to have some processing on top of what the IDE does on processing Java in order to get at a internal DSL in Java. The Spoon framework (spoon.gforge.inria.fr/) is a relatively easy way of getting such processing in place. I’m building an internal DSL with Spoon (funkydsl.wiki.sourceforge.net/, no code yet alas) and from what I know, the errors and warnings for the booking example above would be very easy to implement with Spoon. Most certainly worth checking out.

\Ron

• ##### Re: Very Beneficial

Your message is awaiting moderation. Thank you for participating in the discussion.

Thanks for pointing out Nat's blog. The entry nat.truemesh.com/archives/000728.html is exactly what I was looking for in terms of using TestDataBuilders instead of an ObjectMother.

As for the generic pojo builder generation, I would be willing to give up a little readability for a lot less effort. This is especially the case for large generated domain models on small projects where the effort required to maintain TestDataBuilders is not warranted. The main goal I'm looking for in TestDataBuilders is conciseness.

public class Car {    private Color color;    private Model model;    private Manufacturer manufacturer;    // Getters and Setters    ...}Car car = car().color(RED).manufacturer(manufacturer("Ford")).model(model("Fusion")).build();

The above code, similar to the article's dreamCar code, can be generated straight from POJOs where the static builderfactory methods manufacturer and model reflect required constructor parameters. Another approach I like is using variable length argument lists with named parameters instead of chaining with() calls:

Car redFusion = car(color("red"), manufacturer("Ford"), model("Fusion"));Car blueIon = car(year(2005), color("blue"), manufacturer("Saturn"), model("Ion"));

• ##### A Great Article

by John DeHope,

Your message is awaiting moderation. Thank you for participating in the discussion.

A fantastic article. I really enjoyed the discussion about how to actually build a fluent API. You see a lot of blog articles about what they are, and what they look like. This was the first article I read that explained some of the thought process that goes into actually building them up from scratch. Nice!

• ##### Another nice example of an internal DSL for integration

by David Greco,

Your message is awaiting moderation. Thank you for participating in the discussion.

Camel defines an internal DSL based on the enterprise integration patterns. Using this nice DSL it's possible to easily define messaging routes among different endpoints.
activemq.apache.org/camel/

• ##### A dsl for collection manipulation

by Mario Fusco,

Your message is awaiting moderation. Thank you for participating in the discussion.

Another good example of DSL is the one provided by lambdaj ( code.google.com/p/lambdaj/ ) in order to manipulate collection without poorly readable loops.

• ##### Fluent builder generator

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi,
we've created a generator that can help DSL'ize your process of javabeans creating:

hope you like it

• ##### Nice article - a few nitpicks . . .

by Peter Bell,

Your message is awaiting moderation. Thank you for participating in the discussion.

Nice article guys! As you pointed out, IDE support (code completion) is definitely a benefit when creating internal DSLs in Java. I also agree that compile time correctness checking is a benefit of internal DSLs in static rather than dynamic languages. However you can get the same benefits in external DSLs. Whether you're using an XML concrete syntax and using a schema to validate or using tooling like Xtext (part of the Eclipse Modeling Framework now - used to be part of openArchitectureWare) you can also get such compile time validations. Of course you get it much more quickly/cheaply in Java, but it's the speed with which you get it that differentiates a Java internal DSL from an external DSL.

I'd also argue against Java being good for end user readability. I find consistently that business users are distracted by the syntactic noise. I think Java can be an option for creating an internal technical DSL (for example Braintree Payment systems used a build based Java DSL to wrap the API for their payment processing service) but would suggest alternate approaches if readability by non-technical users is a key concern.

by Kevin Wright,

Your message is awaiting moderation. Thank you for participating in the discussion.

You state in both the article and in comment replies that only dynamic languages can offer a cleaner DSL. Implying that the verbosity of your approach is inherent to static typing.

Scala, mirah and Haskell would all like to respectfully disagree... Scala is most definitely *not* dynamically typed.

I also think you'll find that your favourite IDE features, such as code completion, are also well supported for Scala now, so it's unreasonable to claim this as another unique advantage for Java.

• ##### Re: Good stuff

by Lukas Eder,

Your message is awaiting moderation. Thank you for participating in the discussion.

I have recently published an article about my new DSL called jOOQ (for Java Object Oriented Querying). Here is the full article on dzone:
java.dzone.com/announcements/simple-and-intuiti...

As a teaser, please consider how you can express this SQL:

-- Select authors with books that are sold outSELECT *   FROM T_AUTHOR WHERE T_AUTHOR.ID IN (SELECT DISTINCT T_BOOK.AUTHOR_ID                         FROM T_BOOK                        WHERE T_BOOK.STATUS = 'SOLD OUT');

in jOOQ's fluent DSL:
create.select(T_AUTHOR).where(TAuthor.ID.in(create.selectDistinct(TBook.AUTHOR_ID)                           .from(T_BOOK)                           .where(TBook.STATUS.equal(TBookStatus.SOLD_OUT))));

jOOQ is specifically designed for a developer who:

• interfaces Java with huge legacy databases.

• knows SQL well and wants to use it extensively, but not with JDBC.

• doesn't want to learn any new language (HQL, JPQL, etc)

• doesn't want to spend one minute fine-tuning some sophisticated XML-configuration.

• wants little abstraction over SQL, because his software is tightly coupled with his database. Something that I think the guys at Hibernate or JPA seem to have ignored.

• needs a strong but light-weight library for database access. For instance to develop for mobile devices.