High abstraction level of DSLs to reduce the testing burden?

Posted by Sadek Drobi on Oct 18, 2007

Sections

Enterprise Architecture,

Process & Practices,

Development,

Architecture & Design

Topics

Domain Specific Languages ,

Agile ,

Unit Testing ,

Modeling ,

Design ,

Agile Techniques ,

Architecture

Tags

TDD ,

Business Models ,

Collaborative Technologies ,

Language Oriented Programming

Inconsistencies between the user interface and user’s expectations, i.e. the user model, are an important source of bugs. In absence of comprehensive information on how the software works, the user will have an impression that the its behavior is unpredictable. According to Leonardo Vernazza, this results from the fact that the user and the UI do not talk the same language and therefore need translation.

He asserts in his recent blog post that “the bigger the difference between the languages, the bigger is the probability to produce translation errors between them” and the bigger is the testing burden to ensure the consistency of the translation. Hence, in many cases the right approach would be to offer a “high abstraction language” that both the user model and the user inerface would share, i.e. a DSL. This would be instrumental for avoiding translations and the nececessity to test them. More generally speaking, argues Leonardo, upgrading the level of abstraction reduces the testing burden and this can apply to any other models, e.g. software design, implementation, etc.: 

when you express things at the right abstraction level, testing is useless, […] Would you test that the += operator is working properly in C# or Java?. I wouldn't.

Andres Aguiar has also been arguing on his blog that if enough efforts are put in refining the abstraction, there would be no need for TDD and even for Unit Testing. However, some issues should be taken into consideration.

Using DSLs in order to reduce the testing burden requires a thorough testing of the DSL itself. Leonardo Vernazza asserts that “in most of the cases this is a far simpler and better limited task than testing all the user interfaces”. And such approach allows to “redistribute responsibilities”, moving the testing burden onto DSL authors.  Gareth Jones however believes that “many DSL authors won't test a very high number of language variants against their code generators.”

Commenting on Andres Aguiar’s post, Scott Bellware also highlighted, that determining “the inherently optimal level of abstraction” is not an easy task whereas Brad Head points out that this is precisely what TTD is about: “challenging your design abstractions by "trying it out"”. Moreover, Ron Scott argues that TDD is instrumental for finding out whether changes in code break something. Bellware stresses indeed that a design-tuned DSL is “statically locked down” and would not deliver the same value if the business or technology environment evolves.

So what do you think about abstraction vs. testing? Can the use of DSLs reduce the testing burden?

1. Back to top

   it's just a new automation strategy

   by jiang yan

   express both requirements(or say use cases) and testcases in DSL, will of course bring us lots of good effects.
   phase consistent, keywords index, or tags( for category).
   
   but speaking of test execute, which means automation,
   using DSL is nothing more than a new strategy.
   
   Before DSL, we use data-driven strategy, and when everything goes into table, it really looks like a DSL draft.
   actor action result
   admin login index.page login successful
   admin delete something things are gone.
   
   now we have DSL, sequences of a table will be cut into cases/steps, but still. nothing changed.
   
   Problem we face, we still need to face.
   
   Maybe, now we don't have testers to do all these development work, we blame DSL-coders when things go wrong. :)

   Reply

• Older >