BT

Testing Impact of Model Driven Development

| by Ben Linders Follow 25 Followers on Oct 23, 2014. Estimated reading time: 3 minutes |

By using Model Driven Development component tests could be skipped and integration and system testing went a lot smoother, said Bryan Bakker in the presentation Model Driven Development (MDD) and its impact on testing that he gave at the International Conference on Software QA and Testing on Embedded Systems. In his talk he showed how model driven development has been used to redesign software that is used to control an electron microscope. The main results from the MDD approach are a reduction of the amount of testing that is needed to validate the software and increased reliability of the code that was generated from a mathematical model.

The behavior of a part of the electron microscope software was modeled with a modeling tool, and the tool was used to check for certain issues like deadlocks, live-locks, starvation, and race conditions. Since this software is a critical part of the system the reliability needed to be validated with testing.

A model must be precise, complete and correct. To ensure this a model checker was used to validate the model and remove any errors. Then the model was used to generate code. Since the code is generated from the model that has been checked, the code can be guaranteed to be correct.

The model consisted of state machines, sequence diagrams, algorithms. It includes the normal flow and also the exceptions and illegal behavior. It contained both internal interfaces and interfaces to the existing hand written code, using a design by contract approach. Bryan stated that it is important to decide which part of the software will be modeled. He also advised to not try to put everything in a model, such a model will become too complicated.

A question was if the code which has been generated needed to be tested? Since the correctness of the code is guaranteed by the model checker, the assumption was that component tests would no longer be needed for the generated code. Integration tests and system tests would still be needed.

The tests that were done were automated using a GUI for test actions and verifications. System testing was done to test the reliability of the system, those tests were executed for a long time to increase the chance that failures due to for instance race conditions would be happen.

Reliability faults were found in testing but none of them were in the generated code. Some faults turned out to be in the boundary between the generated and hand written code, most of the faults were in the hand written code. Conclusion was that the reliability of the generated code is very high due to using model driven development. With the use of model driven development hardware failures and wear were detected much quicker, after days of testing in stead of months. The testing revealed several major design flaws in the mechanical parts.

Main results from the model driven development approach are that that component testing could be skipped and integration and system test went a lot smoother. When a module is redesigned with MDD the likelihood of defects goes down, which reduces the risk of failures. Reliability testing can be used as a measurement, since less failures are found and the testing will confirm that a product is reliable.

There were several additional benefits gained from using model driven development:

  • Reviewing the models turned out to be easier than reviewing code.
  • Design discussions could be done earlier which helped to prevent defects.
  • Refactoring turned out to be relatively simple and reliable.
  • Maintaining the models appeared to be easier than maintaining code.

Model driven development fits well into an agile approach. A sprint would deliver hand written code, parts of the model that could be used to generate code, or both. The model can be demonstrated to the stakeholders in the demo.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

The model still needs to be tested by Shane Hastie

I think an important message here is that the models used to generate the code need to be tested thoroughly - moving the testing emphasis from the code level to the model level.
I suspect a risk is not testing the models properly, which will potentially mean the code which is generated works properly (does what the model says) but the product it produces is wrong.
So model driven development moves the component testing to a different set of artefacts and highlights the need for system testing.

Re: The model still needs to be tested by Bryan Bakker

You are right Shane! It is important to make sure the models are indeed correct, by reviewing and testing them thoroughly. Everything which is incorrectly modeled, will result in the same incorrect behavior in the code (rubbish in --> rubbish out).
Component white box tests do not add a lot, as in fact the code generation part would be tested.
There is no less testing, but different testing.

Re: The model still needs to be tested by Sebastian Gabmeyer

Unfortunately, testing the models (or verifying them) is not enough. If I use a generator to produce executable code from the tested/verified model, I still need to test/verify that my generator doesn't introduce any bugs; otherwise, the results obtained by testing/verifying the model might not hold any longer.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

3 Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT