BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

TDD/BDD Leading To Incomplete Unit Tests?

by Mike Bria on Feb 14, 2008 |
Peter Ritchie recently raised concern about what he considers a tendency for adherence to TDD and BDD to keep practitioners from writing good unit tests. In particular, it is the mantra of "interaction testing" that he suggests has the effect of creating incomplete unit tests; tests that fail to show proof that a unit - an object - works under any conditions it could potentially be used. Most interesting about Peter's thoughts may be the counter view they present to the core intentions of TDD and BDD in the first place.

Peter uses the idea of classes being autonomous abstractions of real-world concepts as a foundation for his thoughts that good unit testing is about verifying these autonomous classes and further that TDD and BDD may lead people against this:
One problem I see with Test-Driven Development (TDD) and Behaviour-Driven Development (BDD) is that practitioners simply just center on interaction of the parts of the system and really don't do any "unit testing". They get caught up in the mantras that are TDD and BDD and fail to see the trees for the forest and fall into testing by rote. Unit testing tests individual units, the smallest testable part of an application.
In reference to an example found on Wikipedia's BDD entry, Peter presented the following in support of his case:
The tests detailed only test 13 of 4,294,967,296 possibilities. These tests may very well test the expected behaviour of one system, but don't really test EratosthenesPrimesCalculator as a unit. If the system only allows that behaviour, then these tests prove that it will work. But, if at some point EratosthenesPrimesCalculator is used outside that behaviour (and that's really the purpose of encapsulating code into classes: reuse) not much about EratosthenesPrimesCalculator has been validated.
In large part, the example relies on the thought that a unit's usefulness/correctness is purely an inherent quality of what it's name may imply in the real world. It is this point that many TDD practitioners may challenge, primarily based on the idea that a unit's usefulness is definable only in the context of the environment (system) that is being used. Further, it is the desire to expose these "uses" that TDD/BDD practitioners propose is the primary benefit of the interaction testing Peter refers to in his post. Steve Freeman, co-creator of JMock:
The idea [of test-first interaction testing] is to drive out the relationships that an object has with its environment. For example, you're mocking a DAO, but a DAO is not part of the domain of the application, it's part of the domain of the implementation.
Said another way, many in the TDD school would counter that the primary utility of writing the unit test in the first place is that it is as the explicit specification of what a unit should or should not do. From a discussion comparing TDD to Design By Conctract Mario Gleichmann:
Unit tests as an instrument in the sense of Test Driven Development (TDD) aren’t that much about verification of a correct implementation rather than about a specification of how a unit should behave. in fact, it’s the specification (not the verification), that will drive development. You can see the comeback of this core idea in the rise of Behaviour Driven Development (BDD) that mainly tries to find an adequate vocabulary to write down specifications (that of course can be verified automatically) in an easy natural way, refocussing on how a component should behave under certain conditions.
An often cited corollary to this "a unit is defined by it's context" view is the reminder that unit testing, by definition, does not provide indication to the quality or utility of the system as a whole, and as such should be accompanied during development with adequate levels of acceptance testing. JS Greenwood:
Poor integration tests, as everything is being tested in isolation - we can end up with a system where the constituent parts are clean, isolated, well tested, and known to be correct. But how they fit together is a greyer (or even blacker) area unless [isolated unit testing] is accompanied with a complement of integration tests.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Isn't it some kind of continuum? by Torreborre Eric

My experience of BDD is that specification/verification/testing is a continuum. You start by defining what you want to achieve, then you implement it: that's the specification phase. Then you specify all the implicit/hidden/forgotten assumptions about how your unit should behave: it's the testing phase.

This is why in the specs library there are different techniques for specification and testing:

- a BDD-like vocabulary:
"my system" should {
"behave like this" in {...}
"behave like that" in {...}
}


- a ScalaCheck integration to be able to thoroughly test properties and generate a lot of the missing cases. You can find a blog entry here

- a JMock integration to isolate the relevant parts.

Eric.

Unit Testing is not Test Driven Design by Mario Gleichmann

I think we should distinguish the concepts of pure 'Unit Testing' from 'Test Driven Design'(as they are used interchangeably in the post).
While (Unit-) Testing is in fact not about specifying the intented behaviour of a component (but ensuring that the affected unit under test works properly under any conceivable situation / condition), Test Driven Design is more about 'Design' than 'Test' (at least initially intended as such). Under this Point of View, BDD is only a 'trend' trying to refocus TDD from 'Test' to 'Design' (resp. specifying the Design by the intended behaviour)
I think a good part of the mentioned confusion / complaints is coming from mixing these two ideas.

Greetings

Mario

Re: Unit Testing is not Test Driven Design by Erick Dovale

I couldn't agree more with you Mario.
I also fail to see how not testing for boundary cases is in any way related to TDD or BDD for that matter.

I wish I had this problem by Bruce Rennie

Not to be flippant but if I had this problem, boy, would I be in a happy place. I can't wait. :)

Re: Unit Testing is not Test Driven Design by Al Tenhundfeld

Automated tests are a component of TDD, but automated tests are not necessarily unit tests.

BDD/TDD automated tests should focus on the API (external behavior) of a class. They are fairly dumb and ignorant of the class internals.

Unit tests should focus on covering all of the code and can be written to test specific aspects of the internal class implementation.

That's my general distinction. However, I can understand the article. It's easy to get into the habit of writing your tests, implementing your class, refactoring, etc. and then moving on to the next use case/user story/behavior/whatever. That's part of the reason why code coverage tools are so useful; they can remind you that you neglected unit tests.

-Al

Re: Unit Testing is not Test Driven Design by Michael James

I have some concerns that unit testing is seen as the be-all and end-all of automated testing. Certainly it's a good start, but failing systems are often built from perfectly good components.

more here:

danube.com/blog/michaeljames/junit_is_not_just_...
and
danube.com/blog/michaeljames/mock_objects_consi...

--mj

The Wikipedia page by Elizabeth Keogh

is now updated. Hopefully the new contents may help shed some light.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

7 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT