Peter Ritchie recently raised concern about what he considers a tendency for adherence to TDD and BDD to keep practitioners from writing good unit tests. In particular, it is the mantra of "interaction testing" that he suggests has the effect of creating incomplete unit tests; tests that fail to show proof that a unit - an object - works under any conditions it could potentially be used. Most interesting about Peter's thoughts may be the counter view they present to the core intentions of TDD and BDD in the first place.
Peter uses the idea of classes being autonomous abstractions of real-world concepts as a foundation for his thoughts that good unit testing is about verifying these autonomous classes and further that TDD and BDD may lead people against this:
Peter uses the idea of classes being autonomous abstractions of real-world concepts as a foundation for his thoughts that good unit testing is about verifying these autonomous classes and further that TDD and BDD may lead people against this:
One problem I see with Test-Driven Development (TDD) and Behaviour-Driven Development (BDD) is that practitioners simply just center on interaction of the parts of the system and really don't do any "unit testing". They get caught up in the mantras that are TDD and BDD and fail to see the trees for the forest and fall into testing by rote. Unit testing tests individual units, the smallest testable part of an application.In reference to an example found on Wikipedia's BDD entry, Peter presented the following in support of his case:
The tests detailed only test 13 of 4,294,967,296 possibilities. These tests may very well test the expected behaviour of one system, but don't really test EratosthenesPrimesCalculator as a unit. If the system only allows that behaviour, then these tests prove that it will work. But, if at some point EratosthenesPrimesCalculator is used outside that behaviour (and that's really the purpose of encapsulating code into classes: reuse) not much about EratosthenesPrimesCalculator has been validated.In large part, the example relies on the thought that a unit's usefulness/correctness is purely an inherent quality of what it's name may imply in the real world. It is this point that many TDD practitioners may challenge, primarily based on the idea that a unit's usefulness is definable only in the context of the environment (system) that is being used. Further, it is the desire to expose these "uses" that TDD/BDD practitioners propose is the primary benefit of the interaction testing Peter refers to in his post. Steve Freeman, co-creator of JMock:
The idea [of test-first interaction testing] is to drive out the relationships that an object has with its environment. For example, you're mocking a DAO, but a DAO is not part of the domain of the application, it's part of the domain of the implementation.Said another way, many in the TDD school would counter that the primary utility of writing the unit test in the first place is that it is as the explicit specification of what a unit should or should not do. From a discussion comparing TDD to Design By Conctract Mario Gleichmann:
Unit tests as an instrument in the sense of Test Driven Development (TDD) aren’t that much about verification of a correct implementation rather than about a specification of how a unit should behave. in fact, it’s the specification (not the verification), that will drive development. You can see the comeback of this core idea in the rise of Behaviour Driven Development (BDD) that mainly tries to find an adequate vocabulary to write down specifications (that of course can be verified automatically) in an easy natural way, refocussing on how a component should behave under certain conditions.An often cited corollary to this "a unit is defined by it's context" view is the reminder that unit testing, by definition, does not provide indication to the quality or utility of the system as a whole, and as such should be accompanied during development with adequate levels of acceptance testing. JS Greenwood:
Poor integration tests, as everything is being tested in isolation - we can end up with a system where the constituent parts are clean, isolated, well tested, and known to be correct. But how they fit together is a greyer (or even blacker) area unless [isolated unit testing] is accompanied with a complement of integration tests.