Giving Up on TDD
This post summarizes the experience of a university professor who gave up on TDD and Uncle Bob’s rebuttal of his arguments.
Ian Sommerville, a retired university professor of software engineering, has written several books including Software Engineering, 10th edition. Chapter 8 of the book is dedicated to Software Testing, and sub-chapter 8.2 to TDD in particular. Reiterating some of the ideas in the respective chapter, Sommerville said in a recent post “TDD is a an important step forward in software engineering. There are some classes of system where it is clearly appropriate,” listing the following systems as “TDD-friendly”:
- A layered architecture
- A system with agreed success criteria, building the tests around these criteria.
- A controllable operating environment, not having to interact with systems beyond one’s control.
TDD works well for such systems, added Sommerville, but he found it hard to use for other types of systems:
I started with TDD on a class of system that met these criteria and I liked it. It worked well for me. Then, I moved on to a system which was concerned with visualizing complex linked structures. Now, the thing about visualization is that (a) it’s often much more difficult to have clearly separate layers – the UI is the program and (b) it’s very hard to have pre-defined success criteria – you basically have to program by experiment and see what works and what doesn’t. TDD didn’t work.
Sommerville also wrote the post Giving up on test-first development, sharing his recent experience with TDD on a personal project. He found TDD helpful in the beginning, but later, when starting working on the GUI, he found out that it got harder to write tests and he did not think the time spent writing tests was spent well. As a result, he sometimes wrote implementation code before tests and added some tests later. Eventually he gave up on TDD, explaining himself:
I’m not giving up on TDD because of the (well-known) difficulties in writing automated tests for GUIs. I’m giving up because I think it encourages conservatism in programming, it encourages you to make design decisions that can be tested rather than the right decisions for the program users, it makes you focus on detail rather than structure and it doesn’t work very well for a major class of program problems – those caused by unexpected data.
Sommerville continued with more detail:
- TDD encourages conservatism: one is more reluctant to make major changes to an application knowing that many tests will be broken and they will have to be rewritten.
- “Sometimes, the best design is one that’s hard to test”.
- TDD places a focus on detail rather than the whole. “With TDD, you dive into the detail in different parts of the program and rarely step back and look at the big picture.”
- It’s hard to write tests for unexpected data.
So, he decided to write code first, tests second. But his post did not go unnoticed. Robert C. Martin, aka Uncle Bob, weighed in with a rebuttal of Sommerville’s arguments on giving up on TDD. Uncle Bob thinks Sommerville experienced the beginner’s troubles and he gave up too soon. The reason why changes in one component of the system continually breaks another component is because the two parts of the system are tightly coupled together. When one is afraid to refactor the code because it will break all the tests, it means the tests are too coupled with the production code. One should organize the system into independent layers, and each layer should provide access via interfaces, these interfaces being also used by tests.
Uncle Bob discarded the idea that “The best design is one that’s hard to test,” considering it rubbish. A design that cannot be tested probably won’t work when one needs it. For difficult cases, such as testing a GUI or a device driver, he mentioned the option of using The Humble Dialog Box pattern.
Uncle Bob does not agree that TDD places too much emphasis on detail. He argues that a developer must always design:
No matter what you are writing; whether a unit test, or an acceptance test, or production code, or a mock, or a stub, you have to DESIGN….
We are programmers! We design! We create structures with high cohesion and low coupling. We manage dependencies. We isolate modules. WE. DESIGN.
Programmers sometimes forget to design, continued Uncle Bob. Especially novices, in any field, tend to focus on certain low level details because they are in the process of learning. He stressed the importance of having in mind the larger picture by bringing to remembrance Ron Jeffries’ -co-author of XP and an Agile Manifesto signatory.-advice: “Act locally. Think globally.”
Finally, Uncle Bob agrees that is quite difficult to account for unexpected cases, but that is not TDD’s fault:
Apollo 1 caught fire. Apollo 13 exploded half-way to the Moon. Challenger blew up just after launch. Columbia broke apart during reentry. Why? Because despite the thousands of brains trying to think of everything, something unanticipated happened….
Deal with it. There will always be risk. Don't blame TDD, and don't give up.
Israel Fermin Montilla, a Software Engineer, took a more conciliatory position, commenting on Sommerville’s post:
I believe the thing about writing tests is not to write them first or last, is just write them and have the discipline to maintain them and be strict about the amount of code coverage you want for a certain project. Write them first has obvious benefits, but it’s a technique that might not work for everybody or for every project. In my experience, there are sometimes some time constraints which forces you to implement fast and then test only the critical bits. I’ve work only for startups and internet companies, so, it’s a very fast paced/marketing-focused environment, so, sometimes you acquire huge amounts of technical debt (specially on startups on its early stages) for the sake of releasing and gather data to evaluate.
Uncle Bob being Uncle Bob
Then Uncle Bob responds with his usual defensiveness, affirming that he didn't try enough (aka he is lazy), that he shouldn't blame TDD for his failure (aka he is incompetent). And does this by a blog post in the form of a imaginary conversation with someone that asked questions that aren't at all the critics made by Sommerville in the original article. Talking with himself. This could be signs of senility, he should see a doctor ASAP! :)
I fully agree with Sommerville. I've seen very good developers making very convoluted code, just so that it is easier to test (due the technology used, due to the kind of system).
Testability is a good characteristic to have in a system, no doubt. But it is not the most important, just another one. The problem with Uncle Bob, and other TDD preachers, is that they want us to believe that TDD intrinsically lead to other qualities, or that it never 'competes' with other qualities, which simply is not true.
Well, I still admire Bob for his knowledge, experience and ideas, but I definitely won't be a TDD convert any time soon :)
Yousef Awad May 16, 2016
Jason McGee of IBM Talks about Open Source Projects and the Interactions at the Collaboration Summit
Jason McGee May 15, 2016
Srini Penchikala May 15, 2016