BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

TDD: Essential Skill or Architectural Landmine?

by Deborah Hartmann Preuss on Feb 20, 2008 |
At JAOO '07 Bob Martin asserted: "it is irresponsible for a developer to ship a line of code he has not executed in a unit test." In this twenty-minute InfoQ video, Coplien and Martin Debate TDD, CDD and Professionalism, filmed at JAOO 2007, Martin debated with another well respected software thought leader, Jim Coplien, on this and other topics. Discussion covered the definition of Test-Driven-Design (TDD), whether TDD imposes a procedural approach on o-o design, Design by Contract vs. TDD and how much up-front architecture is needed to keep a system consistent with the business domain model.

Both men are careful about their words and ideas, as displayed in their courteous discourse and the care taken to be clear about terms and concepts. Coplien invited Martin to set the stage by sharing his specific definition of TDD:
Bob Martin: So, I have 3 laws of test driven development:
  1. a test driven developer does not write a line of production code until he has written a failing unit test, and no production code can be written until there is a failing unit test;
     
  2. you do not write more of a unit test than is sufficient to fail, and “not compiling” is failing. So you cannot write very much of the unit test before you must write production code;
     
  3. you cannot write more production code than is sufficient to pass the currently failing test. So you cannot write a little bit of a unit test and then run off and write a whole bunch of production code.

These 3 laws lock you into a cycle that is perhaps 30 second long, and that means you are actually writing test and production code concurrently, with the tests perhaps 30 seconds to a minute ahead. That is my definition.

Although Coplien has been taking issue with XP and TDD in recent months, he asserted that he had no problem with any of these basics! For Coplien the problem seemed to lie in how these are applied within projects, particularly in relation to architecture and usability.

Coplien: ...today the units of organization of the software are objects and we’re testing procedures and there is a little bit of a mismatch. ... one of the things we see a lot, in a lot of projects, is that projects go south on about their 3rd sprint and they crash and burn because they cannot go any further, because they have cornered themselves architecturally. And you can’t refactor your way out of this because the refactoring has to be across class categories, across class hierarchies, and you no longer can have any assurances about having the same functionality.

The other problem we’ve seen is that this destroys the GUI ... you have this procedural architecture kind-of in a JAVA class wrapper; you no longer are driving the structure according to domain knowledge and things that are in the user’s conceptual model of the world, which is where object orientation came from...
After looking at where the problem lies, they did briefly look at the issue of whether TDD is an essential skill for anyone calling themselves a professional programmer these days. Coplien categorically disagrees with this proposal, however when Martin walked him through the creation of a fictitious new system, they didn't differ much in how they would approach it. It almost looks like the trick lies in calling it, or not calling, it TDD - the practices are sound enough in the abstract, but the term TDD itself seems to carry baggage that offends some sensibilities.

Coplien admitted that their positions are not so far apart, summing up it this way:
I think when I am running into people that are doing things right, that avoid the kind of problems [we] talked about earlier, it’s not TDD out-of-the-book or TDD out-of-the-box. So, people have found a way to move to what Dan North now calls BDD, for example, which I think is really cool (if you ignore the RSpec part and all the stuff which is kind of dragging it back to too low of a level).
The short dialogue also touched on Design by Contract and Contract-Driven-Design vs. TDD.

View this short video dialogue between two respected software developers and authors: Coplien and Martin Debate TDD, CDD and Professionalism

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Are we Growing or Building Systems? by Juan Bernabo

Hi,

I just wanted to say that I´m a fan of both guys, I´m also an architecture freak that is learning to try to let it go, and do architectural decisions to be made in a more just in time way.

When Jim talks about TDD he talks about his assumption on TDD as testing units like in isolation not connected with business value, but pretty much TDD is evolving to include and start with the tests that drive an user story, pretty much like an executable specification, and not as just a unit test of a class in isolation, but exercising all the architecture by each functional test that specifies a user scenery.

I think the bottom of this discussion is what is the right approach build or grow a system?

And if we grow it, can we refactor the architecture as nature does when a tree grows from a seed to it´s final form?

Why nature goes refactoring things, could it simply go trying to build trees instead of growing them?

Christopher Alexander in his "The Nature of Order" collection talks about a process for architecture were they are generated and not searched, he even gets to say that alive or valid configurations were mathematically not possible to get by searching the configuration space, so the ONLY way to have systems that are healthy requires a generative approach.

So he talks about two processes of creating an architecture, by search, something done upfront, and something done by generation, what he calls whole extension transformations, pretty much what I think is TDD, the only trouble with this is that the next transformation must be decided by us just in time "by evaluating the level of life that particular part of the system has", this is made by looking into ourselves and feel if that make us feel more alive as more beauty in it.

I think this skill is not teached, and most of the great developers and architects that I know, they can tell, or evaluate this just by looking at code and say this "sucks" or this is beautyfull, but most developers don't use this information to guide their design decisions, and that maybe is getting us into trouble with TDD.

Today technologies allows us to do architecture agnostic code, by using late wiring and IoC, so much more easily than 10 or 15 years ago we can effectively delay the decision about how to wire things even as late as in runtime. This allows me to be free on some of the decisions about architecture, maybe this were architectural decisions already made.

So maybe Agile as in other disciplines is getting us to rethink our assumptions or methafors about all disciplines and we got to architecture.

Probably architects should work in code driven by most important or architecture significant user stories or use cases to implement them and evolve architecture in a test driven way to evolve and stabilize them before they can talk about a settled architecture and look continuously for opportunities.

Juan.

Re: Are we Growing or Building Systems? by Henri Frilund

I like gardening and I think there's some analogy in gardening to making software systems :)

To get tomatoes from your own garden, you can't really build them. But it doesn't work that well if you just throw tomato seeds everywhere and start waiting either. Well yes, you might eventually get a tomato strand that will grow by itself, but it'll take a long time and you'll probably get really small tomatoes.

So what I do when I want big tomatoes fast is I build an environment upfront that makes it possible for my tomatoes to grow fast and after that I nurture them by giving water etc.

I think this principle applies for software making quite well because you really don't usually have the time to wait for the architecture to "grow" naturally so you set up an environment (domain model, main contracts, architectural borders etc) and then start "growing" the system using TDD or WhateverDD. This applies especially if the underlying domain is established (which it usually is).

-Henri

DbC an aspect of TDD by Michael Sawicky

It is great to see these topics come into broader conversation; thank you! These days I am more and more surprised that we continue to frame Design by Contract and TDD unit testing as “doing virtually the same thing.” These practices are complementary, not exclusive of one another.

The unit testing aspect of TDD provides specification-context that informs design and helps us control the increments of our design/development cycle. DbC on the other hand is a concise way to inspect and enforce design detail – using a mechanism that is more concise and efficient than external unit tests. Although it’s true that unit tests can be used to do what DbC does, it can be a lot more work to do so.

DbC also provides value beyond the context of unit testing since assertions continue to be exercised during system testing and functional testing. Actually, when given a robust implementation, assertions can also be available for analyzing faults in production systems.

Perhaps it would be helpful to describe DbC as an *aspect* of TDD, with the best “TDD value” being achieved when both unit testing and DbC are combined. Who knows, maybe having the discipline to use both practices together is a sign of a “highly evolved” professional?

;-)

Thanks again for covering these interesting topics!
-MSaw

Growing vs Building, Bottom up vs Top down by Paul Beckford

Firstly great article. This type of stuff is the meat of what programming is all about and more articles like this would be welcomed. Thanks InfoQ.

Bob and Jim have highlighted the issues as have most of the posters. I think it isn't either/or and that you need to do both. Where XP stepped in is it re-addressed the balance after years of focusing on top down methodologies like Booch, OMT, RUP etc. The truth is you often need very little top down design/architecture and that design should be validated through real stories and bottom up (TDD) code. I've heard this called producing a "walking skeleton" in the earlier iterations that provides a first pass architecture and provides a framework to hang other stories on TDD style.

From what Jim has said, it sounds as though he does something similar. I never really understood Kent Becks Metaphor idea much, but an architectural "skeleton" is an easier idea to grasp and can serve the same purpose by communicating the "design in the large" to the entire team.

To me the power of validating top down decisions (architecture) with bottom up code (TDD), is that you will get signs that your architecture doesn't fit your problem, very early on. These will manifest themselves as "architectural smells", and these smells provide an opportunity to evolve your architecture to something better (often simpler) at very low cost, since you haven't made a huge commitment (deferred commitment).

Couple of points that haven't been raised yet:

1) Domain Driven Design (DDD) plays well with TDD/BDD. Your "ubiquitous domain language" and high level domain model is often the starting place for your top down architecture/design, again validated bottom up.
2) Ron Jefferies website is a great place to learn how to do TDD right from an expert. TDD provides opportunities for learning an discovery by "listening to your code". This "listening" is a skill and goes way beyond any hard and fast rules. To develop skill in anything takes time and practice:

www.xprogramming.com/

Ron discusses these issues also, and explores them through "experiments in code". He calls what Jim calls "architecture" "programming by intent". Which is a name I like because it stresses the fact that you think you know what your code should do. After doing TDD for a number of years I am still surprised by just how often my intent is actually sub optimal or just plain wrong.

Paul.

More on the TDD Controversy by Martin Jul

TDD will ruin your architecture! When Jim Coplien said this in his presentation about Scrum and Architecture at the JAOO 2007 conference he really got people's attention. His statement sparked one of the most passionate debates at the conference. I posted a summary of the discussion on our blog here: The TDD Controversy.

Fantastic by Stuart Charlton

Two of my heroes!

Jim's points should be remembered: Good architecture requires "stable intermediate forms" to evolve: structure matters! Secondly, you can't create an structure without denoting what it means to some degree. But as Bob says, just don't go too crazy with it.

As for the 'professionalism' angle, I'm concerned this is rather harsh. I fear that it's true, though.

The vast majority of software developers in my experience across multiple industries don't practice TDD, or if they do, they only make a token effort of it. Does that make them unprofessional? Perhaps. I'm not sure what good would come out of pointing it out.

If you want to change the behaviour of the industry, start with providing incentives to IT hiring managers and project managers whose mindsets lead to dysfunctional behaviour in their projects. Calling their teams "unprofessional" might scare them into action, or it might just make them defensive ("we don't need the best, just what we can get" is one line I've heard numerous times.)

Cedric Beust weighs in on Test-First, -Last, etc. by Deborah Hartmann

I noticed that in our recent video on Design for Testability, Cedric Beust raised the topic of TDD, calling it "the elephant in the room." He asked those who don't write all their tests up front:
Don't you feel a bit dirty? What with all those books ... telling you you should be doing TDD, and otherwise you are not professional?

Is it just me? I try to do it... sometimes it feels right, sometimes it doesn't..."

... am I doing something wrong, or is it ok?


Of his own work on TestNG he notes that, although he does value TDD he actually seems to write only 10% of his tests up front, and he went on to look at some of his concerns with TDD.

You can view this part of the talk around 32 minutes into the presentation.

Cause and effect by Gordon Reynolds

I'll start by admitting I haven't yet watched this debate, so I'm really just reflecting on the statements I see in other replies to this thread, e.g., that TDD/GUT = professionalism.

I have been trying - in vain - for months, if not years, now to introduce a unit testing "ethic" in the team I work with. Easily the most common "reason" given is that there "just isn't time". But I detect an undertone of skepticism that is really the root of the problem: the developers here just don't see any positive value in it.

What I finally realized is that it is insufficient to simply state that (1) it's an industry best practice, that (2) the likes of Martin Fowler and Robert C. Martin are advocates of the practice, so (3) please unit test. All such arguments are bound to fail in one way or another without first discussing what objective you hope to achieve by adopting the practice - and by showing that what developers are doing now FAILS to achieve those objectives, e.g., fewer bugs, better design, refactoring freedom. Advocating a practice in the abstract is tantamount to saying "just do it because I say so".

So, IMHO, professionalism is not defined by practices, it is defined by the objectives those practices hope to achieve, and the mindset of the developer who adopts them. After all, it is fairly easy to generate useless unit tests; does that make you a "professional"?

CDD vs. TDD by Tim Stewart

Contracts from CDD are passive in that a contract violation is caught while running the product. Someone or something still has to exercise the code to catch the contract violations. That something might be a unit test. If you choose to exercise the code manually, you have your work cut out for you as you try to exercise all of the branches. CDD by itself is not enough. You need people or other code to exercise the system. If you use people, you need to document the various interactions required to test all of the branches of the code.

BTW, when a contract violation occurs, can you continue running the product or is some exception thrown? Also, when a contract is violated, you may have a hard time tracing down why the contract failed. TDD addresses this very nicely by having isolated, stateless tests.

Tests from TDD are active in that they can be run automatically. When a test fails, subsequent tests can still be run. When a test fails, it's easier to find out why.

Re: CDD vs. TDD by Johannes Brodwall

I have also only seen "passive contracts". But Cope hints in this interview (and elsewhere) about teams using some sort of tooling to exercise the contracts. I would very much like to see these tools demonstrated.

My second difficulty with CDD is the fact that many methods are coupled in some way. Even with the humble Stack.push, how do you write a sufficient contract that doesn't mess up the object state in the process? (If you consider an answer, remember both to check "Stack.top == argument" and "Stack.pop; Stack.top = @pre.top")

Re: CDD vs. TDD by Steven Shaw

CDD sounded a lot like QuickCheck to me. These 'checks' are not performed at runtime but runs as 'tests'. Instead of choosing specific values to test, 100s or 1000s of test cases can be generated that conform to the constraints (preconditions), code is then run and failure to match the postconditions results in a failure where the generated values are reported. At least that's my understanding. QuickCheck like test frameworks are available in many languages.

Re: Cedric Beust weighs in on Test-First, -Last, etc. by Esko Luontola

"Of his own work on TestNG he notes that, although he does value TDD he actually seems to write only 10% of his tests up front, and he went on to look at some of his concerns with TDD."

Could that and this be somehow connected? michaelfeathers.typepad.com/michael_feathers_bl...

Also I don't agree with his notion that TDD would be useful only for beginners. On the other hand, I think that it requires lots of skill and practice to be able to use TDD effectively. Corey Haines sums it up quite nicely at www.vimeo.com/groups/7657/videos/3756344

Re: Cause and effect by Dan Austin

Watching this debate years later, I'm motivated to respond to this comment by the mention of Fowler and "Uncle Bob" in such reverent tones. These two individuals have made a career out of positioning themselves as thought leaders. The sheer arrogance of the moniker "Uncle Bob", the 8th Light "you can be a master like me" vibe is sickening. I find it particularly galling that in this interview, Martin dares to directly say, in effect, that anyone who doesn't follow the brain droppings of people like him and Fowler should be considered "unprofessional." People like you swallow that hook, line, and sinker, and then badger your "less professional" coworkers. These people have caused untold damage in their misguided attempts to promote the One True Way of development, with themselves as the guiding lights. Coplien here seems much more practical and balanced. Martin and Fowler should dry up and blow away. Balance in all things. TDD is not The Way(tm). Software is too complex to have a single best way to do everything. Anyone who says otherwise is a moron, despite any good ideas they may have.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

13 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT