In Automating "All" Tests Ron Jeffries discusses the ideas that:
- No one really automates all tests
- No one really could automate all the tests that are needed
- It would be cost prohibitive to do so
- There are many important examinations of the product that can only be carried out "by hand", under the watchful eye of someone with good intuition, good product knowledge, and a generally skeptical nature.
His advice is to set out to automate all tests, accept the fact that you'll never get there, but that you will definitely improve the quality of your work along the way.
Community comments
Testing "Everything that could possibly break" is a game of balance
by Dan Bunea,
The cost of maintenance...
by Deborah (Hartmann) Preuss,
Re: The cost of maintenance...
by Alex Popescu,
Re: The cost of maintenance...
by Deborah (Hartmann) Preuss,
Testing "Everything that could possibly break" is a game of balance
by Dan Bunea,
Your message is awaiting moderation. Thank you for participating in the discussion.
I wrote on the same discussion list:
"Automated test suites, have the great gift of fast feedback: I call them
break detectors. The more break detectors you have, the better the
feedback, but there can be cases with too many tests. Is there such a
thing like too many tests: yes, because tests are code after all and the
rule of running from complexity applies with tests also. Too many tests,
can be a case where you have many tests and they are hard to change. A
suite of tests ensures in agile projects the possibility to make changes
faster, to have more courage. When this advantage is gone, and it is hard
to change the tests, you have duplication in them, they are too complex,
then there's a problem. This kind of problems cannot be explained to
someone that never heard of agility, or has little idea.
Automated tests can be a great weapon. But as a friend of mine said, the
more powerfull the weapon the bigger the chances are for you to kill
yourself with it, if the attention doesn't counterbalance the power.
Probably how much automated testing should there is a complex matter of
balance. Automated tests should be used with a very clear goal, and always
keeping in mind that they can become easily a problem themselves if not
used right."
The cost of maintenance...
by Deborah (Hartmann) Preuss,
Your message is awaiting moderation. Thank you for participating in the discussion.
Scott, to Ron's list:
* No one really automates all tests
* No one really could automate all the tests ...
* It would be cost prohibitive to do so
* There are many important examinations of the product that can only be carried out "by hand" ...
I would add:
* Even if they tried to "write all tests", what team can afford to maintain them?
deb
Re: The cost of maintenance...
by Alex Popescu,
Your message is awaiting moderation. Thank you for participating in the discussion.
... and after this one, I would add:
* Even if they write them all, who is gonna pay for all that work and for their maintenance.
./alex
--
.w( the_mindstorm )p.
Re: The cost of maintenance...
by Deborah (Hartmann) Preuss,
Your message is awaiting moderation. Thank you for participating in the discussion.
So, now that we are all agreeing with Ron - anyone know other good references on the subject? I'm relatively new to the subject myself. --deb