What is a good test? How do we know if we're writing good tests?
Kent Beck posited, that tests should be:
- Isolated (unaffected by the presence, absence, or results of other tests)
- Automated
- Quick to write
- Quick to run
- Unique (providing confidence not provided by other tests/uncorrelated
with other tests)
Roy Osherove adds that good tests have three fundamental properties: maintainable, trustworthy and readable.
Mike Hill has much longer list:
- It is short, typically under a dozen lines of code.
- It does not test the object inside the running app, but instead in a purpose-built testing application.
- It invokes only a tiny portion of the code, most usually a single branch of a single function.
- It is written gray-box, i.e. it reads as if it were black-box, but sometimes takes advantage of white-box knowledge. (Typically a critical factor in avoiding combinatoric issues.)
- It is coded to the same standard as shipping code, i.e. the team's best current understanding of coding excellence.
- In combination with all other microtests of an app, it serves as a 'gateway-to-commit'. That is, a developer is encouraged to commit anytime all microtests run green, and discouraged (strongly, even nastily) to commit otherwise.
- It takes complete control of the object-under-test and is therefore self-contained, i.e. running with no dependencies on anything other than the testing code and its dependency graph.
- It runs in an extremely short time.
- It is generally written before the code-change it is meant to test.
- It avoids most or all usage of 'awkward' collaborators via a variety of slip-and-fake techniques.
- ...
Mike and Ron Jeffries remind us that the key value of TDD is simplifying design and improving productivity. Improvements in code quality and a reduction in bugs are an important side effect.
Jeremy Miller adds that good unit tests should be:
- Order Independent and Isolated - it should be possible to run the tests in whatever order the test runner chooses.
- Intention Revealing - the best unit tests make it clear to the reader how an objects API is intended to be used.
- Easy to Setup
Finally Ed Burnette writes: Make your unit tests repeatable in every aspect; Test your boundary conditions and Always keep your tests passing at 100%.
Community comments
What the qualities of proper English?
by Shane Witbeck,
Re: What the qualities of proper English?
by Mark Levison,
Additions...
by Kevin E. Schlabach,
Two more requirements
by Stefan Roock,
The title is missing the word "Unit"
by Ilja Preuß,
What the qualities of proper English?
by Shane Witbeck,
Your message is awaiting moderation. Thank you for participating in the discussion.
...now read the title out loud
Additions...
by Kevin E. Schlabach,
Your message is awaiting moderation. Thank you for participating in the discussion.
If you aren't pairing, there are some non-technical ideas to improve the test cycle using your peers in the 5 for 5 game.
Since testing is more than TDD or other automated testing approaches, part of quality in testing is tied to when you run the tests.
And the test shouldn't suddenly appear green the moment right before a sprint review.
Re: What the qualities of proper English?
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Thanks for catching that. Another occasion where haste made a real mess of things.
Two more requirements
by Stefan Roock,
Your message is awaiting moderation. Thank you for participating in the discussion.
1. If logic is changed, a test should show that - means: test fails.
2. If logic is changed, only few (ideal: one) tests fail.
(1) provides the security net for refactoring.
(2) ensures that the security net will not hinder refactorings since you have to modify hundereds or thousands of tests for one logic change.
Jester uses the approach (1) to find missing test cases.
The title is missing the word "Unit"
by Ilja Preuß,
Your message is awaiting moderation. Thank you for participating in the discussion.
... as that's obviously what this article is talking about. Other kinds of tests, such as acceptance tests, will need other qualities.