What are the Qualities of a Good Test?

Posted by Mark Levison on Oct 03, 2008 09:53 AM

Community

Agile

Topics

Unit Testing ,

Delivering Quality

Tags

Best Practices ,

TDD

What is a good test? How do we know if we're writing good tests?

Kent Beck posited, that tests should be:

1. Isolated (unaffected by the presence, absence, or results of other tests)
2. Automated
3. Quick to write
4. Quick to run
5. Unique (providing confidence not provided by other tests/uncorrelated
   with other tests)

Roy Osherove adds that good tests have three fundamental properties: maintainable, trustworthy and readable.

Mike Hill has much longer list:

• It is short, typically under a dozen lines of code.
• It does not test the object inside the running app, but instead in a purpose-built testing application.
• It invokes only a tiny portion of the code, most usually a single branch of a single function.
• It is written gray-box, i.e. it reads as if it were black-box, but sometimes takes advantage of white-box knowledge. (Typically a critical factor in avoiding combinatoric issues.)
• It is coded to the same standard as shipping code, i.e. the team's best current understanding of coding excellence.
• In combination with all other microtests of an app, it serves as a 'gateway-to-commit'. That is, a developer is encouraged to commit anytime all microtests run green, and discouraged (strongly, even nastily) to commit otherwise.
• It takes complete control of the object-under-test and is therefore self-contained, i.e. running with no dependencies on anything other than the testing code and its dependency graph.
• It runs in an extremely short time.
• It is generally written before the code-change it is meant to test.
• It avoids most or all usage of 'awkward' collaborators via a variety of slip-and-fake techniques.
• ...

Mike and Ron Jeffries remind us that the key value of TDD is simplifying design and improving productivity. Improvements in code quality and a reduction in bugs are an important side effect.

Jeremy Miller adds that good unit tests should be:

• Order Independent and Isolated - it should be possible to run the tests in whatever order the test runner chooses.
• Intention Revealing - the best unit tests make it clear to the reader how an objects API is intended to be used.
• Easy to Setup

Finally Ed Burnette writes: Make your unit tests repeatable in every aspect; Test your boundary conditions and Always keep your tests passing at 100%.

1. Back to top

   What the qualities of proper English?

   Oct 3, 2008 11:01 AM by Shane Witbeck

   ...now read the title out loud

   Reply

2. Back to top

   Additions...

   Oct 3, 2008 11:23 AM by Kevin E. Schlabach

   If you aren't pairing, there are some non-technical ideas to improve the test cycle using your peers in the 5 for 5 game.
   
   Since testing is more than TDD or other automated testing approaches, part of quality in testing is tied to when you run the tests.
   
   And the test shouldn't suddenly appear green the moment right before a sprint review.

   Reply

3. Back to top

   Re: What the qualities of proper English?

   Oct 3, 2008 12:51 PM by Mark Levison

   Thanks for catching that. Another occasion where haste made a real mess of things.

   Reply

4. Back to top

   Two more requirements

   Oct 5, 2008 5:28 AM by Stefan Roock

   1. If logic is changed, a test should show that - means: test fails.
   2. If logic is changed, only few (ideal: one) tests fail.
   
   (1) provides the security net for refactoring.
   (2) ensures that the security net will not hinder refactorings since you have to modify hundereds or thousands of tests for one logic change.
   
   Jester uses the approach (1) to find missing test cases.

   Reply

5. Back to top

   The title is missing the word "Unit"

   Dec 22, 2008 10:00 AM by Ilja Preuß

   ... as that's obviously what this article is talking about. Other kinds of tests, such as acceptance tests, will need other qualities.

   Reply

• Older >