BT

What are the Qualities of a Good Test?

by Mark Levison on Oct 03, 2008 |

What is a good test? How do we know if we're writing good tests?

Kent Beck posited, that tests should be:

  1. Isolated (unaffected by the presence, absence, or results of other tests)
  2. Automated
  3. Quick to write
  4. Quick to run
  5. Unique (providing confidence not provided by other tests/uncorrelated
    with other tests)

Roy Osherove adds that good tests have three fundamental properties: maintainable, trustworthy and readable.

Mike Hill has much longer list:

  • It is short, typically under a dozen lines of code.
  • It does not test the object inside the running app, but instead in a purpose-built testing application.
  • It invokes only a tiny portion of the code, most usually a single branch of a single function.
  • It is written gray-box, i.e. it reads as if it were black-box, but sometimes takes advantage of white-box knowledge. (Typically a critical factor in avoiding combinatoric issues.)
  • It is coded to the same standard as shipping code, i.e. the team's best current understanding of coding excellence.
  • In combination with all other microtests of an app, it serves as a 'gateway-to-commit'. That is, a developer is encouraged to commit anytime all microtests run green, and discouraged (strongly, even nastily) to commit otherwise.
  • It takes complete control of the object-under-test and is therefore self-contained, i.e. running with no dependencies on anything other than the testing code and its dependency graph.
  • It runs in an extremely short time.
  • It is generally written before the code-change it is meant to test.
  • It avoids most or all usage of 'awkward' collaborators via a variety of slip-and-fake techniques.
  • ...

Mike and Ron Jeffries remind us that the key value of TDD is simplifying design and improving productivity. Improvements in code quality and a reduction in bugs are an important side effect.

Jeremy Miller adds that good unit tests should be:

  • Order Independent and Isolated - it should be possible to run the tests in whatever order the test runner chooses.
  • Intention Revealing - the best unit tests make it clear to the reader how an objects API is intended to be used.
  • Easy to Setup

Finally Ed Burnette writes: Make your unit tests repeatable in every aspect; Test your boundary conditions and Always keep your tests passing at 100%.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

What the qualities of proper English? by Shane Witbeck

...now read the title out loud

Additions... by Kevin E. Schlabach

If you aren't pairing, there are some non-technical ideas to improve the test cycle using your peers in the 5 for 5 game.




Since testing is more than TDD or other automated testing approaches, part of quality in testing is tied to when you run the tests.



And the test shouldn't suddenly appear green the moment right before a sprint review.

Re: What the qualities of proper English? by Mark Levison

Thanks for catching that. Another occasion where haste made a real mess of things.

Two more requirements by Stefan Roock

1. If logic is changed, a test should show that - means: test fails.


2. If logic is changed, only few (ideal: one) tests fail.



(1) provides the security net for refactoring.


(2) ensures that the security net will not hinder refactorings since you have to modify hundereds or thousands of tests for one logic change.




Jester uses the approach (1) to find missing test cases.

The title is missing the word "Unit" by Ilja Preuß

... as that's obviously what this article is talking about. Other kinds of tests, such as acceptance tests, will need other qualities.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

5 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT