BT

Categorizing Tests

| by Amr Elssamadisy Follow 0 Followers on Aug 17, 2009. Estimated reading time: 2 minutes |

In a recent discussion on the test driven development Yahoo! group Carlos Ble shared his understanding of test categories after doing his research:

Eventually the picture I've got in my mind is:
Developer tests:
Unit tests : Isolated, atomic, innocuous: exercised with xUnit
Integration tests:
Isolated tests that might change the state of the system, i.e: saving into database, writing file... An integration test does not represent a functional requirement as is. Can be written for xUnit. They check the integration of our code with a third party tool or with the different layers of our own code, i.e: the business logic layer requires the data access layer
Functional tests: (also known as System tests)
A test that excercises part of the system as a whole, some functional requirement. It might change the status of the system.
Product Owner tests:
Acceptance tests: Functional tests which input and output can be validated by a non-technical person, the product owner.

 

John Donaldson shared a multi-dimensional model of tests that look at test roles and test types:

I like the test view you give. But I think it's an instance of a bigger model, where you have (at least) actor-roles and test-types.

Actor-roles: developer, tester, QA, user, sponsor, etc.

Test-types: unit, integration, functional, system, acceptance, soak, smoke, etc.

In any given situation you probably which role gets to do which test - but in the next project it may be different.

Dale Emery suggested that not knowing what kind of test you are writing is a code smell and indicates lack of clarity. At the same time, a test may be fall under several categories and what is important is your current point of view:

The challenge, I think, is that any given test could be reasonably classified in numerous ways, depending on what dimensions you're focusing on. And there are a lot of dimensions that people use to classify tests. I've identified some here: http://cwd.dhemery.com/2004/04/dimensions/

So I'm less interested in knowing "what kind" of test it is, and more interested in knowing where a given test falls along the dimensions that are most important to me, depending on my purposes at any given moment. The ones that I think about most often are:

  • What "unit" does this test define and test? (What system, subsystem, object, collaboration...) - What feature does the test define and test?
  • Who are this test's primary customers? Who most cares about the results of executing this test?
  • What decisions will be made based on the results of executing this test?

Charlie Poole gives a detailed analysis of Carlos' categorization and suggests:

In my opinion, the most important distinction is between tests of developer intent and tests of customer intent.

This discussion highlights the fact that test categorization can be very confusing - especially to the newcomer.  The majority opinion is that a dimensional approach to test categorization is the way to go and that the type of categorization that is relevant is dependent on the user's current intentions and context.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

The full test life cycle of software by Guo Du

I summarised some time ago give more details about those tests:
duguo.org/en/The_full_test_life_cycle_of_software

Re: The full test life cycle of software by Amr Elssamadisy

Thanks for your input. I think what you'll find is that there is no consensus on even the definitions, so your description of functional tests:
code-facing, test features by sample data inside component
doesn't match that of Charlie Poole's (for example) where he sees functional tests are synonyms to story tests.

Categorizing Tests by rey bumalay

Great article.

I was laughing coz I had a talk with few people who thinks integration testing is unit testing. Hope this article helps them understand.

Perhaps I'm an oxymoron, but... by Stuart Stern

I recently authored the Infoq article, "FlexMonkey brings unit testing to Flex User Interface developers" (www.infoq.com/articles/flexmonkey-ui-unit-testing) and several people I know took issue with my characterization of any sort of UI testing as "unit" testing. In the article, however, I lay out the case (convincingly, of course!) for how it can be sufficient to divide testing into two broad categories: "Developer Tests" vs "QA Tests".

Stu Stern
Gorilla Logic CEO and Inventor of FlexMonkey

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

4 Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT