In a recent discussion on the test driven development Yahoo! group Carlos Ble shared his understanding of test categories after doing his research:
Eventually the picture I've got in my mind is:
Developer tests:
Unit tests : Isolated, atomic, innocuous: exercised with xUnit
Integration tests:
Isolated tests that might change the state of the system, i.e: saving into database, writing file... An integration test does not represent a functional requirement as is. Can be written for xUnit. They check the integration of our code with a third party tool or with the different layers of our own code, i.e: the business logic layer requires the data access layer
Functional tests: (also known as System tests)
A test that excercises part of the system as a whole, some functional requirement. It might change the status of the system.
Product Owner tests:
Acceptance tests: Functional tests which input and output can be validated by a non-technical person, the product owner.
John Donaldson shared a multi-dimensional model of tests that look at test roles and test types:
I like the test view you give. But I think it's an instance of a bigger model, where you have (at least) actor-roles and test-types.
Actor-roles: developer, tester, QA, user, sponsor, etc.
Test-types: unit, integration, functional, system, acceptance, soak, smoke, etc.
In any given situation you probably which role gets to do which test - but in the next project it may be different.
Dale Emery suggested that not knowing what kind of test you are writing is a code smell and indicates lack of clarity. At the same time, a test may be fall under several categories and what is important is your current point of view:
The challenge, I think, is that any given test could be reasonably classified in numerous ways, depending on what dimensions you're focusing on. And there are a lot of dimensions that people use to classify tests. I've identified some here: http://cwd.dhemery.com/2004/04/dimensions/
So I'm less interested in knowing "what kind" of test it is, and more interested in knowing where a given test falls along the dimensions that are most important to me, depending on my purposes at any given moment. The ones that I think about most often are:
- What "unit" does this test define and test? (What system, subsystem, object, collaboration...) - What feature does the test define and test?
- Who are this test's primary customers? Who most cares about the results of executing this test?
- What decisions will be made based on the results of executing this test?
Charlie Poole gives a detailed analysis of Carlos' categorization and suggests:
In my opinion, the most important distinction is between tests of developer intent and tests of customer intent.
This discussion highlights the fact that test categorization can be very confusing - especially to the newcomer. The majority opinion is that a dimensional approach to test categorization is the way to go and that the type of categorization that is relevant is dependent on the user's current intentions and context.