The Correct Ratio of Agile Testers to Developers? It Depends.

Posted by Chris Sims on Jan 05, 2009 11:30 AM

Community

Agile

Topics

Software Testing

Tags

Quality ,

Testing

An long-standing question in the software development world is: what is the correct ratio of testers to developers? A recent thread on the Scrum Development list asked how agile impacts this ratio. The answer to the first question seems to be 'It depends'. The answer to the second question, according to Elisabeth Hendrickson, is that agile teams can do more testing, with fewer testers.

What is the Correct Ratio of Testers to Developers?

Over the years, there has been much interest in finding the 'right' ratio of testers to developers. Microsoft employs a 1-to-1 ratio of testers to developers, according to the book 'Microsoft Secrets.' In an informal poll of participants in a conference session, Randall Rice found the most common ratio was 1 tester to 3 developers. A paper published by Cem Kaner, Elisabeth Hendrickson, and Jennifer Smith-Brock found that ratios such as these are surprisingly meaningless. The responsibilities and tasks assigned to each of these roles vary greatly from project to project. For instance, is the build-master a developer or tester?

In addition to the accounting problems, the group found that variations in project circumstances make comparisons between projects less relevant. Such factors include:

• The incoming reliability of the project
• The breadth of configurations that must be tested
• The testability of the software
• The availability of tools
• The experience of the testers and developers
• The quality standards that must be maintained

Johanna Rothman came to a similar conclusion in the article: It Depends: Deciding on the Correct Ratio of Developers to Testers. Randall Rice, in his article The Elusive Tester to Developer Ratio, also found industry ratios of dubious value:

Let it be clearly understood that I don't completely discount the use of ratios in planning if they are your ratios, based in your experience, your technology and your organizational structure. What I do see as a risk is when an organization takes another organization's ratios and applies them to their project without regard to differences in technology, process maturity, and skill levels.

How does Agile Impact the Ratio of Tester to Developers?

In a recent webcast, both Elisabeth Hendrickson and Lisa Crispin described agile environments as 'testing nirvana'. Elisabeth recalled working in traditional environments where the software that was given to the QA group from development was often D.O.A., uninstallable, or crashed on star-up. This never occurs in her work with agile teams.  On agile teams, testers are able to add much greater value by doing exploratory testing, creating test automation, and working closely with product owners to refine requirments and acceptance criteria.

Elisabeth has seen agile teams functioning effectively with significantly lower tester to developer ratios. This doesn't indicate that testing is less important, however. In her experience, agile teams need testing skills at least as much as traditional teams. The difference is that these skills, and the responsibility for ensuring quality, does not rest with a few people called testers. The entire team is working to build quality in to the product, as opposed to counting on a QA group to test quality into the product.

How does your team handle testing responsibilities? Leave a comment and share your experience.

1. Back to top

   My Experience with Agile Testing

   Jan 6, 2009 2:35 PM by Steve Willcox

   I've been working with the same Agile development team since 2003 and we have tried several testing approaches. Currently, the developers write and maintain all of automated tests (JUNIT and Selenium) for the stories they work on. The QA team focuses more on manual and automated system integration tests (tests encompassing multiple stories). And lastly, we have a dedicated engineer working on load/performance tests. With this structure, we have 12 developers and 3 QA resources.

   Reply

2. Back to top

   Re: My Experience with Agile Testing

   Jan 7, 2009 5:11 AM by Suresh K

   Selenium Tests can greatly help the QA people with the generation of test data sets. Some of our QA people use selenium to generate initial dataset specially when testing UI features and role based access testing. On paper, we have one-to-one developer-to-tester ratio but it is not always possible. All non-developers chip in with testing but not on a full time basis.

   Reply

3. Back to top

   Re: My Experience with Agile Testing

   Jan 7, 2009 6:57 AM by Henrik Leion

   Sounds like an excellent setup to me. In a previous project we pushed the border even further to include a few automated system tests in the DefinitionOfDone for the developers, just to make sure that the testers got working releases from the build server. A dedicated engineer for load/performance was high on my wish list in that project. We had about 10-14 developers, 1 tester and 1 automation scripter, but we definitely felt the need for an additional full-time tester.

   Reply

• Older >