InfoQ

News

The Correct Ratio of Agile Testers to Developers? It Depends.

Posted by Chris Sims on Jan 05, 2009 11:30 AM

Community
Agile
Topics
Software Testing
Tags
Quality ,
Testing

An long-standing question in the software development world is: what is the correct ratio of testers to developers? A recent thread on the Scrum Development list asked how agile impacts this ratio. The answer to the first question seems to be 'It depends'. The answer to the second question, according to Elisabeth Hendrickson, is that agile teams can do more testing, with fewer testers.

What is the Correct Ratio of Testers to Developers?

Over the years, there has been much interest in finding the 'right' ratio of testers to developers. Microsoft employs a 1-to-1 ratio of testers to developers, according to the book 'Microsoft Secrets.' In an informal poll of participants in a conference session, Randall Rice found the most common ratio was 1 tester to 3 developers. A paper published by Cem Kaner, Elisabeth Hendrickson, and Jennifer Smith-Brock found that ratios such as these are surprisingly meaningless. The responsibilities and tasks assigned to each of these roles vary greatly from project to project. For instance, is the build-master a developer or tester?

In addition to the accounting problems, the group found that variations in project circumstances make comparisons between projects less relevant. Such factors include:

  • The incoming reliability of the project
  • The breadth of configurations that must be tested
  • The testability of the software
  • The availability of tools
  • The experience of the testers and developers
  • The quality standards that must be maintained

Johanna Rothman came to a similar conclusion in the article: It Depends: Deciding on the Correct Ratio of Developers to Testers. Randall Rice, in his article The Elusive Tester to Developer Ratio, also found industry ratios of dubious value:

Let it be clearly understood that I don't completely discount the use of ratios in planning if they are your ratios, based in your experience, your technology and your organizational structure. What I do see as a risk is when an organization takes another organization's ratios and applies them to their project without regard to differences in technology, process maturity, and skill levels.

How does Agile Impact the Ratio of Tester to Developers?

In a recent webcast, both Elisabeth Hendrickson and Lisa Crispin described agile environments as 'testing nirvana'. Elisabeth recalled working in traditional environments where the software that was given to the QA group from development was often D.O.A., uninstallable, or crashed on star-up. This never occurs in her work with agile teams.  On agile teams, testers are able to add much greater value by doing exploratory testing, creating test automation, and working closely with product owners to refine requirments and acceptance criteria.

Elisabeth has seen agile teams functioning effectively with significantly lower tester to developer ratios. This doesn't indicate that testing is less important, however. In her experience, agile teams need testing skills at least as much as traditional teams. The difference is that these skills, and the responsibility for ensuring quality, does not rest with a few people called testers. The entire team is working to build quality in to the product, as opposed to counting on a QA group to test quality into the product.

How does your team handle testing responsibilities? Leave a comment and share your experience.

Related Sponsor

VersionOne is recognized by Agile practitioners as the leader in Agile project management tools. Companies such as Adobe, BBC, CNN, Dow, HP, IBM, Sony and 3M have turned to VersionOne to help deliver greater value to their customers.
My Experience with Agile Testing by Steve Willcox Posted Jan 6, 2009 2:35 PM
Re: My Experience with Agile Testing by Suresh K Posted Jan 7, 2009 5:11 AM
Re: My Experience with Agile Testing by Henrik Leion Posted Jan 7, 2009 6:57 AM
  1. Back to top

    My Experience with Agile Testing

    Jan 6, 2009 2:35 PM by Steve Willcox

    I've been working with the same Agile development team since 2003 and we have tried several testing approaches. Currently, the developers write and maintain all of automated tests (JUNIT and Selenium) for the stories they work on. The QA team focuses more on manual and automated system integration tests (tests encompassing multiple stories). And lastly, we have a dedicated engineer working on load/performance tests. With this structure, we have 12 developers and 3 QA resources.

  2. Back to top

    Re: My Experience with Agile Testing

    Jan 7, 2009 5:11 AM by Suresh K

    Selenium Tests can greatly help the QA people with the generation of test data sets. Some of our QA people use selenium to generate initial dataset specially when testing UI features and role based access testing. On paper, we have one-to-one developer-to-tester ratio but it is not always possible. All non-developers chip in with testing but not on a full time basis.

  3. Back to top

    Re: My Experience with Agile Testing

    Jan 7, 2009 6:57 AM by Henrik Leion

    Sounds like an excellent setup to me. In a previous project we pushed the border even further to include a few automated system tests in the DefinitionOfDone for the developers, just to make sure that the testers got working releases from the build server. A dedicated engineer for load/performance was high on my wish list in that project. We had about 10-14 developers, 1 tester and 1 automation scripter, but we definitely felt the need for an additional full-time tester.

Educational Content

Bindings, Platforms, and Innovation

This presentation focuses on the Internet and separating myth from fact, history from the future, and the mundane from the imaginative. Bob Frankston presents a vision of what could and should be.

Orchestrating Long Running Activities with JBoss / JBPM

This article explores the use of JBoss and jBPM to implement design solutions that effectively address the issue of orchestrating long running activities.

Neo4j - The Benefits of Graph Databases

This presentation covers the use of graph databases as an optimal solution for data that is difficult to fit in static tables, rapidly evolving data or data that has a lot of optional attributes.

Realistic about Risk: Software development with Real Options

This session introduces Real Options and shows how it can help in running your project. Real Options is a decision-making process that can be used to manage risk.

Communication Flexibility Using Bindings

This article discusses the use of bindings on services and references (including the instance of non-configured bindings) as the means to implement SCA communications in a Web and SOA environment.

Writing DSLs in Groovy

After a short introduction to DSLs, Scott Davis plays with the keyboard showing how to approach the creation of a DSL by typing working snippets of Groovy code that get executed.

Scaling Agile with C/ALM (Collaborative Application Lifecycle Management)

IBM Rational and InfoQ present, Scaling Agile with C/ALM, an eBook showing organizations how to become “finely tuned software delivery machines” by enabling team integration and scaling.

Concurrent Programming with Microsoft F#

Amanda Laucher presents a real life enterprise application written in F#. She shows actual code snippets, explaining design decisions and suggesting how to use some of the F# constructs.