What is a Good Agile Metric?

Posted by Mark Levison on Nov 05, 2009

Sections

Process & Practices

Topics

Agile ,

Agile in the Enterprise

Tags

Management ,

Measurement ,

Value & Metrics

Agile Coaches and Consultants frequently warn their clients that traditional measures such as Earned Value, Hours Worked, Lines of Code, Code Coverage for Tests are not well suited to Agile Projects. But then our clients are left with the question what are good Agile Metrics? How would I tell a good metric from a bad one? Are there contexts where a good metric is bad?

The classic metric for an XP/Scrum Team is of course velocity or how much work did the team complete in the last iteration? It was originally created to help a team decide how much they can plan for the next iteration. However the question is frequently asked can we use Velocity to measure the productivity of a team? To compare two teams? Hiren Doshi, points out velocity metric is a very 'team' specific metric. In addition, Peter Stevens, Agile Consultant, asks if the teams would have a reason to game the measure: “Is this story a 2 or a 3? That lies purely in the judgment of team. If the team feels a need to deliver as many SP's as possible, then it's obviously a three, and maybe a five.”

Dave Nicolette, Agile/Lean Coach, warns us that poorly designed metrics lead to poor outcomes. For example the business that rewards bug fixing and fire fighting – produces people who write bugs and start fires.

In Appropriate Agile Measurement, Deborah Hartmann Preuss, Agile Coach, and Robin Dymond, Agile Management Consultant, offer some heuristics for good Agile measurements:

• Affirms and reinforces Lean and Agile principles
• Measures outcome, not output
• Follows trends, not numbers
• Belongs to a small set of metrics and diagnostics
• Is easy to collect
• Reveals, rather than conceals, its context and significant variables
• Provides fuel for meaningful conversation
• May measure Value (Product) or Process
• Encourages "good-enough" quality

What are good Agile Measures?

Ron Jeffries has offered Running Tested Features:

1. The desired software is broken down into named features (requirements, stories) which are part of what it means to deliver the desired system.
2. For each named feature, there are one or more automated acceptance tests which, when they work, will show that the feature in question is implemented.
3. The RTF metric shows, at every moment in the project, how many features are passing all their acceptance tests

Peter Hundermark, Scrum Coach, suggests that Running Automated Tests are one measure:

Within limits, the more running (i.e. passing) automated tests a team has in place is a positive
measure of quality. Beyond a certain level, this will cease to be true, but we have not yet met a
team that has reached this point. (We hope to!)
...
Anecdotally, this was one of the prime metrics that salesforce.com put in place during its bigbang
transition to Agile.

In addition he offers Work In Progress:

Items (stories) in-progress is a productivity metric. It seeks to help a team track whether they
are working collaboratively or not. The idea in an Agile team is for the whole team, as far as is
reasonably possible, to collaborate on a single work item until it is ‘done’. This increases the
rate of output, quality and cross-learning. It decreases the risk of unfinished items at the end of
the Sprint, which results in waste.

Simply by tracking on a daily basis how many items the team has in-progress will make visible
the extent to which they are collaborating. The chart tracks stories in-progress against days. It
is agnostic of Sprint boundaries. It should trend towards 1 over time. Any value higher than 2 is
cause for action by the ScrumMaster.

Finally Deborah and Robin remind us when designing a metric we should consider not only when to use it, but when to stop using it and how can it be gamed.

Previously on InfoQ: Metrics in an Agile World and Agile EVM

1. Back to top

   more recommended reading

   by Laz S

   I'd recommend Michaels Marco-measurement white paper: danube.com/system/files/WP_Macromeasurements.pdf
   
   and Dans EVB white paper: danube.com/system/files/WP_AgileEVM_and_Earned_...

   Reply

2. Back to top

   Lead time

   by Aslak Hellesøy

   I have found that measuring lead time (time to market) is a much more valuable metric than velocity and burndown. You can read more about that here: open.bekk.no/2009/11/03/cumulative-flow-diagram...

   Reply

3. Back to top

   Re: Lead time

   by Mark Levison

   We're in agreement. In fact its the one measure I like to use and keep for the long term. For some reason I didn't have a good quote for it when I wrote the item - but I don't remember why now.
   
   Thanks for adding it.

   Reply

• Older >