BT

What is a Good Agile Metric?

by Mark Levison on Nov 05, 2009 |

MeasureAgile Coaches and Consultants frequently warn their clients that traditional measures such as Earned Value, Hours Worked, Lines of Code, Code Coverage for Tests are not well suited to Agile Projects. But then our clients are left with the question what are good Agile Metrics? How would I tell a good metric from a bad one? Are there contexts where a good metric is bad?

The classic metric for an XP/Scrum Team is of course velocity or how much work did the team complete in the last iteration? It was originally created to help a team decide how much they can plan for the next iteration. However the question is frequently asked can we use Velocity to measure the productivity of a team? To compare two teams? Hiren Doshi, points out velocity metric is a very 'team' specific metric. In addition, Peter Stevens, Agile Consultant, asks if the teams would have a reason to game the measure: “Is this story a 2 or a 3? That lies purely in the judgment of team. If the team feels a need to deliver as many SP's as possible, then it's obviously a three, and maybe a five.”

Dave Nicolette, Agile/Lean Coach, warns us that poorly designed metrics lead to poor outcomes. For example the business that rewards bug fixing and fire fighting – produces people who write bugs and start fires.

In Appropriate Agile Measurement, Deborah Hartmann Preuss, Agile Coach, and Robin Dymond, Agile Management Consultant, offer some heuristics for good Agile measurements:

  • Affirms and reinforces Lean and Agile principles
  • Measures outcome, not output
  • Follows trends, not numbers
  • Belongs to a small set of metrics and diagnostics
  • Is easy to collect
  • Reveals, rather than conceals, its context and significant variables
  • Provides fuel for meaningful conversation
  • May measure Value (Product) or Process
  • Encourages "good-enough" quality

What are good Agile Measures?

Ron Jeffries has offered Running Tested Features:

  1. The desired software is broken down into named features (requirements, stories) which are part of what it means to deliver the desired system.
  2. For each named feature, there are one or more automated acceptance tests which, when they work, will show that the feature in question is implemented.
  3. The RTF metric shows, at every moment in the project, how many features are passing all their acceptance tests

Peter Hundermark, Scrum Coach, suggests that Running Automated Tests are one measure:

Within limits, the more running (i.e. passing) automated tests a team has in place is a positive
measure of quality. Beyond a certain level, this will cease to be true, but we have not yet met a
team that has reached this point. (We hope to!)
...
Anecdotally, this was one of the prime metrics that salesforce.com put in place during its bigbang
transition to Agile.

In addition he offers Work In Progress:

Items (stories) in-progress is a productivity metric. It seeks to help a team track whether they
are working collaboratively or not. The idea in an Agile team is for the whole team, as far as is
reasonably possible, to collaborate on a single work item until it is ‘done’. This increases the
rate of output, quality and cross-learning. It decreases the risk of unfinished items at the end of
the Sprint, which results in waste.


Simply by tracking on a daily basis how many items the team has in-progress will make visible
the extent to which they are collaborating. The chart tracks stories in-progress against days. It
is agnostic of Sprint boundaries. It should trend towards 1 over time. Any value higher than 2 is
cause for action by the ScrumMaster.

Finally Deborah and Robin remind us when designing a metric we should consider not only when to use it, but when to stop using it and how can it be gamed.

Previously on InfoQ: Metrics in an Agile World and Agile EVM

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

more recommended reading by Laz S

I'd recommend Michaels Marco-measurement white paper: danube.com/system/files/WP_Macromeasurements.pdf

and Dans EVB white paper: danube.com/system/files/WP_AgileEVM_and_Earned_...

Lead time by Aslak Hellesøy

I have found that measuring lead time (time to market) is a much more valuable metric than velocity and burndown. You can read more about that here: open.bekk.no/2009/11/03/cumulative-flow-diagram...

Re: Lead time by Mark Levison

We're in agreement. In fact its the one measure I like to use and keep for the long term. For some reason I didn't have a good quote for it when I wrote the item - but I don't remember why now.

Thanks for adding it.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

3 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT