Measure Agile Productivity in $
Earlier Scott Ambler posted an article of how to measure productivity on agile teams by utilizing acceleration. Recently he followed up with another post where he answers some frequently asked questions related to agile productivity and acceleration. Specifically one question answers how to measure the amount of $ saved by an accelerating team.
Ambler argues that:
If you can easily measure productivity you can easily identify what is working for you in given situations, or what is not working for you, and adjust accordingly.
He starts by saying that most agile teams use velocity to measure how a team is performing, then goes on and asks (and answers) the question:
Is it possible to use velocity as a measure of productivity?
You can't compare the velocity of the two teams because they're measuring in different units. Team A is reporting in their points and B in their points, so you can't compare them directly.
Instead he proposes a different solution:
Instead of comparing velocities you instead calculate the acceleration of each team.
Team A's velocity is increasing over time whereas team B's velocity is trending downwards. All things being equal, you can assume that team A's productivity is increasing whereas B's is decreasing. Of course it's not wise to manage simply by the numbers, so instead of assuming what is going on I would rather go and talk with the people on the two teams. Doing so I might find out that team A has adopted quality-oriented practices such as continuous integration and static code analysis which team B has not, indicating that I might want to help team B adopt these practices and hopefully increase their productivity.
To calculate the acceleration of a team, Ambler suggests using the following formula:
(velocity iteration x1 – velocity iteration x2) / velocity iteration x2 = team acceleration
where iteration x1 is some iteration back in time before iteration x2. As an example he uses two teams, where one team has positive acceleration and the other negative acceleration:
…the acceleration of team A from iteration 1 to iteration 6 is (20-17)/17 = 0.176 whereas for team B it is (45-51)/51 = -.118
Ambler points out some advantages and disadvantages using this method:
- It's easy to calculate
- It is inexpensive
- It is unlikely to be gamed
- It is easy to automate
- It offers the opportunity for more effective governance
- You can easily adjust for changing team size
- You can easily monetize this metric
- It is an indirect measure of productivity
- You actually need to measure what you're interested in
- Management must be flexible
- Your existing measurement program may be questioned
- The terminology sounds scientific
In his FAQ post he goes into how you can monetize acceleration:
…assume your acceleration is 0.007 (0.7%), there are five people on the team, your annual burdened cost per person is $150,000, and you have two week iterations.
So, per iteration the average burdened cost per person must be $150,000/26 = $5,770. Productivity improvement per iteration for this team must be $5,770 * 5 * .007 = $202. If the acceleration stayed constant at 0.7% the overall productivity improvement for the year would be (1.007)^26 (assuming the team works all 52 weeks of the year) which would be 1.198 or 19.8%. This would be a savings of $148,500
The number 26 is the number of iterations per year (52 weeks / 2 week iterations).
Would you want to start using this method for calculating acceleration on your team(s)?
I would check one more thing...
Unfortunately, I've seen too many teams that have accelerated impressively based purely on story points, but they've actually been re-estimating and increasing the sizes of the stories. So the appearance is acceleration, but the reality is that stories have been de-valued and their relative tracking towards release may not have improved at all.
I'm always amazed by how hard it can be to convince people of the false economy of increasing estimate sizes to guarantee a planning velocity is attained, when doing so effectively increases the size of the project (in terms of storypoints) as well.
Acceleration is good, but can not be constant
My main KPI is User Stories delivered. For stabile teams, User Stories tend to normalize in size over time. Presuming the PO has prioritized the most valuable User Stories to the top of the backlog, the more User Stories the team gets done equals more value delivered. If the amount of User Stories done is trending downwards, there's need to investigate. The team may not be at fault, but something in their environment is causing less value produced.