Measure Teams, Not Individuals
In a late 2007 post, Dubakov raised the issue of agile teams' desire to measure individual velocity. Using an example of two developers, Ted and Jerry, Dubakov illustrated his view that a set of historical 'individual velocity' measurements will provide a team no better mechanism to plan future iterations than does the team's overall velocity measurement:
If Ted completed several tasks with 40 hrs of effort in total and Jerry completed tasks with 25 hrs effort only, we should say that Ted has better velocity in last iteration. Does it mean that Ted is a better/faster developer? Not at all. There are hundreds of reasons why Jerry completed less ... OK, but what about average velocity? Surprisingly, Jerry’s average velocity is 54 hrs per iteration. Gosh! What happened with Jerry last two weeks? Will his average velocity help us to make correct iteration plan? If we sum up individual velocities all developers will it help us to create a better iteration plan? No, since we already have Iteration Velocity metric and it will be exactly the same.More to his point, he continued by outlining two ways this metric is likely also to work against the ideal operational goals of an agile team:
- a harmful focus on individual performance, over team performance, which he notes is likely to discourage team members taking time to help one another out
- an increased tendency towards individual work assignment, over team commitment
Velocity is not about [assessing] performance… it’s about providing your customer with a close to accurate amount of “points” they can spend on features this iteration. Remember that.In a more recent post Dubakov revisited the subject, this time adding a warning against the measurement of individual estimate accuracy. He first pointed out that this metric isn't even feasibly useful unless, first, the estimates were given by individuals and, second, the team tracks time spent for all tasks. The major problem with these conditions, many in the agile community would assert, is that both are likely to work against fundamental agile principles fostering teamwork and simplicity.
To illustrate the misleading effect which measurement of individual estimate accuracy may have, Dubakov goes back to the hypothetical developer Ted:
We can calculate all Ted’s assignments and spent time and define estimate accuracy for next iteration, let’s say it is 0.7.InfoQ's Deborah Hartmann takes Michael's point a step further, questioning the measurement of any time-based estimate accuracy at all, individual or team:
OK, but how we can use this metric? Well, if Ted subscribed to 60 hrs work it means he will spend 85 hours and for 2 weeks iteration it means at least 5 hrs overtime. Ted should take this information into consideration and remove some tasks from his ToDo. This works if Ted estimate accuracy remains the same, but is that always the case? In reality Ted’s accuracy may vary from 0.5 to 0.9 and exactly for next iteration it may be 0.9 and in this case Ted will be able to do all committed work.
To calculate this kind of estimate accuracy, teams would have to expend effort to capture detailed "actual" hours, a practice not advocated by any Agile approach I've seen. The classic "units of work planned" vs "units of work fully completed" measures estimate accuracy in units of more value to the customer: work delivered (story points, ideal hours, bananas, whatever).Dubakov, Carr, and Hartmann all agree, with regard to both individual velocity and individual estimate accuracy, that measurement of these metrics not only provides no more useful information than is already available with their team-level equivalents, but may also have a tendency to encourage teams into behaviors that work against true agility.
The tracking of estimate accuracy in terms of actual hours expended not only adds no more information to the team, it also imposes a new form of waste! I agree with Dubakov, Carr and others: I don't see how this could be a helpful metric for most teams, and I'm happy to see that it was quickly removed from TargetProcess once the point was raised. This is the kind of responsiveness to change that we expect Agile teams to exhibit.
Cultural Barriers are High
Why not measure actual hours?
To calculate this kind of estimate accuracy, teams would have to expend effort to capture detailed "actual" hours
Our product line (6th Sense Analytics) is designed to measure the actual time invested by each developer without slowing the developer down. With a product like ours installed, the cost of measurement goes down to nearly zero.
If you could get actual hours for free, would it be a meaningful metric? Would it be worth something to you to know in hours, instead of magic beans or bananas? I've found it's a very valuable piece of information.
We do focus heavily on the team metric, not the individual. But it's much harder to build a meaningful team metric without measuring the individual.
Here's our Use Case page for more information.
Re: Why not measure actual hours?
The real reason we don't measure actual hours is the amount of time, and therefore dollars, it takes to capture actual time.
Uh, no. The real reason is that bosses scrutinizing individuals is harmful to teamwork. I'm not inclined to mentor my teammates on their tasks if my boss is using some kind of analytical tool to measure my productivity on my own tasks.