Agilists in the community are debating the use of story points and velocity, making people question the use of these in estimation and measurement of overall progress. The common underlying reason is these metrics are often prone to abuse and are seldom used effectively.
Joshua Kerievsky wrote a detailed account of his experiences in using story points and how they are prone to abuse. Here is an example he quotes where story points are prone to irrational inflation.
One day in 2004 Jim exhorted the team to go faster. This team had an average velocity around 52 points per iteration.Their velocity would fluctuate by a few points, but generally remained steady. Yet just weeks after Jim asked the team to "sprint", the team's velocity jumped up into the high 80s!
I asked someone what happened.
She looked at me funny and said "These days around here if you sneeze, you get a story point." I shook my head, amazed at how a mature agile team, a team that had been assessed, trained and coached by two excellent Industrial Logic coaches, and myself could so suddenly inflate their story point estimates to appear like they were going faster.
My confidence in story points and velocity calculations began to erode after that experience.
Joshua also highlighted how comparing teams by points is an anti pattern
Over the years I've heard several managers at different companies ask, "Why is team X getting 24 story points done per sprint, while team Y is only getting 12 story points done? Those teams are roughly the same size, so why the discrepancy?"
Instead of treating velocity as a team-specific, capacity indicator, such managers fall into the trap of thinking of velocity as a performance measurement
Uncle Bob Martin further confirms this in Joshua’s blog comments
Many teams do struggle with story points. I have certainly seen teams devalue those points when their project managers try to get something for nothing by asking the team to go faster. I've also seen teams fake their velocity charts for the benefit of project managers who were more interested in form over function. For such teams, a different approach may be warranted.
On a recent email thread on Scrum Alliance group Ron Jeffries was critical about Velocity as a metric.
- Velocity can be misused, and usually is;
- Velocity can be gamed, and usually is;
And most of all, because it is working on the short end of the lever. What's important in Agile is steering the project by selecting things to do and things to defer, not working the cost side of the equation and trying to improve it.
A team that is focusing on velocity is not focusing on value. I wish I had never invented velocity, if in fact I did.
He elaborates further in the thread saying
We find that most of the questions here are about how to improve estimates, so that they will be accurate. When we drill in, we see teams who are projecting when they'll be done and then pushing the teams to complete everything. We see them measuring teams on accuracy.
We see product owners who are just doling out the backlog items with little or no control over anything, just "following the plan".
Scrum works best when the Product Owner uses the backlog creatively to deliver the best possible product. I have found that focus on estimates is not helpful to getting there.
Mike Cohn recently posted on his blog an example conversation with one customer where he considered estimation to be important. The customer is quoted saying
First of all, in order for me to even consider giving future funding I need to know how close our estimating process got us this go-round. When I ask the stakeholders for more money they’re going to think it’s going into a black hole if we only accomplish ½ of what we said we would. I will manage this but I need to know we have a reasonable approach for coming up with high level estimates. Second – to get follow on funding, I’m going to need to know roughly how much we need to collect (this is a long lead time activity). We’ll have to improve our estimates if they’re nowhere near accurate and I’ll beat the pavement to get the funding. That said, money doesn’t come for free. I have to have some level of metrics that tell me throughout the project that we’re tracking to completion. In other words – I’ll look at the project burndown and watch our costs throughout the project.
Vasco Duarte suggests an alternative to story points .which is to estimate by counting stories .
For the purposes of looking at the long term (remember: this only applies on the long term, i.e. more than 3 sprints into the future) estimation/progress of the project, you can assume that all stories are the same size, and can therefore measure progress by measuring the number of items completed per Sprint.
Mike Cohn also commented on similar lines
I’d agree that Kanban teams do less explicit estimation. That’s often because they’ve made implicit estimates that all work is about the same size.
Neil Killick offers another approach to pricing that does not need estimation. He contrasts an example to build a website for himself where the vendor builds the best possible solution given the revealed budget available and the Neil can chose to end the relationship if he is dissatisfied at any point in time.
He says that this option
Requires no estimation, design can change and emerge as we go along, embraces changes as I see the site evolve and shows a company wanting to work closely with me to achieve a result I am delighted with. One that is prepared to front extra risk (of losing money on the contract) because they are so confident in the quality of work they do and of the relationships they form with their customers.
Have you noticed practices such as velocity and story points estimation encouraging incorrect behaviour in teams ? What are your thoughts on some of the alternatives being suggested by the leaders in our community ?