Do all agile teams understand how story points work? Apparently not. A recent post from Mike Cohn of Mountain Goat Software reiterated that estimating features with story points is about gauging relative effort, not about ranking features in order of their complexity.
To illustrate the difference between ordinal ranking and relative effort Mike used the example of buying a car. A team using ordinal ranking would list the cars from least expensive to most expensive assigning one additional point per car so that a Tata Nano would be one point and a Bugatti would be 10 points. The example illustrates how using story points to rank a product backlog will lead to a misunderstanding in terms of cost; the Bugatti being nearly 960 times the cost of the Nano.
But why did this come up? Were there agile teams out there who truly didn't understand that estimation of features with story points was about gauging effort, cost...not ranking features? That's exaclty what Thomas V asked:
I have to ask: what prompted you to write this post? Are there really people doing this?
And Mike Cohn confirmed. Yep, folks are apparently doing this:
Hi Thomas- Yes, oddly people are doing this. The prompt was an email on Saturday from someone I respect who told me he does this. I was quite surprised. I can’t see a single reason to rank product backlog items in this way, but yes it is done.
Turning it over to our readers; have you seen teams misusing the story point estimation technique? Have you experienced teams using story points to rank...rather than estimate a backlog of features?
Community comments
Story Point
by Theron James /
Re: Story Point
by Chris Goldsbury /
Re: Story Point
by luke stephenson /
Re: Story Point
by Chris Goldsbury /
Re: Story Point
by luke stephenson /
Re: Story Point
by Chris Goldsbury /
Re: Story Point
by luke stephenson /
Yes
by Dave Nicolette /
Re: Yes
by Chris Goldsbury /
Re: Yes
by Dave Nicolette /
It all evens out in the wash
by Tommy Norman /
Re: It all evens out in the wash
by Chris Goldsbury /
Re: It all evens out in the wash
by Tommy Norman /
Re: It all evens out in the wash
by Tommy Norman /
Story point
by tushar jain /
Story Point
by Theron James /
Your message is awaiting moderation. Thank you for participating in the discussion.
The biggest problem I have seen with majority of teams that have been "doing Scrum" before I got to them is equating points with time. A BIG mistake and defeats the purpose of points. I can not tell you how many time I hear something like, "A 5 for us is a 8 hours". Coincidentally, I find estimating tasks in unit time total muda. Story point rule for accuracy.
Re: Story Point
by Chris Goldsbury /
Your message is awaiting moderation. Thank you for participating in the discussion.
Theron....you're bringing up some interesting points. Do some teams use story points as a way to get out of delivering on a timeline? Further...you mention accuracy, but in what way are story points more accurate? Here's what I mean: with time...I can estimate 10 days. If it takes more than 10 days then we know the estimate was not accurate. Same would be true if it took less than 10 days. This acknowledges the truth about estimating anything: it's imperfect, and carries a degree of error. But with story points its all relative. There's no standard for comparison. This is part of the reason, at least in my experience, that management and leadership distrust story points altogether. If I say it takes 120 story points to complete and at the end of the task, story I mark it done and acknowledge it took 120 story points..should I call my estimate 100% accurate? No.
Re: Story Point
by luke stephenson /
Your message is awaiting moderation. Thank you for participating in the discussion.
Hi Chris, Mike Cohn has some brilliant books (Succeeding with Agile: Software Development Using Scrum) outlining the benefit of story points. It's probably best you read that, but I'll take a stab at providing my opinion in a few lines. As you say, estimating is imperfect. However, it's much easier to estimate size relative to another story. Even if I have just joined a project, I can quickly have a fairly good idea of how large one story is relative to another. How long that would take to develop I'd have very little idea.
As for the standard for comparison, all the work the team has historically estimated using story points and completed is the best comparison you can find for determining what you can achieve in the next iteration / release. If the team has estimated all of the stories and manages to complete X story points one iteration, then that serves as a very good indicator of what we can achieve the next.
Theron.. I've also heard so many teams linking story points back to a unit of time during estimation meetings.
Re: Story Point
by Chris Goldsbury /
Your message is awaiting moderation. Thank you for participating in the discussion.
Thanks Luke. Aren't you just illustrating velocity? I'm not questioning whether a team can gauge its velocity, how that's calculated or how we use that as a standard to compare team performance across iterations. That's a fairly well proven concept, isn't it? My question to Theron is how one knows any one story with a point estimate against it was accurate or inaccurate?
Take an example:
4 stories are put into an iteration. The team has gauged through time that they can run at 120-140 points an iteration ( velocity ).
Story 1 = 10 points.
Story 2 = 60 points.
Story 3 = 40 points.
Story 4 = 20 points.
They complete all the stories except Story 1. So, was story 1 estimated inaccurately? No. The team just mis-estimated how much work they could do in an iteration. Story 1 will never actually be inaccurately estimated *UNLESS* you are tying it to time/cost.
Perhaps Theron just meant that story points provide more accuracy in estimating velocity, not necessarily accuracy in estimating any one story.
Sorry for any confusion. You're right. I don't know as much as Mike Cohn. Never will.
Re: Story Point
by luke stephenson /
Your message is awaiting moderation. Thank you for participating in the discussion.
Hi Chris. If story 1 ended up being as complicated as a 60 point story (a random example), then I would say if was estimated inaccurately. At the retrospective the team should look back and reflect on why it couldn't deliver all the stories it had planned for the iteration. This may be because they underestimated the complexity of a particular story, or may have been other external factors (illness, unplanned support). So I still think it's possible to inaccurately estimate using story points, but that it's less likely compared to using time based estimates.
I don't claim to be an expert though :)
Yes
by Dave Nicolette /
Your message is awaiting moderation. Thank you for participating in the discussion.
In answer to the author's question about whether readers have seen abuse of story points: Yes.
The two most common examples have been described by others already: Equating points with time, and using story sizes to rank the backlog. What the latter has meant IME is not exactly "ranking" the backlog, but rather scheduling stories into the release. I've seen teams that chose the smallest stories so that they could maintain the appearance of progress. That left larger stories unaddressed, regardless of their actual business priority. Apparently they found this easier than learning how to split the larger stories. Go figure.
It all evens out in the wash
by Tommy Norman /
Your message is awaiting moderation. Thank you for participating in the discussion.
I would say I see less of the sequential ordering as Mike mentioned originally and more of the time based issues everyone else has mentioned in the comments. Along with that there are common questions around if the estimate is for the entire team or a team member, do team members vote on just their part, etc. I commonly have teams sorting through those concepts as well.
On the issue of accuracy that came up, story points and time based estimates are both fallible since they are provided by we the imperfect human. Where story points (and iterative development/planning) provide an overall better chance for accuracy is in the fact that you are constantly looking back at your past performance and adjusting your plans based on empirical evidence. A bloated velocity in one sprint due to a poorly sized story will only add a minor blip on your overall velocity average so if it is not a common occurrence then it does truly even out in the grand scheme of things.
If your team makes sure to frequently compare stories they are estimating to stories they have done in the past to triangulate their relative sizes, then you should have less instances where items are significantly larger or smaller than originally planned. The trick is to make sure they look at the complexity and activities around the story and not the hours. But of course this will happen from time to time and again it should not cause a significant issue with your overall release planning.
Re: Story Point
by Chris Goldsbury /
Your message is awaiting moderation. Thank you for participating in the discussion.
Gotcha. So a lower story point estimate is accurate whereas a higher story point estimate is not accurate....given the context of the environment you're in. Am I understanding you correctly?
Re: Yes
by Chris Goldsbury /
Your message is awaiting moderation. Thank you for participating in the discussion.
Dave,
So in your example a team uses the estimates as a way to tackle the easy stuff first. How does that play out toward the end of the project?
Re: It all evens out in the wash
by Chris Goldsbury /
Your message is awaiting moderation. Thank you for participating in the discussion.
Good point. So estimation accuracy, irrespective of method ( hours, story points, ideal days, or pine cones ) is a function of setting a baseline, tracking actuals, assessing reasons for variance, and correcting for the future. Would you agree?
Re: It all evens out in the wash
by Tommy Norman /
Your message is awaiting moderation. Thank you for participating in the discussion.
Chris,
I would be careful with the term "tracking actuals". Part of the charm of story points is that they are not supposed to be precise. The actual hours of work for a story that is a 1 may have a decent amount of overlap into the range of actual hours worked for a 2 point story and so on. Striving to be consistent in how many you commit to in an iteration/sprint is a better goal. As I mentioned before, the time boxing and incemental nature of practices like Scrum go hand in hand with this type of estimating. Story pointing a traditional waterfall project would make very little sense because you would not have built in check points in which you routinely inspect and adapt.
Re: It all evens out in the wash
by Tommy Norman /
Your message is awaiting moderation. Thank you for participating in the discussion.
Sorry, posted that reply before I had added my last point: Engineers are going to naturally want to be precise which normally results in them wanting to know everything up front and plan every last detail. When initially grooming and estimating stories, strive to get enough detail to provide a high level estimate and prioritize it. Any design decisions that do not affect scope (like should it be blue or red) can be left to a later time.
Re: Story Point
by luke stephenson /
Your message is awaiting moderation. Thank you for participating in the discussion.
This is the hypothetical I was try to describe. The team had 6 stories in an iteration, all of them initially estimated at 2 points. If all of those stories took approximately the same amount of effort, but one took 5 times as long, then I would say the team underestimated the complexity of that story relative to the others.
I wasn't trying to imply a blanket rule that a lower estimate is accurate and a higher estimate is inaccurate (and will need to be more careful of my wording if it came across that way).
Story point
by tushar jain /
Your message is awaiting moderation. Thank you for participating in the discussion.
Biggest issue I see is not in identifying what does not affect story point but what affect. May some one help me to list factors which affect story points?
Few of the factors I have though of:
Technical complexity
Business complexity
Efforts required to complete the story ( dev +test+ deployment + ...)
Dependency within scrum team
Dependency within project (A project might have multiple scrum teams)
Dependency across different projects ( some of the projects might be running on waterfall model)
Clarity in requirements
Clarity in scope
Urgency of story
Re: Yes
by Dave Nicolette /
Your message is awaiting moderation. Thank you for participating in the discussion.
Death march.