BT

Sprint Planning: Story Points Versus Hours

by Vikas Hazrati on Sep 22, 2009 |

There is a constant, long drawn debate on the benefits of using either story points or hours for sprint planning. Each camp seems to have its own set of reasons of taking one approach over the other. Mike Cohn is big on breaking User Stories down into tasks, which are then estimated in hours. Jeff Sutherland on the other hand suggested that some of the best teams that he has worked with burn down story points. Many agilists have expressed their view point on what they prefer and why.

According to Mike, he does not prefer using story points on sprint planning as they tend to be a long term measure which are not helpful in the short run. According to him,

Suppose a basketball team is in the middle of their season. They’ve scored an average of 98 points per game through the 41 games thus far. It would be appropriate for them to say “We will probably average 98 points per game the rest of the season.” But they should not say before any one game, “Our average is 98 therefore we will score 98 tonight.” 

This is why I say velocity is a useful long-term predictor but is not a useful short-term predictor.

Tara Lee Whitaker does not agree that story points is a short term measure. According to her,

If each story is Small enough to be ‘accurately’ estimated, and Testable enough for one to create Test Confirmations then there may be little benefit achieved through breaking it down into smaller composite parts or re-estimating them in hours.

She shared a big concern about estimating the story in hours,

The main concerns I had when we initially discussed the idea of burning down in hours was that we’d not benefit from the early warning signs provided by the burndown and that we would only discover a story was taking longer than expected when it was too late.

Jim Schiel suggested that it may be possible to do sprint planning both in story points and hours. However, the return on investment by estimating in hours is not high to make that a serious candidate. According to him,

Now, let’s look at the Scrum team that commits to ten 2-story point PBIs. If they finish them all, they’ll have achieved a velocity of 20 story points that Sprint. Next Sprint, they’ll probably try 20 story points again. The Sprint after that may be a little more or a little less depending on what happens during the Sprint. This continues from Sprint to Sprint, with the team averaging a velocity that will likely be somewhere around 18 to 22. 

Can you do the same thing with hours? Yes, but it’ll cost you a whole lot more money to do it right. What do YOU want to pay for – completed, working software or very accurate estimates?

Jack Milunsky further reiterated the significance of story points when he mentioned the following advantages.

  • Universal Measurement – Story point is a universal measurement across the team. It is not biased by the experience or skills or any individual on the team.
  • Steady State – After the 3rd or 4th sprint, the team reaches a steady state and it becomes easier for the product owner to fill the backlog with the steady state story points. Nothing more, nothing less.
  • In the zone – Once the team reaches steady state, business believes the technical team and this puts the team in the zone of high motivation and confidence.

Tomas Björkholm mentioned the following reasons for going the story point way.

  • Reason 1. Estimation is a way of telling the size of a story, not how long it takes to implement it.
  • Reason 2. Ideal man days is a size that varies over time depending on team performance. Story points are relatively constant.
  • Reason 3. It's proven that relative estimation is more correct than absolute and since ideal man days is a time measure it's easy to make absolute estimations even though it could be used relatively as well.

Tomas added,

At a speech about Pomodoro Technique Staffan Nöteberg mentioned that most people feel uncomfortable about giving estimates in actual time. Aha, I thought. Uncomfortable people are less productive so estimation in days are less productive.

Mike Cohn suggested that there is no linear correlation between a story point and the number of hours. According to him, there is a distribution range for each story based on some standard deviation.

One point equals a distribution with a mean of x and some standard deviation. The same is true, of course, for two-point stories, and so on…

Hence, one cannot tell the stakeholders, given a certain velocity measured in story points, that the team would finish on a certain day. It has to be a range.

That range can a date-range such as “We can finish every item on your product backlog but it will be May or June by the time we finish.” Or it can be a scope-range, “We can finish on May 20, like you want, and we’ll get somewhere between 120 and 140 story points by then, which will be between this one and that one on the product backlog.”

Mike Cohn also suggested an alternative way, which might be in line with lean principles, called commitment-driven sprint planning. In this approach, the teams do not discuss story points or velocity. They simply take the high priority items from the backlog, task them out and estimate them in hours as per their capacity and just fulfill their commitment.

Thus, there are pros and cons of both the techniques used for sprint planning. It might just boil down to what the team is comfortable with. Which technique would you prefer and why?

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

This is like quantum computing... by Jim Leonardo

This debate comes up so often, I wonder if it isn't just a filler topic. "I don't know what else to talk about, so I talk about one of the old topics."
Even an hours based approach is subject to some level of ranging. I think the key is what you're doing, not whether you're expressing that as hours or story points. The difference here is more one of top-down vs. bottom-up estimating. I would venture to say that the most important factor to consider in choosing approach is who your management is. If they want to completely understand all the work before starting (especially when budget/time are important), than take the task oriented, bottom-up approach. If they are happy to accept some fuzziness and the risk that maybe something was overlooked that may add to the time line, than go top-down and don't waste your time on detailed planning.
In short, what we prefer to do must align with the needs of the people signing the checks.

Re: This is like quantum computing... by Tadatoshi Takahashi

Jim,

First of all, I enjoyed reading your comment.

But could you elaborate about "If they want to completely understand all the work before starting"? If it's possible to "completely understand all the work before starting", how come we are using Agile or Scrum? My understanding is that the reason why we use Agile or Scrum approach is that it is impossible to do so.

This debate comes up so often, I wonder if it isn't just a filler topic. "I don't know what else to talk about, so I talk about one of the old topics."
Even an hours based approach is subject to some level of ranging. I think the key is what you're doing, not whether you're expressing that as hours or story points. The difference here is more one of top-down vs. bottom-up estimating. I would venture to say that the most important factor to consider in choosing approach is who your management is. If they want to completely understand all the work before starting (especially when budget/time are important), than take the task oriented, bottom-up approach. If they are happy to accept some fuzziness and the risk that maybe something was overlooked that may add to the time line, than go top-down and don't waste your time on detailed planning.
In short, what we prefer to do must align with the needs of the people signing the checks.

A few comments by David karlsen

About "ideal days" measurement - the same variance is present for story points.
Humans do the work - humans have varying days.
The burn down measured over a [short] timeperiod will thus vary the same way.
So - when talking about time - or storypoints over a give timeframe - we have to use the effective time (e.g. ideal time). It's time on the task - not time as such (when measuring efficiency).

Regarding "Universal Measurement. Story point is a universal measurement across the team. It is not biased by the experience or skills or any individual on the team."
The same bias-ness is present whatever we measure it in.
If one teammember thinks it easier than another the same person will probably solve it in less time (for instance if they know the domain better or the technology at hand up front).
What it's measured in isn't the significant thing. This is probably why more even teams seem to work "better/more accurate" in an agile setting (when whoever in the team may pick the task).
Pairing might be a good idea if the team isn't well aligned - to even it out in the long term.

If you had the same team doing the same estimation - once in hours - once in story points - you could probably express the one unit as a function of the other.


A last point:
The evilness lies in the absolute figures.
Noting is X points nor X hours. It's an X with an uncertainty-level. (as somebody already have mentioned).
This of course gets more and more accurate the smaller the task is and the more accurate/distinctive it's expressed.

Measure by Test Cases that pass by Richard Perfect

The technique we like to use is based on using "Planned Test Cases". At the start of a sprint/iteration we estimate how many test cases it will take to complete the required functionality and then measure progress by counting the test cases that pass.

In the past we have found it difficult and time-consuming to to get developers to estimate time to completion. Developers are terrible at estimating because the work they do isn't the same each time - either it's a new business problem, or they are dealing with a new technology. Developers also have a tendancy to "hoard" left-over time, if they think they have been allocated so many hours to do something then the work "expands to fill the available time".

Counting test cases when they pass avoids a number of these problems. Even though you need to estimate the initial collection of planned test cases this is just as easy to estimate as the number of hours or points that something takes. The advantage is that you are estimating something that you can count. Also since the tester's generally write their scripts in parrallel with the developers the planned number becomes more accurate and tangible quite quickly. We even wrote a product about it - www.traceanalyst.com

Re: Measure by Test Cases that pass by Bruce Rennie


I actually really like the test case measurement.

Unfortunately, in my experience both with suggestions to do something like a test case measurement or even basic story points, most people just don't get it. I mean really don't get it. I've seen teams spend most of their time trying to correlate points to some time measurement and once they've made the correlation, time is all they talk about.

Some teams do get it and most of those would have no trouble with a measurement like passing test cases.

Re: This is like quantum computing... by Vikas Hazrati

Hi Jim,

This debate comes up so often, I wonder if it isn't just a filler topic. "I don't know what else to talk about, so I talk about one of the old topics."


I would not disagree with your point that this is an often discussed topic. If you would note the start of the post, that is how it starts.
There is a constant, long drawn debate on the benefits of using either story points or hours for sprint planning.


Till date there seems to be no right way of estimating in Sprint Planning. Both the camps seem to have valid reasons. The idea of this post is to look deeper into the community so that the Agile community can respond back with what is working for them. What are the benefits and pitfalls of one approach over the other?

Re: This is like quantum computing... by Jim Leonardo

Tadatoshi,
I didn't mean to imply that you would take the whole project to a task level before starting anything. I meant more "all the work for a story or feature". For my current project, our management doesn't want us to actually start on a given feature until we've done task level planning (and our tasks aren't very granular.. they are on the order of a day not on the order of 2 hours). They want to know reasonably well what they are getting into before they commit on their end to allowing the work to go ahead. My point was just to illustrate that we should be considering what the needs (fears) of the customer are.

Not a menu of choices, but a continuum of practice by Dave Nicolette

I don't think this recurring topic concerns different ways to do short-term planning, each of which is complete and final, to be selected from a menu of options and locked in as a permanent practice. I suspect people see it as such because they're looking for The Right Way to do short-term planning. In the spirit of continuous improvement, IMHO practitioners ought to be alert to opportunities to reduce their process overhead (without losing effectiveness, of course). From that perspective, these different methods of planning iterative work seem to be points on a continuum of practice, rather than a menu of choices.

Because of a wide range of factors, different teams in different environments may do their short-term planning in different ways. Some teams may be capable of working in a very lightweight fashion, but are constrained by organizational factors or upstream processes. Some organizations may be able to support very lightweight methods, but teams aren't ready to move away from comfortable assumptions about micro-level estimation, or managers aren't ready to give up some of their favorite (obsolete) metrics. None of that is right or wrong in itself; problems arise when people settle into a routine and forget to think about continuous improvement.

If a team can pull the highest-priority item from the backlog and drive it through to completion without bothering to estimate it, that's a pretty lightweight way to handle short-term planning. They spend all their time adding value and no time trying to guess how long it will take them to add value. This requires a high level of maturity in the application of agile thinking in the organization.

Many people are afraid to let go of conventional assumptions about the importance of fine-grained estimation for short-term planning. Sometimes it's the development team and sometimes it's their manager who clings to old assumptions. There is also the case when the backlog is not groomed ahead of each iteration in line with the idea of multiple planning horizons. The backlog consists of a sort of "wish list" with little or no analysis right up to the last minute. This constrains teams from dispensing with story-level sizing because they have no idea what is just about to be dropped onto them from one iteration to the next. In those cases, if a team can size user stories relatively, using an abstraction like story points, then this can be the most effective way to plan - at the present time, due to constraints, and not universally or eternally. The fact they have to do story sizing at all should be an indicator for them to look for opportunities to improve the process.

Yet another subset of practitioners haven't learned to separate the concept of story size from the concept of time. They may have to do estimation in terms of ideal time, not because it's The Best Way but because it's the best way the particular team can manage - at the present time, until they've learned to lighten up. People who ask how they can correlate story points with ideal time have not yet learned to separate the concepts properly. The correct answer to that question is to say that it is a non sequitur; the question itself is meaningless. The fact a team estimates in terms of ideal time should be an indicator for them to look for opportunities to improve the process. Teams should be alert to the common pitfall of equating a story point with some number of ideal hours, and then pretending they're sizing stories in terms of points. They may be hiding an opportunity for improvement from themselves by playing with words.

If a team is not using the lightest-weight method currently known, they should ask themselves what is constraining them from doing so, and direct their self-improvement efforts (or organizational change efforts) accordingly. If the team is functioning according to the lightest-weight method currently known, then they should ask themselves how they can push the envelope and light the way to the next step forward for the rest of us.

When people see this as a question of personal preferences, it's possible they will not think about estimation or sizing as an area for continuous improvement. Once you stop considering any given area of practice as potentially improvable, you have ceased to practice agile thinking.

Re: Not a menu of choices, but a continuum of practice by Chris Chapman

+1 Dave - Precisely.

Human way of estimating by Nick Oostvogels

The biggest advantage in using story points is the big analogy with the human way of estimating.
For instance, I never estimate my travel time exactly up to the minute.
I compare with previous similar journeys, based on how long they took plus some extra considerations (like a rainy day). I estimate if my travel time will be the same or not.
Basically, that's all I need to know. The exact minutes are not really relevant.
Same for sprint planning, as a product owner I want to know how many features the team will deliver, not how many hours each of them take. If I get an idea of the relative size, I can prioritize and plan accordingly.

Re: Sprint estimating in points vs hours and burn down chart by Tanvir Ahmed

when it comes to brun down chart, I am for displaying left over work in points, not hours hours are deceiving and they do not mean exaclty work done. they seem to represent as if hours spent = work done ( which is not true)

if you have sprint tasks estimation in hours and user stoties in points. you wont know for sure, how many points you completed end of the day when updating sprint backlog after stand up meetings.
for example:
you picked 5 user stories (10 points each)total 50 points for a sprint iteration of 4 weeks. each story takes average more than 3 days to complete. how would you know the progress on daily basis and draw sprint burn down chart in points unless you have estimated tasks in points as well.

thanks for clrifying this for me.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

11 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT