BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Is it Time to Stop Estimating User Stories?

by Todd Charron on Oct 01, 2011 |

Most new Agile teams transition from hours based estimates to relative estimation using story points, but do we even need estimates at all?

Michael Dubakov suggests the following reasons why you should stop estimating user stories:

  1. You don’t waste time on estimation
  2. You shouldn’t explain to higher managers why it took soooo loooooooong
  3. You don’t give promises that are hard to keep
  4. You don’t put additional pressure on development team
  5. You focus on really important things

Stephan Schwab, worries about how estimation can result in silos and prevent team based solutions:

Imagine an organization where teams are expected to have a fully estimated backlog in order to determine the cost for the project. At first glance such an approach seems to be a good idea. The people who will do the work will provide the estimate and thus based on what they say the true cost of the work will be known.

But then is the true cost of the project really known? What about all the discoveries that will be made once a team of smart people starts solving a problem? It seems unlikely that a few analysts will be able to analyze a problem and create a good solution expressed in small story cards for a 6 months project within a few weeks. They may be able to create 500 story cards over a few weeks but I think that will be all based on early assumptions. If the problem can be solved in a few weeks, then there is no need to pay the whole team over 6 months.

So to me it seems more like the attempt of predict the future, create a plan and then manage to the plan.

The fact that a team is asked to provide a fully estimated backlog – and a detailed one! – creates the prescriptive behavior and discourages the technical team members from developing a solution using the input from analysts, user experience person and product owner. In the end it should be no surprise, if the quality of the resulting “solution” is lower than expected. The team has been prevented from doing a good job.

Quality has been traded for false predictability and it is likely that this happened based on requests from the very same stakeholders who expect a high quality product.

What is much more important than to “calculate” cost is to build something of value. Something that can be used and in the end makes the stakeholders to want come back to the same people for extensions or with new ideas.

Mike Cottmeyer comes to the defence of estimates:

Since we don’t like managers and we don’t like death marches, we conclude the creation of software is an unpredictable process, that estimates are bad, and we should never estimate at all. Is it possible that we don’t really have an estimating problem? Is it possible that we have a bad management problem?

I don’t see any reason to stop estimating. In fact, unless your business model supports totally emergent outcomes, chances are you have some sort of goal, some sort of business outcome you are tracking toward that is tied to a hard, somewhat pre-defined deliverable. Chances are you’ve sold something to some customer and now you have to make good on that commitment.

We can debate this business model all day long, but if that’s your reality, you need estimates.

Estimates have to have ranges and probabilities. Assumptions have to be managed and risks have to be mitigated. We have to be able to measure how wrong our estimates are so that we can start to forecast that error forward.

We ignore what the data is telling us. We ignore the team’s actual performance against the estimate. There is so much competitive pressure to deliver, so much pressure to add those extra features by the end of next month so we can make the sale, so much pressure to deliver those features our sales team committed to win that big deal. So much pressure that we ignore reality.

The real problem is that, far too often, we oversell the organizations ability to deliver. The real problem lies in creating a ‘you have to deliver at all costs’ culture that doesn’t respect the teams established capacity to build working tested software.

Bad estimates become a problem, bad management becomes a problem, in the face of unyielding pressure to deliver against unreasonable expectations and inflexible project schedules and commitments.

Bob Marshall in the same article adds his comments:

Further, I regularly feel compelled to ask “why estimate?”. Until folks grok the requirements for which estimation is (most often) an implicit solution, how can anyone say with any certainty that estimation has any value?

Many believe Kanban has done away with estimates, Karl Scotland clarifies the situation:

Kanban teams do not eschew estimation simply because it is hard. Some teams choose not to estimate because they can realise the same benefits that estimation gives with a lower cost. The old joke does still apply (“Doctor, Doctor it hurts when I do this”, “Don’t do that then”). If it hurts to estimate, then find other ways to encourage whole team interaction and collaboration, learn about past capability and forecast future capability. On the other hand, if estimation doesn’t hurt, or costs too much, then it may be the right thing to do in your context.

When we are planning, we should decompose work for understanding, rather than sizing.

Jeff Anderson gives another take on estimating in Kanban:

As teams progress, work accepted will typically become categorized into better understood work types that have less variability. i.e. team A might typically work on java enhancements, emergency defects, and canned reports.

The most important part of estimating is breaking up the work into small pieces, once you've done that, the only reason to care about differences in the small range is to provoke discussion.

While it seems the value of the output of estimation may be in doubt, it seems the real benefit of estimation comes from the whole team conversations that occur along the way. What have your experiences been?

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Estimation is not optional by Josh Gough

I believe, at least in most organizations, that estimation is not optional.

What's more, estimates usually become targets, and usually become commitments.

Steve McConnell's book, "Software Estimation: Demystifying the Black Art" advises us as rule #1 to "Distinguish between estimates, targets, and commitments", however.

A reality that is consistently faced is that a team, company, or maybe just a single developer will be brought into a troubled situation. A product owner or stakeholder will ask, "How long is this going to take?"

The answer is usually something like "About two weeks", or "About 3 months", or, "About a year", etc.

As developers, we understand these to be "guesstimates" or even "wishtimates", but what are they based on? Are they based on historical data? Are they based on relative size estimates?

Usually not. More often than not they are based on political and psychological factors.

Of course, inexperienced leadership will be happy to hear the answers that give them a false sense of a short time-line.

McConnell and others have worked with or developed historical models and algorithms for helping "forecast" schedules. (See en.wikipedia.org/wiki/Software_development_effo...)

As a simplistic summary: one of the most important things when estimating is to start to simply "count up" the number of screens, buttons, reports, widgets, inputs, outputs, etc, etc, etc, etc.

This helps size things, but it certainly does not account for variability in situational complexity or rework, or developer skill, etc, etc. McConnell's company provides a free tool called "Estimate" that helps people do this and it computes possible outcomes over a number of samplings, based on industry data or your own historical data. See www.construx.com/Page.aspx?nid=68.

But, what do you do when managers or stakeholders don't seem interested in applying estimation models or using as close to "scientific" methods as can be? What can you do when they won't accept a "range with risk", but want a "hard date"?

I cannot pretend to have any magic solution or advice for this problem. It *is a problem*, and I don't see it going away soon.

As an example: a team I work with has recently been in this exact situation (new management being brought in). What has helped us, but not solved this problem, has been that some of our new management and stakeholders have been exposed to the sheer complexity and volume of work that remains in our project in order for it (a large scale rewrite) to be considered DONE-DONE. They have a strong interest in seeing this done properly, and as thus have been sitting with our team for about two weeks while we (and THEY) draw on whiteboards and fill up flip charts with "TODOs", bring up workflow block diagrams, state-transition diagrams, etc.

All of this VISIBILITY has helped them begin to grasp the volume.

Early on we heard talk of arbitrary, magical dates, such as "June 30th, 2012". Now, even the person who was at first the most interested in a magic date has recently said, "We know this is going to take *at least* a year."

So, agile is all about visibility and collaboration. But that's perfect-world for most of us. Even in "waterfall mentality" places, you can still work collaboratively and help people SEE and grasp the volume of your work.

Resist, as best you can, the temptation to quickly name a "single date". If at all possible, take time to make counts, compare, compute, etc, etc.

If someone is pressuring you to "convert" an estimate into a target and fast-track that into a commitment, make sure you make the effort to do whatever you can to improve those estimates with real empirical data and numbers. Help them visualize the complexity, the testing, the risk.

Best of luck

Re: Estimation is not optional by Tim Schraepen

Mike Cottmeyer speaks truth.
Nice addition Josh Gough.
Here's some "experience" of my own.

As a developer I hate estimating. Especially on fixed-price projects.

We create a backlog that is estimated when we make a proposal, and give some sort of a margin that can swing either way (in favor of the customer or us). But this margin still isn't enough sometimes.
What it usually comes down to is developers that need to estimate are very weary about giving estimates, so they "add a little" here and there. But of course, when asked why something "so simple" can take "so long", developers can never provide arguments. They can only say "Well, we based our estimates on complexity", and sooner or later figure out they have to bring down their estimates. Knowingly screwing themselves for the future.

I have asked myself maybe a million times why it happens everytime and what we can do about it.
Maybe we should try and analyse the stories more before estimating. But we try and re-estimate before we really start a project, and even in that phase we usually underestimate. We also overestimate some other stories though, and I think it sort of equals out in the end.
During the project though, there's always a lot of stress when the team wasn't able to finish the scope of the sprint, or when one particular story is not doing so well "ETC"-wise.

Considering this experience, it sounds a lot like we are indeed using estimates both as a budget and planning metric (targets and commitments).
And I will keep this in mind next time we are re-estimating our stories for the next sprint.

Oh, and I agree about the visibility Josh mentioned. This really is a key ingredient to be able to present valid arguments for stories that took longer than estimated. But in itself it (the visibility) doesn't really have anything to do with estimation itself. I'd rather have good estimates, or none at all.

Another thing I'd like to remark is that customers usually WANT their project to be delivered within a certain period of time, within a certain budget, with a certain quality. They want this, but since we can't really guarantee anything, there's too little trust to allow us to build their project on "times and material", and opt for the "safe" "fixed-price" model.

Re: Estimation is not optional by Josh Gough

Tim, good observations.

Regarding the self analysis and wondering why, I do the same thing. The most successful experience I had with this was when flipping the game around.

Instead of dreading estimation, I started talking about how important it was to the manager who was saying, "We need to pick up estimation as a practice." Although, I feel like he meant something more like: "You guys need to deliver more quickly so that the off-the-cuff wisthimate I gave the CEO can happen, and I look like a genius."

I brought Mike Cohn's book into the office, and our team watched his SF XP videos on YouTube. Now, of course, that book is more about a collaborative, conversational estimation process, not just a "you better work overtime for my sake" commitment dictation.

So, it ended up helping both sides be more realistic.

But, that was for a commerce company, not a project based consultancy. So, the problem of unrealistic schedules and wishes persists.

I wonder if there are any kinds of simple games that could illustrate the reason why things will "take longer" than someone wishes.

I often love the "If I came to you and said paint my house, and you cannot tell me how long it will take or cost, why I hire you?"-pseudo-analogy to software estimation.

When I get the chance to respond to these kind of half-baked analogies, I usually say:

You make a great point, and there is something very important to highlight: when you come to me with that request, I never tell you a flat rate or time. Instead, I ask some simple questions like: "How many rooms do you have? What are the measurements? How much furniture is in the way? What color?" If you do not know the answers to these things, such as the room measurements, then I would need to go measure myself. You are, right, estimating this project is similar, except it is a lot more complicated, and has a lot more moving parts, so that is why the best thing we can say unless you have some more detailed specifications that you do not plan to change is that it will fall within a certain range. I know this range is wide, but unless the specification is very clear, it is the most honest and responsible thing I could tell you right now. You might find someone who could tell you a date off the cuff, but ask yourself, would you really want that guy painting your house if he told you "about a week" before he bothered to ask you how many rooms and what they measured?

To demonstrate agile, empircally based do-some-first-then-forecast style estimation, I asked a stakeholder if he would feel comfortable signing to read a 100 page book in 60 minutes. He said no. I said, ok but what if I told you that each page is on average the same number of words and complexity? He said he could read five to ten pages then forecast the rest.

I said that was exactly what agile planning is with software, except instead "Page Velocity", we talk about "story points" and story velocity.

That worked with him, and has started reading up on agile.

It does not work with everyone, least of all with people who themselves have people above them in towers of power casting down their own arbitrary Magic Dates.

Re: Estimation is not optional by Adam Nemeth


when asked why something "so simple" can take "so long", developers can never provide arguments. [..]
I have asked myself maybe a million times why it happens everytime and what we can do about it.


Well, that's why you should use a model-based estimation. Do whatever fits, I still use FPA, with some additional factors depending on the project, and I can simply defend "Look, you told us you need 3 features, each involving 2-3 different classes of data, one of them is involving some external systems we don't have a clue about, and also you need 4-5 customer facing and 1-2 administrative interfaces for each. Also, it usually takes 2-3 days per interface to get from the design department, and this external system is so unknown that it might end up being a month to solve it. Here's the excel sheet, the number I quoted is in the corner, I know it's a lot, but you asked us how much will it take, and that's the best answer we could tell you based on our experience on how much these things take, I'm sorry."

Estimation is needed as business people have to make promises to other business people. Estimation is possible once you know what you do. If you don't know what you do, you should either estimate it with a high margin, or find someone who has experience on the given field. You should not turn the world around just simply because you accepted a job you have not enough experience to foresee what'll happen.

When changes are happening, estimations can change too. Just make sure when to advertise this fact.

Re: Estimation is not optional by chen chuan

Josh, I love your "painting house" example. I will borrow it next time talking about estimation :)

Re: Estimation is not optional by Philopator Ptolemy

I personally agree with 100% of what you wrote. The points that you brought make perfect sense, but, unfortunately, they appeal only to rational and honest individuals. Quite often estimates are rejected because of hidden or emotional reasons like "You guys need to deliver more quickly so that the off-the-cuff wisthimate I gave the CEO can happen, and I look like a genius."

I often found that no amount of requirements elaboration, or FPA analysis, or explanations about the Cone of Uncertainty helps in convincing that DESIRED estimates cannot be reliable (accurate) or realistic (fit within perceived time frame).

But they just want it...

As McConnell wrote in his Rapid Development "In a few instances, the customer's interest in rapid development masks a desire to improve rapid development's bottom line by eking out as much unpaid overtime as possible".

Re: Estimation is not optional by Philopator Ptolemy

Estimation is possible once you know what you do. If you don't know what you do, you should either estimate it with a high margin, or find someone who has experience on the given field. You should not turn the world around just simply because you accepted a job you have not enough experience to foresee what'll happen.


Unfortunately, it is very hard to include the "factor of unfamiliarity" in the estimates. The customers want estimates as if you've done 10 similar systems, with the similar team and with the similar technologies.

Confusion between essential complexity and accidental complexity by Philopator Ptolemy

I have also noticed that very often customers, PMs, etc. confuse accidental complexity with essential complexity. Something that is conceptually simple is not necessarily easy or fast to implement. As Mike Cohn said "licking 1000 timestamps is much simpler than performing a brain surgery, but it takes more time" (not a literal quote).

Re: Estimation is not optional by Adam Nemeth

Unfortunately, it is very hard to include the "factor of unfamiliarity" in the estimates. The customers want estimates as if you've done 10 similar systems, with the similar team and with the similar technologies.


Wait 5-10 years, and this will come to you. If you look at the right variables, it might come to you sooner.

After all, an information processing application is an information processing application - you can call a rose by any name...

Estimation & Complexity by Harold Shinsato

Thought it might be helpful to contrast the natural desire for estimates and a deadline and the difficulty for providing a certain date in the face of complexity and uncertainty.

Three point estimation gives some exposure to such concepts when estimating - where you say when you give a best case, likely case, and worst case scenario. I've observed greatly improved estimations when probabilities are allowed for rather than ignored. But even with 3 point estimates there were times I was tempted to say infinity for the worst case. Some problems might not be solvable and then a task will need to be scrapped and replaced with a new approach. Good car mechanics will refuse to give estimates in times of uncertainty. It would be an improvement to software engineering if we had the discipline to refuse to give estimates in the face of such uncertainties.

This topic was clarified further for me because of Simon Bennett's recent talk at Agile 2011 about combining the complexity framework Cynefin. His presentation, "Scrum as a Thinking Toolkit", currently has no slides attached so instead maybe look at this blog: www.cognitive-edge.com/blogs/guest/2011/06/is_s....

It helps to understand Dave Snowden's Cynefin framework (en.wikipedia.org/wiki/Cynefin) which divides the domain of issues and problems into simple, complicated, complex, chaotic, and disordered. My sense is that breaking large user stories into smaller parallel stories helps move things closer to the simple or at lease complicated area in the framework, which helps make a sprint iteration estimable. But it might be important to recognize the variability especially when communicating likelihood of hitting a larger extended plan - and perhaps it is also why shorter release cycles can help order emerge in complex domains much more successfully than long release cycles.

Re: Estimation & Complexity by Harold Shinsato

The blog entry was not from Simon Bennett but only mentions him - though it still is an interesting article. Here's a PDF from a workshop Simon taught with Rowan Bunning at the Agile Australia 2011 conference: www.slatteryit.com.au/Agile2011/preso/Agile-Aus...

Burn down points, not hours! by Theron James

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

12 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT