Is it Time to Stop Estimating User Stories?

I believe, at least in most organizations, that estimation is not optional.

What's more, estimates usually become targets, and usually become commitments.

Steve McConnell's book, "Software Estimation: Demystifying the Black Art" advises us as rule #1 to "Distinguish between estimates, targets, and commitments", however.

A reality that is consistently faced is that a team, company, or maybe just a single developer will be brought into a troubled situation. A product owner or stakeholder will ask, "How long is this going to take?"

The answer is usually something like "About two weeks", or "About 3 months", or, "About a year", etc.

As developers, we understand these to be "guesstimates" or even "wishtimates", but what are they based on? Are they based on historical data? Are they based on relative size estimates?

Usually not. More often than not they are based on political and psychological factors.

Of course, inexperienced leadership will be happy to hear the answers that give them a false sense of a short time-line.

McConnell and others have worked with or developed historical models and algorithms for helping "forecast" schedules. (See en.wikipedia.org/wiki/Software_development_effo...)

As a simplistic summary: one of the most important things when estimating is to start to simply "count up" the number of screens, buttons, reports, widgets, inputs, outputs, etc, etc, etc, etc.

This helps size things, but it certainly does not account for variability in situational complexity or rework, or developer skill, etc, etc. McConnell's company provides a free tool called "Estimate" that helps people do this and it computes possible outcomes over a number of samplings, based on industry data or your own historical data. See www.construx.com/Page.aspx?nid=68.

But, what do you do when managers or stakeholders don't seem interested in applying estimation models or using as close to "scientific" methods as can be? What can you do when they won't accept a "range with risk", but want a "hard date"?

I cannot pretend to have any magic solution or advice for this problem. It *is a problem*, and I don't see it going away soon.

As an example: a team I work with has recently been in this exact situation (new management being brought in). What has helped us, but not solved this problem, has been that some of our new management and stakeholders have been exposed to the sheer complexity and volume of work that remains in our project in order for it (a large scale rewrite) to be considered DONE-DONE. They have a strong interest in seeing this done properly, and as thus have been sitting with our team for about two weeks while we (and THEY) draw on whiteboards and fill up flip charts with "TODOs", bring up workflow block diagrams, state-transition diagrams, etc.

All of this VISIBILITY has helped them begin to grasp the volume.

Early on we heard talk of arbitrary, magical dates, such as "June 30th, 2012". Now, even the person who was at first the most interested in a magic date has recently said, "We know this is going to take *at least* a year."

So, agile is all about visibility and collaboration. But that's perfect-world for most of us. Even in "waterfall mentality" places, you can still work collaboratively and help people SEE and grasp the volume of your work.

Resist, as best you can, the temptation to quickly name a "single date". If at all possible, take time to make counts, compare, compute, etc, etc.

If someone is pressuring you to "convert" an estimate into a target and fast-track that into a commitment, make sure you make the effort to do whatever you can to improve those estimates with real empirical data and numbers. Help them visualize the complexity, the testing, the risk.

Best of luck

Mike Cottmeyer speaks truth.
Nice addition Josh Gough.
Here's some "experience" of my own.

As a developer I hate estimating. Especially on fixed-price projects.

We create a backlog that is estimated when we make a proposal, and give some sort of a margin that can swing either way (in favor of the customer or us). But this margin still isn't enough sometimes.
What it usually comes down to is developers that need to estimate are very weary about giving estimates, so they "add a little" here and there. But of course, when asked why something "so simple" can take "so long", developers can never provide arguments. They can only say "Well, we based our estimates on complexity", and sooner or later figure out they have to bring down their estimates. Knowingly screwing themselves for the future.

I have asked myself maybe a million times why it happens everytime and what we can do about it.
Maybe we should try and analyse the stories more before estimating. But we try and re-estimate before we really start a project, and even in that phase we usually underestimate. We also overestimate some other stories though, and I think it sort of equals out in the end.
During the project though, there's always a lot of stress when the team wasn't able to finish the scope of the sprint, or when one particular story is not doing so well "ETC"-wise.

Considering this experience, it sounds a lot like we are indeed using estimates both as a budget and planning metric (targets and commitments).
And I will keep this in mind next time we are re-estimating our stories for the next sprint.

Oh, and I agree about the visibility Josh mentioned. This really is a key ingredient to be able to present valid arguments for stories that took longer than estimated. But in itself it (the visibility) doesn't really have anything to do with estimation itself. I'd rather have good estimates, or none at all.

Another thing I'd like to remark is that customers usually WANT their project to be delivered within a certain period of time, within a certain budget, with a certain quality. They want this, but since we can't really guarantee anything, there's too little trust to allow us to build their project on "times and material", and opt for the "safe" "fixed-price" model.

Tim, good observations.

Regarding the self analysis and wondering why, I do the same thing. The most successful experience I had with this was when flipping the game around.

Instead of dreading estimation, I started talking about how important it was to the manager who was saying, "We need to pick up estimation as a practice." Although, I feel like he meant something more like: "You guys need to deliver more quickly so that the off-the-cuff wisthimate I gave the CEO can happen, and I look like a genius."

I brought Mike Cohn's book into the office, and our team watched his SF XP videos on YouTube. Now, of course, that book is more about a collaborative, conversational estimation process, not just a "you better work overtime for my sake" commitment dictation.

So, it ended up helping both sides be more realistic.

But, that was for a commerce company, not a project based consultancy. So, the problem of unrealistic schedules and wishes persists.

I wonder if there are any kinds of simple games that could illustrate the reason why things will "take longer" than someone wishes.

I often love the "If I came to you and said paint my house, and you cannot tell me how long it will take or cost, why I hire you?"-pseudo-analogy to software estimation.

When I get the chance to respond to these kind of half-baked analogies, I usually say:

You make a great point, and there is something very important to highlight: when you come to me with that request, I never tell you a flat rate or time. Instead, I ask some simple questions like: "How many rooms do you have? What are the measurements? How much furniture is in the way? What color?" If you do not know the answers to these things, such as the room measurements, then I would need to go measure myself. You are, right, estimating this project is similar, except it is a lot more complicated, and has a lot more moving parts, so that is why the best thing we can say unless you have some more detailed specifications that you do not plan to change is that it will fall within a certain range. I know this range is wide, but unless the specification is very clear, it is the most honest and responsible thing I could tell you right now. You might find someone who could tell you a date off the cuff, but ask yourself, would you really want that guy painting your house if he told you "about a week" before he bothered to ask you how many rooms and what they measured?

To demonstrate agile, empircally based do-some-first-then-forecast style estimation, I asked a stakeholder if he would feel comfortable signing to read a 100 page book in 60 minutes. He said no. I said, ok but what if I told you that each page is on average the same number of words and complexity? He said he could read five to ten pages then forecast the rest.

I said that was exactly what agile planning is with software, except instead "Page Velocity", we talk about "story points" and story velocity.

That worked with him, and has started reading up on agile.

It does not work with everyone, least of all with people who themselves have people above them in towers of power casting down their own arbitrary Magic Dates.

when asked why something "so simple" can take "so long", developers can never provide arguments. [..]
I have asked myself maybe a million times why it happens everytime and what we can do about it.

Josh, I love your "painting house" example. I will borrow it next time talking about estimation :)

I personally agree with 100% of what you wrote. The points that you brought make perfect sense, but, unfortunately, they appeal only to rational and honest individuals. Quite often estimates are rejected because of hidden or emotional reasons like "You guys need to deliver more quickly so that the off-the-cuff wisthimate I gave the CEO can happen, and I look like a genius."

I often found that no amount of requirements elaboration, or FPA analysis, or explanations about the Cone of Uncertainty helps in convincing that DESIRED estimates cannot be reliable (accurate) or realistic (fit within perceived time frame).

But they just want it...

As McConnell wrote in his Rapid Development "In a few instances, the customer's interest in rapid development masks a desire to improve rapid development's bottom line by eking out as much unpaid overtime as possible".

Estimation is possible once you know what you do. If you don't know what you do, you should either estimate it with a high margin, or find someone who has experience on the given field. You should not turn the world around just simply because you accepted a job you have not enough experience to foresee what'll happen.

I have also noticed that very often customers, PMs, etc. confuse accidental complexity with essential complexity. Something that is conceptually simple is not necessarily easy or fast to implement. As Mike Cohn said "licking 1000 timestamps is much simpler than performing a brain surgery, but it takes more time" (not a literal quote).

Unfortunately, it is very hard to include the "factor of unfamiliarity" in the estimates. The customers want estimates as if you've done 10 similar systems, with the similar team and with the similar technologies.

Thought it might be helpful to contrast the natural desire for estimates and a deadline and the difficulty for providing a certain date in the face of complexity and uncertainty.

Three point estimation gives some exposure to such concepts when estimating - where you say when you give a best case, likely case, and worst case scenario. I've observed greatly improved estimations when probabilities are allowed for rather than ignored. But even with 3 point estimates there were times I was tempted to say infinity for the worst case. Some problems might not be solvable and then a task will need to be scrapped and replaced with a new approach. Good car mechanics will refuse to give estimates in times of uncertainty. It would be an improvement to software engineering if we had the discipline to refuse to give estimates in the face of such uncertainties.

This topic was clarified further for me because of Simon Bennett's recent talk at Agile 2011 about combining the complexity framework Cynefin. His presentation, "Scrum as a Thinking Toolkit", currently has no slides attached so instead maybe look at this blog: www.cognitive-edge.com/blogs/guest/2011/06/is_s....

It helps to understand Dave Snowden's Cynefin framework (en.wikipedia.org/wiki/Cynefin) which divides the domain of issues and problems into simple, complicated, complex, chaotic, and disordered. My sense is that breaking large user stories into smaller parallel stories helps move things closer to the simple or at lease complicated area in the framework, which helps make a sprint iteration estimable. But it might be important to recognize the variability especially when communicating likelihood of hitting a larger extended plan - and perhaps it is also why shorter release cycles can help order emerge in complex domains much more successfully than long release cycles.

The blog entry was not from Simon Bennett but only mentions him - though it still is an interesting article. Here's a PDF from a workshop Simon taught with Rowan Bunning at the Agile Australia 2011 conference: www.slatteryit.com.au/Agile2011/preso/Agile-Aus...

rapidscrum.com/20110322%20AgileIndy%20PDF.pdf