Accurate Estimates - the ultimate oxymoron?

Posted by Ben Hughes on May 20, 2007

Sections

Process & Practices,

Architecture & Design

Topics

Delivering Quality ,

Quality ,

Delivering Value ,

Agile in the Enterprise ,

Software Craftsmanship ,

Lean ,

Agile Techniques ,

Value & Metrics ,

Agile ,

Productivity ,

Simplicity

In a two part post on his blog, Amit Rathore discusses the concept of an extreme lean approach - the art of estimating and the value it has in the software development process, by looking at the real impact that time based task estimates have on the software produced. Following a fictitious Web 2.0 start up and assuming quality is non negotiable, some key points are raised:

• What value is gained by translating the gut feelings of experts into hard estimates?
• Traditional planning assumes a small number of variables, whereas the reality (like the weather) is much more complex;
• Questions the value of applying time based estimates  - based on the fact that time spent estimating adds no value to either the customer or the software.
  

The thing to note, also, is that if we’d spent the time to do detailed estimation (at the task level for each story in the sprint) – we’d have lost another half-day to a day every iteration, and we’d still have no real change in the outcome. Sure we’d have data as to how accurate (or entirely off) our development team was with their estimates, but that would be about it… it wouldn’t change the amount of time they’d have taken. And we’d be short about 5 to 10 percent of the time to actually build the software. Oh boy, we sure made a good decision of not going even slower!

Amit goes on to propose simply using the velocity as a measure of when the project is going to deliver - drawing the parallels of one Redmond based software manufacturer....

After all, what software team ever announced a date that they actually met? These days, many companies just stick the number of the year at the end of the name of their product - Office 2007, Windows 2003, Pocket PC 2006. They’ve given up the charade - they’re now saying they’ll just deliver it sometime in the year.

In summary, given the lean approach and the agile's focus on quality and value delivery the author raises some valid points on the value that time based task-level estimating brings to the party and questions whether they should be invited at all.

1. Back to top

   Estimates allow a team to determine their real velocity

   by Deborah Hartmann

   > simply using the velocity as a measure
   
   Though I've note yet read the blog entry, still I'll venture this:
   
   I teach teams to estimate and track how they deliver against that estimate (using StoryPoints or IdealHours and a Sprint burndown), precisely because that's how we learn (over time) our real velocity.
   
   It's true that mature teams sometimes stop estimating and use only velocity to plan. The first time they do this, they are using "yesterday's weather", a familiar Agile pattern. The second time they do it, they are using "last week's weather", and the next time "last month's weather". This analogy suggests that eventually they could find themselves using "Christmas's weather" in July :-)
   
   But I'll take the time to read the entry when I get home, too :-) I'm at RoCoCoCamp in Montreal and having a blast!

   Reply

2. Back to top

   Estimates

   by fryse lager

   I dont buy the argument:
   "it wouldn’t change the amount of time they’d have taken. And we’d be short about 5 to 10 percent of the time to actually build the software. Oh boy, we sure made a good decision of not going even slower!"
   
   Not doing estimates is like fombeling around in the dark.

   Reply

3. Back to top

   Re: Estimates

   by Ben Hughes

   I wouldn't agree - I think we have to really look at the value a process delivers that is (almost) always flawed (see Google's take on performance appraisals). By flawed I mean usually fails/results in conflict of some kind.
   
   As mentioned in the above comment, there is much more value in "Yesterday's weather" and gut feel than there is punting on exactly how many days it will take to complete a task.

   Reply

4. Back to top

   Estimates are often needed to charter a project...

   by Robert Merrill

   Here are a couple of reasons why estimation is still time well spent.
   
   1. An overall estimate is usually needed to actually start a project within a business. It's certainly needed for contracting situations, which is where I come from.
   
   2. Estimating stories/features is a good way for a team to work through the stories and get to know them.
   
   3. Estimation at the story or feature level is also necessary to allow the customer to prioritize, which is a critical component of an agile process.
   
   4. I've seen a lot of teams respond well to a reasonable target, especially one they've set themselves. Setting those cycle targets and hitting them builds confidence all around.
   
   Once you've done these things, the overall estimate to complete a given scope just falls right out, with no extra work. Estimating at the story level is something you need to do anyway.

   Reply

5. Back to top

   Re: Estimates are often needed to charter a project...

   by Alex Popescu

   I have always considered project estimation discussion a chicken and egg problem. As people have already mentioned, most of the time the business will not start without estimates. But estimates upfront are most of the time guesstimates, cause you will not know exactly without building it. These initial estimates are usually based on experience (and at best on prototyping), but most of the time you are not asked to rebuild the system X, so that your experience will serve you well 100%. And a prototype is still a prototype. What is more important in my opinion is the re-adjustment process, the weekly/iteration based re-adjustment. Unfortunately, not everybody likes this part and will almost always mention your initial estimates as the "promises" you made.
   
   ./alex
   --
   .w( the_mindstorm )p.
   ________________________
   Alexandru Popescu
   Senior Software Eng.
   InfoQ TechLead&CoFounder

   Reply

6. Back to top

   Re: Estimates are often needed to charter a project...

   by Deborah Hartmann

   Until we estimate at the story level, we don't have any idea whether a feature is this big ### or this big ################# because of the assumptions unrevealed at that point. As long as this uncertainty is made very clear, up front estimates can serve some general purpose. For a brief moment :-) until things change, lol.
   deb

   Reply

7. Back to top

   Why are programmers special?

   by Bruce Rennie

   I've often wondered why we, as a profession, struggle so much with this. When I call a plumber or take my car to a mechanic, they're not always sure of what they'll find but I'd be crazy to just let them start work with a "when it's done, it's done".
   
   In fact, mechanics are pretty agile. They're give an estimate and, if they find that the work is going to exceed that initial estimate, they'll call me and get a thumbs up on the new work.
   
   It's not that estimating is bad, it's our industry's expectation that an estimate is a commitment, not an estimate.
   
   I also agree with Deb. If you don't estimate up front, how exactly are you computing your velocity? Estimating in story points is still estimating.

   Reply

8. Back to top

   Re: Why are programmers special?

   by Mark Peterson

   The difference between a programmer and an auto mechanic is, like you said, when the mechanic finds that it's going to exceed the estimate he is allowed to give a new estimate. Programmers, more often than not, are held to the original estimate no matter how big the project grows. For some reason people have the mindset that since we're not creating solid objects we should be able to manipulate applications with little or not added time.

   Reply