BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Estimating on agile projects: what’s the story, what’s the point?

Posted by David Morris on Nov 21, 2013 |

Introduction

Before you commission a painter to decorate your home or a mechanic to fix your car, you get an estimate from them, right? You need to know how much it’s likely to cost and how long it might take. It’s just common sense.

What does experience tell us, however? How close are those original estimates to the final bill? It’s all too likely that the painter will find loose plaster that needs removing and the wall needs rendering and re-plastering. The mechanic is sure to find additional work required to get your car roadworthy again. In a 1951 cartoon for the New Yorker magazine, Syd Hoff drew a mechanic saying to his customer,

“Of course, that’s only an estimate; the actual cost will be more”.

If the painter or mechanic tells us in enough time, we can choose not to take on the extra work … and yet, all too often, we feel we have to fix these additional things. Who wants to live in a house with a potentially damp wall or drive in a car with potentially faulty steering?

How do we overcome this? A common reaction is to insist on a full and final fixed-price estimate, or ‘quote’ as it’s commonly called; so they work harder and longer to get more accurate estimates. Yet however hard they work, they still cannot really predict the unexpected.

In the life of projects, this is just the same

Traditionally, project managers tend to focus on creating detailed estimates that can withstand scrutiny from the finance team. Of course, this is based on ‘known knowns’ with some contingency for the ‘known unknowns’ … and as Donald Rumsfeld is famously quoted “there are also unknown unknowns – there are things we do not know we don't know”. Like the tradespeople above, we can never really predict the unexpected.

The more we invest in creating elaborate estimates, however, the more problems we cause. Detailed estimates can be seen as a binding quote, a target that distracts from delivering value, and focusing on delivering something—or even anything—to the agreed date and cost.

We beat ourselves up in post-implementation reviews, and tell ourselves to just try harder; Einstein, however, defined insanity as “doing the same thing over and over and expecting different results”; so there must be a better way.

Surely this isn’t true for agile projects? Don’t we just start with a high-level scope and work it out as we go along? Well, yes and no (or as they would say here in New Zealand, “yeah … nah”). While we don’t hamstring ourselves by creating detailed estimates, it is still vital that we get a feel for the size of the work we’re considering, and here’s why …

Why we need estimates

Forget about estimates being used to build beautifully crafted Gantt charts that force us to focus on tasks and not outcomes. There are three outcomes that need us to get a rough handle on how big something is.

  1. When we’re considering the justification for a proposed project, we need to understand the likely costs in advance, so that we can decide whether it is worth the investment.
  2. When we’re launching new or improved products to market, we need some idea of roughly when significant features might be ready for release, so we can plan associated activities.
  3. When we’re prioritising work, the product owner needs to understand the cost, and the team needs to understand the value, of each story (or backlog item).

It’s also interesting to note that estimating can be a really healthy activity, so long as the whole team collaborates on it together. It helps foster a buy-in from the whole team and ensure that everyone gets a common understanding of the scope and value to be delivered.

However the label of ‘estimates’ can be distracting.

Use sizes rather than estimates

To avoid setting an expectation that we’re talking about cost and time, when we estimate the complexity of a story, some of us like to refer to this as ‘sizing’ rather than estimating. In the 90’s when I first used Scrum and XP—before they were even called agile practices—we sized stories using t-shirt sizes (S, M, L, and XL).

Now, however, we use story points – a way of sizing stories relative to each other – so we find what will be the simplest story, size that as 1-point story, then compare another story to that, if it’s more complex than maybe it’s a 3-point story.

To make things more interesting, we don’t use a simple sequence like 1, 2, 3, 4, 5, etc. – instead we use a modified form of the Fibonacci sequence like 1, 2, 3, 5, 8, 13, etc. (as seen in ‘The Da Vinci Code’). This makes the jump between numbers larger the higher we go and drives us to choose which is smaller or larger.

While this is not an exact science, it is more than good enough for the three outcomes mentioned above, and as John Maynard Keynes said, “it is better to be roughly right, than precisely wrong”. This means, though, we still need to translate story points to rough time and rough cost.

Another practice central to agile projects is establishing the ‘definition of done’, that is to have a comprehensive understanding of everything required to say that a story is done and releasable, including items like user documentation, translation, advertising, etc. etc.

Provided we have a good definition of done, we can then take a couple of sample stories and calculate the effort required. From that we can get rough cost estimates for an investment decision, rough timing for release planning, and enough understanding to assist in story prioritisation.

Some, however, still find estimating by story points a distraction, and one response has been the debate around #NoEstimates.

So what’s with the #NoEstimates debate

If you have been following any of the major influencers on Twitter, you might have noticed some getting involved in discussions with the hashtag of #NoEstimates. While this sounds like a call to cease estimating altogether, it is really a call to stop sizing stories and instead just start developing.

This discussion arose, and has gained momentum, because the experience of those working on large projects has been that whether you size by story points or just count the number of stories, the tracking of throughput is about the same.

While this might partly be down to the more experienced delivery teams decomposing stories to consistently smaller sizes on a just-in-time basis—as this enables greater throughput of potentially shippable increments—the reporting looks similar even on projects still working with a range of very small to moderately large stories in each iteration.

While this makes story prioritisation simpler and enables the delivery team to get on with it quicker; our poor product owners and sponsors still need to know roughly how much it will cost and how long it will take. The good news is that we can still do this based on counting the number of stories, once we have sufficient metrics to make this feasible.

For any new team starting out on your agile journey, however, I would still strongly recommend you size with story points until you’ve reached the experience and maturity levels of these teams.

Conclusions

So to sum up, and to mangle a famous quote from General Dwight D. Eisenhower: "In preparing for [projects] I have always found that [estimates] are useless, but [estimating] is indispensable"; we still need to do it, whatever we call it and whatever we’re counting when we do it.

I am indebted to Esther Derby (estimates become targets), Mike Cohn (estimates for release planning and prioritisation), Ahmed Sidky (whole-team estimation), Martin Fowler (story points), Ian Mitchell (definition of done), Vasco Duarte (story count vs story points), Stephen Forte (metrics vs estimates), Neil Killick (experienced teams define smaller stories), amongst many others, for sharing their thinking on this topic – and I have linked to their respective articles in context above.

About the Author

David Morris is an independent business agility practitioner, coach, and instructor; working through his company Sophorum, delivering business analysis, team leading, coaching, and training services to customers throughout New Zealand. With nearly 30 years’ experience in project delivery, he has worked in and led teams and run his own business across strategic, business, and technical projects following structured, iterative, and agile methodologies.

David is a qualified CBAP, ICAgile Certified Practitioner, Certified ScrumMaster, and an IT Certified Professional. He runs study groups, professional development workshops, and bootcamps; helps organise events for Agile Auckland and IIBA NZ; has spoken at conferences and events in Europe and Australasia; contributed to several books (including the ‘Agile Extension to the BA Body of Knowledge’); and is an active blogger, tweeter, and Wikipedia editor.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Apply the 5 whys by Steven Gordon

The key to what any truly agile team agrees to do is understanding who will use what they do and for what purpose. Then the team can propose more effective and cost-effective ways to accomplish the real goal. Agile teams should not just blindly fulfill orders.

So the key point in the article is:

"WHY WE NEED ESTIMATES



1. When we’re considering the justification for a proposed project, we need to understand the likely costs in advance, so that we can decide whether it is worth the investment.

2. When we’re launching new or improved products to market, we need some idea of roughly when significant features might be ready for release, so we can plan associated activities.

3. When we’re prioritising work, the product owner needs to understand the cost, and the team needs to understand the value, of each story (or backlog item)."

I dispute that all three needs are best addressed by estimates:

1. In practice, any software development project that has so narrow a profit margin is doomed to failure. Software development cost is unpredictable, not just because developers do not know what they do not know, but also because when given working software customers will always be discovering that they really did not need exactly what they asked for but they do need something they did not know to ask for until they tried using the software. Also, people, markets, competitor, etc. change.

A software development project is an investment. If the ROI does not potentially dwarf the cost, it is best to deploy your investment in something that will. Furthermore, every month into the project, you should have a much better idea of whether things will work out than you did the month before. Just start, deliver working software quickly, obtain and respond to feedback quickly, and pull the plug quickly if necessary. Start, inspect and adapt is much more realistic and much less expensive than wasting time and money building unreliable estimation models (which, by the way, are even more inaccurate on the return side than the cost side anyway).

2. Again, by getting started immediately and delivering working software, within a month or two, you will have a much better idea of when any particular feature will be delivered than playing the estimation game. The fact is that you will almost always have the feature ready and working by the release date - it just might not have all the bells and whistles.

3. Doing the first vertical slice of a feature will tell you much more about the cost of getting the feature totally working than story pointing will. And you will be 2 weeks closer to delivering it, to boot. There is nothing wrong with implementing the first vertical slice of several features in an iteration, and then making an informed decision about which one(s) to complete first.

The real reasons that businesses want cost estimations are:

A. Antiquated, big-bang budgeting practices.

B. CYA - so the business can always blame development if things do not work out.

Re: Apply the 5 whys by Neil Killick

Hear hear Steven! The "need" for estimates is a perceived one based on "that's the way it's always been done". There are alternatives. And in my opinion they are way better than making big decisions based on estimates. In the software world, "estimates" are little better than guesses, and in any case the act of doing them drives the wrong kind of behaviour when we start the work. We start caring purely about "are we on track?", meaning "are we going to deliver all the agreed scope on the agreed date". This is poor risk management and makes it very likely that we will deliver the wrong thing, and late!

The "need" for estimates by Glen Alleman

Counter to Neil's comment, the "need" for estimates is not based on "that's the way it's always been done." It's based on the foundation of all business process. Exchange of capital for value. Those with the money are obligated to spend it wisely. To do so they need to know the "value at risk" for their investment. Without this knowledge the very essence of business investment - Return on Investment" cannot function.

ROI = (Value - Cost)/Cost

As for the issue of accounting needing an accurate estimate for a project. This statement confuses Budget with Funding. Accounting needs to know the "funding" for the project. The management of the project is done in units of "budgets." Budgets overrun all the time. More funding is applied. The management of projects with budget provides feedback from comparing actual to budget for all the elements of the project. Mike Clayton has a nice "periodic table" for the elements of a project mikeclayton.files.wordpress.com/2013/11/pm-peri...

Mixing financial accounting and project budgets is common in some domains. Don't let that happen. Flow down the budget, let finance handle the money. This is the basis of all government and commercial contracting budget <> funding.

Finally the Rumsfeld quote is simply bad management on his part and the part of those using it. If Rumsfeld had read history, literally "The Histories" en.wikipedia.org/wiki/Histories_(Herodotus) he would have read the advice of Alexandr the Great - "Don't Go There."

There are two kinds of uncertainty in the project domain (actually in the mathematics of uncertainty) - reducible and irreducible. Both create risk to the project. Reducible is addressed by spending money or setting aside Management Reserve. Irreducible is address by "margin," either schedule, cost, or technical.

In order to have a credible plan to handle anything on a project, including the probabilistic estimates needed for proper governance, there are three approaches

1. We don't know upfront - we didn't sufficiently plan
2. We don't know during execute - the uncertainty emerged
3. We don't want to know - knowing will cause the project to be canceled before it starts.

The notion of "not wanting" to estimate is number three, since the "need" for estimates is fundamental at any business process - we can't know the value of anything until we know what it costs. This basic notion of business seems to be intentionally ignored in the No Estimates community.

I like story points by Tony Heap

I've been using story points on a project recently for an organization that aren't really used to agile. The stakeholders would have really struggled if we hadn't given them any sort of plan - we never would have heard the end of it. So we spent maybe an hour putting rough estimates against each story (some were way bigger than others so this was probably better than just counting stories) and then I created a burn-up chart.

We told the stakeholders we needed 3 weeks before we could give them an "end date" - and we spent 3 weeks building, until we got to a point where we could measure our velocity (in points per week). Then I made projections - optimistic was velocity + 20% and pessimistic was velocity - 20%, giving a date range of when we would be "done" (knowing full well of course that things would change over the coming weeks as we got to understanding the scope better).

The amount of effort expended was minimal, and it was just about scientific enough to stop the stakeholders worrying (and basically keep them off our backs so we could get on with it!). For me it's a nice balance because it is being honest about uncertainty (no fixed date, just a range) - but it's way more palatable to a stakeholder than "it will be finished when it's finished".

Re: Apply the 5 whys by David Morris

Interestingly, I agree with the remedies your propose on all three points, however I would like to test my understanding of two of your assertions:

1) Your points seem predicated on a perception that sizing should be considered as "wasting time and money building unrealistic estimation models". It does not take long to ask the team how complex they think a piece of functionality is, flashing some planning poker cards (or fingers, whatever), agreeing a size or discussing what's involved if the variance is large. The team also benefits from gaining a better understanding of work coming up, so their horizon is not too close. This is beneficial and doesn't stop the team from getting started, so I see this as a win-win.

2) The issues does not have to be so much on whether the ROI on an individual project is worthwhile, and is more usually because organisations are assessing a portfolio of suggested product and business changes, and not many organisations have the budget and people to do everything they want all at once. They need a way of comparing unlike projects, and one way of doing that is to have a rough feel for the costs and timescales, as well as the likely benefits.

Re: Apply the 5 whys by David Morris

Agreed; all data points can be used or misused. However, many teams working on product development find that they are required to work with the product management and marketing team to assess when major features might be ready to launch to market. If you are in a fast-paced market, announcing changes only when they're ready is really not good enough; and having a feel for a high-level release map can provide a degree of confidence to management, shareholders, the public, and the financial markets. While some of this is outside the development team's remit, it's certainly important in the world of those making decisions about which products to launch to market and which projects to fund.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

6 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT