Scrum Project Estimation Beyond the Near-Term?
Michael Sync has a question about how to estimate large projects or large features:
I heard that we should not estimate more than one or two sprints because we will have to change them later. But it doesn't really work in reality. We need to estimate the whole project or a few features in advance so we can negotiate the target date and adjust/prioritize the features. In order to estimate more accurately, we need to break down the user story into the task before creating the sprint. How should I handle or estimate for that?
Are you being asked for estimates or for commitments? An estimate is an educated guess based on the information currently available. Estimates are always either high or low, and usually low when building complex software. They are not commitments and can't be used as commitments.
The consensus of the Scrum Development mailing list seems to be that project effort estimation isn't a particularly useful activity, or one that you should spend a lot of time doing. "woynam" writes:
Organizations that really get this stuff stop worrying about how much something is going to cost, and shift toward variable scope work. If you know how much you are willing to spend, and how much time you're willing to invest, then the only variable is how many prioritized features you'll get for your money. The key is that if you are "off" on your estimates, you're still guaranteed to get the most valuable features first.
George Dinwiddie writes:
If your profit or loss is dependent on a precise, accurate prediction of the cost, then you've chosen the wrong product. The upside potential is too small.
Often I find that the people on the top of an organization understand this, but the managers in the middle often do not.
Estimates are not a good thing in most organizations: they are used as commitments and bludgeons. They also lead to a mechanistic approach to deciding how much work to take on, adding up points or hours, rather than a deeper look at what the team can accomplish, and how.
I recommend not estimating stories at all, beyond making them all "a couple of days work for a couple of people". That is, of course, still a kind of estimate, but not one that is easy to misuse.
I like having the team estimate the big stories in weeks, then add them up. What this will do is give a general sense of the size of the project. THIS CANNOT BE A PROMISE, but it can be a point at with you can ask the important question, which is something like: With N dollars and M months, can we get something worth shipping?
I agree with Ron
Of course the biggest problem with estimates is that they give an estimate of effort rather than elapsed time. (Which is where velocity comes in I suppose.) The business investor wants to know about elapsed time ( times run rate ) which gives the true cost.
More detail doesn't lead to better estimates
This is independent of the process (Scrum, XP, Waterfall, etc.) you are using.
That's a bit like the weather forecast: It works pretty well for the next few hours. It gets far worse for the next few days and it's completely unpredictable what the weather will be like in two weeks from now (with the premise that you don't live in the desert or at some place where the weather hardly ever changes).
So you have to accept that you get bad estimates. Sometimes it helps to compare projects with previous projects. Sometimes it is better to have a more or less fixed
budget and you do your best to squeeze in the most important features.
It's a bit like Heisenberg's uncertainty principle. There is a natural limit for estimates and you cannot get beyond it.
Re: I agree with Ron
Not estimating stories???
A recurring question
I think there are two obvious points here (but I may be wrong about the "obvious" part):
(1) Macro-level estimation with the goal of making a go / no-go decision about an entire initiative or about a major release is necessary and useful. Scrum does not address this at all. There are other disciplines that address it.
(2) Micro-level estimation of individual work items or tasks or User Stories is a complete waste of time. If a team isn't (yet) skilled enough at splitting stories to arrive at a reasonably consistent story size, then they may use relative sizing at the sprint planning level (like T-shirt sizes or a Fibonacci scale). This is an interim technique that keeps the work moving until the team learns to craft stories more skilfully.
"Reasonably consistent story size" means (IMHO) that the variation in size is smaller than the natural variation of fine-grained task estimates. That implies the variation in size is not significant enough to have an impact on short-term planning. Short-term planning doesn't have to be dead-on accurate. It only has to keep the work on track.
I suspect this question keeps coming up because people don't recognize the difference between these two levels of estimation and their distinct purposes.
When people say "Scrum doesn't say to do this" (or other statements with a similar structure), it strikes me that they are trying to adhere to a model (in this case, Scrum) too rigidly. The fact Scrum doesn't deal with macro-level estimation doesn't mean a Scrum team is "not allowed" to do any macro-level estimation. Yet, people seem to be afraid to do anything their chosen model doesn't explicitly define for them.
Sometimes I call this behavior Camelot Syndrome, after the scene in Monty Python and the Holy Grail in which the knights decide not to go to Camelot after all, because it's "only a model." Scrum is only a model, too. Use it if it helps you, but don't let /it/ use /you/.
Enterprise adoption challenges for Agile