Sprint Burndowns - Are We Measuring the Wrong Things?

by Mark Levison on Sep 22, 2010 |

 Sprint Burndown ChartAccording to the Scrum Guide, traditional Sprint Burndown charts measure:

The amount of work remaining for a Sprint is the sum of the work remaining for all of Sprint Backlog. Keep track of these sums by day and use them to create a graph that shows the work remaining over time. … Duration is not considered in Scrum. Work remaining and date are the only variables of interest.

Traditionally people have been taught to measure the remaining work in hours. As Mark Woyna points out when tracking task hours it's possible to burn down many hours without completing any stories. In that case the burndown is hiding the real progress of the team. He goes on to suggest that many teams track story points completed. When Adam Sroka sees this problem is usually because the team has started working on too many things at the same time.

This reporter notes that tracking task hours encourages us to complete tasks whether they’re still relevant to the completion of a story, encouraging clients not to estimate tasks in hours and not to use a burndown chart. Instead we watch the progress of stories and tasks across the team’s sprint board.

Johanna Rothman measures: Stories completed, stories remaining and total stories, both at the sprint and release level.

Aslak Hellesøy recommends Culmulative Flow Diagrams over Burndowns, noting that they provide more information.

Do you still use Sprint Burndown Charts? If not what have you replaced them with?

Related Editorial

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Estimating and tracking hours is waste by Stefan Schubert

An important point of agile is to deliver frequently. So it is better to track the speed of delivery (results) instead of the >estimated< energy wasted on that. If your delivery is slow, the team will recognize that if they track story points. They will not if they track hours. For ideal estimation you get a burndown of 8 hours per developer per day. So you measure the quality of estimation of tasks. But who cares about that as soon as the sprint is over or in retrospectives? Estimation accuracy should be good enough, not as good as possible. It's an estimate only, after all.

The funny thing is that our Scrum coach taught that to us too and a half year ago. I thought it would be state-of-the art and task estimation would be some kind of agile dinosaur :-)
I can only advocate doing that!

Do not estimate tasks at all! Cut your tasks small. Better not bigger then a day, better a team member can accomplish two or three tasks a day. If tasks survive a day, make a dot on them. If you want to track your quality of sprint planning, identify bottlenecks or impediments track the tasks and dots.

For burn-down purpose hour tracking is not making delivery problems transparent but only problems in estimation. So you can think about that as waste or in Scrum language: An impediment that makes your team slower.

Hours Remaining != Hours Estimated - Hours Worked by Amy Thorne

If you want to burn down hours, I think you can do so without hiding the real progress of the team. You just have to recognize that you are burning down estimated hours remaining, which isn't necessarily the same as difference between the original estimate and the hours worked. Team members _might_ track the number of hours they have worked on the story, but they _must_ track the number of hours they think are left.

I'm not necessarily endorsing burning down in hours--currently the team I'm working on is burning down in story points, where story points only get counted in the burndown when the story is complete, and I still think this might be more estimation than it's worth--but I am saying that if you track estimated hours remaining, I think you might not encounter problems where the burndown hides the real progress of the team.

Story size by Mark Schumann

If you're burning a lot of hours but not completing any stories, <em>your stories are too big.</em> Heck, part of the definition of a "story" is that it's something that doesn't take a lot of hours.

I recently saw a post on "Slicing and Dicing Epic User Stories" that offers a few ways to solve that problem.

agilecoach.typepad.com/agile-coaching/2010/09/i...

Re: Estimating and tracking hours is waste by Mark Levison

Stefan - the state of the art is always changing and I can still find a few coaches/trainers who see task estimation in hours as the way to go. Personally I've just seen teams debate the precise number of hours involved in a task even when I limit them to factors of 2. The real value in task estimation is in having a conversation around the task itself. Once they team agree on the what the task represents stop debating and just get it done.

Or as you say perhaps the burndown represents waste. Either way I find the watching the progress of stories far more informative than a burndown chart.

Agile-V by Guy Beaver

Burning down hours only hides queues (WIP) as discussed here...

www.agilejournal.com/articles/columns/column-ar...

Lean flow suggests that a steady stream of closed stories can best be achieved by managing WIP.

Depends on culture and objectives by Vamsidhar Bethanabatla

What we measure greatly depends on the org. culture and the objectives of the planning exercise. Irrespective of that, I feel, our measurements should help us grasp the 'amount' of work remaining/accomplished in the sprint.

Personally, I feel no. of hours is a flawed measurement. Basically, if my measurement says that 5 out 10 hours of work is done, it is not giving any joy to me. Rather I would be happy to see myself 'complete' 5 out of 10 sub tasks in a task. There is a sense of completion. The stake-holders "feel" the progress.

vamsi.typepad.com/blog/2010/09/random-thought-a...

Re: Hours Remaining != Hours Estimated - Hours Worked by Mark Levison

Thanks Amy - you're right if you burn down hours it must be hours remaining. However that still mask problems. You can burn down a lot of hours by completing the first two tasks from every story and yet have completed none. Pathological perhaps but I have seen cases where the burndown looks fine and but the reality is different.

Cheers
Mark Levison
Agile Pain Relief Consulting

Re: Hours Remaining != Hours Estimated - Hours Worked by Tomi Tuomainen

We are not burning down hours, we are measuring time remaining. We could as well estimate percentage remaining for the story. But that way we could easily end up with too big tasks (several days). When we are talking about hours (or days) remaining, it forces team members to think about the size each task.

If you are burning down tasks but completed none stories, then your estimates are just wrong for the remaining tasks. But we could have two charts: one for work remaining, one for completed stories.

Re: Story size by Mark Levison

I agree that slicing your stories small is a key part of making this a success. However nothing I know (or teach) about stories says that they "don't take a lot of hours". Even the INVEST criteria (see xp123.com/xplor/xp0308/index.shtml): "Good stories tend to be small. Stories typically represent at most a few person-weeks worth of work. (Some teams restrict them to a few person-days of work.) Above this size, and it seems to be too hard to know what's in the story's scope. Saying, "it would take me more than month" often implicitly adds, "as I don't understand what-all it would entail." Smaller stories tend to get more accurate estimates."

Cheers
Mark Levison
Agile Pain Relief Consulting

Re: Depends on culture and objectives by Mark Levison

Vamsidhar - I agree there is no joy in saying a task has 5hrs remaining. The joy is in getting things done. I prefer to focus on the stories and not the tasks because its the stories that deliver business value.

Cheers
Mark Levison
Agile Pain Relief Consulting

Re: Story size by Mark Levison

I meant to mention in my own training I tell people that the largest story should smaller than what two people can complete in half an iteration. I use "two people" to remind team members that a story shouldn't be completed by one person alone, that I expect team members to pair and collaborate.

Half an iteration is the upper limit, because stories larger than that size become bottlenecks and impede the flow of the team. When teams pick stories larger than that in size they generally don't finished in the sprint. However most teams when they're starting to do Scrum struggle to split their stories into small enough chunks.

Re: Hours Remaining != Hours Estimated - Hours Worked by Mark Levison

Tomi - first up if it works for you great don't change a thing. In general I no longer see the benefit of estimating task size. Instead I get teams to discuss whether a task is smaller or larger than one day.

Cheers
Mark Levison
Agile Pain Relief Consulting

Re: Story size by Mark Levison

Mark - this conversation inspired me to write a blog post on the subject: agilepainrelief.com/notesfromatooluser/2010/09/...

Cheers
Mark Levison
Agile Pain Relief Consulting

On plotting time against time by Kevlin Henney

The issue of what is being measured and tracked on a burndown is something I discuss in the following talk, "Individuals and Interactions over Processes and Tools", from NDC 2010:

streaming.ndc2010.no/tcs/?id=1272299E-1DD1-4C11...

In particular the pitfalls of plotting time on both the vertical axis (hours) and the horizontal axis (days), which indicates utilisation rather than velocity or scope coverage.

Answering the "When will it be done?" question by Gary Elliott

I absolutely hate time estimation in any form. This is mostly because I'm terrible at it. But no matter how I track a project, by story, feature, bugs fixed, issues closed, etc. some manager is always going to ask the dreaded question, "When are you going to be done?" Somehow they are never understand anything that doesn't include a month day and year in it (and sometimes the time of day).

I see heads nod in approval as I explain how we have feature X, Y and Z all completed. But because feature A, B and C still remain, they ask, you know, the QUESTION. And my eyes roll back in my head as if to peer into my brain somewhere for an answer from which I have nothing.

Now giving a date for when a particular feature will be ready is usually not too bad. Giving a real date for a project with a long list of features is not possible for my low Mhz single core CPU brain.

If time spent or remaining means nothing to real progress (and I tend to agree), then why does everyone keep asking me about time?

Re: Answering the "When will it be done?" question by Yuri Khomich

Because projects are limited by time. It is surprising to see people do not understand this simple fact.

Re: Answering the "When will it be done?" question by Jinze Buissing

Yuri, i totally agree. Projects also are limited in budget and resources. So, unless there is fixed scope and fixed budget contract, i do understand why the manager and the customer are interested in the remaining hours (i.e. costs). Of course, it's all about delivering "done" stories and business value. Some customers have trouble with prioritizing their stories...

Re: Answering the by Mark Levison

Gary - first you're not alone, it turns out that humans are not good at absolute estimation. Consider: Your spouse calls and asks you to pick up the kids from school, buy some groceries and then buy some wine from the liquor store. Precisely how long will it take and when will you be home? Please make your estimate accurate +/- 20%. Unless this is what you do everyday your estimate will be wildly out.

Relative estimation plays to your strength's is this bigger or small than this other thing. Estimation in story points leverages the latter approach. So assuming you have your backlog estimated in points and your velocity, when is a very simple calculation. Divide the # of points remaining by the velocity and that will tell how many more sprints you need assuming there are no scope changes. At this moment management have a meltdown and tell you the project must be completed by date X. You can use date X to show just how much work will be completed as a line in the backlog again.

Remember the key trick - its all relative.

Cheers
Mark Levison
Agile Pain Relief Consulting

Re: Answering the Question by Gary Elliott

Hey, how do you know my wife?

Seriously though, I like your relative view. That I might be able to handle.

Re: Estimating and tracking hours is waste by Tom S

For us it's a question of purpose.

According to the scrum guide, the purpose of the sprint burndown is so that "the Team can manage it's progress in completing a Sprint's work". So the sprint burndown is for the Team to evaluate their progress towards delivering on their commitments, it is not a tool for the product owner to measure how much "value" the team have added to the product.

Measuring your burndown in storypoints, number of tasks, value points etc is prone to peaks and troughs which, while they give a better illustration of what is "done", their reduced granularity makes them are less accurate at illustrating progress at any specific point in time.

We also try not to get hung up on "hours", as Mark mentioned it is all relative and "task hours remaining" bears no correlation to "work hours remaining", tasks could equally be measured in points, spoons, vegetables or whatever.

For the purpose of helping the product owner to understand what has been added to their product, we also maintain a burnup chart showing how many storypoints and/or value points are Done.

Re: Estimating and tracking hours is waste by Mark Levison

Tom - I think you've nailed it. The keys is helping the team understand where it is and keeping the product owner informed of progress in terms of value delivered.

Cheers
Mark Levison
Agile Pain Relief Consulting

Tried both hours and story point burndown by sachin kundu

We have tried both the hours and story points burndown. Did not matter much in how much better we are in saying if we are on schedule. So we stopped putting hours on tasks as they just take time. For some times in sprint there is no visible progress on the burndown chart so we talk about it in the daily scrum but besides that burning story points help carry more meaning for the product owner and less work for the team

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

22 comments

Educational Content

General Feedback
Bugs
Advertising
Editorial
 InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy