Are Product Backlogs Wasteful?

Posted by Geoffrey Wiseman on Oct 17, 2007

Sections

Process & Practices

Topics

Agile ,

Agile Techniques

Tags

User Stories

Planning the features to be developed is an important part of software development, particularly software product development, and most methods/processes have some way of managing and planning those features such that the business and developers can get an understanding of "what's next". In Scrum, the list of features desired but not yet implemented is typically called the backlog (or product backlog). This is meant to be lightweight, but can it still be wasteful?

In a stereotypical waterfall development process, this can rapidly move from being a list of desired features into a long set of fleshed-out use cases, high-level and low-level specifications, and so on: an endless stream of documents whose weight is only surpassed by its outdatedness.

Keen to emphasize delivering working software over comprehensive documentation, most agile processes try and reduce the amount of effort put into documenting features up-front, particularly when the nature and priority of those features has a tendency to shift. However, even if a product backlog is lighter-weight than a functional specification, can it still be wasteful? When waste-averse lean software developers cross-pollenate with scrum afficionados, this question is bound to arise: Is a Backlog Wasteful?

Simply put by Alan Shalloway, backlog could be seen as inventory:

Work put into the product backlog is inventory. It is wasteful. But it may be necessary work to mitigate/manage risk. This is not inconsistent with agile methods. We analyze those stories that are about to get built. The rest of the backlog should be analyzed as little as possible so there is as little waste as possible.

Mary Poppendieck suggests some mitigation strategies:

I recommend that a LIMITED backlog equivalent to a couple or three cycles of work is adequate for making sure there is always something to do and it is the current highest priority work. One way to discover the appropriate length of this queue is to relentlessly remove the work that has an age older than the average age of work that is getting through the queue. Those items have fallen to the bottom of the queue and will very likely never get done. Another way to discover how short a queue can be is to keep on shortening it relentlessly until it is clear that it can't be any shorter.

The way to determine here if your backlog has gotten too long or too detailed too soon is to determine what % of so-called "requirements" are changed from the time they are specified until the system is implemented. This is often called requirements "churn". If this churn is greater than -— say — 10%, then the detailed specifications are being done too soon.

David Starr itemizes a few other ways to reduce backlog waste:

1. Set a sight horizon to trim the backlog and remove items beyond that horizon.
2. Use a FDD based backlog where features are defined at a higer level than button clicks. Use cases work well for this.
3. Enforce a estimation process for the teams that does NOT ALLOW large amounts of time spent sizing backlog items.
4. Use the higher numbers in Poker Planning ( 20, 40, 100, 300, whatever). This is usually the area of the backlog where teams waste time. BL owners want high level sizing and engineers like to break it down into smaller and smaller bits. Move on.

Paul Oldfield adds to that list:

break down the 'higher number' features into smaller ones just before they go into an iteration / sprint, so you can estimate better and so you only do the higher value aspects of the feature at this time

Alan further suggests there may be another danger to a lengthy backlog — a mindset shift that implies everything in the backlog should be completed:

What has happened now is that we've slipped into project thinking instead of product thinking. We are not looking to see if we actually need to continue on with the backlog. This is a subtle danger as it tends to make projects longer than they need to be. It impedes management of the products being developed/enhanced from looking to see if there is a better thing for the team to be working on.

Have you seen other forms of waste in managing a product backlog? How do you balance the need to plan with the waste of investing too much effort into an ever-changing backlog? Read more about this and other agile topics here at InfoQ.

1. Back to top

   Is this really your biggest problem?

   by Bruce Rennie

   Lean doesn't claim to eliminate waste. That's impossible. Lean recognizes that there are two types of "muda": necessary and unnecessary waste.
   
   It's true that a backlog IS inventory, but even Toyota has to operate with some inventory on the floor, to smooth out spikes in flow.
   
   It feels, to me, that moving from a waterfall style set of requirements documents to a backlog eliminates the major part of waste in the requirements areas of a project. Sure, you want to make sure your backlog doesn't become polluted with cruft but I suspect it's not going to be the biggest problem a team faces.

   Reply

2. Back to top

   Re: Is this really your biggest problem?

   by Amr Elssamadisy

   That's a good point. If this isn't the biggest problem, then working on it will not lead to increased throughput.
   
   Lean is good at telling us to eliminate as much waste as we can, but not very good at pointing to which waste to attack first. Theory of Constraints is a complementary way of looking at the same problem and tells us to focus on the bottleneck(s). Any work done to improve a part of the system that is not the bottleneck will not increase throughput. ( More here.)

   Reply

3. Back to top

   Mary said it all

   by Michael Nygard

   The whole point of being Agile is to release features as rapidly as possible.
   
   If a feature sits in a backlog (queue) for many iterations, it can take a very long time to reach production.
   
   It's not hard to envision a biweekly iteration rhythm that still takes 12 months for a feature to make it out of the backlog into production. Meantime, understanding all the other backlog items, thinking about them, prioritizing them, re-prioritizing them, and maintaining the backlog itself is exactly equivalent to picking up crates of raw materials and moving them around the shop floor. It is muda.

   Reply

4. Back to top

   Re: Is this really your biggest problem?

   by Bruce Rennie

   The interesting thing is that ToC requires you to put inventory in front of your constraint in order to ensure that it's never idle. Inventory may be waste, but it's also occasionally necessary.

   Reply

5. Back to top

   Re: Mary said it all

   by Bruce Rennie

   Of course it's muda. The question is: is it type 1 or type 2?

   Reply

6. Back to top

   Re: Mary said it all

   by Michael Nygard

   In my experience, anything more than 3 iterations worth of features is unnecessary waste. They linger on the bottom of the backlog, sounding like worthwhile features, but somehow never become important enough to get above the line.

   Reply

7. Back to top

   Re: Mary said it all

   by Paul Oldfield

   In reality, we're trying to optimise between two wastes. There's the waste of maintaining information that may never get used. We need to balance this against the possible waste of discovering the information several times if we discard it.

   Reply

8. Back to top

   How long does it take you to create a Release backlog?

   by Amr Elssamadisy

   The discussion here and on the mailing list seems to indicate that there is a very large investment in creating a backlog. That has not been my experience, here's what I've seen typically happens:
   
   1) Domain expert(s) understand the business and various needs and write out small stories (single card explanations), and enough to be able to have a high-level conversation about the requirement.
   
   2) A (typically 1/2 - 2 day) workshop with all of the pigs & chickens takes place to perform effort and value estimates which enable the initial Release backlog to be created.
   
   And that's it. With this investment, a team has created the initial backlog for a release. So, if we trim this effort, how much will we *really* help the project? Save a day?

   Reply

9. Back to top

   Re: Mary said it all

   by Deborah Hartmann

   Imo, if teams are following Scrum as taught, doc of future iterations should be increasingly sketchy... further than 3 sprints into the future, a story should consist of a title only. As such, where is the waste? This sketchy future backlog serves as a kind of project memory, and keep product managers from maintaining a separate wish list in a different tool, which then introduces the muda of transcription.

   Reply

10. Back to top

    Re: How long does it take you to create a Release backlog?

    by Deborah Hartmann

    Perhaps we should hold off doing t-shirt sizing on more than 3 months' worth? There's a place to save, since sizing requires discussion, however brief, among multiple parties. So, beyond 3 months, the backlog would simply be a collection mechanism, allowing for relative business prioritization.

    Reply

11. Back to top

    Re: How long does it take you to create a Release backlog?

    by Bruce Rennie

    I think that's a good idea, but it does assume that you're operating in a truly agile environment. I'm sure there's plenty of implementations of Scrum where the backlog is closer to a contract: we want to get the following features into the next release. There's nothing in Scrum that specifically forbids this, so I think, again, this falls into the category of necessary waste.

    Reply

• Older >