Coping with Bugs on an Agile/Scrum Project

| by Mark Levison Follow 0 Followers on Jul 15, 2009. Estimated reading time: 3 minutes |

BugsAn often asked question is how does Scrum recommend a team to handle bugs? Should they be placed on the product backlog? Or on a separate bug list? If they’re on the backlog, does the Product Owner get to set their priority or are they automatically the most important items? Should there be a separate bug fixing sprint?

On Pascal Maugeri’s team, even after improving their definition of done and doing “proper testing/unit-testing”, they’re still finding bugs that escape the sprint. He asks how to resolve this problem.

George Dinwiddie, Agile Coach, suggests raising the issue with the team during the retrospective – he has worked with teams that had vanishingly small bug rates. Mark Levison (this reporter) suggested “I would start to ask the question why weren't they found and fixed within the sprint that they occurred? My focus is on reducing the time it takes to discover (and then fix) problems. After all, if we discover a bug in the story during the sprint it is written then the PO shouldn’t accept it as being done. In addition early discovery will make it much easier to fix as the code is still fresh in the mind of the development team.”

Jim Schiel, CST with Artisan Consulting, just puts bugs in the product backlog to be prioritized by the Product Owner - “unless it’s a dead simple fix, in which case you determine the solution during Sprint Planning and execute the solution during the Sprint.”

Bruce Kantelis says it’s all about developing a culture: “We categorize defects. Priority 1, blocking the user from working get immediate attention, the interrupt work for a fix and patch, all others become stories that are placed at the top of the list for the next sprint. Over time, teams realize that quality measures and behaviors really impact their workday and rally to minimize the interruption.”

Mike Cohn reminds us that bugs found within the sprint are best handled by yelling across the team room and describing the bug. Failing that, a card describing the  bug can be added to the board. However for bugs that escape the sprint – he prefers to add them to product backlog, leaving it up to the product owner to prioritize them. Many existing teams still have a bug database that they need to continue using. In that case, he counsels maintaining a separate bug backlog, where product owner simply says what his priorities are from each queue: i.e. the first two items from the product backlog, then the following bugs and finally the next two items from the backlog.

Kevlin Henney argues that this approach is akin to treating a bug as a feature with a negative value:

If defects are viewed as features with negative value, they become managed as if they were features. Development groups store up prioritized repositories of bugs, treat bugs as user stories, outsource the fixing of bugs, and so on. While each of those can be a useful technique or perspective in dealing with a project in transition or crisis, it is not a long-term view that should be encouraged. After all, as the "Manifesto for Agile Software Development" says, "Working software is the primary measure of progress." It is a little disingenuous to pass off a feature with known defects as complete and working — "Yes, it's done... but there are a few bugs."

Ron Jeffries turns the whole problem on its ear saying that fixing a defect in a feature after it's finished is always more expensive than doing it right the first time.

So when we write the software incorrectly and then fix it, the customer pays more: she pays whatever she paid before, plus the bug fixing penalty.

She should be really ticked off about that. I like to encourage the customer to prioritize all the defects, to ensure that she'll feel the pain of the team's inadequate software process. I feel sure that she'll express that pain and that the team has a better chance of getting the idea that doing things well is better.

Do you avoid bugs altogether? Put them on the Product Backlog? Do you see the problems Kevlin outlines?

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Fix bugs as soon as they are found by Rob Elliot

We operate on the basis that a bug is written up on a pink card that goes straight to the top of the priority list, so it's the next card taken out by a free pair.

Prioritized by Giorgio Migliaccio

Of course, we try to minimize bugs at the origin, so at development time. Nevertheless, since no person and no methodology is perfect, and especially since we're doing distributed development in India, which doesn't permit to follow up everything adequately, some bugs might get through to the 'internal release', which will be tested by our testing team in Brasil.
We make use of a bug tracker system to post the issues and give them a degree of severity.
Now, the development team just takes 3 days every sprint (4weeks) to solve issues. So they have a pick from the most severe ones, complemented with some minor, easy fixable stuff. This every sprint again.
This seems to work for us.
Although, sometimes, very major issues, which are mostly design related, may need to be placed on the Product Backlog, because it actually is a requirement change or rewrite, but we only had to do this once until now.

Fix bug now ! by Anton Antonov

Recently I thought on 'How can I reduce the bugs to minimum ?'
  • My decision was to execute these steps :
  • 1) Stop and fix the bug if it is serious or log it for a later inspection.

  • Note : See TPS (Toyota Production System) and why they stop the production line when there is defect.

  • 2) Ask 5 Why regarding this bug.

  • Note : I search the root of the problem(bug) I don't want to fix only the bug. The decision will give us a principle in a lot of situations, not a fix for the bug in a single case.

  • 3) Describe the bug in a file and its decision. If the bug is related with some principle then I fixed the bug perfectly.

  • Note : It's very easy to forget the problem, the fix ,or principle which fixes all group of bugs.

  • 4) Make a revision of you principles and the way you mustn't do something if you want to reduce the mistakes. Probably It's good idea to talk on every teem meeting for the bugs written in the file.

  • Note: Item 3 and 4 are practices are taken from the book "TEAM SECRETS OF THE NAVY SEALS' "

  • Now I have perfect practices for the job I do! I reckoned that the time for fixing a bug will be reduced a lot because I will not do a lot of bugs in future ! :D

  • I know this approach needs from me to be orderly but If you have the best one tell me. :D
  • Bug-driven development is evil by Piers Thompson

    There's a guiding principle that everyone should apply - "Bug driven development is evil". But it's common. And bugs on the backlog are a varient of bug driven development.

    Why is it evil? Because bugs are impossible to estimate. OK, maybe impossible is too strong, but I'd argue that "several orders of maginitude harder to estimate than features" is accurate. If you plan your project using a list of bugs then your estimated timescales are going to be utterly useless.

    What's the antidote? What's the opposite of "bug driven development"? It's much easier to estimate how long it will take to test something than it is to estimate how long it will take to fix bugs in it. So if you've got high priority bugs that have escaped the sprint then put tests for them on the backlog. It's not the conventional meaning of "test driven development", but it works for me.


    Re: Fix bugs as soon as they are found by Mark Levison

    So far so good - what happens if many low priority bugs are found? Do you fix them in preference to doing work that the PO wants?

    Why wait? by Mike Bria

    #, First, to reiterate, do everything you can to never let the code leave the developers desk with bugs. Duh.

    Second, if you find it during the iteration, fix it. Verbal communication to coordinate this is best.

    Finally, if it's found post iteration - fix it somewhere between "now" and "first thing next iteration" (at the latest), favoring "now" whenever possible, depending on the criticality of the bug.

    Consider this: if you could have predicted the bug when you first slated the story for development, would you ever have considered prioritizing that bug for "sometime later"? Of course not, you'd have demanded the bug never make it in. Why allow a different attitude when the bug comes in later?

    Bottom line, keeping growing repositories of bugs (whether in bug trackers or backlogs) is a bane to productivity. Make it a top priority to avoid.



    Re: Fix bugs as soon as they are found by Mike Bria

    #, What does "low priority bug" really mean?

    My take is this: if you have a "bug", meaning the software aint doing what was expected, then it either gets fixed ASAP (see my other response), or it's not a "bug". If it can "prioritized for later", then it becomes a real-deal story.

    Re: Prioritized by Mark Levison

    So far so good - but do you ask why the bug was written in the first place? Your doing well the challenge is always about getting to the next stage.

    Re: Fix bug now ! by Mark Levison

    I think this approach is on the money and in the long run you will have as George said "vanishingly few bugs". Most teams I see shy away from this approach saying but we will never get any work on features done. Sad. Keep at this Anton you're going in the right direction.

    Re: Why wait? by Mike Bria

    #, ..and, yes, for any bug - inspect (into why it occurred) and adapt.

    Legacy projects differ by Mikkel Toudal Kristiansen

    I agree with the ones of you guys saying, that bugs should never escape the developers during a sprint, as long as we are talking green field projects, or at least projects that has been running scrum from the beginning.

    In "brown field" projects, and projects having recently made the switch from something else to scrum, it's a very different ball game.

    Typically projects like that are blessed with a large number of known bugs, and perhaps even more unknown ones. Bug tracking systems are often in place, and the customer is acustomed to discovering new bugs within every release. Unit tests are absent or at least way too few, and testers are often a scarce resource, so it's only on rare occasions, that the entire system is tested properly.

    On projects like these, it is often not feasible to have a strategy to fix a bug as soon as it is found, because of the hard-to-estimate nature of bugs - the team will not get any features "done" in the sprint.

    If the customer is paying per implemented feature, and not for bug fixes, it actually works quite well to treat bugs as "features with negative value", as Kevlin puts it. That way the customer can prioritize wether to have the team focusing on stabilizing the product or adding new functionality.

    I would strongly recommend teams on projects like this apply the techniques explained by Michael Feathers in the book "Working Effectively with Legacy Code", so the code coverage of the system increases sprint by sprint.

    Just my humble opinion :-)

    Re: Legacy projects differ by Mikkel Toudal Kristiansen

    Sorry, this was my first post in here, and my formatting of the message went poof ...

    Re: Fix bug now ! by Anton Antonov

    By the way we have a joke with my colleague "If you want to write software do it at home not in the office!". The reason is that nobody wants to invest time in what ever activity which doesn't return value now ! ;-)

    Re: Legacy projects differ by Mark Levison

    Legacy code is only a bit different. My focus never make the situation worse, always try to make it a bit better. In this case I would suggest no new bugs should escape the sprint.

    As to the Michael Feathers recommendation "Working Effectively with Legacy Code" - I second it.

    Re: Fix bugs as soon as they are found by Jim Leonardo

    You hit the nail on the head. This is the first question you must ask... is it a bug, or is it a change? I've seen a lot of bugs that have come up that were "We asked you for x, never thinking about y, so could you please change the system so that y is covered?" It's a business scenario, so there's little reason to expect a dev or tester to anticipate/test it.
    I know where we've struggled is in injecting items into the product backlog, we tend to classify it as a bug and so we end up with a lot of bugs, but not a lot of change in the back log. That's exactly what I think we're supposed to be avoiding in Scrum or any other agile methodology. As we see change, we should be taking it on as a feature and prioritizing it accordingly. Only when it's something broken should it be called a bug.

    Re: Fix bugs as soon as they are found by Mike Bria


    Combine defect-fixing with 3rd-level tech support... by Douglas Stein

    At my last company, we took the following approach (the app was a single-instance SaaS serving about 2.2 million K-8 students and their teachers):

    • * Each release (approx 3-4 2-week sprints) a team member was assigned to the 3rd-level support role. The rest of the team focused solely on the current release.

    • * The person assigned to 3rd level support was permitted to work on applied R&D projects of his/her choosing that we agreed were worthwhile to the company. You can think of these as "spikes" to increase stored value in using new technology we were considering to solve problems further out on the roadmap. Sometimes it was simply paying down any "technology debt" that had accrued.

    • * These projects had to be shelved immediately if any defects appeared in production and were affecting customers. The person was permitted to spend up to one day without prior approval from the product owner. During that time if it couldn't be solved it had to be characterized via a clear test case and an estimate on how it impacted customers. The product manager could decide whether it became a story or whether the person doing 3rd level support could continue for a bit longer.

    • * Fixing the bug included identifying whether the defect was based on fragile architecture, insufficient testing, or imperfect understanding of what the customer's expectations would be. (When you have a large customer base, you find folks who use the product in a way that doesn't match the original scenarios.)

    One of the benefits of this approach is that everyone has a positive incentive for minimizing defects - more time to pursue ideas that increase the team's velocity significantly when it's "your turn" to take the 3rd level support role.

    Of course, this approach assumes the defect level is low enough that the person doesn't become Lucy at the chocolate factory.

    Re: Combine defect-fixing with 3rd-level tech support... by Mark Levison

    Wow Doug - I think you've found a great approach to this problem. You've also identified the biggest weakness that I can see.

    Bugs vs Features by Daniel Sobral

    One shouldn't classify things into bugs vs features, in first place. It leads to wasted time spent arguing whether something is a bug or not, make people feel and treat them differently, and is awkward to manage, as, after all, it takes exactly the same work to do either.

    What exists is an unsatisfied requirement. If the user says the requirement is part of something that was already done, then you pick the whole story, bring it back to the product backlog, do whatever further detail the team thinks necessary -- if they think it necessary, and prioritize it right along everything else.

    That gets rid of the blame game, focus on the fact that the requisite is not "done", and makes it possible to prioritize workload appropriately.

    Now, if that is not viable because the contract says bug correction is free, everything else is not, then you have a bad contract. Instead of trying to get someone else to pay up for badly written software -- since you'll not get that anyway, the price tag will just be added to other expenses one way or another -- you should focus on getting good developers, good tools and good process.

    Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

    Email me replies to any of my messages in this thread

    Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

    Email me replies to any of my messages in this thread

    19 Discuss