Throw Away Your Bug Tracking System?

Posted by Mike Bria on Mar 18, 2009

Sections

Process & Practices

Topics

Agile ,

Delivering Quality

Tags

Quality ,

Planning

Elisabeth Hendrickson, A.K.A "testObsessed", presents a thought-provoking stance on triaging bugs in an agile project. She discusses her feelings that problems found during the iteration are not "bugs", that only the Product Owner has the right to call something "bug", and that a healthy agile team might likely have no need for a bug tracking system.

Hendrickson leads off her discussion with a definition of what a "bug" really is:

In an Agile context, I define a bug as behavior in a “Done” story that violates valid expectations of the Product Owner.

After clarifying how she defines "Product Owner" and discussing her understanding of "expectations", Hendrickson presents her stance on things found in the software before it's "DONE" that don't match these "Product Owner expectations". She explains that these should not to be tagged as "bugs", and more to the point that the only thing one needs to do with them is to fix them immediately:

Before we declare a story “Done,” if we find something that would violate the Product Owner’s expectations, we fix it. We don’t argue about it, we don’t debate or triage, we just fix it. This is what it means to have a zero tolerance for bugs.
...
And since we just fix them as we find them, we don’t need a name for these things. We don’t need to prioritize them. We don’t need to track them in a bug tracking system. We just take care of them right away.

Having established this, Hendrickson explains what she feels really are "bugs" and what to do with them:

After the story is Done and Accepted, we may learn about circumstances in which the completed stories don’t live up to the Product Owner’s expectations. That’s when we have a bug.

If we’re doing things right, there should not be very many of those things. Triaging and tracking bugs in a fancy bug database does not make sense if there are something like 5 open issues at any given time. The Product Owner will prioritize fixing those bugs against other items in the product backlog and the team will move on.

And if we’re not doing things right, we may find out that there are an overwhelming number of the little critters escaping. That’s when we know that we have a real problem with our process. Rather than wasting all that time trying to manage the escaping bugs, we need to step back and figure out what’s causing the infestation. Stop the bugs at the source instead of trying to corral and manage the little critters.

Also in the article, Hendrickson addresses the question of what to do with the things that someone thinks might not be right with the software, but that the Product Owner deems as "not a problem". She advises against keeping record of these things:

Many of the traditional teams I worked with (back before I started working with Agile teams) had bug databases that were overflowing with bugs that would never be fixed. Usually these were things that had been reported by people on the team, generally testers, and prioritized as “cosmetic” or “low priority.”

Such collections of low priority issues never added value: we never did anything with all that information. And yet we lugged that data forward from release to release in the mistaken belief that there was value in tracking every single time someone reported some nit picky thing that the business just didn’t care about.

The database became more like a security blanket than a project asset. We spent hours and hours in meetings discussing the issues, making lists of issues to fix, and tweaking the severity and priority settings, only to have all that decision making undone when the next critical feature request or bug came in. If that sounds familiar, it’s time to admit it: that information is not helping move the project forward. So stop carrying it around. It’s costing you more than it’s gaining you.

In a nutshell, Hendrickson challenges us to be more stingy with the things we'll allow ourselves to call a "bug". More specifically really, she challenges us to greatly reduce the things we'll record as "to be fixed later"; to reduce them to the point where using any sort of bug tracking system becomes overkill. And finally, she encourages us to acknowledge that when you've got enough [true] bugs hanging around that you think you do need complex tracking, then you'd be better to re-examine and improve your development process then to go out and get a bug tracking system.

Possibly radical thinking to some. Nonetheless, be encouraged to read Hendrickson's article in its entirety (as only a small portion is excerpted here), think hard about it's message, and add to the discussion with your own thoughts and experiences.

1. Back to top

   Related (conflicting?) news

   by Mike Bria

   I'd also like to note that there have also been 2 recent blog entries on Agile Software Development site by Jack Milunsky (with many interesting responses) focused on the same topic of "agile bug triaging/tracking":
   
   Accounting For Bugs In An Agile World, Part 2
   Accounting For Bugs In An Agile World
   
   I'll leave this up to you to decide whether these conflict with Elisabeth's message, and if so where you stand between them.
   
   Cheers
   MB

   Reply

2. Back to top

   Re: Related (conflicting?) news

   by Philip Jarrell

   To me, taking in Elisabeth's and Jack's comments about bugs in agile development, means balance must be struck. While Elisabeth's overall theme of kill bugs as you encounter them is valid, as Jack points out, you will encounter bugs that take time. Hence, a "bug" may need to make it into the backlog if it is a large enough effort to warrant it. Otherwise, just kill them as you encounter them. This seems to be the balanced approach. Setting clear guidelines for the team will help them to progress quickly without getting tied up on "is it a bug or not?" and "when should it be fixed?". My point is, be balanced.

   Reply

3. Back to top

   Re: Related (conflicting?) news

   by Mike Bria

   Hi Philip --
   
   So, I agree with what you're saying: "balance the bugs. If you can kill them immediately, then absolutely do so. If absolutely necessary, plan it for a future fix".
   
   I think though that if you read both articles in full you'll probably note that BOTH of Elisabeth and Jack agree on that particular point.
   
   Where I personally think these two diverge is regarding the details of these points:
   

   
   
   • - For "in-process / pre-done" bugs, Elisabeth says just get em fixed right away without any fuss or formality at all, where it seems Jack suggests these be tracked as official "tasks" in a backlog. Truth is, this just may be a "Scrum vs XP" disconnect, so maybe not here nor there or even worth taking about here.
   
   
   • - (More notably) For those "we'll fix em later" bugs, Elisabeth seems to talk about treating these more akin to how you'd treat "TBD user stories", whereas Jack seems to insist on treating bugs as "tasks" no matter what. In fact, Jack & I discuss/debate this a bit in the commentary on the "Part 2" blog.
   
   

   
   
   I tend to agree more with Elisabeth in this case, as is probably obvious!
   
   Cheers
   MB

   Reply

4. Back to top

   This is a point I've been making for years...

   by Antony Marcano

   Her points are valuable but not entirely original.
   
   Interesting that I blogged about this 5 years ago and debated it extensively on the Agile Testing mailing list - as referenced in Lisa Crispin and Janet Gregory's book "Agile Testing" (p426) as the "Hidden Backlog" - and presented it at a workshop (LEWT03 in 2006) in Elisabeth's presence... at which Elisabeth told me that it helped her reach a profound realisation about the negative effects of a bug tracking system and the value of tests in that context...
   
   Interesting that Lisa and Janet recalled it enough to ask me to write about it in their book despite not knowing me personally for even half as long... yet Elisabeth cannot recall either the workshop (despite using the word "profound" in her reaction to my talk on it), the blogpost or the section in Janet and Lisa's book enough to reference me, if only as someone who triggered the ideas she now has on the subject.
   
   As I said - the points are valuable... but not entirely original.

   Reply

• Older >