Stabilization Sprints, A Necessary Evil or Pure Waste?
Dushy had heard of Stabilization Sprints and wondered if they were norm in the Agile world. Stabilization sprints are an additional number of sprints added to the end of the normal development cycle before shipping the product. As the name suggests, they’re usually added to shake down the product one last time and drive the last of the bugs.
Ilja Preuss says that: “A stabilization sprint is a sign that your definition of done is incomplete or not followed.”
Sarath Kummamuru suggests that he sees some cases where stabilizations sprints are appropriate:
1. Taking care of technical debt that might have accrued due to the hurry to complete a sprint! (basically re-factoring existing code bases or may be improving unit test coverage, etc).
2. Take care of QA debt that the teams might have accrued because of lack of complete automation and regression every sprint. Very common with companies working on legacy code bases which do not have as much automation.
3. Testing and certification of the product that is being released on various platforms (say to certify it on different application servers, to certify that the product is working on different OS Platforms, etc).
4. If there is any packaging that needs to be done (say CDs to be released, etc) these are also typically done in these release/stabilization sprints.
This reporter noted that accepting stabilization sprints as necessary is akin to giving people a crutch and never helping them to walk again. It may take years to automate the testing of all the existing code but that’s no excuse to live with the current state of affairs. In addition, any code that doesn’t have automated acceptance and unit tests is of unknown quality. We don’t know if there are bugs hiding in there - effectively it's waste (from a lean point of view).
Edward Arunaj notes: “Typically if anything were pending then we are accruing debt. Many times you may need more than 1 sprint for stabilization and that can lead to unpredictability of the release. (stakeholders dislike unpredictability more than delay)”.
Mark Woyna gives an example where it's not currently economically viable to eliminate stabilization sprints. In this case the test environment has 800 servers (worth several million dollars), needs to perform 300,000 operations per second. Tests on these servers take 3-4 weeks to run as the teams need to simulate what happens as an upgrade is rolled out slowly across the servers. However Mark points out this is special case: “I agree with you that work performed during a stabilization sprint is simply work that wasn't done … If the software is defective, you'd rather find out earlier,
rather than later”
Finally, Steve Gordon says:
Fixing the root causes of this defective work is what will improve things. Yes, we also need to fix the immediate problem, but if that is all you do, the problems will recur and you will always be living with incomplete work and 'unexpected" defects.
Accepting "stabilization sprints" as a regular, normal practice is just fixing the immediate problem and accepting that the same kinds of problems will occur again.
Previously on InfoQ: Does "Done" Mean "Shippable"?
Agree and disagree
What makes scrum and agile so great is that THE TEAM can inspect and adapt and if that means you need to have stabilization sprints than so be it or maybe more ideally you just need them from time to time or eventually never.
I personally have seen if you create a sprint buffer (stabilization sprint) early on in the release, this stabilization still gets filled with features. I do agree in a theoretical scrum definition that yes it does indicate an "incomplete definition of done," but there are so many unique team situations that could drive the need for some stabilization sprints. Teams have many problems or out of the normal scrum defined situations, I see the "definition of done" is only one of them.
I do agree trying to use a stabilization sprint as a last case option and trying to solve the problem earlier is more ideal, but I would not "enforce" other solutions if this is what THE TEAM feels it needs to meet the project deadlines in the quality level required by the project. If the team learns to solve the problem the "right way" THE TEAM will say they are un-necessary.
Other reasons for stabilization sprints could be. Product owner and team/QA do not have great communication and you need to "stabilize" to make sure system works as expected. PO being on other time zone can add complexity. Also performance could require re-factoring/stabilization. So this brings up a good point "What is stabilization?" Really I see it as a time for the dev team to set priorities higher than the product owner on what get's completed, assuming they weren't able to do so earlier in the sprint cycle, it's a retrospective adjustment at the end of the release.
Obviously this is my 2 cents... Good luck happy scruming.
I'm doing the best I can....
Is it sub-optimal? Of course. Does it mean that the team can't use Scrum? I'd like to hope that isn't so. Rome wasn't built in a day, either.
Nobody is ever really finished improving their delivery cycle.
Are your Stabilization Sprints shrinking in size and effort? If so you are definitely applying Agile best practices and should not worry.
Teams and projects are all different. That is what the power of the retrospective is for.
There are a multitude of reasons why this could be needed. Much of the time it IS a case of "let's not let this happen again", but sometimes it is unavoidable and there is either nothing else reasonable you can do or this really IS the best solution to the situation (imagine that?!).
I would add the very basic case I came across where the customer has many small, "hidden" requirements which are revealed later. So the team concept of "done" was met, but the customer has moved the target and thus tweaks are required.
You can go on and on about how this should not be the case and how you can better manage the customer etc etc. Talk is cheap after all. The customer I am thinking of would have you in a padded cell raving long before you got there.
You could also rename this list of tweaks and changes as "new" work and call it a story, but that is just semantics. Deliverable needing to be changed before release means it was wrong. End of Story.
In this case Agile was amazing for organising our workstreams and delivery what we said on time. Organising the extra work into a later sprint was the best way to deal with it. End of story again IMO.
Slow down when the product starts to wobble
Kent Beck has a nice example for it in his early XP books: He tells a story about himself at the airport, rushing from one flight to the next, pulling his suitcase behind him that runs on small, narrow wheels. When he goes faster, the suitcase begins to wobble. When he goes too fast, the suitcase falls down and all the saved time is immediately lost. The habit he developed: As soon as the suitcase starts to wobble, Kent slows down until it runs smoothly again.
So, slow down and take a stabilization sprint as soon as your product starts to wobble. In the retrospective of that stabilization sprint, the team should find out what went so well in that sprint and do more of that in all the "regular" sprints. After a while, you should not need stabilization sprints any more.
No. And yes.
Unfortunately, I don't know anybody who lives or works in an ideal world. This means that there will be bugs, even with all of the TDD, nightly builds, CI, UATs and so on.
So yes, resources (time and effort) must be invested to fix these bugs that accrue.
Should an entire sprint be dedicated to this? Maybe. If enough bugs are found, that are deemed more important than adding the next feature, then an entire sprint will be dedicated to them. That is by intent, even if not by name, a stabilization sprint. So the long answer is Yes - if the need arises.
I would say that fixing these bugs are stories, not to be treated any differently than a story for a new feature, if not for one thing. The presence of enough bugs to fill a sprint indicates a fault in the production line, that must be addressed. Before anything else is done.
Re: I'm doing the best I can....
Of course you're doing Scrum. You may not be able to automate all of your acceptance testing for the code written in the past today - but at a minimum you can stop digging a deeper hole. In this case the team might agree that no new functionality will be written without automated acceptance tests. When you get good at that then you can back fill.
The Agile Consortium
Re: Agree and disagree
Example: with the communication problems - have stabilization sprints to start - but work very quickly to improve the communication. Otherwise you're Agile project is only marginal better than a waterfall project.
Mike Hartington Jul 26, 2015