Continuous Deployment, In Practice

Posted by Mike Bria on Jan 27, 2010

Sections

Process & Practices,

Architecture & Design,

Operations & Infrastructure

Topics

Agile ,

Deployment / Datacenter ,

Delivering Value ,

Delivering Quality

Tags

Monitoring ,

Releases ,

Lean ,

Continuous Integration ,

Continuous Deployment

Continuous deployment, described by Ash Maurya as "a process by which software is released several times throughout the day – in minutes versus days, weeks, or months", has gained a recent buzz in the Lean-slanted "eliminate work-in-progess" movement. But while many may find this an intriguing and logically worthwhile objective, many less can visualize how this might actually be achieved. Ash Maurya helps to fill this gap by describing his experience with making it happen at his company.

Maurya on why his previous 1-week staged release process didn't work for him:

The release process took at least half a day and sometimes the whole day. Dedicating up to 20% of the week on releasing software is incredibly wasteful for a small team. This is not counting the ongoing coordination effort also needed in prioritizing the ever-changing release content for the week as new critical issues are discovered.

His transition started by following the advice in Eric Reis' 5 step continuous deployment primer, which helped him get his tools in order. From there his story centers around what he considered the most challenging aspect: simply "getting comfortable with releasing all the time."

[We] took things easy at first - made small changes and audited the release process maniacally. I started relying heavily on functional tests (over unit tests) which allowed me to test changes as a user would. I also identified a set of events that would indicate something terribly going wrong (e.g. no users on the system) and built real-time alerting around them (using nagios/ganglia). As we built confidence, we started committing bigger and multi-part changes, each time building up our suite of testing and monitoring scripts. After a few iterations, our fear level was actually lower than how we used to feel after a staged release. Because we were committing less code per release, we could correlate issues to a release with certainty.

Maurya describes his Continuous Deployment process with the following principles/practices:

• "Don’t push features": a fundamental Lean mantra. Let customer feedback about your MVP "pull" new features onto your radar
• "Code in small batches": for Maurya, 2 hour coding sessions that result in check-in (which triggers the automated build)
• "Prefer functional tests over unit tests whenever possible": Muarya uses Selenium
• "Always test the User Activation flow": be absolutely sure your user's "critical path towards initial gratification" is working
• "Utilize automagic software updates": users should be able to receive updates as seamlessly as possible; Maurya describes exactly how he does this for his OSGI-based application
• "Build in alerts and monitoring": Maurya uses nagios and ganglia to get notified of any abnormal dips in system usage
• "Build in application level diagnostics": an application that can check its own health to find things tests and users might not be able to tell you
• "Tolerate unexpected errors exactly once": take the time to understand root causes and implement real fixes

If this interests you, take a moment to read Maurya's full experience report summarized here.

• Older >