Devops in AWS LiveLessons, published by Addison-Wesley Professional, is a video course aimed at infrastructure developers and Sys Ops engineers who have a goal of creating a fully-automated continuous delivery system in Amazon Web Services (AWS). The 4+ hour course is especially focused on leveraging key features of AWS, such as programmable infrastructure, elasticity and ephemeral resources, while also presenting them in the framework of DevOps best practices and tools. InfoQ has spoken with course author Paul Duvall.
The course sets off by considering the importance of learning the motivation of stakeholders and putting in place the proper communication tools to get access to their assets. Once this is accomplished, according to Duvall, the next step is assessing the current state of the software delivery system. This entails documenting all steps of the current software delivery process as a pre-requisite to its final automation.
The rest of the course is devoted to a very thourough demonstration of all the steps that are required to setup a CI infrastructure in AWS, from the creation of network resources using AWS Virtual Private Cloud and the definition of subnets, route table, security groups and so on, to the set up of a proper CI pipeline across all of its stages. For each stage in the CI pipeline-commit, acceptance, capacity, exploratory, pre-production, and production-Duvall explains what its purpose is and what role it plays.
The final lessons address how to automate the process itself of setting up a new environment and deployment system; processes that are not specific to any given stage in the pipeline, such as monitoring and logging; and ongoing activities that can be helpful for the whole team.
InfoQ has interviewed Paul Duvall.
InfoQ: Hi Paul. Could you please shortly introduce yourself and describe your experience with Continuous Delivery?
I’m the co-founder of Stelligent and the primary author of the book on Continuous Integration. Our business objective at Stelligent is to implement Continuous Delivery solutions in Amazon Web Services for our customers so that they can release software whenever they choose to do so. I’ve been practicing Continuous Integration since the early 2000s on software development projects and began practicing what is now referred to as Continuous Delivery a few years after that. Like most things, it’s been a bit of an evolution toward more self-service, more automation and a better user experience for those who are developing and delivering software systems.
I’ve got a particular passion for automating repetitive and error-prone processes so that we humans can focus on higher-level activities. I find that the more I ask myself the question, “how can I systematize this process and make it self-service?”, the more I test the limits of what’s possible with automation. I believe we’re still at “Day 1” when it comes to automation around the software delivery process.
InfoQ: Can you briefly define Continuous Delivery and what benefits it can bring to an organization?
CD provides the benefits of having always-releasable software so decisions to release software are driven by business needs rather than operational constraints. Moreover, CD reduces the time between when a defect is introduced and when it’s discovered and fixed. CD embraces the DevOps mindset which, at its core, is about increasing collaboration and communication across an organization and reducing team silos that stifle this collaboration.
InfoQ: When should an organization adopt a continuous delivery model?
Organizations should implement a continuous delivery model when they have a need to regularly release software and/or they want to reduce the heroic efforts when they do release software. Releasing software less tends to make the release process more complex and prone to error, which calls for heroic efforts that can often otherwise be avoided. In other words, even if there isn’t a business need to release once a month, there’s often a compelling cost and quality of life motivation to move toward Continuous Delivery regardless of how often releases occur.
Organizations that haven’t incorporated Continuous Delivery and practices into their teams usually experience at least some of the following symptoms:
- Team Silos - Teams segmented by functional area
- Manual Instructions - Using manual instructions as the canonical source for infrastructure and other configuration
- Tribal Knowledge - Knowledge is shared from one person to others and not formally institutionalized
- Email - Managing release activities through emails
- Different Tools Across Teams: Different teams across the lifecycle use different tools to deliver software
- Issues/Tickets - Using issues/tickets as a means of communicating and assigning build, test, deployment and release-related tasks
- Meetings - Meetings are used as a weak attempt to get different teams on the “same page”
These Symptoms Often Generate These Results:
- Errors - When a full system isn’t integrated frequently, environments throughout the lifecycle are different, often leading to errors
- Increased costs - Errors lead to increased costs
- Delays - Weeks to get access to even just development and testing environments; Increased wait times as teams attempt to communicate across team silos
- Release less frequently - Release during off-hours, weekends, calling for hero efforts
InfoQ: How would you describe the kind of mentality change that should go along with adopting a CD system?
On an individual level, people need to be moving away from the “it’s not my job” mentality and move toward a “systems thinking” mentality. People should be continually asking themselves “how will this change affect the rest of the system?” This kind of systems thinking should manifest into more holistic thinking for the benefit of the overall system.
When I refer to the whole team, I mean a cross-functional team that consists of people who are 100% dedicated to the project. This team’s external team dependencies should be minimal and they should have the ability to release the software whenever there’s a business decision to do so without going through a separate team.
When I refer to the whole process, I’m referring to a heuristic that we’ve found to work well when thinking about “systematizing” a process. This heuristic is: document, test, code, version and continuous. This translates to:
- Document - Document the process with the idea that you will automate away most of the steps you’re documenting. We refer to this as “document to automate”.
- Test - Write automated tests that will verify that the behavior you’re automating is working.
- Code - Codify the behavior based on the tests and/or documentation.
- Version - Commit the code and tests to the version-control repository.
- Continuous - Once the code and tests are versioned, ensure it can be run headless (e.g. a single command taking into account any necessary dependencies) and then configured to run as part of a single path to production through a continuous integration system.
When people think in terms of the whole system, they need to extend beyond just the application/service code, and include the configuration, infrastructure, data and all the supporting and dependent resources such as build, tests, deployments, binaries, etc. All of these components need to be documented, tested, codified, versioned and run continuously with each and every commit.
InfoQ: In your LiveLessons, you stress the importance of a number of initial steps, such as identifying all stakeholders, performing a discovery process, setting up communication management tools, moving to a cross-functional organization and so on. Could you elaborate on the importance of those steps?
As an engineer, it’d be easy for me to gloss over these types of activities for the more interesting hands-on coding exercises. While other lessons in the video do focus on the hands-on coding, I was seeking to show viewers all of the steps that we typically go through at Stelligent when implementing CD with a customer. For example, we find that when teams don’t take the time to determine the current state of their processes, they perform unnecessary sub-optimization on things that don’t provide the most benefit.
For instance, let’s say your process from code commit to production takes three months and there are multiple days just waiting for people downstream to just click or approve something, why would you first spend days or weeks optimizing the process time of the one of the steps in your delivery process? Instead, you might spend some time determining why there are multi-day bottlenecks.
This is why it’s essential to do things like value-stream mapping so that everyone gets on the same page about the current state (either on the software project under development or other projects in the organization if it’s a new development effort) so that you’re spending time on optimizing the most critical bottlenecks first.
InfoQ: Among all the steps you go through to set up a CD solution within an organization, what are the critical ones in order to get CD right?
- Value-stream mapping - Create a left-to-right illustration of the current state containing all the steps in your process including the overall lead time, process times and wait times. Illustrate the anticipated future state.
- Self Service - create fully-automated pull-based systems so that any authorized team member can get environments, tools or other resources without human intervention.
- Cross-functional teams - for example: application developers, testing/QA, DBAs, operations, business analysts as part of the same small team
- Feedback - get the right information to the right people at the right time. This may include real-time information radiators, emails and other communication mechanisms. Instill “stopping the line” into the team culture so that errors are fixed soon after they’re introduced.
InfoQ: You focus on AWS in your LiveLessons. What is the advantage of using it to implement an organization’s CD system?
At Stelligent, we’ve focused exclusively on building CI/CD solutions and have been working in the cloud since 2009. We’d worked with multiple cloud providers in the first year or so. For one customer, we used and analyzed something like 15 cloud providers, at the time. We determined that AWS was the most feature-rich, stable, cost-effective solution for their needs and we decided that it was the best solution for our needs.
InfoQ: How would AWS stack up in a hypothetical Continuous Delivery Contest against its main competitors, such as Microsoft Azure, IBM SmartCloud, Google Cloud Platform, and so on?
For what it’s worth, I’d include the Google Cloud Platform before I’d include the IBM SmartCloud. AWS is far ahead of any of the other infrastructure-as-a-service providers on the market. For example, there’s no real equivalent to AWS OpsWorks - particularly in the context of an integrated IaaS provider. Because AWS has a large suite of services, it provides and then exposes them through AWS CloudFormation, and the number of resources you can automatically provision is much more than any other provider.
Moreover, there are no genuinely comparable solutions to the new Continuous Delivery-focused services at AWS such as AWS CodeDeploy, AWS CodePipeline and AWS CodeCommit and several other enterprise-focused services.
About the Interviewee
Paul M. Duvall is the Chairman and CTO at Stelligent. Stelligent is an expert in implementing Continuous Delivery solutions in Amazon Web Services (AWS) and has been working with AWS since 2009. Paul is the principal author of Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley, 2007) and a 2008 Jolt Award Winner. Paul is an author of many other books and publications including DevOps in AWS (Addison-Wesley, 2014) and two IBM developerWorks series on topics around automation, DevOps and Cloud Computing. He is passionate about software delivery and the cloud and actively blogs here.