To drive operational maturity you need a microservices architecture, continuous delivery process, DevOps culture and platform automation, argued Casey West. Together these four help you to transform your whole organization for achieving cloud-native operability to continuously deliver additional value to your customers.
Casey West, principal technologist at Pivotal, gave a talk at the Agile and Software Architecture Symposium 2016 about the journey to operational maturity. InfoQ is covering this event with Q&As, summaries, and articles.
Microservices are not an optimization for computer resources, argued West; their aim is to support the architecture and the organization. Adopting microservices helps you to avoid slow and risky monolithic deployment strategies, and to expose inefficiencies in your organization’s culture and deployment strategies.
If we get anything out of DevOps, it’s improving the way we work together, said West. DevOps can be used to create a culture where people collaborate, learn and share.
West suggested that the first sprint that a team does should include setting up a build pipeline and automating production. He mentioned the characteristics of platform automation which can be used to build a minimum viable platform:
- Dynamic DNS, routing and load balancing
- Backing service brokering
- Infrastructure orchestration
- Health management, monitoring, and recovery
- Immutable artifact repository
- Log aggregation
InfoQ spoke with Casey West after his talk at ASAS 2016 and asked him what’s needed to achieve cloud-native operability, establish continuous delivery processes, and a DevOps culture, and how you can assess if an organization is able to develop and deliver cloud-native software.
InfoQ: What is cloud-native operability?
Casey West: Cloud-native operability is a holistic approach to software design and delivery for large-scale systems taking advantage of the cloud. Rather than focus on one modern approach to optimizing development and operations throughput, such as "DevOps" or microservices, I aim to have the audience consider the four key components to operationally mature business-value delivery.
InfoQ: In your talk you stated that you need a microservices architecture, continuous delivery process, DevOps culture and platform automation. Why do you need these four things?
West: We need all four because they all rely on each other. For instance, if you have not automated your path to production with continuous delivery but your engineering team is moving forward with a microservices architecture it will be incredibly difficult for your teams to ship updates to production in a timely and inexpensive fashion. Just as the success of your microservices strategy depends on a streamlined deployment process, it also relies on a culture of shared responsibility and collaboration inherent in the DevOps movement.
Equally important, a mature, fully automated, resilient production environment is required for its success. If production is the worst place for your applications to live, and is constantly breaking requiring manual intervention, you will not succeed in the big picture goal: increase throughput while reducing risk.
InfoQ: What can organizations do if they want to establish a continuous delivery process?
West: Continuous delivery requires coordination and collaboration among several functional teams in an organization. Developers and testers must work together to build reliable, trustworthy, automated test suites. When a change to the software is made, a passing test suite should offer your team a high level of confidence this change will not break the system.
An automated delivery pipeline begins with the quality test suite, and ends with a single step, push button deployment to production. Again, a fully automated deployment strategy is critical, and requires coordination between developers, release engineers, IT, and operations. In most environments, an automated deployment without down time (often called "zero-downtime deployments" or "blue green deployments") is possible when capacity and physical resource allocation is automated, and when the changes made during deployment are small.
These small changes, or small batch sizes, on deployment are the final key to the puzzle, and require coordination with business interests such as product and project management, as well as developers. When organizations are used to modifying production once or twice a year, their batch sizes for deployment are massive. Even if you ship every two weeks, this could still represent thousands of lines of code change for each deployment and dozens of different system changes grouped together. In this case it’s incredibly risky to change the system. If production misbehaves how do you know which change caused the issue? How to you triage, and how do you fix it efficiently?
Small batch sizes offer a less risky option: make incremental changes one at a time and deploy each of them to production in turn. In this way you may be changing just one line of code, or a dozen. We can reason about changes at this scale, and have a clearer understanding of its effects on the system. If it misbehaves we know which change did it, and we can react much faster.
To adopt this strategy you must disassociate changes to production, or deployments, from feature releases. They are not the same thing in continuous delivery, and the business needs to understand this. Development must also plan for this approach with feature flags or A/B testing strategies, so feature releases can be managed independently from code deployments.
InfoQ: Which are the ingredients of a DevOps culture?
West: In my opinion the most important contribution the DevOps movement has made to the efficacy of our software delivery teams is its culture. This is best represented by the acronym "CALMS" which stands for collaboration, automation, learning, measuring, and sharing. These are not specific actions, but rather how we work together. These are the characteristics effective teams are hiring for and cultivating in modern organizations today.
InfoQ: How can you assess if an organization is able to develop and deliver cloud-native software?
West: Organizations iterating on their architecture, process, culture, and automation are on the path. They must be working on all four of these aspects of business-value delivery simultaneously to have a chance at successfully taking advantage of the cloud. As an industry we have seen evidence that organizations with a mature microservices architecture, trustworthy continuous delivery process, responsible DevOps culture, and fully automated platform in production have the greatest success developing and delivering cloud-native software.
InfoQ: Any final advice for organizations that want to achieve cloud-native operability?
West: Continue to heed Fred Brooks’ advice from his seminal essay, "No Silver Bullet" from 1986:
"There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity."
I still believe this is true. However, thanks to recent advancements in cloud-based platforms, organizational culture, automated delivery, and software design, we can combine these efforts and accelerate our productivity, reliability, and simplicity. When delivering large-scale, cloud-native solutions I believe we have no choice but to take advantage of these four key factors to reduce risk, increase throughput, and ultimately stay competitive.