Key Takeaways
- Software delivery performance matters, and it has a significant impact on organizational outcomes such as profitability, market share, quality, customer satisfaction, and achieving organizational and mission goals.
- High performers achieve higher levels of throughput, stability, and quality; they’re not trading off to achieve these attributes (for example, “moving fast and breaking things”).
- You can improve your performance by implementing practices from the lean, agile and devops playbooks.
- Implementing these practices and capabilities also has an impact on your organizational culture, which in turn has an impact on both your software delivery performance and organizational performance.
- There’s still lots of work to do to understand how to improve performance, and we’re excited to see other teams build on our research!
The book Accelerate: Building and Scaling High Performance Technology Organizations by Nicole Forsgren, Jez Humble, and Gene Kim, explores the factors that impact software delivery performance and describes capabilities and practices that help to achieve higher levels of throughput, stability, and quality.
InfoQ readers can download an excerpt from Accelerate.
InfoQ interviewed Humble and Forsgren about how they researched the performance of technology organizations, the main conclusions from the research, how culture impacts performance, how to build in security from the start, what deployment pain is and how to reduce it, how DevOps contributes to job satisfaction, what we don’t know about high performing organizations, and the main trends in lean software and DevOps.
InfoQ: Why did you write this book?
Jez Humble: This book builds on four years working with Puppet on the State of DevOps Report (SODR). Through the SODR, Dr Nicole Forsgren, Gene Kim and I conducted a research program into how to measure performance in the context of software delivery, what high performing software delivery means for organizational performance, and the factors that impact it. This book is a deep dive into that research and how we did it—but it’s also concise, at around 250 pages. We wrote it because this represents a fundamentally new approach to researching software practices and processes, and we wanted to present the results into an easily digestible form that teams can apply straight away and technical executives can leverage to advance their organizations.
InfoQ: For whom is this book intended?
Humble: This book really is for anybody who works in software. We think everyone from practitioners to executives could benefit from it. While the results won’t be a big surprise to anybody who is au faitwith modern trends in software, such as the Agile, Lean and DevOps movements, we cut to the chase by presenting the key capabilities that actually have an impact on performance, including what does and doesn’t work, along with a description of whythey work. We also include information about how to think about measurement and progress in technology to help teams get ahead.
InfoQ: What methods do you use to research the performance of technology organizations?
Humble: We used psychometrics, because this allowed us to collect data from a large population of people quickly (among other reasons, something we discuss in Part II of the book). Studying software is hard because good measures don’t yet exist, and if they do, the tools aren’t always there yet. In addition, software development and delivery are primarily social activities, both in themselves and in the wider context that product development is ultimately about problem solving. It’s practically impossible to run randomized, controlled experiments in the context of studying software because there are so many variables and they’re so hard to control -- plus, companies aren’t willing to accept an experimental condition where they are forced to lose money just to fulfill an aspect of research design. And so we use what are known as quasi-experimental methods, or field research methods: we collect data from organizations doing the work, controlling for some variables, and leveraging scientifically validated, reliable survey measures. I’m really delighted at the strength and power of the results. Our principal investigator, Dr Nicole Forsgren, has been doing this kind of research for many years in an academic context and it’s extremely serendipitous that we all ran into each other and got to work together applying the methods she developed to studying the Agile, Lean and DevOps paradigms.
InfoQ: You explained in the book that in order to improve performance we should focus on capabilities and not on maturity. Can you elaborate why this is, and how this impacts the way we can measure performance?
Nicole Forsgren: We caution organizations against adopting a maturity-based view of technology, because it isn’t appropriate for the complex, ever-changing world we operate in. I’ll focus on one issue here: maturity models assume a destination and not a direction. They assume you can identify a goal state, which, honestly, is incredibly difficult given the rate of change of technology, the competitive landscape, and what our customers expect of us. Once you have arrived at that destination, you declare success (or at least “arrival”), you are done and resources disappear. Resources can be money, time, or attention to the technology work that is increasingly essential for remaining competitive, secure, and reliable.
The best, most innovative organizations know that any time away from continued improvement means time that your competitors are moving beyond you; it’s the old tortoise and hare story. If you don’t keep improving, you will be passed by.
In contrast, capabilities models help teams and organizations understand that the goal is improvement, and then focus on the key capabilities to improve to help them at each step along the way. With the understanding that you will never “arrive,” but instead always be progressing, the capabilities model helps focus efforts on the work that will be the most impactful.
Sometimes I hear the complaint, “but when will this technology transformation be over?” Well, when will your accounting work be over? When will your compliance work be done? When will your core offerings be finished? The answer is, and should be, never. As an organization, you should be working to improve your core processes continuously so that you can delight your customers and win in the market (that’s the piece of your tech that is unique and innovative to you), and you should be executing your foundational functions to keep your organization running at its best (things like accounting and HR and the pieces of your tech that are necessary to run your business, like infrastructure and mail). You’ll notice that tech appeared in both categories. Tech is both a commodity and value differentiator. Thinking that there will be a day when an organization can (or should) stop investing in tech is a fool’s errand.
We cover three more challenges with maturity models and why capabilities are more appropriate ways to address technology transformation in detail in the book.
InfoQ: What are the main conclusions from your research?
Humble: Our main conclusions are that software delivery does impact organizational performance, both for commercial and non-commercial outcomes. Further, high performers are able to achieve higher levels of throughput, stability and quality; they’re not in fact “going faster and breaking things” or trading off throughput against stability and quality. Finally, we’ve shown that—when done right—many of the practices from the Agile, Lean and DevOps movements impact both software delivery and organizational performance.
InfoQ: How does the culture within an organization impact their performance?
Humble: To answer this question you have to first say what you mean by “culture”; there are many models. For our research we picked a model by Dr. Ron Westrum, who has been studying the interplay between culture and safety in the domains of healthcare and aviation. We were able to show that a more generative, performance-oriented culture (what you might also call a mission-oriented culture) impacts both software delivery performance and organizational performance. We also showed, following John Shook’s observation that “the way to change culture is … to start by changing how people behave—what they do,” that implementing practices from continuous delivery and lean management can change culture.
InfoQ: How can we build in security from the start?
Humble: There are a number of ways to achieve this. First, we can make sure developers have access to toolchains which check for common vulnerabilities, ensure that libraries are up-to-date, and so forth. Second, we make sure developers are educated in how to build secure systems, and common security risks (for example the OWASP Top 10). A third approach is to provide developers secure platforms to deploy to for testing and production purposes; for example, the US government’s Technology Transformation Service created a PaaS called cloud.gov which takes care of a large number of information security controls at the platform layer. Most important is for InfoSec to be involved in the governance and design of the software delivery lifecycle rather than stepping in at the end. By that time, it’s extremely expensive to make significant changes to a system that wasn’t designed with security in mind.
InfoQ: What is deployment pain? What can be done to reduce it?
Humble: Deployment pain is the fear and anxiety engineers and technical staff feel when they push code into production. I’ve often said that the goal of continuous delivery (CD) is to make releases boring: deployment pain is the opposite. My favorite CD metric is the number of person-hours teams must spend outside normal business hours performing deployments to production; the goal here is zero. That was a crucial goal of Dave Farley and I writing the Continuous Delivery book: to show how to achieve this using simple patterns and practices. Unfortunately there are still many teams that have to perform complex, big-bang, orchestrated deployments outside of business hours, but there is absolutely no excuse for this in 2018 - although of course it’s still hard to do and requires sustained effort and focus.
Our research shows, as you might expect, that implementing continuous delivery practices such as test and deployment automation and creating a loosely-coupled architecture reduces deployment pain.
InfoQ: How does DevOps contribute to job satisfaction?
Humble: Our research shows that implementing continuous delivery and lean management practices, as well as having a generative or mission-oriented culture, positively impacts job satisfaction. This is important as we also showed that people who recommended their team or organization as a good place to work (as measured by employee net promoter score or eNPS) also identify more strongly with their organization’s goals, and the effort they are willing to put in to make the organization successful. Naturally, high eNPS also helps with recruiting: employees are more likely to refer others as well as being less likely to quit, which puts less pressure on recruiting in the first place. The data doesn’t include detailed stories behind why implementing continuous delivery and lean management increases job satisfaction, but we know that lean practices and continuous delivery reduce overwork and burnout, help teams build better products, and also feel more connected to the rest of the organization and to customers… and I’m sure we’ve all heard stories about how improving our automation, processes, and culture can having lasting effects on how we feel about our work and work/life balance.
InfoQ: Are there things about high performing organizations that we don't know yet?
Humble: Yes! We’re at the beginning of the journey of researching software delivery, and we hope other researchers take inspiration from what we’ve been able to achieve. In the spirit of research, we also look forward to the verification and extension of our research by other teams. In our own research, we’ve been able to confirm some of the results of Google’s Project Aristotle (which we talk about in the book). Meanwhile, we’ve just opened the survey for our 2018 study, and we’d love readers to take the 2018 survey and help us understand more about how to build high performing teams and organizations.
InfoQ: What are the main trends in lean software and DevOps that your research has uncovered? What do you expect the future to bring?
Humble: We’ve confirmed that much of what the Lean Software Development, Lean Startup, Agile and DevOps movements have said works, but often with a twist: for example, limiting work in process is particularly strong in the context of software development and delivery when teams also use visual displays to share information and context. We’ve also tried to shine a light on common scenarios where people claim to be implementing these ideas but have actually missed what’s essential. For example, we find that teams which have the authority to experiment with new ideas do better in terms of both software delivery and organizational performance. Unfortunately, many organizations that claim to have implemented “Agile” still don’t allow this, and their software teams are still fundamentally just order-taking functions rather than partners in creating value.
In the future I expect to see lots of innovation in both tools and new ways of working. My hope is that as an industry we have the wisdom to combine what is known to work with the ideas that withstand the crucible of experimentation. That will require us to get a lot better both at understanding our history as an industry, and at taking a scientific approach to innovation. I hope we’ve been able to contribute to this goal.
About the Book Authors
Jez Humble is co-author of Accelerate,The DevOps Handbook, Lean Enterprise, and the Jolt Award winning Continuous Delivery. He has spent his career tinkering with code, infrastructure, and product development in companies of varying sizes across three continents, most recently working for the US Federal Government at 18F. He is currently researching how to build high performing teams at his startup, DevOps Research and Assessment LLC, and teaching at UC Berkeley. How about adding a short bio from Nicole and Gene too (unsure if InfoQ requires it)?
Dr. Nicole Forsgren is co-founder, CEO and chief scientist at DevOps Research and Assessment. She is best known for her work measuring the technology process and as the lead investigator on the largest DevOps studies to date. She has been a professor, sysadmin, and performance engineer. Forsgren’s work has been published in several peer-reviewed journals. Forsgren earned her PhD in Management Information Systems from the University of Arizona, and is a research affiliate at Clemson University and Florida International University.
Gene Kim is a multi-award winning CTO, researcher, and author. He is the founder of Tripwire and served as CTO for thirteen years. His books include The Phoenix Project, The DevOps Handbook, Beyond the Phoenix Project, Accelerate, The Visible Ops Handbook, and Visible Ops Security. Gene is a huge fan of IT operations and how it can enable developers to maximize throughput of features from “code complete” to “in production” without causing chaos and disruption to the IT environment. He has worked with some of the top Internet companies on improving deployment flow and increasing the rigor around IT operational processes. In 2007, ComputerWorld added Gene to the “40 Innovative IT People to Watch Under the Age of 40” list, and he was named a Computer Science Outstanding Alumnus by Purdue University for achievement and leadership in the profession. Follow Gene on his website or on Twitter @RealGeneKim