Jez Humble and Gene Kim, prominent figures of the DevOps movement, are working with Puppet Labs on the 2013 DevOps Survey Of Practice. The survey's goal is to better understand which IT practices drive an organization to high performance. The survey will close on the 15th of January and everyone is invited to participate.
The 2013 DevOps Survey will try to understand which IT practices enable high-performing organizations. As Jez and Gene explain on a recent article, they established several research hypotheses that will be put to the test and independently evaluated:
- small teams with high trust that span the entire value stream: Dev, QA, IT Operations and Infosec
- shared goals and shared pain that span the entire value stream
- small development batch sizes
- presence of continuous, automated integration and testing
- emphasis on creating a culture of learning, experimentation and innovation
- emphasis on creating resilient systems
The survey will also test two more overarching hypotheses. It will try to establish how lead time correlates with product quality, customer satisfaction and employee happiness. Building upon last year's survey, which correlated adoption of DevOps practices with higher IT performance, the survey will explore how higher IT performance impacts business performance (e.g: return on assets; market share).
Gene told InfoQ why he thinks this survey is relevant:
We're testing many of the brilliant foundational theories and practices laid out by Martin Fowler on continuous integration and delivery over the last 15 years, to hopefully show concrete evidence of how it improves performance and productivity, business value or developer happiness.
The 2012 DevOps Survey Of Practice, which had 4039 respondents, found out that high performing organizations have some common patterns. When compared with low performing organizations, they deploy about 30x faster with shorter change lead times, which can be measured in hours or minutes instead of weeks or months. They also had 50% fewer change failures (e.g.: production outages or service impairments) with a 12x faster mean time to recover.
The DevOps Surveys of Practice are cross-population studies. Gene and Kim's aforementioned article explains that these studies are a "statistical research technique designed to uncover what factors (e.g., practices, cultural norms, etc.) correlate with outcomes (e.g., IT performance)", commonly used, for example, in medical research. Cross-population studies are more rigorous than more common approaches to support hypotheses in the world of IT.
Gene is hoping for a higher percentage of respondents with a developer profile than on the 2012 survey:
We've already gotten 4000+ responses, beating last year. But we're really hoping to get a large number developer respondents this year, to ensure that we can measure the Dev in DevOps. Last year there was a greater emphasis on the Ops side.
The survey will close on January 15th. The results will be published by mid-2014, in time for the Velocity Conference.