The Puppet Labs: State of DevOps Report 2015 shows the current DevOps trends in IT, comparing the high and low performers in terms of deployment success and stability, and observing the link between architecture and developer productivity.
The majority (81%) of almost 5,000 survey respondents belonged to the IT Operations, Development and DevOps departments, 70% of them working in companies with more than 100 servers in their infrastructure, with 18% having more than 1,000 servers. Among others, the survey has evaluated the deployment throughput and service stability across the industry, by inquiring for the following metrics:
- Deployment Frequency – how many times an organization deploys code
- Deployment Lead Time – time needed for a change to pass from code committed to being live in production
- Mean Time to Recover (MTTR) – time needed to restore a service when it goes down due to a failure
- Change Success Rate – the percentage of code changes that make it into production successfully
Evaluating the answers, the survey’s authors have made a distinction between high performers and low performers, the following table showing the differences between them. What has changed from the last year’s report is the MTTR –the high performers recovering much faster from failures- and the Change Success Rate – the high performers having 20 times more code successfully pushed to production:
| Metric | 2015 (high vs. low) | 2014 (high vs low) |
| Deployment Frequency | 30x | 30x |
| Deployment Lead Time | 200x | 200x |
| MTTR | 168x | 48x |
| Change Success Rate | 60x | 3x |
The authors of the reports have used structural equation modeling to measure the impact of lean management and continuous delivery on organizational and IT performance. They concluded that limiting Work in Progress (WIP), using visual methods to monitor quality, productivity and WIP, and using monitoring tools to make business decisions have a direct impact on organizational performance. Also, test and deployment automation, trunk-based development and continuous integration, and keeping app and system configuration in VCS lower the deployment pain and the change fail rates, raising the deployment throughput and stability, resulting in more productivity, market share and profitability.
This year’s survey has also measured the link between software architecture and developer productivity. They tracked the number of deployments per day per developer, noticing that when more developers are involved in a project the productivity lowers for low performers and grows exponentially for high performers, as shown in the following graphic:
/filters:no_upscale()/news/2015/07/puppet-labs-devops-survey-2015/en/resources/devops-survey-2015-1.png "image")
The study has remarked the following factors contributing to high IT performance: “a goal-oriented generative culture, a modular architecture, the engineering practices that enable continuous delivery, and effective leadership.”
One chapter focuses on advice for IT managers on how to help their teams perform better, emphasizing why organizational culture matters, how to build a generative (performance-oriented) culture and how to introduce DevOps. Another chapter is dedicated to employee burnout, outlining the main causes: work overload, lack of control, insufficient rewards, community breakdown, lack of fairness, and value conflicts.