Music streaming giant Spotify has recently shared how it measures the return on investment (ROI) of Backstage, its open platform for building developer portals.
Following the news that Toyota North America have saved $10m with a Backstage implementation, Spotify outlined a methodology for measuring the success of Backstage in an earlier blog post, with a focus on determining success by three metrics:
- Developer productivity - the ratio of work completed compared to the effort and time put in
- Developer efficiency - the resources required in time, computer power and people to generate a unit of output
- Developer effectiveness - a qualitative look at the individuals writing code in the organisation
A number of other metrics are also used, such as the time taken for a new starter to merge their 10th pull request, and many proxy metrics such as plugin contributions, search success rates, click-through rates and reduced context switching.
The company also measures the ROI of Backstage through engagement metrics, including the number of unique users, the frequency of visits, and the time spent on the platform. Additionally, the company conducts regular surveys to gauge employee satisfaction and gather feedback on the platform's features and functionality, through a Backstage plugin - Pulse.
The survey has found that use of Backstage directly translates to quantifiable impact, with stats such as frequent Backstage users being 2.3x more active in GitHub, and deploying twice as often, with these changes taking 17% less time to go live. This translated to a potential saving of three full-time employees in a team of ten developers, saving time and resources for the organisation.
Furthermore, changes deployed by Backstage users tended to stay in production for more than three times as long, suggesting that Backstage use is contributing to higher code quality.
Spotify's commitment to measuring the effectiveness of Backstage also underscores the value of data-driven decision-making in the workplace. By collecting and analyzing engagement metrics and employee feedback, companies can make informed decisions about how to improve their communication and collaboration tools, ultimately leading to a more engaged and productive workforce.
The overall conclusion of the report is that high-frequency Backstage users deploy software more often and in less time, with higher perceived productivity, and employees using it also stay longer with the company. The full post is available to read on the Backstage blog.