Yahoo! Screwdriver–A CD Tool That Scales

| by Abel Avram Follow 5 Followers on Jan 13, 2017. Estimated reading time: 1 minute |

A note to our readers: You asked so we have developed a set of features that allow you to reduce the noise: you can get email and web notifications for topics you are interested in. Learn more about our new features.

Yahoo! has open sourced Screwdriver, their CD tool used internally in production.

Screwdriver is a continuous delivery tool that integrates all steps from code commit to deployment in production. Yahoo! has increasedly used Screwdriver to automate all delivery processes for the last five years, reaching over 25,000 builds and 12,000+ git commits each day. According to Yahoo!, they wanted a CD tool that:

  • Enables developers to easily create deployment pipelines. Pipelines are defined in YAML files and one can revert to an earlier version of the pipeline by discarding changes or using another version of the configuration file.
  • Optimizes for trunk development. The trunk should contain a shippable version of the application. When a pull request is made, the tests run automatically to make sure the code to be accepted is tested.
  • Supports rolling back. Anybody with write permissions can roll back a deployment in case something goes wrong.

Screwdriver has five main components:

  • REST API – an interface for working with pipelines.

  • Web UI – a visual interface for the pipelines API.

  • Launcher – a tool for setting up the environment and execute shell commands.

  • Execution Engine – a pluggable build executor supporting the execution of commands inside of a container (Jenkins, Kubernetes, Mesos, Docker Swarm).

  • Datastore – a pluggable NoSQL storage used to maintain pipeline configuration data (DynamoDB, MongoDB, CouchDB, Postgres). Both the execution engine and the datastore are using a pluggable architecture that allows one to use the engine of their choice.

Yahoo! has open sourced now a trimmed down version of their CD tool, but they plan to add some of the missing components in the months to come: keeping build metadata to use later, metrics collection, log analysis and templates for setting up and executing pipelines.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you