Spotify Moves Infrastructure and Data Services to Google Cloud Platform

| by Kent Weare Follow 10 Followers on Feb 28, 2016. Estimated reading time: 2 minutes |

On February 23rd, 2016 Spotify announced it is moving its technical infrastructure and data services from its existing owned and leased data centers to Google Cloud Platform.

Spotify is a popular music steaming service that powers 2 billion playlists, more than 20 billion hours of music for more than 75 million listeners. As the business has grown year over year, Spotify questioned whether or not they wanted to continue to run their own data centers. In previous years, the company felt the public cloud services could not meet the quality, performance and cost expectations required to make the shift to the cloud.  In a recent blog post, Nicholas Harteau, VP engineering and infrastructure at Spotify, explained some of the rationale for moving to the cloud now: “Storage, compute and network services available from cloud providers are as high quality, high performance and low cost as what the traditional approach provides. This makes the move to the cloud a no-brainer for us.”

In a recent Rightscale Cloud Survey, Google Cloud Platform ranked third in Public Cloud adoption behind leader Amazon Web Services and Microsoft Azure.  Even though Google is not a leader in this space, Harteau explains why Spotify ended up choosing Google: “What really tipped the scales towards Google for us, however, has been our experience with Google’s data platform and tools. Good infrastructure isn’t just about keeping things up and running, it’s about making all of our teams more efficient and more effective, and Google’s data stack does that for us in spades.”

Google, also very excited about the partnership with Spotify, revealed some specific details about how Spotify will be using their services in their own blog post.  From a compute standpoint, Spotify will be relying upon high performance IOPS SSD and local SSD storage capabilities.  They will also be taking advantage of auto scaling capabilities to ensure their infrastructure is able to respond to burst scenarios.  One such burst scenario was last November 13th when Justin Bieber set a record for the most streams in a single day.  Over 36 million streams were listened to on that day.  Using a public cloud model, Spotify can now rely upon Google to provide that elasticity, instead of Spotify building out their own infrastructure to support peak loads.

Spotify will also be using Google’s networking services such as Direct Peering, Cloud VPN and Cloud Router in order to efficiently transfer petabytes of data between the two organizations.

From a data services perspective, Spotify will be moving away from Hadoop, MapReduce, Hive and some custom dashboards in favor of Google Cloud services including Google Cloud Pub/Sub, Google Cloud Dataflow, Google BigQuery and Google Dataproc.  Guillaume Leygues, lead sales engineer at Google Cloud Platform, explains why Spotify is moving to these Google Services: “With BigQuery and Cloud Dataproc, data teams can run complex queries and get answers in a minute or two, rather than hours. This lets Spotify perform more frequent in-depth, interactive analysis, guiding product development, feature testing and more intelligent user-facing features.”

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you