BT
x Share your thoughts on trends and content!

Spotify Moves Infrastructure and Data Services to Google Cloud Platform

by Kent Weare on Feb 28, 2016 |

On February 23rd, 2016 Spotify announced it is moving its technical infrastructure and data services from its existing owned and leased data centers to Google Cloud Platform.

Spotify is a popular music steaming service that powers 2 billion playlists, more than 20 billion hours of music for more than 75 million listeners. As the business has grown year over year, Spotify questioned whether or not they wanted to continue to run their own data centers. In previous years, the company felt the public cloud services could not meet the quality, performance and cost expectations required to make the shift to the cloud.  In a recent blog post, Nicholas Harteau, VP engineering and infrastructure at Spotify, explained some of the rationale for moving to the cloud now: “Storage, compute and network services available from cloud providers are as high quality, high performance and low cost as what the traditional approach provides. This makes the move to the cloud a no-brainer for us.”

In a recent Rightscale Cloud Survey, Google Cloud Platform ranked third in Public Cloud adoption behind leader Amazon Web Services and Microsoft Azure.  Even though Google is not a leader in this space, Harteau explains why Spotify ended up choosing Google: “What really tipped the scales towards Google for us, however, has been our experience with Google’s data platform and tools. Good infrastructure isn’t just about keeping things up and running, it’s about making all of our teams more efficient and more effective, and Google’s data stack does that for us in spades.”

Google, also very excited about the partnership with Spotify, revealed some specific details about how Spotify will be using their services in their own blog post.  From a compute standpoint, Spotify will be relying upon high performance IOPS SSD and local SSD storage capabilities.  They will also be taking advantage of auto scaling capabilities to ensure their infrastructure is able to respond to burst scenarios.  One such burst scenario was last November 13th when Justin Bieber set a record for the most streams in a single day.  Over 36 million streams were listened to on that day.  Using a public cloud model, Spotify can now rely upon Google to provide that elasticity, instead of Spotify building out their own infrastructure to support peak loads.

Spotify will also be using Google’s networking services such as Direct Peering, Cloud VPN and Cloud Router in order to efficiently transfer petabytes of data between the two organizations.

From a data services perspective, Spotify will be moving away from Hadoop, MapReduce, Hive and some custom dashboards in favor of Google Cloud services including Google Cloud Pub/Sub, Google Cloud Dataflow, Google BigQuery and Google Dataproc.  Guillaume Leygues, lead sales engineer at Google Cloud Platform, explains why Spotify is moving to these Google Services: “With BigQuery and Cloud Dataproc, data teams can run complex queries and get answers in a minute or two, rather than hours. This lets Spotify perform more frequent in-depth, interactive analysis, guiding product development, feature testing and more intelligent user-facing features.”

Rate this Article

Relevance
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss
General Feedback
Bugs
Advertising
Editorial
Marketing
InfoQ.com and all content copyright © 2006-2016 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.