BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations Scio: Moving Big Data to Google Cloud, a Spotify Story

Scio: Moving Big Data to Google Cloud, a Spotify Story

Bookmarks
54:50

Summary

Neville Li tells the Spotify’s story of migrating their big data infrastructure to Google Cloud, replacing Hive and Scalding with BigQuery and Scio, which helped them iterate faster.

Bio

Neville Li is a software engineer at Spotify who works mainly on data infrastructure and tools for machine learning and advanced analytics. In the past few years he has been driving the adoption of Scala and new data tools for music recommendation, including Scalding, Spark, Storm and Parquet. Before that he worked on search quality at Yahoo! and old school distributed systems like MPI.

About the conference

Entering its 12th year, the Emerging Technologies Conference has brought the leaders of the open source community to Philadelphia to teach about their projects, their work and how these technologies are changing software development. During this time, ETE has become one of the largest gatherings of developers in the mid-Atlantic.

Recorded at:

May 26, 2017

BT