BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations Scaling Uber's Elasticsearch Clusters

Scaling Uber's Elasticsearch Clusters

Bookmarks
48:18

Summary

Danny Yuan talks about how Uber scaled its Elasticsearch clusters as well as its ingestion pipelines for ingestions, queries, data storage, and operations by a three-person team. He covers topics like federation, query optimization, caching, failure recovery, data fidelity, transition from Lambda architecture to Kappa architecture, and improvements on Elasticsearch internals.

Bio

Danny Yuan is a software engineer in Uber. He’s currently working on streaming systems for Uber’s marketplace platform. Prior to joining Uber, he worked on building Netflix’s cloud platform. His work includes predictive autoscaling, distributed tracing service, real-time data pipeline that scaled to process hundreds of billions of events every day, and Netflix’s low-latency crypto services.

About the conference

Software is changing the world. QCon empowers software development by facilitating the spread of knowledge and innovation in the developer community. A practitioner-driven conference, QCon is designed for technical team leads, architects, engineering directors, and project managers who influence innovation in their teams.

Recorded at:

Apr 11, 2018

BT