InfoQ Homepage Presentations Scaling Uber's Elasticsearch Clusters
Scaling Uber's Elasticsearch Clusters
48:18
Summary
Danny Yuan talks about how Uber scaled its Elasticsearch clusters as well as its ingestion pipelines for ingestions, queries, data storage, and operations by a three-person team. He covers topics like federation, query optimization, caching, failure recovery, data fidelity, transition from Lambda architecture to Kappa architecture, and improvements on Elasticsearch internals.
Bio
Danny Yuan is a software engineer in Uber. He’s currently working on streaming systems for Uber’s marketplace platform. Prior to joining Uber, he worked on building Netflix’s cloud platform. His work includes predictive autoscaling, distributed tracing service, real-time data pipeline that scaled to process hundreds of billions of events every day, and Netflix’s low-latency crypto services.
About the conference
Software is changing the world. QCon empowers software development by facilitating the spread of knowledge and innovation in the developer community. A practitioner-driven conference, QCon is designed for technical team leads, architects, engineering directors, and project managers who influence innovation in their teams.
Apr 11, 2018