BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations Cloud-Native and Scalable Kafka Architecture

Cloud-Native and Scalable Kafka Architecture

Bookmarks
46:58

Summary

Allen Wang talks about how Netflix addresses the issues of stability and scalability in a cloud environment by having many smaller and mostly immutable Kafka clusters with limited state changes. He proves the merit of this architecture in mathematical terms and illustrates how this architecture and additional tooling helps to improve availability, scale and failover.

Bio

Allen Wang is currently with Netflix Real-time Data Infrastructure team where he made significant contribution to Kafka and data infrastructure in AWS. He is a contributor to both Apache Kafka and NetflixOSS and the author of Kafka's rack-aware partition assignment.

About the conference

Software is changing the world. QCon empowers software development by facilitating the spread of knowledge and innovation in the developer community. A practitioner-driven conference, QCon is designed for technical team leads, architects, engineering directors, and project managers who influence innovation in their teams.

Recorded at:

Apr 12, 2018

BT