BT

New Early adopter or innovator? InfoQ has been working on some new features for you. Learn more

Netflix: Dystopia as a Service

| by Saul Caganoff on May 22, 2013. Estimated reading time: 2 minutes |

"As engineers we strive for perfection; to obtain perfect code running on perfect hardware, all perfectly operated." This is the opening from Adrian Cockroft, Cloud Architect at Netflix in his recent Keynote Address at Collaboration Summit 2013. Cockroft continues that the Utopia of perfection takes too long, that time to market trumps quality and so we compromise. But rather than lamenting the loss of "static, better, cheaper" perfection, Cockroft describes how the Netflix "Cloud Native" architecture embraces the Dystopia of "broken and inefficient" to deliver "sooner" and "dynamic". Cockroft says that "the new engineering challenge is not to construct perfection but to construct highly agile and highly available services from ephemeral and often broken components."

Hosted almost entirely on Amazon Web Services (AWS) infrastructure, the Netflix architecture utilises "antifragile patterns" that were inspired by Nassim Taleb's 2012 book "Antifragile: Things That Gain From Disorder" and Michael Nygard's book "Release It!." These patterns include stateless auto-scaled microservices, reactive APIs, circuit breakers, bulkheads and graceful degradation. Antifragility includes the curious concept of Chaos Monkeys; services that randomly terminate virtual machines in production to test and prove the availability and reliability of the whole system. Cockroft describes how Netflix recently deployed their "Chaos Gorilla" that terminated an entire zone out of their three AWS zones, yielding no perceptible outage for Netflix customers.

Cockroft describes the architecture as "Cloud Native". "Master copies of data are cloud resident, everything is dynamically provisioned, all services are ephemeral." says Cockroft. The largest services are autoscaled with an average instance lifetime of 36 hours. At the physical level the Netflix infrastructure is composed of thousands of AWS instances supporting auto-scaled micro-services. Some micro-services provide stateless middleware functions while others provide database storage via a quorum of Cassandra NoSQL database instances.

The final part of Cockroft's address holds ideas and opportunities for the Linux Foundation, sponsors of the Collaboration Summit. What keeps Cockroft "awake at night" are the problems associated with a monoculture. That replicating "the best" as patterns and reducing interaction complexity leads to an epidemic single point of failure. Cockroft cites the example of the leap-second bug that caused major problems for online services last year. Cockroft suggests that the only way to address these "pattern failures" is automated diversity management and to understand the trade-off between efficiency and fragility.

According to a recent report by networking equipment company Sandvine, Netflix is responsible for one third of all the US-based downstream traffic on the internet. Netflix is very open about their architecture and progressive in releasing much of their code as open source software in the form of NetflixOSS. Adrian Cockroft's slides from this talk are posted on slideshare

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT