One of the key achievements of the past decade has been both the wide availability of scalable systems for the masses, especially Cloud based systems, and the super-scalability of some Web applications. Facebook for instance serves 13 millions requests per second on average with peaks at 450 M/s. Even with such results, the concepts and architectures behind scalable systems are still rapidly evolving. About three years ago, Ricky Ho, a software architect based in California, had written blog post detailing the state of the art of scalable systems. Three years later, he felt that it was time to revisit the question.
Scalability is about reducing the adverse impact due to growth on performance, cost, maintainability and many other aspects
In his latest post, he lists patterns:
- Load Balancer
- Scatter and Gather
- Result Cache
- Shared Space (a.k.a the Blackboard)
- Pipe and Filter
- Map Reduce
- Bulk Synchronous Parallel
- Execution Orchestration
If Load Balancing, Result Cache and Map Reduce have been around for a while, some patterns are targeting new problems introduced by social medias. For instance the Bulk Synchronous Parallel, which was invented in the 80s, is now used as part of the Google Pregel Graph Processing project which supports three general processing patterns:
- Capture (e.g. When John is connected to Peter in a social network, a link is created between two Person nodes)
- Query (e.g. Find out all of John's friends of friends whose age is less than 30 and is married)
- Mining (e.g. Find out the most influential person in Silicon Valley)
Ricky introduces also the Execution Orchestrator pattern:
This model is based on an intelligent scheduler / orchestrator to schedule ready-to-run tasks (based on a dependency graph) across a clusters of dumb workers.
He reports that this pattern is used as part of the Microsoft Dryad project, which allows programmers "to use thousands of machines without knowing anything about concurrent programming".
A Dryad programmer writes several sequential programs and connects them using one-way channels. The computation is structured as a directed graph: programs are graph vertices, while the channels are graph edges. A Dryad job is a graph generator which can synthesize any directed acyclic graph. These graphs can even change during execution, in response to important events in the computation.
The kind of scalability that we routinely achieve today was unthinkable just 10 years ago. Where are the next limits? What is your experience in building scalable systems? What is missing?