InfoQ

News

Article: Scalability Principles

Posted by Floyd Marinescu on May 21, 2008 01:06 PM

Community
Architecture
Topics
Performance & Scalability
Tags
Concurrency ,
Performance Evaluation
At the simplest level, scalability is about doing more of something. This could be responding to more user requests, executing more work or handling more data. While designing software has its complexities, making that software capable of doing lots of work presents its own set of problems. In this article, Simon Brown presents some principles and guidelines for building scalable software systems. 

Read Scalability Principles, by Simon Brown

The principles covered in the article include:
  1. Decrease processing time
  2. Partition
  3. Scalability is about concurrency
  4. Requirements must be known
  5. Test continuously
  6. Architect up front
  7. Look at the bigger picture
The majority of the principles for this article have been sourced from some notes taken during a scalability discussion that took place at a private summit for architects held in London, UK, in late 2005. The summit was organized by Alexis Richardson, Floyd Marinescu, Rod Johnson, John Davies, and Steve Ross-Talbot. The video entitled "JP Rangaswami on open source in the enterprise & the future of information" is also from the summit.
Up-front Concurrency Design by Luiz Esmiralha Posted May 23, 2008 6:49 AM
Re: Up-front Concurrency Design by Simon Brown Posted May 23, 2008 10:34 AM
good article by Surender Kumar Posted May 25, 2008 4:39 PM
Re: good article by Simon Brown Posted May 28, 2008 4:55 AM
Re: good article by Surender Kumar Posted Jun 2, 2008 2:01 AM
Good principles by Randy Shoup Posted May 28, 2008 5:43 PM
advises for junior programmers by Giuseppe Proment Posted Jun 4, 2008 8:45 PM
  1. Back to top

    Up-front Concurrency Design

    May 23, 2008 6:49 AM by Luiz Esmiralha

    Thanks for your article, Simon. It will prove useful in my next projects! But, I am somewhat puzzled by your affirmation that "any design for concurrency must be done up-front". Can you ellaborate further on the reasons for that, if possible with a real-world scenario where postponing concurrency design to a later moment proves to be a very expensive decision?

  2. Back to top

    Re: Up-front Concurrency Design

    May 23, 2008 10:34 AM by Simon Brown

    Thanks Luiz, I'm glad that you found the article useful. Adding concurrency in later is definitely possible ... I just think it's trickier. If I write something with concurrency in mind, I tend to *test* it with concurrency in mind. If I add concurrency afterwards, I tend to not test it as thoroughly and/or introduce a bunch of nasty side-effects! This principle can be applied to data too. If you're building a big distributed system, thinking about concurrent data access (e.g. how data will be locked/synchronized/shared) is easier to do when you have a blank canvas. As another example, think about what you might need to do to add concurrency features to a GUI application - you'd need to figure out your concurrency strategy (e.g. pessimistic locking by the user vs optimistic locking by the application) and then modify code right from the GUI through to the back-end. At the end of the day, there's no "right answer". I just find that I make a better job of concurrency if I think about it up front.

  3. Back to top

    good article

    May 25, 2008 4:39 PM by Surender Kumar

    Thanks for a good article Simon. I believe it states the points inline with KISS approach which we tend to overlook in quest for abstraction and loose-coupling. My current simple application gave me a throughput of ~20 requests/second on a single CPU linux server which I definitely want to improve. I may not know what's optimum. Is the count an embarrassment :-) or is it okay? Can you please cite some statistics with respect to this. Also increasing the servers in a cluster I'm sure of increasing it many folds.

  4. Back to top

    Re: good article

    May 28, 2008 4:55 AM by Simon Brown

    Thanks ... exactly, simple is good.

    Unfortunately, the answer to your question is "it depends". ~20 requests/sec doesn't sound much, but you don't say what each of those requests does. If they are very large in nature, then "20" might be an excellent result. Increasing the number of servers will help you scale this number, but it might not provide you with linear scalability. That too depends on things like shared state, contention and so on.

    The best advice I can give is this - if your project sponsors are happy with the performance/scalability of your system, then your job is done. If you need additional scale, then you need to get another server and see what sort of numbers you get out. Stats are useful, but not as useful as testing your software yourself. :-)

  5. Back to top

    Good principles

    May 28, 2008 5:43 PM by Randy Shoup

    Hi, Simon --

    Good summary of some critical scaling principles. You will see your points on partitioning echoed in my article on Scalability Best Practices: Lessons from eBay.

    I particularly like the point that scaling is about concurrency. Very clearly stated. That is, after all, the fundamental reason why partitioning helps.

    Ditto the point that scaling out rarely comes for free. If your only option is to scale up, sooner or later you will run out of runway. eBay has seen this time and again in its history, and the rearchitecture efforts to remove those bottlenecks (first in the database, and then in search) were long and painful. What I would add, though, is that this does not necessarily mean that it is wrong to design such a system -- just that it is important to be aware that such a system will not scale. While it is inarguably cheaper to design in scaling from the beginning, the additional time and effort it requires may not be worth it at that moment. Just make that tradeoff in full recognition of the fact that when the time comes, it will be more expensive than it otherwise would have been.

    Take care,
    -- Randy

  6. Back to top

    Re: good article

    Jun 2, 2008 2:01 AM by Surender Kumar

    thanks for the comments simon. Will test/profile to gauge the optimal throughput. Good work on your site btw. keep up.

  7. Back to top

    advises for junior programmers

    Jun 4, 2008 8:45 PM by Giuseppe Proment

    This article is enough just for junior programmers, what about something more profound like patterns for how to build a scalable domain model ?

Educational Content

Bindings, Platforms, and Innovation

This presentation focuses on the Internet and separating myth from fact, history from the future, and the mundane from the imaginative. Bob Frankston presents a vision of what could and should be.

Orchestrating Long Running Activities with JBoss / JBPM

This article explores the use of JBoss and jBPM to implement design solutions that effectively address the issue of orchestrating long running activities.

Neo4j - The Benefits of Graph Databases

This presentation covers the use of graph databases as an optimal solution for data that is difficult to fit in static tables, rapidly evolving data or data that has a lot of optional attributes.

Realistic about Risk: Software development with Real Options

This session introduces Real Options and shows how it can help in running your project. Real Options is a decision-making process that can be used to manage risk.

Communication Flexibility Using Bindings

This article discusses the use of bindings on services and references (including the instance of non-configured bindings) as the means to implement SCA communications in a Web and SOA environment.

Writing DSLs in Groovy

After a short introduction to DSLs, Scott Davis plays with the keyboard showing how to approach the creation of a DSL by typing working snippets of Groovy code that get executed.

Scaling Agile with C/ALM (Collaborative Application Lifecycle Management)

IBM Rational and InfoQ present, Scaling Agile with C/ALM, an eBook showing organizations how to become “finely tuned software delivery machines” by enabling team integration and scaling.

Concurrent Programming with Microsoft F#

Amanda Laucher presents a real life enterprise application written in F#. She shows actual code snippets, explaining design decisions and suggesting how to use some of the F# constructs.