Building Scalable Web Services

| by Dilip Krishnan on Sep 08, 2009. Estimated reading time: 2 minutes |

Tom Killalea, Vice President of technology with responsibility for infrastructure and distributed systems engineering at wrote an article on ACM queue on building scalable web services. He outlines guiding principles to building scalable web services with a lot of real-world examples, the core theme of which is “build only what you need”.

Caution: Early Optimization

Time and resources spent optimizing scalability might be better spent improving customer experience and driving traffic.

Embrace: Other People's Work

He explains that leaning on other peoples work for frameworks and infrastructure can reduce time to market and helps focus on providing customer value.

Three significant developments have contributed in different ways to this lowering of the barriers: the trend toward SOA (service-oriented architecture), the emergence of cloud computing infrastructure services, and the availability of Web application frameworks such as ASP.NET, Django, Rails, and Spring.

Caution: Overoptimization

He references Nicholas Nassim Taleb’s work on highly improbable and unpredictable events that have massive impact, and suggests using redundancy as a strategy for increased availability; the idea being that providing for redundancy as a means to balance load as opposed to just as a failover mechanism can be more cost effective than over-optimization for statistically less probable events.

Embrace: the Cloud

Tom gives an example of how Animoto, a social web application, hosted in the EC2 infrastructure could suddenly scale out on demand, all the way up to 3500 instances. Which in non-cloud infrastructure would incur considerable cost to maintain during lean traffic periods.

Caution: Target-driven Optimization

There are dangers in modeling expected traffic and then building a very precise scalability plan to meet that target. Good models are hard to build and can suffer from simplifying or happy assumptions that discount variability. […] if your Web service is successful, you'll eventually see greater demand than the target model suggests—maybe not this Black Monday or Super Bowl Sunday, but perhaps soon after and at a less anticipated hour.

Embrace: Ripping the Wings Off

Beyond analyzing what broke first and why”, Tom says “We look for how the given application or service can make progress without the broken or missing pieces, and then rerun the test to determine the next breaking point”.

Tom concludes his article saying “The most difficult challenge in building a scalable Web service is how to handle the trade-offs between durability, reliability, performance, and cost effectiveness in the presence of failures and extreme concurrent access patterns”.

In addition to Toms article, the October 2008 issue has other great articles on building scalable services.

Rate this Article


Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Leveraging existing frameworks by Vijay Narayanan

Nice summary of the article. The point about leveraging existing frameworks for achieving scalability is very valid. Development teams that roll out their own implementations could end up spending a lot of resources getting it to work and be stable. Code reviews can certainly help with achieving this. Why not leverage the work of those that have worked on and solved the same problems?

Be careful about reuse of frameworks by James Richardson

Reuse of software is not a real goal anyway - reducing the cost / speed etc for delivering software is the goal. Some people have been led to believe that reuse is _the_ way to get there. The fact is that software that is suitable for reuse has quite different success criteria than single purpose software.

Anyhow - in any large enterprise a lot of the existing "frameworks" are rubbish - be brave and reject them!

This doesn't mean of course you shouldn't download something from the internet.


Re: Be careful about reuse of frameworks by Dilip Krishnan

Reuse is subtly different from leverage. Leveraging of "mature" frameworks is what the author is talking about. Where the "mature" means that it is best of breed solution for solving a particular problem, for e.g. (N)Hibernate for persistence, Microsoft WCF or Apache Axis2 for communications etc.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

3 Discuss