The Hadoop Summit of 2010 included presentations from a number of large scale users of Hadoop and related technologies. Notably, Facebook presented a keynote and details information about their use of Hive for analytics. Mike Schroepfer, Facebook's VP of Engineering delivered a keynote describing the scale of their data processing with Hadoop.
Steve Huffman, co-founder of Reddit, shares the main lessons he learned scaling Reddit from a small web application to a large social website.
Terracotta and Eucalyptus Integration Provides Data Management and Elastic Provisioning in the Cloud
Terracotta recently announced a partnership with open source private cloud platform vendor Eucalyptus that allows the companies to provision private clouds on Amazon AWS-compatible Eucalyptus cloud platform and take advantage of the elasticity and flexibility of the cloud.
Tom Killalea, Vice President of technology with responsibility for infrastructure and distributed systems engineering at Amazon.com wrote an article on ACM queue on building scalable web services. He outlines guiding principles to building scalable web services with a lot of real-world examples, the core theme of which is “build only what you need”.
Memcached has recently been released in version 1.4 which added new features like the binary protocol. Also: WebCache is a Memcached protocol-compliant hardware solution to boost performance even more.
Martin Fowler talks about ThoughtWorks's experience with using Ruby on client projects for the past three years, and the creation of a Ruby-based product 'Mingle'.
David Pollak, found of the Lift web framework and "Beginning Scala" author, has announced a new initiative "Goat Rodeo" that aims to bring data modeling into the 21st century.
In this interview from QCon London 2009, Rich Hickey talks about Clojure. The discussion includes the ideas behind Clojure's STM support, what other concurrency primitives Clojure supports and which ones might get added in the future. Other topics covered are Clojure's AOT support, the role and implementation of multimethods, Clojure ports to other systems and much more.
New patches by Joe Damato improve the efficiency of Ruby 1.8.x's green threads with heap stacks: instead of copying the entire stack at every context switch, the patches actually switch between different stacks. Ryan Davis released zenprofile and event_hook for efficient profiling. Also: work on a MacRuby Ahead of Time compiler using LLVM has started.
Engine Yard announced Flex at this year's Rails Conf. Flex runs on Amazon's EC2, but unlike its smaller brother Solo, Flex scales over more than one instance. We talked to Michael Mullany, VP of Marketing at Engine Yard to get more information.
37Signals is the latest company to use Erlang in combination with Ruby. The recent Erlang Factory conference also had other examples of Erlang use at EngineYard as well as a talk about Erlectricity, the library that connects Erlang and Ruby.
Heroku debuted a commercial version of their Rails hosting solution last week, after a free beta stage that lasted over a year. Using Heroku, deployment of a new Ruby web application from scratch is accomplished with little more than a handful of commands from your terminal. No emails, phone calls or support tickets needed.
Betfair is the world's largest betting exchange with a transaction volume the equivalent of over half the combined equity trading volume of every major stock exchange in the world. In response to an increase in transaction volume coupled with a decrease in value per transaction, Betfair launched a number of initiatives to dramatically increase transaction processing capacity and reduce cost.
In this interview with Gregg Pollack of Rails Envy Podcasts, Robert Bazinet talks with Gregg about the issues around scaling Rails, his involvement with New Relic and the creation of the Scaling Rails screencast series as well as other keys to scaling Rails.
Nick Gunn provides a practical introduction Using the Concurrency and Coordination Runtime. CCR radically changes the way multi-threaded applications are written in .NET, shifting the focus from threads and locks to lightweight, asynchronous tasks. The Concurrency and Coordination Runtime, also known as CCR, offers actor-style concurrency for .NET applications.