Nanite: A Self Assembling Cluster of Ruby Processes

Posted by Mirko Stocker on Nov 26, 2008

Sections

Operations & Infrastructure,

Enterprise Architecture,

Development,

Architecture & Design

Topics

Ruby ,

Cloud Computing ,

Messaging

Tags

XMPP ,

Erlang

Nanite is Ezra Zygmuntowicz's newest addition to Engine Yard's cloud computing strategy: a "self assembling cluster of ruby processes" to form the backend of a highly scalable web application (see Ezra's announcement).

Nanite consists of two parts: agents and mappers. Agents implement the functionality of your application and are managed by mappers, the control nodes of your system. Agents periodically advertise their status and (per default) their average load to the mappers, so the mappers can distribute work to the least loaded agent. For more information and examples, take a look at the Nanite Readme file.

Nanite is built around RabbitMQ, "a complete and highly reliable Enterprise Messaging system" built with Erlang. According to a comment in his blog, it should also work with any other AMQP compatible messaging middleware.

InfoQ: Ezra, are you using Nanite at Engine Yard? 

Yes Nanite is used as the backend for our new engine yard as a service platform. It is used for autoscaling EC2 instances as your traffic grows as well as being a core part of our service exposed to users so they can use Nanite agents as part of their apps with zero fuss.

InfoQ: Do you see other applications for Nanite besides the web?

Nanite has many applications outside of the web I think. Anytime you need to control a set of resources across a number of servers from Ruby then Nanite is a good fit.

InfoQ: We talked about Vertebra—the next generation cloud computing platform—a few months back, what's the relation between Nanite and Vertebra?

Nanite and Vertebra will eventually be able to work together via the rabbitmq/xmpp bridge. Nanite is more for scaling individual applications in a cloud environment. Vertebra is more for connecting multiple disparate systems together where each agent in the system could be on a different customers servers.

Rabbitmq's security model is based on vhosts, so anything in a vhost can talk to anything else in the same vhost. This works well for having a vhost per application space and scaling apps that way. Nanite is also higher performance messaging for application workloads where users are waiting on results. Vertebra is about coordinating multiple machines while keeping them secure form each other.

Nanite is available on GitHub, and Vertebra should be released as open source before Christmas, around December 15.

• Older >