Flex: Engine Yard's New Cloud Offering

Posted by Mirko Stocker on May 18, 2009

Sections

Enterprise Architecture,

Operations & Infrastructure,

Process & Practices,

Architecture & Design,

Development

Topics

Cloud Computing ,

Ruby ,

Performance & Scalability ,

Announcements

Tags

Scalability ,

Amazon ,

EC2

Engine Yard's new Flex offering "is geared towards professional web applications that need reliability, scaling, and professional support 24x7". Flex is for customers that want more than the single computing instance Solo offers but don't need fully managed and privately running Engine Yard Slices. From the Flex press release:

Engine Yard Flex introduces unique new features like environment cloning and one-button deployment as well as essential features for agile operations, such as automatic deployment from a version control system and application server auto-recovery.

"Large numbers of developers and organizations have already adopted Ruby on Rails because of its superior development productivity, and are increasingly considering cloud services as their default deployment platform," said Tom Mornini, CTO of Engine Yard. "Engine Yard Flex is the natural cloud choice for Ruby on Rails applications that need easy scaling and reliability."

We talked to Michael Mullany, VP of Marketing at Engine Yard to learn more about Flex.

The mayor difference between Solo and Flex are single vs. multiple instances, what else is different between the two offerings?

The philosophy behind Solo is an inexpensive service that you can quickly get a site up an running on. Flex is designed as a peer to our current slice offering, where you run multiple redundant web servers, you have a failover database etc. For example, Flex will have a paid support option whereas Solo is a community support platform. So if you are running on a Flex cluster you will be able to have access to Engine Yard's global support staff 24/7. Flex also has self-healing capabilities meaning when a server goes down, a new one will boot to take over the old one's role.

Can you tell me more about environment cloning, automatic and one-button deployment?

Environment cloniing is a way to clone a running environment including all of the data into a duplicate setup for testing etc. In our system an environment encapsulates a set of gem and unix dependencies, backup policy, applications to run and other information needed to instantiate a server or group of servers.

Once you have an environment running with a single instance or with a cluster, you can click on 'clone' button in order to create a completely isolated copy of the whole environment, including block level copies of all your data. We will snapshot the old servers, boot up identical copies and mount the snapshots and configure the servers identically to the environment you cloned from. We even take care of handling in-process database transactions so you have a consistent database in the cloned environment.

This allows for unbelievable flexibility when testing new deploys that have complex code changes and do large migrations. Almost no one these days keeps a staging environment that is identical to their production environment, it is usually different in many ways. With this system you can get bit for bit identical staging envs to test new deployments on. And on top of that you only pay for what you use so even if you clone a large 5 node production cluster and use it for 1 hour to test a new deployment, it will probably only cost you a few dollars.

How does the application server auto-recovery work? Does it work independently of the running application?

This works independently of the running application. We have written custom monitoring daemons that run on all of the app server nodes. Each app server runs haproxy on port 80 and runs nginx or apache on port 81. The haproxies balance requests to all the other app server nodes so any one of them can become master at any moment. Only one of these nodes will have the main IP address and thus is called the Application Master. All the other app servers are Application Slaves. The app slaves will monitor the health of the app master and if the master fails, the slaves will use a quorum vote in order for one of them to get a lock. The slave that gets the lock will steal the IP address, become the new master and will kill the old master server and boot a new slave in its place with the old disks from the old master. once this new server comes online, haproxy is dynamically reconfigured on all the app servers in order to heal the cluster. This is all transparent to the user and no special code needs to be written. This ensures high availability for your flex clusters.

How does Flex relate to your other  technologies like Vertebra?

Flex is an Engine Yard service offering. Just like Solo, you will be able to sign up for it on our web-site and start deploying rails applications. Vertebra is one of the many open-source projects that we've sponsored or contributed to since we were founded. Some of the bigger projects that we've sponsored and contributed to include Rubinius, a Ruby run-time alternative to the MRI interpreter, and Merb, another Ruby framework, now merging into Rails 3. The Passenger application server port to nginx was another project we sponsored. Right now, we're using Vertebra internally in places at Engine Yard for automation, but we have no plans to offer a commercial open-source version.

Engine Yard Flex will be available on Amazon EC2 in June, at which time the complete feature set and pricing will be announced.

• Older >