Will Amazon Change How Enterprise Applications are Written and Hosted?

Posted by Scott Delap on Sep 05, 2006 01:43 PM

Community

Java,

Architecture,

Ruby,

SOA

Topics

Configuration Management ,

Deployment / Datacenter

Tags

Amazon ,

Deployment

Amazon has quietly been expanding their business as of late. These newest additions are not stores targeted at consumers. Instead Amazon is targeting developers with a trio of services:

Amazon Elastic Compute Cloud (EC2)
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers. Features:
• Elastic - Add Servers in Minutes
  
• Completely Controlled - Root acces to each machine you deploy. Each instance predictably provides the equivalent of a system running Fedora Core with a 1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth.
  

Pricing is based on the time/data that you use.

• $0.10 per instance-hour consumed (or part of an hour consumed).
• $0.20 per GB of data transferred outside of Amazon (i.e., Internet traffic).
• $0.15 per GB-Month of Amazon S3 storage used for your images (charged by Amazon S3).

Data transfer between EC2 servers or Amazon S3 is free.

Amazon Simple Storage Service (S3)
Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. Features:

• Write, read, and delete objects containing from 1 byte to 5 gigabytes of data each. The number of objects you can store is unlimited
  
• Uses standards-based REST and SOAP interfaces
  
• Default download protocol is HTTP. A BitTorrent(TM) protocol interface is provided to lower costs for high-scale distribution.
  

Pricing is based on the data stored/transfered.

• $0.15 per GB-Month of storage used.
  
• $0.20 per GB of data transferred.
  

Amazon Simple Queue Serve (SQS)
Amazon SQS offers a reliable, highly scalable hosted queue for storing messages as they travel between computers. By using Amazon SQS, developers can simply move data between distributed application components performing different tasks, without losing messages or requiring each component to be always available. Amazon SQS works by exposing Amazon's web-scale messaging infrastructure as a web service. Any computer on the Internet can add or read messages without any installed software or special firewall configurations.

• Reliable - To prevent messages from being lost or becoming unavailable, all messages are stored redundantly across multiple servers and data centers.
  
• Simple - Developers can utilize Amazon SQS queues by using only four APIs: CreateQueue, SendMessage, ReceiveMessage, and DeleteMessage. The SOAP, REST and query APIs can be used with virtually any language and platform.
  
• Scalable - Amazon SQS was designed to enable an unlimited number of computers to read and write and unlimited number of messages at any time.
  
• Inexpensive - No up-front or fixed expenses. The only cost of sending messages through Amazon SQS is a small per-message handling fee based on the size of a message.
  

Again pricing is based only on usage.

• $0.10 per 1,000 messages sent ($0.0001 per message sent)
  
• $0.20 per GB of data transferred
  

Questions about privacy concerns and compliance with laws such as Sarbanes-Oxley have been raised about the services.  This has not stopped a number of tutorials from being created detailing getting an EC2 server instance up and running with applications written in languages such as Python and Ruby on Rails.

Given the variety of features and pricing numbers it can be hard to grasp what can be done using Amazon's services. A few quick calculations reveal:

• The cost of running 1 server via EC2 is appoximately $72/month.
  
• The cost of that server delivering 100GB of web pages is $20/month.
  
• The cost of that server receiving 100GB of image uploads is $20/month.
  
• The cost of transfering those images to S3 for long term storage is $0.
  
• The cost of storing those images in S3 is $15/month.
  

The blogsphere is beginning to discuss the ramifications of using these three services in unison as well:

Vinny Carpenter writes -

...Can you imagine being able to provision 1, 2 or 500 additional servers in minutes for your application programmatically?...And so if Digg, Techmeme, Reddit or TechCrunch or the meme of your choice is sending you millions of hits, add a few virtual servers to support your application and then scale back as traffic dissipates.

John Lam writes:

...Capacity planning and operations management are things that are often ignored at startup companies. When all of a sudden traffic spikes and you're unable to scale until the new servers arrive next week you're screwed. But the ability to add additional capacity in minutes is an enormous leveling force. Also, the ability to scale up and down is just as important. You can greatly reduce your operational costs by not paying for the compute power until you need it.

An example of Amazon's services changing how applications are written is the Openfount Queued Server.  The Ajax/GWT client application is served via Amazon's S3.  S3 then serves as a queue providing communication between the client and one or many servers handling the processing of requests. 

1. Back to top

   very compelling

   Sep 5, 2006 3:46 PM by Floyd Marinescu

   This is very compelling. It could transform the webhosting industry - it would cost < $250/month to host your typical 2 server clustered app + one DB machine and < 50 gigs a month transfer. I think the biggest change could result in making it really easy for software writers to also get into the ASP business. Today software vendors aren't typically also offering hosting solutions as that's a very different business model. But if Amazon can make it super easy for them to do that, perhaps in 3 years it will be come the norm to buy an already hosted server instance for any piece of software you might want, instead of downloading the app itself and installing it within your own datacenter. First open source dramatically reduces the cost of software licences for softare projects, now large scale virtualized hosting dramatically reduces the cost of hardware/datacenter...

   Reply

2. Back to top

   Re: very compelling

   Sep 5, 2006 5:05 PM by Dan Diephouse

   I agree - this is very very compelling. I am contemplating switching one of my applications to it. One of my questions is whats the best way to do database stuff (SQL? someothing else?)on top of these apis? There is no permanent filesystem which I could point an sql server too, which makes it difficult.

   Reply

3. Back to top

   Re: very compelling

   Sep 5, 2006 5:21 PM by Floyd Marinescu

   I could be wrong but my understanding is that you get root access to a virtual linux instance on which you can install anything you want and it should have access to its own local virtual hard drive.

   Reply

4. Back to top

   Re: very compelling

   Sep 5, 2006 5:35 PM by Dan Diephouse

   The hard drive is erased once the instance is gone though AFAIK. And its only 160MB of space.

   Reply

5. Back to top

   Re: very compelling

   Sep 5, 2006 5:54 PM by karan malhi

   This is good, but from a developers perspective, it is still not affordable. Atleast I cannot afford a $250 a month server. From a small business perspective, this might be a viable option, but i think there should be cheaper options out there for dedicated hosting, maybe they dont give root access to the machine, but again, from a developers perspective, if i had to choose, i would definitely forego root access than paying $250 per month.

   Reply

6. Back to top

   Re: very compelling

   Sep 5, 2006 10:27 PM by Floyd Marinescu

   Most companies are paying $2500/month and up for a 4 name brand servers at a quality ISP with load balancing, redundancy, etc. $250 is not bad. :) But hard drive space appears to be a problem.

   Reply

7. Back to top

   The answer is yes

   Sep 5, 2006 11:48 PM by Vinny Carpenter

   I think the answer to the question, 'Will Amazon Change How Enterprise Applications are Written and Hosted?' is a resounding YES. I was just thinking about something we do at work where month-end processing takes tons of resources and CPU cycles for maybe 24 hrs straight - What if you could use 1000 virtual instances to multi-task this process and scale horizontally? And if you could get your work done in an hour, it would only cost you $10.00.. Can you imagine that? Instead of getting clusters of servers or big 64 processor boxes, you can leverage EC2. I think this is going to be a compelling offering as long as Amazon is really behind it and continues to support it. SOX, while an issue is not going to be a real issue as long as Amazon really wants to get into the Enterprise market. Being a public company, Amazon already has most of the SOX controls in place and can implement appropriate separation in terms of data, encryption to pass any audit. In fact, they would bring in Price Waterhouse Coopers or some auditing firm like that and get a SOX certification like a SAS 70 certification. --Vinny http://www.j2eegeek.com/blog/

   Reply

8. Back to top

   Re: very compelling

   Sep 6, 2006 7:51 AM by ZedroS Schwartz

   It's written 160 GB on the article above, no MB... It makes quite a difference isn't it ? lol

   Reply

9. Back to top

   Re: very compelling

   Sep 6, 2006 9:56 AM by maninder batth

   As a theoratical concept, ec2 sounds compelling. Does anyone know if amazon provides SLA? Their compute grid is limited by the number of machines. In an environment, where increase in compute capacity is as simple as making a webservice,i am wondering how do they characterize the workload of their grid. This becomes particularly tricky when machines are shared. One simple way is to go dedicated hosting and reserve the machine or machines for x amount of hours. If end user requires more computing power, a webservice call can be made and at that time, based on machine reservation, amazon can provide or deny extra computing... In this case this is not really on demand computing.

   Reply

10. Back to top

    Re: very compelling

    Sep 8, 2006 9:35 PM by Shane Witbeck

    From Amazon's EC2 docs regarding the storage limitations:

    Do not rely on an instance's local storage for valuable, long-term data . Instances can fail, and when they fail, the data on the local disk is lost. You should use a replication strategy across multiple instances to keep your data safe or store your persistent data in Amazon S3.

    From my perspective, this means is that you actually have an unlimited amount of space available when you use EC2 and the S3 services together.

    Reply

• Older >