Last week, 6,000 attendees from around the globe were in Las Vegas, NV for the first-ever Amazon Web Services (AWS) re:Invent conference. InfoQ was there to interview thought leaders and identify the key messages of the conference.
The two-day conference was organized into 15 tracks including Architecture, Big Data, Databases, Enterprise IT, Gaming, Mobile, Security and Tools. While there were plenty of introductory sessions for individual products, a clear theme among sessions was about using best practices, building scalable, secure architectures, and describing the business value of cloud services. Two AWS architects from Japan spoke to a packed room about Cloud Design Patterns and explained a provider-neutral way to describe a variety of patterns for availability, batch processing, data replication, caching, network, maintenance and more. They then revealed an English-language wiki that includes all the various cloud design patterns that they’ve investigated so far. This was one of many sessions that looked at how to take the myriad components of the AWS cloud and assemble them into maintainable, distributed systems. Other sessions that were focused on incorporating best practices included an insightful look at Architecting for High Availability, Accelerating Amazon RDS with ElastiCache, Failures at Scale and How to Ride Through Them, and Building Scalable Applications on Amazon S3. Netflix’s Adrian Cockcroft led a discussion about Highly Available Architecture at Netflix that was so full that people were turned away at the door.
Among sessions targeted at organizations looking for help with articulating business value and defining their strategy, there were excellent sessions such as Defining an Enterprise Cloud Strategy, Amazon.com’s Migration to AWS, Embracing the Cloud, and How Much Can Your Organization Save with AWS. The multiple keynote addresses focused heavily on case studies that demonstrated how customers were using the AWS cloud to operate more efficiently or tackle problems in new ways. The Wednesday keynote from AWS VP Andy Jassy featured a dramatic retelling of the Mars Curiosity project which used the AWS platform to process videos and images from the landing. Representatives from Netflix, Pinterest, and Animoto also described how they were able to scale quickly and operate globally using AWS services. The keynotes also provided a platform for numerous announcements. AWS announced a sizable price drop in S3 storage, a new data warehouse service called Redshift, a data movement tool called Data Pipeline, and a pair of new, massive server instance types.
The keynote from AWS CTO Werner Vogels focused on what he called a 21st century architecture where cost is incorporated into the architectural design. Vogels used the same wording in his keynote as in a recent blog post.
The most important concept is that, when you are growing, your cost should grow over the same dimension your revenue is coming in over. For Amazon.com that dimension is number of orders. If orders go up your cost should be allowed to rise as well. Although if you are architected well, you will be able to exploit economies of scale and your cost will rise less than the rise of your revenue. If you are architected correctly for cost-awareness scale becomes your friend.
Vogels contends that customers can now build the systems they have always wanted, but were prevented from doing so in the past. He said that factors like capital, capacity, geography, physics, people, and scope are no longer constraints to building secure, scalable, fault tolerant, high performing and cost effective applications. Vogels sees everything as a programmable resource: data centers, networks, compute, storage, databases and load balancers. A 21st century architecture embraces automation as a way to increase efficiency and decrease cost. Vogels pointed out four major characteristics of a 21st century architecture.
- Controllable. New systems should be decomposed into small, loosely coupled, stateless building blocks. Software should be comprised of individual units that you can control. These units may pivot on scale, fault tolerance or other critical dimensions. Vogels used an example from IMDB where an initial integration architecture created a tight linkage that forced IMDB to scale whenever Amazon.com did. Instead, they decomposed and decoupled the systems further by using S3 as an intermediary. This allowed each layer to scale independent of the other.
- Resilient. Vogels says that the first priority of any software is to protect the customer. Any sensitive data should be encrypted at rest. As an example, Amazon.com encrypts everything at rest and in transit. Production system should be geographically distributed in order to survive failure in a single data center. Vogels reminded the audience that failure is always around the corner, and he encouraged architects to not think of failures as exceptions, but as another form of deployment.
- Adaptive. Modern software shouldn’t be dependent on fixed resources. Adapt to different circumstances, but don’t be constrained by them. As an example, S3 was originally scoped to hold 20 billion objects, and quickly had to be refactored to address the explosive growth that now has the service with over 1 trillion objects. All of this was done without disrupting the existing customer base. Vogels told the crowd to assume nothing and avoid applying constraints up front that will prevent future changes.
- Be Data Driven. Instrument everything. Don’t rely on predictions, but use reality and data to make decisions. If you don’t collect the data, reminded Vogels, you cannot act upon it. Network services like the AWS Elastic Load Balancer can make decisions based on more than just low-level system metrics, but also business-level data points. Vogels encouraged architects and operational staff to not be shy about collecting data and take advantage of tools like the new Data Pipeline to aggregate the data for analysis.
This was the first annual re:Invent conference, but AWS has already announced plans to hold this conference again next November in Las Vegas. Visit the conference site for additional details and browse the SlideShare repository for conference presentations.