At their annual re:Invent conference in Las Vegas, AWS unleashed a flurry of announcements about upcoming cloud services. Amazon outlined over two dozen new capabilities coming to the public cloud, including directly querying data in S3 object storage, building code as part of deployment pipelines, provisioning cheap virtual private servers, and moving data in bulk, ETL-style.
At the recent AWS Re:Invent event, Amazon announced a new preview service, called AWS Batch. AWS Batch allows organizations to optimize their scheduling and workload execution across a cloud-based landscape. Amazon has built this service in response to many AWS customers building their own batch platforms using EC2 instances, containers and CloudWatch.
AWS recently launched a Docker container image for its Amazon Linux operating system, complementing the EC2 specific Amazon Linux AMI with a versatile deployment option for custom cloud and on-premise environments. The image is available through the Amazon EC2 Container Registry (Amazon ECR), and also as an official repository on Docker Hub.
In late 2015, AWS unveiled the EC2 Run Command feature. It gave operators a single interface for running administrative tasks across a fleet of AWS servers. In June of this year, AWS expanded the scope of the feature to work with servers located in other clouds or data centers.
Netflix has shed light on how the company uses the latest version of their Keystone Data Pipeline, a petabyte-scale real-time event stream processing system for business and product analytics. This news summarizes the three major versions of the pipeline, now used by almost every application at Netflix.
AWS Release ‘Scheduled Reserved Instances’, Allowing EC2 Capacity to be Reserved on a Periodic Basis
Amazon Web Services (AWS) have introduced ‘Scheduled Reserved Instances’, which enables EC2 compute capacity to be reserved at a discounted price for use on a periodic basis. For example, a EC2 instance type can be reserved for daily usage between the hours of 01:00 UTC and 05:00 UTC to perform overnight data analysis, or weekly or monthly to perform compute-intensive calculations.
Last month, Amazon announced EC2 Dedicated Hosts are now generally available. Amazon initially discussed EC2 Dedicated Hosts at its Re:Invent conference in October. Using this new service, customers will have the ability to map Virtual Machines (VMs) to a physical host which runs in AWS.
Amazon Web Services recently introduced VPC endpoints to enable a "private connection between your VPC and another AWS service without requiring access over the Internet, through a NAT instance, a VPN connection, or AWS Direct Connect". VPC endpoint policies provide granular access control to other service's resources. Initially available are connections to S3, other services will be added later.
We published in 2014 the results of TechEmpower’s benchmark of various web frameworks, a term including web platforms and micro-frameworks. A year later, they have published a new set of results outlining important changes in the performance of top 10 web frameworks.
Amazon Web Services announced a new instance type called D2 which is optimised for Massively Parallel Processing (MPP) data warehouses, log processing, and MapReduce jobs.
AWS has simplified the pricing model for Amazon EC2 reserved instances. It has replaced the utilization based pricing model with a simpler model to buy reserved capacity.
TechEmpower has been running benchmarks for the last year, attempting to measure and compare the performance of web frameworks. For these benchmarks the term “framework” is used loosely including platforms and micro-frameworks.
Domino, a Platform-as-a-Service for data science, enables people to do analytical work using languages such as Python or R in the cloud (EC2).
EC2 users can now automate the deployment of Apache Mesos, an open-source tool to share cluster resources between multiple data processing frameworks, at scale through a new web service called Elastic Mesos provided by Big Data startup Mesosphere.
Netflix deploys a hundred times per day, without the use of Chef or Puppet, without a quality assurance department and without release engineers. To do this, Netflix built an advanced in-house PaaS (Platform as a Service) that allows each team to deploy their own part of the infrastructure whenever they want, however many times they require.