Netflix has shed light on how the company uses the latest version of their Keystone Data Pipeline, a petabyte-scale real-time event stream processing system for business and product analytics. This news summarizes the three major versions of the pipeline, now used by almost every application at Netflix.
AWS Release ‘Scheduled Reserved Instances’, Allowing EC2 Capacity to be Reserved on a Periodic Basis
Amazon Web Services (AWS) have introduced ‘Scheduled Reserved Instances’, which enables EC2 compute capacity to be reserved at a discounted price for use on a periodic basis. For example, a EC2 instance type can be reserved for daily usage between the hours of 01:00 UTC and 05:00 UTC to perform overnight data analysis, or weekly or monthly to perform compute-intensive calculations.
Last month, Amazon announced EC2 Dedicated Hosts are now generally available. Amazon initially discussed EC2 Dedicated Hosts at its Re:Invent conference in October. Using this new service, customers will have the ability to map Virtual Machines (VMs) to a physical host which runs in AWS.
Amazon Web Services recently introduced VPC endpoints to enable a "private connection between your VPC and another AWS service without requiring access over the Internet, through a NAT instance, a VPN connection, or AWS Direct Connect". VPC endpoint policies provide granular access control to other service's resources. Initially available are connections to S3, other services will be added later.
We published in 2014 the results of TechEmpower’s benchmark of various web frameworks, a term including web platforms and micro-frameworks. A year later, they have published a new set of results outlining important changes in the performance of top 10 web frameworks.
Amazon Web Services announced a new instance type called D2 which is optimised for Massively Parallel Processing (MPP) data warehouses, log processing, and MapReduce jobs.
AWS has simplified the pricing model for Amazon EC2 reserved instances. It has replaced the utilization based pricing model with a simpler model to buy reserved capacity.
TechEmpower has been running benchmarks for the last year, attempting to measure and compare the performance of web frameworks. For these benchmarks the term “framework” is used loosely including platforms and micro-frameworks.
Domino, a Platform-as-a-Service for data science, enables people to do analytical work using languages such as Python or R in the cloud (EC2).
EC2 users can now automate the deployment of Apache Mesos, an open-source tool to share cluster resources between multiple data processing frameworks, at scale through a new web service called Elastic Mesos provided by Big Data startup Mesosphere.
Netflix deploys a hundred times per day, without the use of Chef or Puppet, without a quality assurance department and without release engineers. To do this, Netflix built an advanced in-house PaaS (Platform as a Service) that allows each team to deploy their own part of the infrastructure whenever they want, however many times they require.
Amazon has announced two licensing deals with Microsoft. The first eliminates the licensing fees for running Microsoft software on EC2 for BizSpark members. The second allows all developers to run the preview of SQL Server 2012 on EC2. It should be noted that normal EC2 service rates still apply for both.
Amazon.com formally added three new capabilities to its cloud computing portfolio with the introduction of Direct Connect and the updates to the Virtual Private Cloud and Identity and Access Management services. These offerings are targeted at organizations looking to construct hybrid or private clouds on the Amazon Web Services platform.
Talend announced at the Cloud Computing Expo last week that it was releasing a Cloud-based version of its Enterprise Unified Integration Platform.
Since April 21, 2011, 1:41 AM PDT Amazon's US-EAST EC2 Datacenter reported major outages (due to failing EBS volumes) affecting many sites like Reddit, Foursquare, Quora, Hootsuite and Heroku which rely on EC2 services. The article links to discussions about reliability of Availability Zones of EC2 datacenters and EC2 SLA's as well as desaster recovery and prevention.