The Twitter Engineering team has recently provided an insight into the evolution and scaling of the core technologies behind their in-house infrastructure that powers the social media service. Core lessons shared included: Architect beyond the original specifications; there is no such a thing as a “temporary change or workaround”; and documenting best practices has been a “force multiplier”.
Docker Inc. launched its answer to Amazon ECS into public beta at the end of last year: an AWS-compatible service for managing and orchestrating Docker containers. Now, Docker for AWS is generally available.
Amazon is the first cloud provider to support Windows Docker containers in their managed container platform - the AWS EC2 Container Service. The new beta service for Windows has several restrictions, but it paves the way to running multi-platform solutions across a single cluster of container hosts.
StorageOS aims to make container storage flexible by providing a single view of the underlying storage and exposing APIs for automation.
Docker Inc, has released version 1.13 of its open source Docker container engine project. This release includes significant restructuring of the Docker CLI, and the introduction of ‘clean-up’ commands to reclaim disk space. Alongside the launch of Docker 1.13, new releases of the supporting toolchain were also made, including: Docker Compose 1.10, Docker Machine 0.9.0, and Notary 0.4.3.
In late 2016, Microsoft announced the general availability of Azure SQL Database In-Memory technologies. In-Memory processing is only available in Azure Premium database tiers and provides performance improvements for On-line Analytical Processing (OLTP), Clustered Columnstore Indexes and Non-clustered Columnstore Indexes for Hybrid Transactional and Analytical Processing (HTAP) scenarios.
Google Cloud Platform has released an open source Zipkin server that allows Zipkin-compatible clients to send traces to Google’s own Stackdriver Trace distributed tracing service for analysis. This Zipkin/Stackdriver Trace integration is aimed at developers whose applications and services are written in a language or framework that Stackdriver Trace doesn’t officially support.
Intel open-sources BigDL, a distributed deep learning library that runs on Apache Spark. It leverages existing Spark clusters to run deep learning computations and simplifies the data loading from big datasets stored in Hadoop.
Yahoo! has open sourced Screwdriver, their CD tool used internally in production.
The 2017 State of Testing survey aims to provide insights into how the testing profession develops. The survey is open throughout January 2017.
At the AWS re:Invent conference, Amazon launched Rekognition, a managed service for Image Recognition and Analysis, powered by Deep Learning. The capabilities that Rekognition provides include Object and Scene detection, Facial Analysis, Face Comparison and Facial Recognition. The service attempts to extract meaning from visual content for the 1.2 Trillion pictures captured annually.
Instacart is an online delivery service for groceries under one hour. Customers order the items on the website or using the mobile app, and a group of Instacart’s shoppers go to local stores, purchase the items and deliver them to the customer. InfoQ interviewed Mathieu Ripert, data scientist at Instacart, to find out how machine learning is leveraged to guarantee a better customer experience.
HashiCorp has released v0.8 of Terraform, an open source tool that enables the building, combining and launching of programmable infrastructure providers such as Amazon Web Services, VMware vSphere, and UltraDNS. Major new functionality includes an interactive console, conditional values, and HashiCorp Vault and Nomad providers.
Project Springfield is a fuzz testing service for finding security critical bugs in software. William Blum, principal software engineering manager on the Springfield team at Microsoft Research, explains how adopting F# helped the team build the cloud service.
Amazon's Simple Queue Service (SQS) recently gained FIFO (first-in, first-out) queues, which are designed to "guarantee that messages are processed exactly once, in the order that they are sent, and without duplicates". AWS rolled out this new queue type in the US East (Ohio) and US West (Oregon) regions and "plans to make it available in many others in early 2017".