Netflix has shed light on how the company uses the latest version of their Keystone Data Pipeline, a petabyte-scale real-time event stream processing system for business and product analytics. This news summarizes the three major versions of the pipeline, now used by almost every application at Netflix.
RethinkDB has launched a Windows port of their database. This version of RethinkDB took a year of development, runs on Windows 64-bit and it is currently in beta.
Hortonworks has quietly made available the DataFlow platform which is based on Apache NiFi and attempts to solve the processing needs of the IoAT.
Twitter has replaced Storm with Heron which provides up to 14 times more throughput and up to 10 times less latency on a word count topology, and helped them reduce the needed hardware to a third.
The first stable ORTC (Object RTC) specification is out. The questions is how is it going to impact WebRTC?
Google has unveiled their new data-warehouse called Mesa. Mesa is a system that scales across multiple data centers and processes petabytes of data, while being able to respond to queries in sub-second time and maintain ACID properties.
DataTorrent is a real-time streaming and analyzing platform that can process over 1B real-time events/sec.
Twitter Engineering has released details about Manhattan, its real-time, multi-tenant distributed database.
Hadoop is definitely the platform of choice for Big Data analysis and computation. While data Volume, Variety and Velocity increases, Hadoop as a batch processing framework cannot cope with the requirement for real time analytics. Spark, Storm and the Lambda Architecture can help bridge the gap between batch and event based processing.
Amazon recently announced Kinesis, a service that allows developers to stream large amounts of data from different sources and process it. The service is currently in limited preview.
This article contains an interview on GenMyModel’s architecture, technology used and future.
Version 0.8.9 of Sails, a real-time MVC framework built atop node.js, was released on April 9th by Austin, TX-based Balderdash in an ongoing effort to bring a Rails-like development platform to modern web apps
Microsoft CU-RTC-Web is an alternative approach to WebRTC meant to show some of WebRTC’s weaknesses and to push it forward.
A new open source project – Dempsy adds one more option for people trying to do real time processing of big data. Comparable to Storm and S4 Dempsy is most applicable to near real time stream processing where latency is more important than guaranteed delivery.