AirFlow recently joined the Apache Incubator program. AirFlow is a workflow and scheduling system designed to manage data pipelines. Developed by AirBnb for their internal usage, it was open sourced last September, as previously reported by InfoQ.
Operational Data Stream and Batch Processing at Netflix with Mantis
Today at GraphConnect Europe 2016, Neo Technology announced the release of Neo4j 3.0, which includes a new binary protocol for transmitting data between server and client, and a new set of standardised drivers for interacting with the database, along with stored procedure support and higher performance and capacity. InfoQ spoke to Neo Technology to find out more.
Late last month Google released an alpha version of their TensorFlow (TF) integrated cloud machine learning service as a response to a growing need to make their Tensor Flow library to run at scale on the Google Cloud Platform (GCP). Google describes several new feature sets around making TF usage scale by integrating several pieces of the GCP like Dataproc, a managed Hadoop and Spark service.
InfoQ's Rags Srinivas caught up with Stephan Ewen, a project committer for Apache Flink about the 1.0.0 Release and the roadmap
Since announcements late last year about Google open-sourcing TensorFlow, the company’s open-source library for machine learning, and previous coverage at InfoQ, the data-science community has had an opportunity to try out TensorFlow for their own projects.
Funnel analysis is used to analyze a sequence of events to help with user engagement on a website or a mobile application. Data Science team at Twitter uses this concept to learn how users interact with user interfaces during sign up or tweeting for improving user engagement with Twitter.
A game simulation at Google's Deep Mind defeated expert humans at Go last month in a breakthrough for AI. Go is considered one of the great unsolved problems in AI.
Last year, Netflix Cloud Database Engineering (CDE) team introduced Dynomite. Dynomite is a proxy layer, aiming to turn any non-distributed database into a sharded, multi-region replication aware distributed database system. Now Netflix released a benchmark using Dynomite with Redis in AWS infrastructure.
Net Promoter Score (NPS) is a customer loyalty metric used to determine the likelihood that a customer will return to a company's website or use their service again. Airbnb uses NPS extensively in measuring the customer loyalty, as a more effective measurement to determine the likelihood that a customer will return to book again or recommend the company to their friends.
Hazelcast has released version 3.6 of their flagship in-memory grid and caching software, featuring numerous performance improvements and new cloud management and container deployment options.
Yahoo has open-sourced DataSketches, a library written in Java for stochastic streaming algorithms. DataSketches is able to perform traditionally expensive operations, like counting distinct occurrences of a variable within a stream, using a fraction of time and memory and with a predictable error margin.
Riley Newman, head of data science at Airbnb, recently published an article describing how the Californian startup defines and uses data science. He explains that data can be seen as the voice of the customers, and data science as an act of interpretation. He also details several initiatives that have been particularly important for scaling data science.
Yahoo! has benchmarked three of the main stream processing frameworks: Apache Flink, Spark and Storm.
MongoDB recently announced the newest version of its NoSQL database synonymous product. Building upon the new features introduced in 3.0 release, 3.2 is expanding and solidifying MongoDB’s interest towards the corporate world.