• DevOps Follow 483 Followers

    When Streams Fail: Implementing a Resilient Apache Kafka Cluster at Goldman Sachs

    by Daniel Bryant Follow 444 Followers on  Feb 13, 2018 1

    At QCon New York, Anton Gorshkov presented “When Streams Fail: Kafka Off the Shore”. The talk shared insight into how a platform team at a large financial institution design and operate shared internal messaging clusters like Apache Kafka, and also how they plan for, and resolve, the inevitable failure that occurs.

  • Data Science Follow 529 Followers

    Migrating Batch ETL to Stream Processing: A Netflix Case Study with Kafka and Flink

    by Daniel Bryant Follow 444 Followers on  Feb 08, 2018

    At QCon New York, Shriya Arora presented “Personalising Netflix with Streaming Datasets” and discussed the trials and tribulations of a recent migration of a Netflix data processing job from the traditional approach of batch-style ETL to stream processing using Apache Flink.

  • Architecture & Design Follow 1342 Followers

    Is Batch ETL Dead, and is Apache Kafka the Future of Data Processing?

    by Daniel Bryant Follow 444 Followers on  Jan 22, 2018 4

    At QCon San Francisco 2016, Neha Narkhede presented “ETL is Dead; Long Live Streams”, and discussed the changing landscape of enterprise data processing. A core premise of the talk was that the open source Apache Kafka streaming platform can provide a flexible and uniform framework that supports modern requirements for data transformation and processing.

Data Science Follow 529 Followers

Traffic Data Monitoring Using IoT, Kafka and Spark Streaming

Posted by Amit Baghel Follow 0 Followers on  Sep 28, 2016

Internet of Things (IoT) is an emerging technology. One of the areas of IoT is the connected vehicles. In this article, we'll use Spark and Kafka to analyse and process IoT connected vehicle's data. 16

Data Science Follow 529 Followers

Chris Fregly on the PANCAKE STACK Workshop and Data Pipelines

Posted by Dylan Raithel Follow 5 Followers on  Aug 29, 2016

InfoQ interviews Chris Fregly, organizer for the 4000+ member Advanced Spark and TensorFlow Meetup about the PANCAKE STACK workshop, Spark and building data pipelines for a machine learning pipeline

Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you