Articles
-
AI, ML and Data Engineering AI, ML and Data Engineering Follow 647 Followers
Migrating Batch ETL to Stream Processing: A Netflix Case Study with Kafka and Flink
At QCon New York, Shriya Arora presented “Personalising Netflix with Streaming Datasets” and discussed the trials and tribulations of a recent migration of a Netflix data processing job from the traditional approach of batch-style ETL to stream processing using Apache Flink.
-
Architecture & Design Architecture & Design Follow 1643 Followers
Exploring the Fundamentals of Stream Processing with the Dataflow Model and Apache Beam
At QCon San Francisco 2016, Frances Perry and Tyler Akidau presented “Fundamentals of Stream Processing with Apache Beam”, and discussed Google's Dataflow model and associated implementation of Apache Beam.
-
Architecture & Design Architecture & Design Follow 1643 Followers
Is Batch ETL Dead, and is Apache Kafka the Future of Data Processing?
At QCon San Francisco 2016, Neha Narkhede presented “ETL is Dead; Long Live Streams”, and discussed the changing landscape of enterprise data processing. A core premise of the talk was that the open source Apache Kafka streaming platform can provide a flexible and uniform framework that supports modern requirements for data transformation and processing.
Video Stream Analytics Using OpenCV, Kafka and Spark Technologies
Processing Streaming Human Trajectories with WSO2 CEP
Extracting useful information from inaccurate data stream, such as human trajectories, is a key issue in IoT applications. This problem can be successfully tackled with Kalman filters.
Key Takeaway Points and Lessons Learned from QCon San Francisco 2016
The 10th annual QCon San Francisco was the biggest yet, bringing together over 1500 team leads, architects, project managers, and engineering directors.
Traffic Data Monitoring Using IoT, Kafka and Spark Streaming
Chris Fregly on the PANCAKE STACK Workshop and Data Pipelines
InfoQ interviews Chris Fregly, organizer for the 4000+ member Advanced Spark and TensorFlow Meetup about the PANCAKE STACK workshop, Spark and building data pipelines for a machine learning pipeline
Storm Applied Review and Q&A with the Authors
Storm Applied is a new book from Manning that aims to provide a practical guide on using Storm, both in a development and in a production setting. InfoQ has spoken with two of the book’s authors.
Big Data Processing with Apache Spark – Part 1: Introduction
Real-Time Stream Processing as Game Changer in a Big Data World with Hadoop and Data Warehouse
Using SEDA to Ensure Service Availability
