InfoQ Homepage Streaming Content on InfoQ
-
ETL Is Dead, Long Live Streams
Neha Narkhede shares the experience at LinkedIn moving from ETL to real-time streams, the challenges of scaling Kafka to hundreds of billions of events/day, supporting thousands of engineers, etc.
/presentations/etl-streams/en/mediumimage/neabig.JPG)
-
Ingest & Stream Processing - What Will You Choose?
Pat Patterson and Ted Malaska talk about current and emerging data processing technologies, and the various ways of achieving "at least once" and "exactly once" timely data processing.
/presentations/ingest-stream-processing/en/mediumimage/PatTedbig.JPG)
-
Apache Beam: The Case for Unifying Streaming APIs
Andrew Psaltis talks about Apache Beam, which aims to provide a unified stream processing model for defining and executing complex data processing, data ingestion and integration workflows.
/presentations/apache-beam/en/mediumimage/Andbig.JPG)
-
Monitoring and Troubleshooting Real-Time Data Pipelines
Alan Ngai and Premal Shah discuss best practices on monitoring distributed real-time data processing frameworks and how DevOps can gain control and visibility over these data pipelines.
/presentations/monitoring-data-pipelines/en/mediumimage/12big.jpg)
-
REST Considered Harmful
Ross Garrett explores how design decisions may be leading to poor UX, discussing the principles of reactive applications and how streaming APIs can deliver significant benefits over RESTful APIs.
/presentations/rest-streaming-api/en/mediumimage/Rosbig.JPG)
-
Server-Less Design Patterns for the Enterprise with AWS Lambda
Tim Wagner defines server-less computing, examines the key trends and innovative ideas behind the technology, and looks at design patterns for big data, event processing, and mobile using AWS Lambda.
/presentations/serverless-computing/en/mediumimage/Timbig.JPG)
-
Handling Streaming Data in Spotify Using the Cloud
Neville Li and Igor Maravić cover the evolution of Spotify’s event delivery system, discussing lessons learned moving it into the cloud using Scio, a high level Scala API for the Dataflow SDK.
/presentations/spotify-streaming-cloud/en/mediumimage/Doi big.jpg)
-
Large-Scale Stream Processing with Apache Kafka
Neha Narkhede explains how Apache Kafka was designed to support capturing and processing distributed data streams by building up the basic primitives needed for a stream processing system.
/presentations/apache-kafka-streams/en/mediumimage/Nebig.JPG)
-
Detecting Anomalies in Streaming Data, Evaluating Algorithms for Real-World Use
Alexander Lavin introduces the Numenta Anomaly Benchmark (NAB), a framework for evaluating anomaly detection algorithms on streaming data.
/presentations/numenta-anomaly-benchmark/en/mediumimage/Albig.JPG)
-
Staying in Sync: from Transactions to Streams
Martin Kleppmann explores using event streams and Kafka for keeping data in sync across heterogeneous systems, and compares this approach to distributed transactions.
/presentations/event-streams-kafka/en/mediumimage/Marbig.JPG)
-
Netflix Keystone - How We Built a 700B/day Stream Processing Cloud Platform in a Year
Peter Bakas presents in detail how Netflix has used Kafka, Samza, Docker, and Linux to implement a multi-tenant pipeline processing 700B events/day in the Amazon AWS cloud.
/presentations/netflix-keystone/en/mediumimage/Pebig.JPG)
-
Compositional I/O Stream in Scala
Runar Bjarnason presents how to get started with the Scalaz-Stream library, shows some examples, and how we can combine functional streams into large distributed systems.
/presentations/funnel-distributed-monitoring/en/mediumimage/Runbig.JPG)