BT

Ready for InfoQ 3.0? Try the new design and let us know what you think!

Older rss
  • AI, ML & Data Engineering Follow 1108 Followers

    Apache Kafka: Ten Best Practices to Optimize Your Deployment

    by Ben Bromhead Follow 0 Followers on  Oct 19, 2018

    Author Ben Bromhead discusses the latest Kafka best practices for developers to manage the data streaming platform more effectively. Best practices include log configuration, proper hardware usage, Zookeeper configuration, replication factor, and partition count.

  • AI, ML & Data Engineering Follow 1108 Followers

    Democratizing Stream Processing with Apache Kafka® and KSQL - Part 2

    by Robin Moffatt Follow 2 Followers on  Sep 07, 2018

    In this article, author Robin Moffatt shows how to use Apache Kafka and KSQL to build data integration and processing applications with the help of an e-commerce sample application. Three use cases discussed: customer operations, operational dashboard, and ad-hoc analytics.

  • Architecture & Design Follow 2632 Followers

    A Critique of Resizable Hash Tables: Riak Core & Random Slicing

    by Scott Lystig Fritchie Follow 0 Followers on  Aug 26, 2018

    This fall, Wallaroo Labs will be releasing a large new feature set to our distributed data stream processing framework, Wallaroo. One of the new features requires a size-adjustable, distributed data structure to support growing & shrinking of compute clusters. It might be a good idea to use a distributed hash table to support the new feature, but what distributed hash algorithm should we choose?

AI, ML & Data Engineering Follow 1108 Followers

How to Choose a Stream Processor for Your App

Posted by Srinath Perera Follow 2 Followers , Miyuru Dayarathna Follow 0 Followers on  Aug 21, 2018

In this article, authors discuss reference architecture of stream processor technologies and key features required by most streaming applications. 2

AI, ML & Data Engineering Follow 1108 Followers

Democratizing Stream Processing with Apache Kafka and KSQL - Part 1

Posted by Michael Noll Follow 0 Followers on  Jun 15, 2018

In this article, author discusses the stream processing with KSQL, the streaming SQL engine for Apache Kafka. Topics covered include challenges of stateful stream processing & how KSQL addresses them. 1

AI, ML & Data Engineering Follow 1108 Followers

Migrating Batch ETL to Stream Processing: A Netflix Case Study with Kafka and Flink

Posted by Daniel Bryant Follow 824 Followers on  Feb 08, 2018

At QCon NY, Shriya Arora presented “Personalising Netflix with Streaming Datasets” and discussed a migration of a data processing job from batch-style ETL to stream processing using Apache Flink.

Architecture & Design Follow 2632 Followers

Exploring the Fundamentals of Stream Processing with the Dataflow Model and Apache Beam

Posted by Daniel Bryant Follow 824 Followers on  Jan 30, 2018

At QCon SF 2016, Frances Perry and Tyler Akidau presented “Fundamentals of Stream Processing with Apache Beam”, and discussed Google's Dataflow model and associated implementation of Apache Beam.

Architecture & Design Follow 2632 Followers

Is Batch ETL Dead, and is Apache Kafka the Future of Data Processing?

Posted by Daniel Bryant Follow 824 Followers on  Jan 22, 2018

At QCon San Francisco 2016, Neha Narkhede presented “ETL is Dead; Long Live Streams”, and discussed the changing landscape of enterprise data processing. 5

Mobile Follow 187 Followers

Processing Streaming Human Trajectories with WSO2 CEP

Posted by Ramindu De Silva Follow 0 Followers , Miyuru Dayarathna Follow 0 Followers on  Feb 27, 2017

Extracting useful information from inaccurate data stream, such as human trajectories, is a key issue in IoT applications. This problem can be successfully tackled with Kalman filters.

Architecture & Design Follow 2632 Followers

Key Takeaway Points and Lessons Learned from QCon San Francisco 2016

Posted by Abel Avram Follow 12 Followers on  Dec 14, 2016

The 10th annual QCon San Francisco was the biggest yet, bringing together over 1500 team leads, architects, project managers, and engineering directors.

AI, ML & Data Engineering Follow 1108 Followers

Big Data Processing with Apache Spark - Part 3: Spark Streaming

Posted by Srini Penchikala Follow 41 Followers on  Jan 07, 2016

In this article, third installment of Apache Spark series, author discusses Apache Spark Streaming framework for processing real-time streaming data using a log analytics sample application. 8

Followers

Storm Applied Review and Q&A with the Authors

Posted by Sergio De Simone Follow 21 Followers on  Jul 27, 2015

Storm Applied is a new book from Manning that aims to provide a practical guide on using Storm, both in a development and in a production setting. InfoQ has spoken with two of the book’s authors.

BT