BT
rss
AI, ML & Data Engineering Follow 1068 Followers

Julien Le Dem on the Future of Column-Oriented Data Processing with Apache Arrow

by Alexandre Rodrigues Follow 1 Followers on  Dec 08, 2016 1

Julien Le Dem, the PMC chair of the Apache Arrow project, presented on Data Eng Conf NY on the future of column-oriented data processing. Apache Arrow is an open-source standard for columnar in-memory execution. InfoQ interviewed Le Dem to find out the differences between Arrow and Parquet.

AI, ML & Data Engineering Follow 1068 Followers

Stream Processing and Lambda Architecture Challenges

by Alexandre Rodrigues Follow 1 Followers on  Oct 19, 2016 4

Lambda architecture has been a popular solution that combines batch and stream processing. Kartik Paramasivam at LinkedIn wrote about how his team addressed stream processing and Lambda architecture challenges using Apache Samza for data processing. The challenges described are the late arrival of events and the processing of duplicated messages.

BT