BT
Architecture & Design Follow 2015 Followers

Scaling Push Messaging for Millions of Devices @Netflix - Susheel Aroskar at QCon NY

by Srini Penchikala Follow 34 Followers on  Jul 16, 2018

Susheel Aroskar from Netflix's Engineering team spoke at the recent QCon New York 2018 Conference about Zuul Push, a scalable push notification service that asynchronously pushes data like personalized movie recommendations from cloud to devices.

Architecture & Design Follow 2015 Followers

Experiences from Building an Event-Sourced System with Kafka Streams

by Jan Stenberg Follow 33 Followers on  Jul 05, 2018

At the recent JEEConf conference in Kiev, Amitay Horwitz described how he and his team implemented an event-sourced invoice system, the challenges they experienced after running in production for 2 ½ years, and how they implemented a new design using Kafka Streams. The new design is still under assessment, but they do heavily use Kafka in production.

Cloud Follow 265 Followers

Microsoft Announces Azure Event Hubs for Kafka Ecosystems in Public Preview

by Steef-Jan Wiggers Follow 6 Followers on  Jun 02, 2018 1

During Build 2018, Microsoft announced it would support Kafka clients to integrate with Azure Event Hubs. The Microsoft engineering team responsible for Azure Event Hubs made a Kafka endpoint available for users of their service to stream event data into it.

Cloud Follow 265 Followers

Managing and Operating Kafka Clusters in Kubernetes

by Srini Penchikala Follow 34 Followers on  Feb 06, 2018

Nenad Bogojevic, platform solutions architect at Amadeus, spoke at KubeCon + CloudNativeCon North America 2017 Conference on how to run and manage Kafka clusters in Kubernetes environment. He talked about provisioning Kafka clusters and configuring them using Kubernetes custom resources or ConfigMaps.

Architecture & Design Follow 2015 Followers

Kevin Webber on Migrating Java to the Cloud

by Srini Penchikala Follow 34 Followers on  Nov 27, 2017

Kevin Webber spoke at Reactive Summit 2017 last month about migrating enterprise Java applications to the cloud by leveraging techniques like Event Storming, Domain Driven Design, and Cloud Native.

AI, ML & Data Engineering Follow 814 Followers

Apache Kafka Reaches 1.0

by Andrew Morgan Follow 3 Followers on  Nov 06, 2017

The Apache Software Foundation has announced Apache Kafka 1.0, adding an improved Streams API, enhanced metrics, improved tolerance for disk failures, general bug fixes, and more.

AI, ML & Data Engineering Follow 814 Followers

Confluent Releases KSQL, a Distributed Streaming SQL Engine for Apache Kafka

by Srini Penchikala Follow 34 Followers on  Oct 25, 2017

Confluent released KSQL: interactive, distributed streaming SQL engine for Apache Kafka. KSQL supports stream processing operations like aggregations, joins, windowing, and sessionization on topics in Apache Kafka. Confluent announced the open source streaming SQL engine at the recent Kafka Summit conference.

Cloud Follow 265 Followers

Integrate 2017 Recap: Adding Intelligence to Integration

by Kent Weare Follow 9 Followers on  Jul 04, 2017

Integrate 2017, an annual integration event focused on Microsoft Integration technologies, took place in London from June 26th – 28th. Some of the key themes that were discussed include the role of cognitive computing in integration, API orchestration, SaaS connectivity, cloud native integration, the impact of serverless on integration and cloud messaging at scale.

Architecture & Design Follow 2015 Followers

Confluent Cloud, Apache Kafka as a Service in AWS

by Alex Giamas Follow 8 Followers on  May 18, 2017

Apache Kafka is a distributed, fault-tolerant pub sub messaging soltuion, originally developed by LinkedIn and open sourced. Confluent was formed by former LinkedIn engineers in the Kafka development group and today announced Confluent Cloud, a fully hosted and managed Apache Kafka as a Service in AWS. We also take a look at Confluent's second annual Streaming Data report and its findings.

Java Follow 887 Followers

Lightbend Speaks to InfoQ on Their Acquisition of OpsClarity

by Michael Redlich Follow 12 Followers on  Feb 24, 2017

Nine months after acquiring BoldRadius, Lightbend announced their acquisition of OpsClarity, a company specializing in monitoring reactive applications. InfoQ interviewed Mark Brewer, president and CEO at Lightbend and Alan Ngai, co-founder of OpsClarity and now VP of cloud services at Lightbend to learn more about this new partnership.

AI, ML & Data Engineering Follow 814 Followers

Apache Flink 1.2 Released with Dynamic Rescaling, Security and Queryable State

by Alexandre Rodrigues Follow 0 Followers on  Feb 15, 2017

Apache Flink 1.2 was announced and features dynamic rescaling, security, queryable state, and more. The release resolved 650 issues, maintains compatibility with all public APIs and ships with Apache Kafka 0.10 and Apache Mesos support. Flink’s dynamic rescaling allows one to change the parallelism of a streaming job or of an operator within the job.

DevOps Follow 766 Followers

Chaperone - A Kafka Auditing Tool from the Uber Engineering Team

by Hrishikesh Barua Follow 14 Followers on  Jan 26, 2017 2

The Uber Engineering team released their Kafka auditing tool called Chaperone as an open-source project. Chaperone allows for auditing and detection of data loss, latency, and duplication of messages in the multi-datacenter and high-volume Kafka setup at Uber.

AI, ML & Data Engineering Follow 814 Followers

Yelp Open-Sources Latest in Data Pipeline Project, Data Pipeline Client Library

by Dylan Raithel Follow 9 Followers on  Jan 06, 2017

Yelp open sources latest component in its data pipeline initiative, a python-based data pipeline client library.

AI, ML & Data Engineering Follow 814 Followers

Microservices and Stream Processing Architecture at Zalando Using Apache Flink

by Srini Penchikala Follow 34 Followers on  Oct 31, 2016 1

Javier Lopez and Mihail Vieru spoke at Reactive Summit 2016 Conference about cloud-based data integration and distribution platform used for stream processing in business intelligence use cases. Their solution is based on technologies such as Flink, Kafka and Elasticsearch.

AI, ML & Data Engineering Follow 814 Followers

Stream Processing and Lambda Architecture Challenges

by Alexandre Rodrigues Follow 0 Followers on  Oct 19, 2016 4

Lambda architecture has been a popular solution that combines batch and stream processing. Kartik Paramasivam at LinkedIn wrote about how his team addressed stream processing and Lambda architecture challenges using Apache Samza for data processing. The challenges described are the late arrival of events and the processing of duplicated messages.

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT