BT
Cloud Follow 239 Followers

Managing and Operating Kafka Clusters in Kubernetes

by Srini Penchikala Follow 33 Followers on  Feb 06, 2018

Nenad Bogojevic, platform solutions architect at Amadeus, spoke at KubeCon + CloudNativeCon North America 2017 Conference on how to run and manage Kafka clusters in Kubernetes environment. He talked about provisioning Kafka clusters and configuring them using Kubernetes custom resources or ConfigMaps.

Architecture & Design Follow 1744 Followers

Kevin Webber on Migrating Java to the Cloud

by Srini Penchikala Follow 33 Followers on  Nov 27, 2017

Kevin Webber spoke at Reactive Summit 2017 last month about migrating enterprise Java applications to the cloud by leveraging techniques like Event Storming, Domain Driven Design, and Cloud Native.

AI, ML & Data Engineering Follow 702 Followers

Apache Kafka Reaches 1.0

by Andrew Morgan Follow 3 Followers on  Nov 06, 2017

The Apache Software Foundation has announced Apache Kafka 1.0, adding an improved Streams API, enhanced metrics, improved tolerance for disk failures, general bug fixes, and more.

AI, ML & Data Engineering Follow 702 Followers

Confluent Releases KSQL, a Distributed Streaming SQL Engine for Apache Kafka

by Srini Penchikala Follow 33 Followers on  Oct 25, 2017

Confluent released KSQL: interactive, distributed streaming SQL engine for Apache Kafka. KSQL supports stream processing operations like aggregations, joins, windowing, and sessionization on topics in Apache Kafka. Confluent announced the open source streaming SQL engine at the recent Kafka Summit conference.

Cloud Follow 239 Followers

Integrate 2017 Recap: Adding Intelligence to Integration

by Kent Weare Follow 9 Followers on  Jul 04, 2017

Integrate 2017, an annual integration event focused on Microsoft Integration technologies, took place in London from June 26th – 28th. Some of the key themes that were discussed include the role of cognitive computing in integration, API orchestration, SaaS connectivity, cloud native integration, the impact of serverless on integration and cloud messaging at scale.

Architecture & Design Follow 1744 Followers

Confluent Cloud, Apache Kafka as a Service in AWS

by Alex Giamas Follow 8 Followers on  May 18, 2017

Apache Kafka is a distributed, fault-tolerant pub sub messaging soltuion, originally developed by LinkedIn and open sourced. Confluent was formed by former LinkedIn engineers in the Kafka development group and today announced Confluent Cloud, a fully hosted and managed Apache Kafka as a Service in AWS. We also take a look at Confluent's second annual Streaming Data report and its findings.

Java Follow 785 Followers

Lightbend Speaks to InfoQ on Their Acquisition of OpsClarity

by Michael Redlich Follow 11 Followers on  Feb 24, 2017

Nine months after acquiring BoldRadius, Lightbend announced their acquisition of OpsClarity, a company specializing in monitoring reactive applications. InfoQ interviewed Mark Brewer, president and CEO at Lightbend and Alan Ngai, co-founder of OpsClarity and now VP of cloud services at Lightbend to learn more about this new partnership.

AI, ML & Data Engineering Follow 702 Followers

Apache Flink 1.2 Released with Dynamic Rescaling, Security and Queryable State

by Alexandre Rodrigues Follow 0 Followers on  Feb 15, 2017

Apache Flink 1.2 was announced and features dynamic rescaling, security, queryable state, and more. The release resolved 650 issues, maintains compatibility with all public APIs and ships with Apache Kafka 0.10 and Apache Mesos support. Flink’s dynamic rescaling allows one to change the parallelism of a streaming job or of an operator within the job.

DevOps Follow 644 Followers

Chaperone - A Kafka Auditing Tool from the Uber Engineering Team

by Hrishikesh Barua Follow 11 Followers on  Jan 26, 2017 2

The Uber Engineering team released their Kafka auditing tool called Chaperone as an open-source project. Chaperone allows for auditing and detection of data loss, latency, and duplication of messages in the multi-datacenter and high-volume Kafka setup at Uber.

AI, ML & Data Engineering Follow 702 Followers

Yelp Open-Sources Latest in Data Pipeline Project, Data Pipeline Client Library

by Dylan Raithel Follow 9 Followers on  Jan 06, 2017

Yelp open sources latest component in its data pipeline initiative, a python-based data pipeline client library.

AI, ML & Data Engineering Follow 702 Followers

Microservices and Stream Processing Architecture at Zalando Using Apache Flink

by Srini Penchikala Follow 33 Followers on  Oct 31, 2016 1

Javier Lopez and Mihail Vieru spoke at Reactive Summit 2016 Conference about cloud-based data integration and distribution platform used for stream processing in business intelligence use cases. Their solution is based on technologies such as Flink, Kafka and Elasticsearch.

AI, ML & Data Engineering Follow 702 Followers

Stream Processing and Lambda Architecture Challenges

by Alexandre Rodrigues Follow 0 Followers on  Oct 19, 2016 4

Lambda architecture has been a popular solution that combines batch and stream processing. Kartik Paramasivam at LinkedIn wrote about how his team addressed stream processing and Lambda architecture challenges using Apache Samza for data processing. The challenges described are the late arrival of events and the processing of duplicated messages.

AI, ML & Data Engineering Follow 702 Followers

Jay Kreps on Distributed Stream Processing with Apache Kafka and Kafka Streams

by Srini Penchikala Follow 33 Followers on  Oct 16, 2016

Apache Kafka and Kafka Streams frameworks help with developing stream-centric architectures and distributed stream processing applications. Jay Kreps, CEO of Confluent, gave the keynote presentation on stream processing and microservices at Reactive Summit 2016 Conference last week.

AI, ML & Data Engineering Follow 702 Followers

Reactive Summit 2016 Conference: Reactive Microservices and Staging Data Pipelines

by Srini Penchikala Follow 33 Followers on  Oct 08, 2016

Reactive microservices, data center scale operating system (DCOS), and staging reactive data pipelines were the highlighted topics at Reactive Summit 2016 Conference held this week. InfoQ team attended the conference and this post is a summary of the first day's events at the conference.

AI, ML & Data Engineering Follow 702 Followers

Confluent Announces Kafka for the Enterprise with Multi-Datacenter Replication

by Srini Penchikala Follow 33 Followers on  Oct 05, 2016

Confluent Enterprise latest version supports multi-datacenter replication, automatic data balancing, and cloud migration capability. Confluent, provider of the Apache Kafka based streaming platform, announced last week the new features for Confluent Enterprise, to help build streaming data pipelines and develop stream processing applications.

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT