BT

Ready for InfoQ 3.0? Try the new design and let us know what you think!

Cloud Follow 375 Followers

Amazon Announces Managed Streaming for Kafka in Public Preview

by Steef-Jan Wiggers Follow 10 Followers on  Dec 06, 2018

At the recent AWS re:Invent 2018 event, Amazon announced a new fully managed service that makes it easy for customers to build and run applications that use Apache Kafka to process streaming data. This new service is called Amazon Managed Streaming for Kafka, Amazon MSK for short, and is now in public preview.

Cloud Follow 375 Followers

Microsoft Announces the General Availability of Azure Event Hubs for Apache Kafka

by Steef-Jan Wiggers Follow 10 Followers on  Nov 09, 2018

Recently, Microsoft announced the general availability of Azure Event Hubs for Apache Kafka. Customers will get the best of both worlds—the ecosystem and tools of Kafka, along with Azure’s security and global scale.

AI, ML & Data Engineering Follow 1107 Followers

William McKnight on Data Platforms and Creating a Modern Data Architecture

by Srini Penchikala Follow 41 Followers on  Oct 15, 2018

William McKnight gave a keynote presentation last week at Data Architecture Summit 2018 Conference on creating a modern data architecture using different data platforms.

Cloud Follow 375 Followers

Netflix Keystone Real-Time Stream Processing Platform

by Alex Giamas Follow 10 Followers on  Sep 30, 2018

Netflix recently published a post in their tech blog discussing the design considerations and insights of Keystone, their Real-time stream processing platform. Keystone has been operational since December 2015 and has grown significantly over the years as Netflix subscribers have grown from 65 to over 130 million in the past 3 years. This article follows on the latest state of Keystone platform...

Cloud Follow 375 Followers

Data Artisans Announces Serializable ACID Transactions on Streaming Data

by Eldert Grootenboer Follow 4 Followers on  Sep 12, 2018

Data Artisans has announced the general availability of Streaming Ledger, which extends Apache Flink with capabilities to perform serializable ACID transactions across tables, keys, and event streams. The patent-pending technology is a proprietary add-on for Flink and allows going beyond the current standard where operations could only consistently work on a single key at a time.

Cloud Follow 375 Followers

NATS Messaging System Gets Kafka-Like Log API via Liftbridge

by Richard Seroter Follow 8 Followers on  Aug 20, 2018

Joining the hot event-driven technology space is Liftbridge, an open-source project that extends the NATS messaging system with a scalable, Kafka-like log API. InfoQ spoke to creator Tyler Treat to learn more about the project, and the changing nature of data integration.

Architecture & Design Follow 2632 Followers

Axon Framework 3.3 with a Subscription Query API and Kafka Support

by Jan Stenberg Follow 39 Followers on  Jul 19, 2018

Version 3.3 of the Axon framework was recently released with a subscription query API for subscribing to query model updates, a manager for scheduling the publishing of deadline messages, and an Axon-Kafka module allowing for the use of Kafka to send and receive events. An updated version, 3.3.2, has also been released, and for those on version 3.3 an upgrade is strongly recommended.

Architecture & Design Follow 2632 Followers

QCon NY: Jonas Bonér on Designing Events-First Microservices

by Srini Penchikala Follow 41 Followers on  Jul 05, 2018

Events-first domain driven design (DDD) and event streaming are critical in developing a resilient and scalable microservices architecture. Jonas Bonér from LightBend engineering team spoke at QCon New York 2018 Conference last week about the events-first design.

Architecture & Design Follow 2632 Followers

Experiences from Building an Event-Sourced System with Kafka Streams

by Jan Stenberg Follow 39 Followers on  Jul 05, 2018

At the recent JEEConf conference in Kiev, Amitay Horwitz described how he and his team implemented an event-sourced invoice system, the challenges they experienced after running in production for 2 ½ years, and how they implemented a new design using Kafka Streams. The new design is still under assessment, but they do heavily use Kafka in production.

AI, ML & Data Engineering Follow 1107 Followers

Distributed Messaging Framework Apache Pulsar 2.0 Supports Schema Registry and Topic Compaction

by Srini Penchikala Follow 41 Followers on  Jun 25, 2018

The latest version of open-source distributed pub-sub messaging framework Apache Pulsar enables companies to move “beyond batch” by acting on data in motion. Streamlio recently announced the availability of Apache Pulsar 2.0 streaming messaging solution. The new version supports Pulsar Functions, Schema Registry and Topic Compaction.

Cloud Follow 375 Followers

Microsoft Announces Azure Event Hubs for Kafka Ecosystems in Public Preview

by Steef-Jan Wiggers Follow 10 Followers on  Jun 02, 2018 1

During Build 2018, Microsoft announced it would support Kafka clients to integrate with Azure Event Hubs. The Microsoft engineering team responsible for Azure Event Hubs made a Kafka endpoint available for users of their service to stream event data into it.

AI, ML & Data Engineering Follow 1107 Followers

eBay's Accelerator Data Processing Framework Provides Parallel Execution and Live Recommendations

by Srini Penchikala Follow 41 Followers on  May 31, 2018

eBay's Accelerator data processing framework provides parallel execution and automatic organization of source code, input data, and results. It can be used for data analysis, and algorithm development, as well as a live recommendation system.

Java Follow 1207 Followers

Hazelcast Releases Jet 0.6 for Stream and Fast Batch Processing

by Ben Evans Follow 35 Followers on  Apr 24, 2018 1

Hazelcast, maker of distributing computing technologies and tools, have released a new major version (version 0.6) of Jet, their open-source streaming processing engine.

Cloud Follow 375 Followers

Spring Cloud Stream 2.0 Released with Focus on Performance, Flexibility and Consistency

by Daniel Bryant Follow 823 Followers on  Apr 17, 2018

Pivotal has announced the General Availability release of the Spring Cloud Stream 2.0. This release includes a complete revamp of content-type negotiation functionality (allowing user-defined message converters), polling consumers, micrometer metrics support, enhanced Apache Kafka Streams support, and more.

Architecture & Design Follow 2632 Followers

Retroactive and Future Events in an Event Sourced System

by Jan Stenberg Follow 39 Followers on  Feb 09, 2018

When Thomas Pierrain started a new project with an asset management company, one important requirement was the ability to go back in time to understand why they took decisions that today look strange. At the recent DDD Europe 2018 conference in Amsterdam, Pierrain discussed his experiences when building an event sourced system that included some temporal challenges.

BT