InfoQ Homepage Streaming Content on InfoQ
-
Swim Open Sources Platform That Challenges Conventional Wisdom in Distributed Computing
If one reviews the talks at a modern conference, it'd be reasonable to assume that many of today's software systems are made up of stateless compute, distributed databases, and high-throughput message brokers. Swim recently open-sourced their platform that uses stateful "digital twins" to analyze streaming data in real time without depending on databases, message brokers, app servers, or storage.
-
The Importance of Event-First Thinking
For global businesses to meet today’s architectural challenges with constant change and extreme scale, we need to go back to the basic principles of system design. The common element in the problems we face is the notion of events driving both actions and reactions, Neil Avery writes in a series of blog posts describing why events are so important and the advantages of an event-first approach .
-
Amazon Announces Managed Streaming for Kafka in Public Preview
At the recent AWS re:Invent 2018 event, Amazon announced a new fully managed service that makes it easy for customers to build and run applications that use Apache Kafka to process streaming data. This new service is called Amazon Managed Streaming for Kafka, Amazon MSK for short, and is now in public preview.
-
Microsoft Announces the General Availability of Azure Event Hubs for Apache Kafka
Recently, Microsoft announced the general availability of Azure Event Hubs for Apache Kafka. Customers will get the best of both worlds—the ecosystem and tools of Kafka, along with Azure’s security and global scale.
-
Event Sourcing to the Cloud at HomeAway
Adam Haines, Data Architect at HomeAway, recently spoke at the Data Architecture Summit 2018 Conference about how his team leverages event sourcing cloud design pattern to accelerate the big data initiatives in their organization.
-
William McKnight on Data Platforms and Creating a Modern Data Architecture
William McKnight gave a keynote presentation last week at Data Architecture Summit 2018 Conference on creating a modern data architecture using different data platforms.
-
Netflix Keystone Real-Time Stream Processing Platform
Netflix recently published a post in their tech blog discussing the design considerations and insights of Keystone, their Real-time stream processing platform. Keystone has been operational since December 2015 and has grown significantly over the years as Netflix subscribers have grown from 65 to over 130 million in the past 3 years. This article follows on the latest state of Keystone platform...
-
Data Artisans Announces Serializable ACID Transactions on Streaming Data
Data Artisans has announced the general availability of Streaming Ledger, which extends Apache Flink with capabilities to perform serializable ACID transactions across tables, keys, and event streams. The patent-pending technology is a proprietary add-on for Flink and allows going beyond the current standard where operations could only consistently work on a single key at a time.
-
NATS Messaging System Gets Kafka-Like Log API via Liftbridge
Joining the hot event-driven technology space is Liftbridge, an open-source project that extends the NATS messaging system with a scalable, Kafka-like log API. InfoQ spoke to creator Tyler Treat to learn more about the project, and the changing nature of data integration.
-
Axon Framework 3.3 with a Subscription Query API and Kafka Support
Version 3.3 of the Axon framework was recently released with a subscription query API for subscribing to query model updates, a manager for scheduling the publishing of deadline messages, and an Axon-Kafka module allowing for the use of Kafka to send and receive events. An updated version, 3.3.2, has also been released, and for those on version 3.3 an upgrade is strongly recommended.
-
QCon NY: Jonas Bonér on Designing Events-First Microservices
Events-first domain driven design (DDD) and event streaming are critical in developing a resilient and scalable microservices architecture. Jonas Bonér from LightBend engineering team spoke at QCon New York 2018 Conference last week about the events-first design.
-
Experiences from Building an Event-Sourced System with Kafka Streams
At the recent JEEConf conference in Kiev, Amitay Horwitz described how he and his team implemented an event-sourced invoice system, the challenges they experienced after running in production for 2 ½ years, and how they implemented a new design using Kafka Streams. The new design is still under assessment, but they do heavily use Kafka in production.
-
Distributed Messaging Framework Apache Pulsar 2.0 Supports Schema Registry and Topic Compaction
The latest version of open-source distributed pub-sub messaging framework Apache Pulsar enables companies to move “beyond batch” by acting on data in motion. Streamlio recently announced the availability of Apache Pulsar 2.0 streaming messaging solution. The new version supports Pulsar Functions, Schema Registry and Topic Compaction.
-
Microsoft Announces Azure Event Hubs for Kafka Ecosystems in Public Preview
During Build 2018, Microsoft announced it would support Kafka clients to integrate with Azure Event Hubs. The Microsoft engineering team responsible for Azure Event Hubs made a Kafka endpoint available for users of their service to stream event data into it.
-
eBay's Accelerator Data Processing Framework Provides Parallel Execution and Live Recommendations
eBay's Accelerator data processing framework provides parallel execution and automatic organization of source code, input data, and results. It can be used for data analysis, and algorithm development, as well as a live recommendation system.