InfoQ Homepage Streaming Content on InfoQ
-
Confluent Announces Apache Flink on Confluent Cloud in Open Preview
Confluent recently announced the open preview of Apache Flink on Confluent Cloud as a fully managed service for stream processing. The company claims that the managed service will make it easier for companies to filter, join, and enrich data streams with Flink.
-
Digital Ocean Launches its Managed Kafka Service
Digital Ocean enters the arena of fully managed Kafka services with its new offering aimed at simplifying management and maintenance of the popular event streaming platform. Digital Ocean Kafka targets startups and SMBs by offering them an all-inclusive, flat-rate pricing model.
-
Allegro Uses Control Theory for Workload Balancing in Its Apache Kafka PubSub Platform
Allegro, the largest eCommerce platform in Poland, implemented dynamic workload balancing in Hermes, its open-source publish-subscribe message broker, built on top of Apache Kafka. The new workload balancing algorithm achieves more uniform resource utilization and lower infrastructure costs.
-
Running Apache Flink Applications on AWS KDA: Lessons Learnt at Deliveroo
Deliveroo introduced Apache Flink into its technology stack for enriching and merging events consumed from Apache Kafka or Kinesis Streams. The company opted to use AWS Kinesis Data Analytics (KDA) service to manage Apache Flink clusters on AWS and shared its experiences from running Flink applications on KDA.
-
Grab Reduces Traffic Cost for Kafka Consumers on AWS to Zero
Grab took advantage of the ability of Apache Kafka consumers to connect to the broker node in the same availability zone (AZ) introduced in Kafka 2.3 and reduced the traffic cost on AWS to zero for reconfigured consumers. The change has substantially reduced overall infrastructure costs for running Apache Kafka on AWS.
-
Grammarly Replaces its in-House Data Lake with Databricks Platform Using Medallion Architecture
Grammarly adopted the medallion architecture while migrating from their in-house data lake, storing Parquet files in AWS S3, to the Delta Lake lakehouse. The company created a new event store for over 6000 event types from 40 internal and external clients and, in the process, improved data quality and reduced the data-delivery time by 94%.
-
Yelp Rebuilds Corrupted Cassandra Cluster Using Its Data Streaming Architecture
Yelp created a solution to sanitize data from the corrupted Apache Cassandra cluster utilizing its data streaming architecture. The team explored many potential options to address the data corruption issue, however, ultimately had to move the data into a new cluster to remove corrupted records in the process.
-
Public Preview of JSON Schema Support in Azure Event Hubs Schema Registry for Kafka Applications
Microsoft recently announced that the Azure Event Hubs schema registry now includes JSON schema support, providing Kafka applications with a centralized repository for schema documents used in messaging-centric and event-driven applications. The JSON schema support is currently in public preview.
-
Airbnb Improves Web Performance with HTTP Streaming
Airbnb introduced HTTP streaming to improve the page-loading performance of their website. They reduced the First Contentful Paint (FCP) metric by around 100 milliseconds on every page tested, including the homepage. They also minimized the impact of slow backend queries on the loading times.
-
Apache Pulsar 3.0 Delivers a New LTS Version and Efficiency Improvements
The Apache Software Foundation has released version 3.0 of Apache Pulsar, the distributed messaging and streaming platform. Pulsar 3.0 introduces the Long-Term Support release and many performance and scalability improvements.
-
AWS Lambda Introduces Response Payload Streaming
Amazon recently announced that AWS Lambda functions can progressively stream response payloads back to the client. The new feature improves performance for web and mobile applications and is currently available for Node.js 14.x and newer runtimes, as well as custom ones.
-
Amazon EventBridge Pipes Support Point-to-Point Integrations between Event Producers and Consumers
At re:Invent, AWS introduced Amazon EventBridge Pipes, a new feature in Amazon EventBridge providing developers a more straightforward way to connect events from multiple services.
-
AWS Introduces Amazon EventBridge Scheduler
AWS recently introduced Amazon EventBridge Scheduler, a new capability from Amazon EventBridge that allows organizations to create, run, and manage scheduled tasks at scale.
-
Microsoft Releases Stream Analytics No-Code Editor into General Availability
During the Ignite Conference, Microsoft released Azure Stream Analytics no-code editor, a drag-and-drop canvas for developing jobs for stream processing scenarios such as streaming ETL, ingestion, and materializing data to data into general availability. The no-code editor is hosted in the company’s big-data streaming platform and event ingestion service, Azure Event Hubs.
-
Confluent Introduces Stream Governance Advanced to Safely Extend Data Streaming Power
Confluent recently announced new enhancements to its Stream Governance product that will improve engineering teams' ability to discover, understand, and trust real-time data. Organizations can use Stream Governance Advanced to resolve issues within complex pipelines more easily with point-in-time lineage.