InfoQ Homepage Streaming Content on InfoQ
-
Grammarly Replaces its in-House Data Lake with Databricks Platform Using Medallion Architecture
Grammarly adopted the medallion architecture while migrating from their in-house data lake, storing Parquet files in AWS S3, to the Delta Lake lakehouse. The company created a new event store for over 6000 event types from 40 internal and external clients and, in the process, improved data quality and reduced the data-delivery time by 94%.
-
Yelp Rebuilds Corrupted Cassandra Cluster Using Its Data Streaming Architecture
Yelp created a solution to sanitize data from the corrupted Apache Cassandra cluster utilizing its data streaming architecture. The team explored many potential options to address the data corruption issue, however, ultimately had to move the data into a new cluster to remove corrupted records in the process.
-
Public Preview of JSON Schema Support in Azure Event Hubs Schema Registry for Kafka Applications
Microsoft recently announced that the Azure Event Hubs schema registry now includes JSON schema support, providing Kafka applications with a centralized repository for schema documents used in messaging-centric and event-driven applications. The JSON schema support is currently in public preview.
-
Airbnb Improves Web Performance with HTTP Streaming
Airbnb introduced HTTP streaming to improve the page-loading performance of their website. They reduced the First Contentful Paint (FCP) metric by around 100 milliseconds on every page tested, including the homepage. They also minimized the impact of slow backend queries on the loading times.
-
Apache Pulsar 3.0 Delivers a New LTS Version and Efficiency Improvements
The Apache Software Foundation has released version 3.0 of Apache Pulsar, the distributed messaging and streaming platform. Pulsar 3.0 introduces the Long-Term Support release and many performance and scalability improvements.
-
AWS Lambda Introduces Response Payload Streaming
Amazon recently announced that AWS Lambda functions can progressively stream response payloads back to the client. The new feature improves performance for web and mobile applications and is currently available for Node.js 14.x and newer runtimes, as well as custom ones.
-
Amazon EventBridge Pipes Support Point-to-Point Integrations between Event Producers and Consumers
At re:Invent, AWS introduced Amazon EventBridge Pipes, a new feature in Amazon EventBridge providing developers a more straightforward way to connect events from multiple services.
-
AWS Introduces Amazon EventBridge Scheduler
AWS recently introduced Amazon EventBridge Scheduler, a new capability from Amazon EventBridge that allows organizations to create, run, and manage scheduled tasks at scale.
-
Microsoft Releases Stream Analytics No-Code Editor into General Availability
During the Ignite Conference, Microsoft released Azure Stream Analytics no-code editor, a drag-and-drop canvas for developing jobs for stream processing scenarios such as streaming ETL, ingestion, and materializing data to data into general availability. The no-code editor is hosted in the company’s big-data streaming platform and event ingestion service, Azure Event Hubs.
-
Confluent Introduces Stream Governance Advanced to Safely Extend Data Streaming Power
Confluent recently announced new enhancements to its Stream Governance product that will improve engineering teams' ability to discover, understand, and trust real-time data. Organizations can use Stream Governance Advanced to resolve issues within complex pipelines more easily with point-in-time lineage.
-
Confluent Ships Stream Designer Democratizing Data Streams
Confluent recently released Stream Designer, a visual interface that lets developers quickly build and deploy streaming data pipelines.
-
Write Directly from Cloud Pub/Sub to BigQuery with BigQuery Subscription
Recently Google introduced a new type of Pub/Sub subscription called a “BigQuery subscription,” allowing to write directly from Cloud Pub/Sub to BigQuery. The company claims that this new extract, load, and transform (ELT) path will be able to simplify event-driven architectures.
-
Fitting Presto to Large-Scale Apache Kafka at Uber
The need for ad-hoc real-time data analysis has been growing at Uber. They run a large Apache Kafka deployment and need to analyse data going through the many workflows it supports. Solutions like stream processing and OLAP datastores were deemed unsuitable. An article was published recently detailing why Uber chose Presto for this purpose and what it had to do to make it performant at scale.
-
Amazon Rekognition Introduces Streaming Video Events
AWS recently announced the general availability of Streaming Video Events, a new feature of Amazon Rekognition to provide real-time alerts on live video streams.
-
Quine Aims to Simplify Event Processing on Data in Motion
Developed at thatDot, Quine is an open source streaming graph solution aimed at high-volume event processing. Quine combines graph data and streaming technologies to enable the creation of real-time, complex event processing workflows at scale, says thatDot.