InfoQ Homepage Apache Kafka Content on InfoQ
-
Oracle Expands Cloud Native Services, Adds Kafka Streaming, API Gateway and Logging Support
In a recent blog post, Oracle announced the limited availability of three news service offerings in its Oracle Cloud Native Services platform. The three new services include Kafka Compatibility for Oracle Streaming, an API Gateway for managing connectivity to serverless components and containers and a Logging service that supports log management and analytics across resources and applications.
-
The Future of Data Engineering: Chris Riccomini at QCon San Francisco
At QCon San Francisco 2019, Chris Riccomini presented “The Future of Data Engineering”. The key takeaway of his talk is about reaching an end goal with data engineering, which is having a fully automated decentralized data warehouse.
-
Delta – a Data Synchronization and Enrichment Platform by Netflix
Large systems often utilize numerous data stores. There is sometimes a need to keep some of these data stores in sync, and to enrich data in a store by calling external services. To address these needs, Netflix has created Delta, an eventual consistent, event-driven data synchronization and enrichment platform. In a blog post, the team behind Delta gives an overview of their design.
-
Jagadish Venkatraman on LinkedIn's Journey to Samza 1.0
At the recent ApacheCon North America, Jagadish Venkatraman spoke about how LinkedIn developed Apache Samza 1.0 to handle stream processing at scale. He described LinkedIn's use cases involving trillions of events and petabytes of data, then highlighted the features added for the 1.0 release, including: stateful processing, high-level APIs, and a flexible deployment model.
-
Celia Kung on LinkedIn's Brooklin Data Streaming Service
Celia Kung from LinkedIn's team spoke at the QCon New York 2019 Conference last week about Brooklin data streaming service that supports pluggable sources and destinations. These can be data stores or messaging systems making the solution flexible and extensible. Brooklin is part of the streams infrastructure platform developed at LinkedIn.
-
Amazon Managed Kafka Aims to Simplify Kafka Streaming Setup and Use
Introduced as a public preview at AWS re:invent 2018, Amazon Managed Streaming for Kafka (MSK) is now generally available. Amazon MSK aims to make it easy to build and run streaming applications based on Kafka.
-
Expo: Real Time A/B Testing and Monitoring with Spark Streaming and Kafka at Walmart Labs
The WalmartLabs engineering team developed a real time A/B testing tool called Expo that collects and analyzes user engagement metrics. It uses Spark Structured Streaming to process the incoming data and stores the metrics in KairosDB.
-
Allegro Launches Hermes 1.0, a REST-based Message Broker Built on Top of Kafka
Allegro has launched version 1.0 of Hermes, a rest API based message broker built on top of Apache Kafka. Whilst not containing any new features, this first major version release is given to the current stable codebase.
-
QCon NY (Jun 24-28): New Talks, a Focus on the Skills That Matter & Why You Should Join Us This Year
In the recent Stack Overflow 9th annual survey of over 90,000 software developers, we learned that non-development work remains a productivity challenge for software managers and leaders. At QCon New York, the conference for senior software developers, we have many sessions to help you learn how others have overcome those challenges.
-
HiveMQ Goes Open Source, Brings MQTT and Kafka Together
Recently announced by HiveMQ, HiveMQ Enterprise Extension for Kafka aims to integrate Kafka and MQTT to enable real-time streaming for IoT applications.
-
License Changes for Confluent Platform Restricting Cloud Vendor Usage
Confluent has announced changes to the license for some components of their Confluent Platform, a streaming platform which provides capabilities to transport data, and tools to connect systems and data sources. The license changes specifically focus on restricting the usage of these components by SaaS providers.
-
Scaling Apache Kafka at Pinterest
Apache Kafka is used at Pinterest for transporting data for real time streaming applications, logging and visibility metrics for monitoring. Hosted on AWS, Pinterest’s Kafka installation uses the MirrorMaker and DoctorKafka tools for replication and high availability.
-
Amazon Announces Managed Streaming for Kafka in Public Preview
At the recent AWS re:Invent 2018 event, Amazon announced a new fully managed service that makes it easy for customers to build and run applications that use Apache Kafka to process streaming data. This new service is called Amazon Managed Streaming for Kafka, Amazon MSK for short, and is now in public preview.
-
Microsoft Announces the General Availability of Azure Event Hubs for Apache Kafka
Recently, Microsoft announced the general availability of Azure Event Hubs for Apache Kafka. Customers will get the best of both worlds—the ecosystem and tools of Kafka, along with Azure’s security and global scale.
-
Q&A with Microsoft's Arindam Chatterjee Discussing Azure HDInsight 4.0
InfoQ caught up with Arindam Chatterjee, principal group manager at Microsoft, regarding the announcements about HDInsight at Microsoft Ignite.