InfoQ Homepage Apache Kafka Content on InfoQ
-
/articles/quarkus-with-kafka-streams/en/smallimage/kafka-streams-and-quarkus-real-time-processing-events-small-1656083680635.jpg)
Kafka Streams and Quarkus: Real-Time Processing Events
Consuming Kafka messages is simple; you get them as long as they are produced, but nothing more. But if you need real-time processing of the data (filtering, joining, or manipulating events), just using the Kafka-consuming API might not be the best approach as the resulting code becomes complex. Kafka Streams and Quarkus are the perfect matches to start processing Kafka events in real-time.
-
/articles/data-with-quarkus-kafka/en/smallimage/UaUXvhM4-1651071035514.jpg)
Getting Started to Quarkus Reactive Messaging with Apache Kafka
How data is processed/consumed nowadays is different from how it was once practiced. In the past, data was stored in a database and it was batch processed for analytics. Apache Kafka is a distributed event store and stream-processing platform for storing, consuming, and processing data streams in real-time. In this post, we’ll learn how to produce and consume data using Apache Kafka and Quarkus.
-
/articles/microservices-inside-out/en/smallimage/turning-microservices-inside-out-small-image-1627634447180.jpg)
Turning Microservices Inside-Out
Turning microservices inside-out means moving past a single, request/response API to designing microservices with an inbound API for queries and commands, an outbound APIs to emit events, and a meta API to describe them both. A database can be supplemented with Apache Kafka via a connecting tissue such as Debezium.
-
/articles/database-audit-system-kafka/en/smallimage/Building-a-SQL-Database-Audit-System-Using-Maxwell-logo-small-1609154420148.jpg)
Building an SQL Database Audit System Using Kafka, MongoDB and Maxwell's Daemon
In this article, the author discusses the importance of a database audit logging system outside of traditional built-in data replication, using technologies like Kafka, MongoDB, and Maxwell's Daemon.
-
/articles/whats-the-next-step-for-data-management/en/smallimage/database-1605276684430.jpeg)
Beyond the Database, and beyond the Stream Processor: What's the Next Step for Data Management?
Databases have been around forever with the same shape: you make a request to your data and then you receive an answer. Now, stream processors came along with a different approach: data isn’t locked up, it is in motion. Understand how stream processors and databases relate and why there is an emerging new category of databases that focus on data that stays in place as well as data that moves.
-
/articles/real-time-api-kafka/en/smallimage/real-time-api-kafka-s-1602600487985.jpeg)
Real Time APIs in the Context of Apache Kafka
Events offer a Goldilocks-style approach in which real-time APIs can be used as the foundation for applications which is flexible yet performant; loosely-coupled yet efficient. Apache Kafka offers a scalable event streaming platform with which you can build applications around the powerful concept of events.
-
/articles/applied-probability-unstructured-events-theta-sketches/en/smallimage/applied-probability-unstructured-events-theta-sketches-s-1593199758266.jpg)
Applied Probability - Counting Large Set of Unstructured Events with Theta Sketches
In this article, author Ronen Cohen discusses the solution to processing the event data using Theta Sketches and technologies like HBase and Kafka.
-
/articles/building-scalable-iot-application-with-apache-kafka/en/smallimage/The-Kongo-Problem-Building-a-Scalable-IoT-Application-with-Apache-Kafka-logo-1581595522263.jpg)
The Kongo Problem: Building a Scalable IoT Application with Apache Kafka
In this article, author Paul Brebner discusses the best practices for developing IoT projects using Apache Kafka and Kafka Streams technologies and how to maximize Kafka scalability.
-
/articles/prometheus-monitor-applications-at-scale/en/smallimage/logo-1560849701397.jpg)
How to Use Open Source Prometheus to Monitor Applications at Scale
In this article, the author discusses how to collect metrics and achieve anomaly detection from streaming data using Prometheus, Apache Kafka and Apache Cassandra technologies.
-
/articles/apache-kafka-best-practices-to-optimize-your-deployment/en/smallimage/apache-kafka-best-practices-to-optimize-your-deployment-s-1539792752614.jpeg)
Apache Kafka: Ten Best Practices to Optimize Your Deployment
Author Ben Bromhead discusses the latest Kafka best practices for developers to manage the data streaming platform more effectively. Best practices include log configuration, proper hardware usage, Zookeeper configuration, replication factor, and partition count.
-
/articles/democratizing-stream-processing-kafka-ksql-part2/en/smallimage/Democratizing-Stream-Processing-Apache-Kafka-KSQL-Part-2-logo-small-1536080637584.jpg)
Democratizing Stream Processing with Apache Kafka® and KSQL - Part 2
In this article, author Robin Moffatt shows how to use Apache Kafka and KSQL to build data integration and processing applications with the help of an e-commerce sample application. Three use cases discussed: customer operations, operational dashboard, and ad-hoc analytics.
-
/articles/democratizing-stream-processing-apache-kafka-ksql/en/smallimage/kafka-streaming-logo-2-1528901042755.jpg)
Democratizing Stream Processing with Apache Kafka and KSQL - Part 1
In this article, author Michael Noll discusses the stream processing with KSQL, the streaming SQL engine for Apache Kafka. Topics covered include challenges of stateful stream processing and how KSQL addresses them, and how KSQL helps to bridge the world of streams and databases through streams and tables.