InfoQ Homepage Apache Kafka Content on InfoQ
-
/articles/strimzi-the-gitops-way/en/smallimage/strimzi-gitops-small-logo-1664297048552.jpg)
Moving Kafka and Debezium to Kubernetes Using Strimzi - the GitOps Way
Deploying an Apache Kafka cluster to a Kubernetes is not an easy task. There are a lot of pieces to configure like the zookeeper, the Kafka cluster, topics, and users. Strimzi is a Kubernetes controller making the deployment process of Kafka a child game. Moreover, Strimzi lets you manage Kafka using GitOps methodology as everything is executed using a Kubernetes YAML file.
-
/articles/change-data-capture-debezium/en/smallimage/logo-1660294927422.jpg)
Debezium and Quarkus: Change Data Capture Patterns to Avoid Dual-Writes Problems
It’s common in microservices to write data in two places, a database and then send the content to another microservice. One approach to tackle this problem is dual writes, but you may lose data because of concurrent writes. Debezium is an open-source project for change data capture using the log scanner approach to avoid dual writes and communicate persisted data correctly between services.
-
/articles/quarkus-with-kafka-streams/en/smallimage/kafka-streams-and-quarkus-real-time-processing-events-small-1656083680635.jpg)
Kafka Streams and Quarkus: Real-Time Processing Events
Consuming Kafka messages is simple; you get them as long as they are produced, but nothing more. But if you need real-time processing of the data (filtering, joining, or manipulating events), just using the Kafka-consuming API might not be the best approach as the resulting code becomes complex. Kafka Streams and Quarkus are the perfect matches to start processing Kafka events in real-time.
-
/articles/data-with-quarkus-kafka/en/smallimage/UaUXvhM4-1651071035514.jpg)
Getting Started to Quarkus Reactive Messaging with Apache Kafka
How data is processed/consumed nowadays is different from how it was once practiced. In the past, data was stored in a database and it was batch processed for analytics. Apache Kafka is a distributed event store and stream-processing platform for storing, consuming, and processing data streams in real-time. In this post, we’ll learn how to produce and consume data using Apache Kafka and Quarkus.
-
/articles/microservices-inside-out/en/smallimage/turning-microservices-inside-out-small-image-1627634447180.jpg)
Turning Microservices Inside-Out
Turning microservices inside-out means moving past a single, request/response API to designing microservices with an inbound API for queries and commands, an outbound APIs to emit events, and a meta API to describe them both. A database can be supplemented with Apache Kafka via a connecting tissue such as Debezium.
-
/articles/database-audit-system-kafka/en/smallimage/Building-a-SQL-Database-Audit-System-Using-Maxwell-logo-small-1609154420148.jpg)
Building an SQL Database Audit System Using Kafka, MongoDB and Maxwell's Daemon
In this article, the author discusses the importance of a database audit logging system outside of traditional built-in data replication, using technologies like Kafka, MongoDB, and Maxwell's Daemon.
-
/articles/whats-the-next-step-for-data-management/en/smallimage/database-1605276684430.jpeg)
Beyond the Database, and beyond the Stream Processor: What's the Next Step for Data Management?
Databases have been around forever with the same shape: you make a request to your data and then you receive an answer. Now, stream processors came along with a different approach: data isn’t locked up, it is in motion. Understand how stream processors and databases relate and why there is an emerging new category of databases that focus on data that stays in place as well as data that moves.
-
/articles/real-time-api-kafka/en/smallimage/real-time-api-kafka-s-1602600487985.jpeg)
Real Time APIs in the Context of Apache Kafka
Events offer a Goldilocks-style approach in which real-time APIs can be used as the foundation for applications which is flexible yet performant; loosely-coupled yet efficient. Apache Kafka offers a scalable event streaming platform with which you can build applications around the powerful concept of events.
-
/articles/applied-probability-unstructured-events-theta-sketches/en/smallimage/applied-probability-unstructured-events-theta-sketches-s-1593199758266.jpg)
Applied Probability - Counting Large Set of Unstructured Events with Theta Sketches
In this article, author Ronen Cohen discusses the solution to processing the event data using Theta Sketches and technologies like HBase and Kafka.
-
/articles/building-scalable-iot-application-with-apache-kafka/en/smallimage/The-Kongo-Problem-Building-a-Scalable-IoT-Application-with-Apache-Kafka-logo-1581595522263.jpg)
The Kongo Problem: Building a Scalable IoT Application with Apache Kafka
In this article, author Paul Brebner discusses the best practices for developing IoT projects using Apache Kafka and Kafka Streams technologies and how to maximize Kafka scalability.
-
/articles/prometheus-monitor-applications-at-scale/en/smallimage/logo-1560849701397.jpg)
How to Use Open Source Prometheus to Monitor Applications at Scale
In this article, the author discusses how to collect metrics and achieve anomaly detection from streaming data using Prometheus, Apache Kafka and Apache Cassandra technologies.
-
/articles/apache-kafka-best-practices-to-optimize-your-deployment/en/smallimage/apache-kafka-best-practices-to-optimize-your-deployment-s-1539792752614.jpeg)
Apache Kafka: Ten Best Practices to Optimize Your Deployment
Author Ben Bromhead discusses the latest Kafka best practices for developers to manage the data streaming platform more effectively. Best practices include log configuration, proper hardware usage, Zookeeper configuration, replication factor, and partition count.