InfoQ Homepage Database Content on InfoQ
-
/articles/time-series-storage-design/en/smallimage/Time-Series-Storage-Design-Choices-That-Shape-Cost-and-Performance-thumb-1778155792101.jpg)
Time-Series Storage: Design Choices That Shape Cost and Performance
Every time-series database makes a set of storage design decisions: how to lay out rows, when to compress, what to partition on. These decisions determine cost and query performance more than the choice of database itself. This article works through those fundamentals from first principles, using widely available tools like PostgreSQL and Apache Parquet to make each trade-off measurable.
-
/articles/micro-batch-streaming-lessons-learned/en/smallimage/micro-batch-streaming-lessons-learned-thumbnail-1777381781538.jpg)
From Batch to Micro-Batch Streaming: Lessons Learned the Hard Way in a Delta Index Pipeline
This article describes how a production delta-index pipeline migrated from scheduled batch to micro-batch Spark Structured Streaming. It covers why record-level streaming was rejected, how partition-based watermarks replaced fragile S3 completion markers, overlap-window correctness, and restart-as-design strategies for better predictability in object-store–based ingestion systems.
-
/articles/lakehouse-sql-identifier-rules/en/smallimage/lakehouse-sql-identifier-rules-thumbnail-1776241856705.jpg)
Lakehouse Tower of Babel: Handling Identifier Resolution Rules across Database Engines
Lakehouse architectures enable multiple engines to operate on shared data using open table formats such as Apache Iceberg. However, differences in SQL identifier resolution and catalog naming rules create interoperability failures. This article examines these behaviors and explains why enforcing consistent naming conventions and cross-engine validation is critical.
-
/articles/replacing-database-sequences/en/smallimage/replacing-database-sequences-thumbnail-1774603762739.jpg)
Replacing Database Sequences at Scale without Breaking 100+ Services
The article discusses the challenges faced during a migration from a relational database to NoSQL, focusing on the importance of database sequences for unique identifiers. It outlines the development of a new sequence service using DynamoDB and a two-tier caching architecture.
-
/articles/jakartaee-12-milestone-2/en/smallimage/jakartaee-12-milestone-2-thumbnail-1770367607817.jpg)
Jakarta EE 12 Milestone 2: Advent of the Data Age along with Consistency and Configuration
Jakarta EE 12 Milestone 2 marks the beginning of the next generation of enterprise Java. It introduces Jakarta Query, a unified query language across Persistence, Data, and NoSQL, while aligning the platform with Java 21. This milestone focuses on integration, modernization, and improving developer productivity for cloud-native enterprise applications.
-
/articles/agent-reinforcement-learning-apache-spark/en/smallimage/thumbnail-1769431334097.jpg)
Autonomous Big Data Optimization: Multi-Agent Reinforcement Learning to Achieve Self-Tuning Apache Spark
This article introduces a reinforcement learning (RL) approach grounded in Apache Spark that enables distributed computing systems to learn optimal configurations autonomously, much like an apprentice engineer who learns by doing. The author also implements a lightweight agent as a driver-side component that uses RL to choose configuration settings before a job runs.
-
/articles/nextgen-search-ai-opensearch-mcp/en/smallimage/nextgen-search-ai-thumbnail-1765802308459.jpg)
NextGen Search - Where AI Meets OpenSearch through MCP
In this article, authors Srikanth Daggumalli and Arun Lakshmanan discuss next-generation context-aware conversational search using OpenSearch and AI agents powered by Large Language Models (LLMs) and Model Context Protocol (MCP).
-
/articles/reducing-false-positives-retrieval-augmented-generation/en/smallimage/reducing-false-positives-retrieval-augmented-generation-thumbnail-1762786576150.jpg)
Reducing False Positives in Retrieval-Augmented Generation (RAG) Semantic Caching: a Banking Case Study
In this article, author Elakkiya Daivam discusses why Retrieval Augmented Generation (RAG) and semantic caching techniques are powerful levers for reducing false positives in AI powered applications. She shares the insights from a production-grade evaluation with 1,000 query variations tested across seven bi-encoder models.
-
/articles/rag-with-spring-mongo-open-ai/en/smallimage/rag-with-spring-mongo-open-ai-thumbnail-1761292137478.jpg)
Building a RAG Application with Spring Boot, Spring AI, MongoDB Atlas Vector Search, and OpenAI
The RAG paradigm redefines AI: it combines generative models and business data for accurate, contextualised responses. The article shows how to integrate Spring Boot, Spring AI, MongoDB Atlas and OpenAI into a powerful and flexible pipeline capable of transforming the way businesses access and create value from data, with applications ranging from finance and healthcare to customer service.
-
/articles/ai-ml-data-engineering-trends-2025/en/smallimage/the-infoq-trends-report-500x500-logo-1758095546194.jpeg)
InfoQ AI, ML and Data Engineering Trends Report - 2025
This InfoQ Trends Report offers readers a comprehensive overview of emerging trends and technologies in the areas of AI, ML, and Data Engineering. This report summarizes the InfoQ editorial team’s and external guests' view on the current trends in AI and ML technologies and what to look out for in the next 12 months.
-
/articles/timeseries-db-rust/en/smallimage/time-series-database-using-open-source-rebuilding-InfluxDB-in-apache-arrow-rust-thumb-1757058446201.jpg)
Engineering a Time Series Database Using Open Source: Rebuilding InfluxDB 3 in Apache Arrow and Rust
At times, to evolve your product, you need to rebuild it from scratch. The article provides the story behind the rewrite of InfluxDB from scratch using a different programming language - Rust - and stack - Apache Flight, Data Fusion, Apache Arrow and Parquet (FDAP). It emphasises the benefits, as well as the mechanics behind its operation and the different versions of the product.
-
/articles/jakarta-ee-11-overview/en/smallimage/jakarta-ee-11-overview-thumbnail-1753179997354.jpg)
Jakarta EE 11 Overview: Virtual Threads, Records, and the Future of Persistence
Jakarta EE 11 delivers enhancements that include support for Java 17 and 21, integration with Java records and virtual threads, and the introduction of the Jakarta Data specification for unified SQL and NoSQL persistence. This release simplifies enterprise Java and establishes the groundwork for Jakarta EE 12, which will advance capabilities in data management.