InfoQ Homepage Database Content on InfoQ
-
/articles/continuous-authorization-cloud/en/smallimage/continuous-authorization-cloud-thumbnail-1781599988842.jpg)
Designing Continuous Authorization for Sensitive Cloud Systems
Most cloud systems make one authorization decision at login. Everything after runs on trust established at authentication time. For systems handling regulated data, that gap is where breaches happen. This article presents a continuous authorization architecture covering risk-tiered evaluation, behavioral baselines, privacy-preserving audit trails, and a phased and incremental rollout.
-
/articles/spark-oom-kubernetes-misconfigurations/en/smallimage/spark-oom-kubernetes-misconfigurations-thumbnail-1780044756757.jpg)
Two Misconfigurations That Caused Spark OOM Failures on Kubernetes
After migrating Spark pipelines to Azure Kubernetes Service, two infrastructure settings interacted destructively: spark.kubernetes.local.dirs.tmpfs=true backed shuffle spill with RAM instead of disk, and a hard podAffinity rule forced all executors onto one node. Together, they caused repeated OOM kills invisible to standard diagnostics.
-
/articles/vector-search-hybrid-retrieval-rag/en/smallimage/vector-search-hybrid-retrieval-rag-thumbnail-1779972811121.jpg)
Why Vector Search Alone Isn't Enough: Hybrid Retrieval for RAG
In this article, author Aaditya Chauhan discusses the limitations of RAG pipelines based purely on vector search and how an internal omni-search application using Reciprocal Rank Fusion (RRF) that combines BM25 and vector results, can enhance the search solution.
-
/articles/time-series-storage-design/en/smallimage/Time-Series-Storage-Design-Choices-That-Shape-Cost-and-Performance-thumb-1778155792101.jpg)
Time-Series Storage: Design Choices That Shape Cost and Performance
Every time-series database makes a set of storage design decisions: how to lay out rows, when to compress, what to partition on. These decisions determine cost and query performance more than the choice of database itself. This article works through those fundamentals from first principles, using widely available tools like PostgreSQL and Apache Parquet to make each trade-off measurable.
-
/articles/micro-batch-streaming-lessons-learned/en/smallimage/micro-batch-streaming-lessons-learned-thumbnail-1777381781538.jpg)
From Batch to Micro-Batch Streaming: Lessons Learned the Hard Way in a Delta Index Pipeline
This article describes how a production delta-index pipeline migrated from scheduled batch to micro-batch Spark Structured Streaming. It covers why record-level streaming was rejected, how partition-based watermarks replaced fragile S3 completion markers, overlap-window correctness, and restart-as-design strategies for better predictability in object-store–based ingestion systems.
-
/articles/lakehouse-sql-identifier-rules/en/smallimage/lakehouse-sql-identifier-rules-thumbnail-1776241856705.jpg)
Lakehouse Tower of Babel: Handling Identifier Resolution Rules across Database Engines
Lakehouse architectures enable multiple engines to operate on shared data using open table formats such as Apache Iceberg. However, differences in SQL identifier resolution and catalog naming rules create interoperability failures. This article examines these behaviors and explains why enforcing consistent naming conventions and cross-engine validation is critical.
-
/articles/replacing-database-sequences/en/smallimage/replacing-database-sequences-thumbnail-1774603762739.jpg)
Replacing Database Sequences at Scale without Breaking 100+ Services
The article discusses the challenges faced during a migration from a relational database to NoSQL, focusing on the importance of database sequences for unique identifiers. It outlines the development of a new sequence service using DynamoDB and a two-tier caching architecture.
-
/articles/jakartaee-12-milestone-2/en/smallimage/jakartaee-12-milestone-2-thumbnail-1770367607817.jpg)
Jakarta EE 12 Milestone 2: Advent of the Data Age along with Consistency and Configuration
Jakarta EE 12 Milestone 2 marks the beginning of the next generation of enterprise Java. It introduces Jakarta Query, a unified query language across Persistence, Data, and NoSQL, while aligning the platform with Java 21. This milestone focuses on integration, modernization, and improving developer productivity for cloud-native enterprise applications.
-
/articles/agent-reinforcement-learning-apache-spark/en/smallimage/thumbnail-1769431334097.jpg)
Autonomous Big Data Optimization: Multi-Agent Reinforcement Learning to Achieve Self-Tuning Apache Spark
This article introduces a reinforcement learning (RL) approach grounded in Apache Spark that enables distributed computing systems to learn optimal configurations autonomously, much like an apprentice engineer who learns by doing. The author also implements a lightweight agent as a driver-side component that uses RL to choose configuration settings before a job runs.
-
/articles/nextgen-search-ai-opensearch-mcp/en/smallimage/nextgen-search-ai-thumbnail-1765802308459.jpg)
NextGen Search - Where AI Meets OpenSearch through MCP
In this article, authors Srikanth Daggumalli and Arun Lakshmanan discuss next-generation context-aware conversational search using OpenSearch and AI agents powered by Large Language Models (LLMs) and Model Context Protocol (MCP).
-
/articles/reducing-false-positives-retrieval-augmented-generation/en/smallimage/reducing-false-positives-retrieval-augmented-generation-thumbnail-1762786576150.jpg)
Reducing False Positives in Retrieval-Augmented Generation (RAG) Semantic Caching: a Banking Case Study
In this article, author Elakkiya Daivam discusses why Retrieval Augmented Generation (RAG) and semantic caching techniques are powerful levers for reducing false positives in AI powered applications. She shares the insights from a production-grade evaluation with 1,000 query variations tested across seven bi-encoder models.
-
/articles/rag-with-spring-mongo-open-ai/en/smallimage/rag-with-spring-mongo-open-ai-thumbnail-1761292137478.jpg)
Building a RAG Application with Spring Boot, Spring AI, MongoDB Atlas Vector Search, and OpenAI
The RAG paradigm redefines AI: it combines generative models and business data for accurate, contextualised responses. The article shows how to integrate Spring Boot, Spring AI, MongoDB Atlas and OpenAI into a powerful and flexible pipeline capable of transforming the way businesses access and create value from data, with applications ranging from finance and healthcare to customer service.