InfoQ Homepage Infrastructure Content on InfoQ
-
/articles/time-series-storage-design/en/smallimage/Time-Series-Storage-Design-Choices-That-Shape-Cost-and-Performance-thumb-1778155792101.jpg)
Time-Series Storage: Design Choices That Shape Cost and Performance
Every time-series database makes a set of storage design decisions: how to lay out rows, when to compress, what to partition on. These decisions determine cost and query performance more than the choice of database itself. This article works through those fundamentals from first principles, using widely available tools like PostgreSQL and Apache Parquet to make each trade-off measurable.
-
/articles/micro-batch-streaming-lessons-learned/en/smallimage/micro-batch-streaming-lessons-learned-thumbnail-1777381781538.jpg)
From Batch to Micro-Batch Streaming: Lessons Learned the Hard Way in a Delta Index Pipeline
This article describes how a production delta-index pipeline migrated from scheduled batch to micro-batch Spark Structured Streaming. It covers why record-level streaming was rejected, how partition-based watermarks replaced fragile S3 completion markers, overlap-window correctness, and restart-as-design strategies for better predictability in object-store–based ingestion systems.
-
/articles/mcp-java-architectural-strategy-llm-integrations/en/smallimage/mcp-java-architectural-strategy-llm-integrations-thumbnail-1776772947180.jpg)
MCP in the Java World: Bringing Architectural Strategy to LLM Integrations
Discover how the Model Context Protocol (MCP) Java SDK is establishing a new architectural discipline for enterprise LLM integrations. By defining explicit contracts and leveraging MCP servers as anti-corruption layers, it ensures governance, loose coupling, and security alignment with the JVM ecosystem and existing operational practices, moving integrations beyond fragility to resilience.
-
/articles/sovereign-fault-domains-cloud-resilience/en/smallimage/sovereign-fault-domains-cloud-resilience-thumbnail-1776430533702.jpg)
When a Cloud Region Fails: Rethinking High Availability in a Geopolitically Unstable World
Sovereign fault domains are failure boundaries defined by legal, political, or physical jurisdiction rather than hardware topology. The article maps geopolitical events to known distributed-systems failure modes, argues multi-region should replace multi-AZ as the HA baseline for systems crossing jurisdictions, and outlines design patterns, chaos experiments, and an ALE model to justify the spend.
-
/articles/replacing-database-sequences/en/smallimage/replacing-database-sequences-thumbnail-1774603762739.jpg)
Replacing Database Sequences at Scale without Breaking 100+ Services
The article discusses the challenges faced during a migration from a relational database to NoSQL, focusing on the importance of database sequences for unique identifiers. It outlines the development of a new sequence service using DynamoDB and a two-tier caching architecture.
-
/articles/agent-reinforcement-learning-apache-spark/en/smallimage/thumbnail-1769431334097.jpg)
Autonomous Big Data Optimization: Multi-Agent Reinforcement Learning to Achieve Self-Tuning Apache Spark
This article introduces a reinforcement learning (RL) approach grounded in Apache Spark that enables distributed computing systems to learn optimal configurations autonomously, much like an apprentice engineer who learns by doing. The author also implements a lightweight agent as a driver-side component that uses RL to choose configuration settings before a job runs.
-
/articles/durable-objects-handle-inflight-requests/en/smallimage/durable-objects-handle-inflight-requests-thumbnail-1769161248259.jpg)
One Cache to Rule Them All: Handling Responses and In-Flight Requests with Durable Objects
Traditional caching fails to stop "thundering herds" where multiple clients trigger the same work during a miss. This article proposes using Cloudflare Durable Objects to treat in-flight work and finished results as two states of one cache entry. By routing to a single owner, systems eliminate redundant tasks. This pattern replaces complex locks with simple promises, simplifying the system design.
-
/articles/reducing-false-positives-retrieval-augmented-generation/en/smallimage/reducing-false-positives-retrieval-augmented-generation-thumbnail-1762786576150.jpg)
Reducing False Positives in Retrieval-Augmented Generation (RAG) Semantic Caching: a Banking Case Study
In this article, author Elakkiya Daivam discusses why Retrieval Augmented Generation (RAG) and semantic caching techniques are powerful levers for reducing false positives in AI powered applications. She shares the insights from a production-grade evaluation with 1,000 query variations tested across seven bi-encoder models.
-
/articles/building-resilient-platforms-mission-critical-infrastructure/en/smallimage/building-resilient-platforms-mission-critical-infrastructure-thumbnail-1762169439707.jpg)
Building Resilient Platforms: Insights from over Twenty Years in Mission-Critical Infrastructure
Building resilient platforms requires understanding the art and science of creating infrastructure that others depend on for critical applications. This perspective applies to anyone who builds software consumed by others at scale. Whether developing infrastructure platforms, software development platforms, or messaging systems, principles address how to build software that others consume at scale
-
/articles/llms-evolution-ai-infrastructure/en/smallimage/llms-evolution-ai-infrastructure-thumbnail-1758096401785.jpg)
Disaggregation in Large Language Models: the Next Evolution in AI Infrastructure
Large Language Model (LLM) inference faces a fundamental challenge: the same hardware that excels at processing input prompts struggles with generating responses, and vice versa. Disaggregated serving architectures solve this by separating these distinct computational phases, delivering throughput improvements and better resource utilization while reducing costs.
-
/articles/ransomware-resilient-storage-cyber-defense/en/smallimage/ransomware-resilient-storage-cyber-defense-thumbnail-1755589602893.jpg)
Ransomware-Resilient Storage: the New Frontline Defense in a High-Stakes Cyber Battle
Cybersecurity has evolved, with ransomware now primarily targeting data storage and backups. To combat this, modern defense strategies focus on making storage systems more resilient. Key tactics include using immutable storage that prevents data from being altered or deleted, employing AI-powered detection, and implementing air-gapping to create isolated, tamper-proof recovery points.
-
/articles/zero-downtime-cloud-upgrades/en/smallimage/zero-downtime-cloud-upgrades-thumbnail-1754398638654.jpg)
Zero-Downtime Critical Cloud Infrastructure Upgrades at Scale
Engineers can avoid common pitfalls in large-scale infrastructure upgrades by studying others' experiences. The article provides lessons learned from big firms like eBay and Snowflake, offering solutions for legacy systems, performance validation, and rollback planning. It emphasizes systematic preparation and clear communication to handle challenges and ensure zero-downtime upgrades at scale.