InfoQ Homepage AI, ML & Data Engineering Content on InfoQ
-
/articles/secure-mcp-server-aws/en/smallimage/secure-mcp-server-aws-thumbnail-1778765095736.jpg)
Building a Secure MCP Server on AWS for a Million-Company B2B Platform
We wanted to expose a B2B intelligence platform built on more than one million company profiles to an LLM client through an MCP server so a user can ask “find SaaS companies in Germany with 50-200 employees” and receive results through the LLM client. The engineering problem was: How do you make that workflow useful without creating an unsafe bridge between an LLM and production data?
-
/articles/time-series-storage-design/en/smallimage/Time-Series-Storage-Design-Choices-That-Shape-Cost-and-Performance-thumb-1778155792101.jpg)
Time-Series Storage: Design Choices That Shape Cost and Performance
Every time-series database makes a set of storage design decisions: how to lay out rows, when to compress, what to partition on. These decisions determine cost and query performance more than the choice of database itself. This article works through those fundamentals from first principles, using widely available tools like PostgreSQL and Apache Parquet to make each trade-off measurable.
-
/articles/local-first-ai-inference-cloud/en/smallimage/Local-First-AI-Inference-A-Cloud-Architecture-Pattern-for-Cost-Effective-Document-Processing-thumb-image-1778141518292.jpg)
Local-First AI Inference: a Cloud Architecture Pattern for Cost-Effective Document Processing
The Local-First AI Inference pattern routes 70–80% of documents to deterministic local extraction at zero API cost, reserving Azure OpenAI calls for edge cases and flagging low-confidence results for human review. Deployed on 4,700 engineering drawing PDFs, it cut API costs by 75% and processing time by 55%, while bounding errors through a human review tier.
-
/articles/micro-batch-streaming-lessons-learned/en/smallimage/micro-batch-streaming-lessons-learned-thumbnail-1777381781538.jpg)
From Batch to Micro-Batch Streaming: Lessons Learned the Hard Way in a Delta Index Pipeline
This article describes how a production delta-index pipeline migrated from scheduled batch to micro-batch Spark Structured Streaming. It covers why record-level streaming was rejected, how partition-based watermarks replaced fragile S3 completion markers, overlap-window correctness, and restart-as-design strategies for better predictability in object-store–based ingestion systems.
-
/articles/securing-autonomous-ai-agents-kubernetes/en/smallimage/securing-autonomous-ai-agents-kubernetes-thumbnail-1777378848477.jpg)
Securing Autonomous AI Agents on Kubernetes: Trust Boundaries, Secrets, and Observability for a New Category of Cloud Workload
Autonomous AI agents break Kubernetes security assumptions with dynamic dependencies, multi-domain credentials, and unpredictable resource use. This article covers production-tested patterns: Job-based isolation, Vault for scoped short-lived credentials, a four-phase trust model from shadow mode to autonomous operation, and observability for non-deterministic reasoning cycles.
-
/articles/ai-code-guardian/en/smallimage/ai-code-guardian-thumbnail-1776157217464.jpg)
CodeGuardian: a Model Context Protocol Server for AI-Assisted Code Quality Analysis and Security Scanning
CodeGuardian is an MCP server that extends AI coding assistants with comprehensive code quality and security analysis capabilities. By implementing eleven specialized tools, CodeGuardian enables developers to access enterprise-grade analysis directly through their AI assistant, eliminating context-switching and reducing friction in adopting secure coding practices.
-
/articles/mcp-java-architectural-strategy-llm-integrations/en/smallimage/mcp-java-architectural-strategy-llm-integrations-thumbnail-1776772947180.jpg)
MCP in the Java World: Bringing Architectural Strategy to LLM Integrations
Discover how the Model Context Protocol (MCP) Java SDK is establishing a new architectural discipline for enterprise LLM integrations. By defining explicit contracts and leveraging MCP servers as anti-corruption layers, it ensures governance, loose coupling, and security alignment with the JVM ecosystem and existing operational practices, moving integrations beyond fragility to resilience.
-
/articles/orchestrating-agentic-multimodal-ai-pipelines-apache-camel/en/smallimage/orchestrating-agentic-multimodal-ai-pipelines-apache-camel-thumbnail-1776763980414.jpg)
Orchestrating Agentic and Multimodal AI Pipelines with Apache Camel
In this article, author Vignesh Durai discusses how agentic and multimodal AI systems can be engineered using Apache Camel and LangChain4j technologies. The key components in the solution include LLM-based reasoning, retrieval-augmented generation (RAG), and image classification.
-
/articles/lakehouse-sql-identifier-rules/en/smallimage/lakehouse-sql-identifier-rules-thumbnail-1776241856705.jpg)
Lakehouse Tower of Babel: Handling Identifier Resolution Rules across Database Engines
Lakehouse architectures enable multiple engines to operate on shared data using open table formats such as Apache Iceberg. However, differences in SQL identifier resolution and catalog naming rules create interoperability failures. This article examines these behaviors and explains why enforcing consistent naming conventions and cross-engine validation is critical.
-
/articles/building-hierarchical-agentic-rag-systems/en/smallimage/building-hierarchical-agentic-rag-systems-thumbnail-1775040657142.jpg)
Building Hierarchical Agentic RAG Systems: Multi-Modal Reasoning with Autonomous Error Recovery
In this article, the author explores how hierarchical agentic RAG systems coordinate specialized workers through structured orchestration to improve accuracy, reliability, and explainability in complex enterprise analytics workflows. The article uses Protocol-H as a to show how deterministic routing, reflective retry, and modality-aware reasoning support safer multi-source query execution.
-
/articles/ai-agent-transport-layer/en/smallimage/ai-agent-transport-layer-thumbnail-1775031603285.jpg)
Stateful Continuation for AI Agents: Why Transport Layers Now Matter
Agent workflows make transport a first-order concern. Multi-turn, tool-heavy loops amplify overhead that is negligible in single-turn LLM use. Stateful continuation cuts overhead dramatically. Caching context server-side can reduce client-sent data by 80%+ and improve execution time by 15–29% .
-
/articles/optimization-in-automated-driving/en/smallimage/optimization-in-automated-driving-thumbnail-1774348229685.jpg)
Optimization in Automated Driving: from Complexity to Real-Time Engineering
In this article, author Avraam Tolmidis discusses technical architecture of autonomous vehicles, with focus on optimization techniques like context-aware sensor fusion and Model Predictive Control (MPC) solvers to help with processing raw sensor data into safe control commands.