InfoQ Homepage AI, ML & Data Engineering Content on InfoQ
-
/articles/why-ml-projects-fail-production/en/smallimage/why-ml-projects-fail-production-thumbnail-1769072877092.jpg)
Why Most Machine Learning Projects Fail to Reach Production
In this article, the author diagnoses common failures in ML initiatives, including weak problem framing and the persistent prototype-to-production gap. The piece provides practical, experience-based guidance on setting clear business goals, treating data as a product, and aligning cross-functional teams for reliable, production-ready ML delivery.
-
/articles/agent-reinforcement-learning-apache-spark/en/smallimage/thumbnail-1769431334097.jpg)
Autonomous Big Data Optimization: Multi-Agent Reinforcement Learning to Achieve Self-Tuning Apache Spark
This article introduces a reinforcement learning (RL) approach grounded in Apache Spark that enables distributed computing systems to learn optimal configurations autonomously, much like an apprentice engineer who learns by doing. The author also implements a lightweight agent as a driver-side component that uses RL to choose configuration settings before a job runs.
-
/articles/ai-developers-rewriting-software-process/en/smallimage/ai-developers-rewriting-software-process-thumbnail-1768398859653.jpg)
Virtual Panel - AI in the Trenches: How Developers Are Rewriting the Software Process
This virtual panel brings together engineers, architects, and technical leaders to explore how AI is changing the landscape of software development. Practitioners share their insights on successes and failures when AI is incorporated into daily workflows, emphasizing the significance of context, validation, and cultural adaptation in making AI a sustainable element of modern engineering practices.
-
/articles/ai-assisted-development-series/en/smallimage/ai-assisted-development-thumb-image-1768388299998.jpg)
Article Series: AI-Assisted Development: Real World Patterns, Pitfalls, and Production Readiness
In this series, we examine what happens after the proof of concept and how AI becomes part of the software delivery pipeline. As AI transitions from proof of concept to production, teams are discovering that the challenge extends beyond model performance to include architecture, process, and accountability. This transition is redefining what constitutes good software engineering.
-
/articles/agentic-terminal-cli-agents/en/smallimage/agentic-terminal-cli-agents-thumbnail-1767616966806.jpg)
Agentic Terminal - How Your Terminal Comes Alive with CLI Agents
In this article author Sachin Joglekar discusses the transformation of CLI terminals becoming agentic where developers can state goals while the AI agents plan, call tools, iterate, ask for approval where needed, and execute the requests. He also explains the planning styles for three different CLI tools: Gemini, Claude, and Auto-GPT.
-
/articles/architects-ai-era/en/smallimage/Where-Architects-Sit-in-the-Era-of-AI-thumb-image-1765888921801.jpg)
Where Architects Sit in the Era of AI
As AI evolves from tool to collaborator, architects must shift from manual design to meta-design. This article introduces the "Three Loops" framework (In, On, Out) to help navigate this transition. It explores how to balance oversight with delegation, mitigate risks like skill atrophy, and design the governance structures that keep AI-augmented systems safe and aligned with human intent.
-
/articles/architecture-ai-augmented-change/en/smallimage/Architecture-in-a-Flow-of-AI-Augmented-Change-thumb-image-1765883092263.jpg)
Architecture in a Flow of AI-Augmented Change
While AI adoption is surging, most organizations fail to scale past pilots. The solution lies in organizational structure, not just technology. This article details how architects can enable "fast flow" by defining clear domains and guardrails. Learn how to shift from controlling outcomes to curating context, allowing AI to drive continuous, valuable business change.
-
/articles/nextgen-search-ai-opensearch-mcp/en/smallimage/nextgen-search-ai-thumbnail-1765802308459.jpg)
NextGen Search - Where AI Meets OpenSearch through MCP
In this article, authors Srikanth Daggumalli and Arun Lakshmanan discuss next-generation context-aware conversational search using OpenSearch and AI agents powered by Large Language Models (LLMs) and Model Context Protocol (MCP).
-
/articles/reducing-false-positives-retrieval-augmented-generation/en/smallimage/reducing-false-positives-retrieval-augmented-generation-thumbnail-1762786576150.jpg)
Reducing False Positives in Retrieval-Augmented Generation (RAG) Semantic Caching: a Banking Case Study
In this article, author Elakkiya Daivam discusses why Retrieval Augmented Generation (RAG) and semantic caching techniques are powerful levers for reducing false positives in AI powered applications. She shares the insights from a production-grade evaluation with 1,000 query variations tested across seven bi-encoder models.
-
/articles/training-data-preprocessing-for-text-to-video-models/en/smallimage/training-data-preprocessing-for-text-to-video-models-thumbnail-1762244675850.jpg)
Training Data Preprocessing for Text-to-Video Models
In this article, author Aleksandr Rezanov discusses the data preparation for generative text-to-image models to accelerate work on video generation services to be used in TV series and films. He explains how data is prepared and can serve as a starting point for creating custom datasets to develop proprietary models.
-
/articles/PDCA-AI-code-generation/en/smallimage/AI-code-gen-with-a-formal-and-repeatable-PDCA-cycle-thumb-image-1760692414560.jpg)
A Plan-Do-Check-Act Framework for AI Code Generation
AI code generation tools promise faster development but often create quality issues, integration problems, and delivery delays. A structured Plan-Do-Check-Act cycle can maintain code quality while leveraging AI capabilities. Through working agreements, structured prompts, and continuous retrospection, it asserts accountability over code while guiding AI to produce tested, maintainable software.
-
/articles/llms-evolution-ai-infrastructure/en/smallimage/llms-evolution-ai-infrastructure-thumbnail-1758096401785.jpg)
Disaggregation in Large Language Models: the Next Evolution in AI Infrastructure
Large Language Model (LLM) inference faces a fundamental challenge: the same hardware that excels at processing input prompts struggles with generating responses, and vice versa. Disaggregated serving architectures solve this by separating these distinct computational phases, delivering throughput improvements and better resource utilization while reducing costs.