InfoQ Homepage Large language models Content on InfoQ
-
Scaling LLM-Based Ranking Systems for Latency-Critical Search & Recommendation Workloads
Sundara Raman Ramachandran explains how LinkedIn scales LLM-based ranking to 1.3B members, detailing model distillation to 400M parameters, context compression, and deep Python/GPU runtime tuning.
-
Opening the Black Box: Critical Explainability for AI Agent Tool Selection with Kiji Inspector
Hannes Hapke explains how to move beyond post-hoc narratives by using Sparse Autoencoders with Kiji Inspector to uncover true causal mechanistic interpretability in production LLM agent decisions.
-
Batch Intelligence at Scale: Cost-Efficient Multi-Agent LLM Workflows with Built-In Resilience
Aditya Mulik shares how Walmart scaled resilient multi-agent LLM batch workflows across 4,700 stores, cutting monthly inference costs by 90% while enforcing semantic validation and bounded retries.
-
The Ladder is Missing Rungs: Engineering Progression When AI Ate the Middle
Alasdair Allan shares data on the "missing rungs" of software engineering, explaining how AI coding tools stunt junior developer skill growth and disrupt future technical leadership pipelines.
-
Serving LLMs at Scale: the Hidden KV Cache Advantage
Khawaja Shams explains how distributed systems engineering solves modern LLM inference bottlenecks, covering KV caching, memory cliffs, prompt prefilling, and disaggregated execution.
-
Beyond the Prototype: Scaling Framework Agnostic AI Agent Infrastructure with Ray
The speakers explain how to run and autoscale LLM agents in production using Ray, detailing system architecture, control planes, multi-region deployments, and CI/CD pipelines with LLM evals.
-
Building Reusable Evaluation Frameworks for Agentic AI Products
Susan Chang shares how Elastic built a production-grade AI agent evaluation framework, detailing lessons on tracing, LLM-as-a-judge, programmatic evals to prevent regression across engineering teams.
-
Building GenAI Platform at DoorDash
Swaroop Chitlur and Sidd Kodwani explain how DoorDash built and scaled an enterprise GenAI platform, revealing key architectural bets, pivots, and operational principles driving real business impact.
-
From Consumers to Builders: Turning 200 of our Team into Agent Creators in Two Weeks
Ben Maraney explains how Forter enabled R&D teams to build internal AI agents fast by simplifying tools, using custom MCP servers, and removing organizational roadblocks to maximize engineering ROI.
-
Designing Fast, Delightful UX with LLMs for Mobile Frontends
Balakrishnan Ramdoss explains how to scale AI-driven conversational app experiences using server-driven UI, BFF architectures, optimized prompting & high-performance on-device models for low latency.
-
The Agent Harness: Control Planes, Invariants, and Approval Boundaries for Production AI Agents
OpenAI’s Vinoth Govindarajan explains how to build reliable AI agent harnesses in production, focusing on state ownership, mutation ordering, scoped authority, and user-visible proof of action.
-
Decision Models in Agentic Architectures: from Production to Agent Skills
Alex Porcelli explains how combining deterministic DMN decision models with agentic AI resolves enterprise AI challenges around accountability, rule governance, and non-deterministic logic.