BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Register Sign in

Unlock the full InfoQ experience

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources.

Log In

or

Don't have an InfoQ account?

Stay updated on topics and peers that matter to youReceive instant alerts on the latest insights and trends.
Quickly access free resources for continuous learningMinibooks, videos with transcripts, and training materials.
Save articles and read at anytimeBookmark articles to read whenever youre ready.

Logo - Back to homepage

News Articles Presentations Podcasts Guides

Topics

Development

Featured in Development

Accelerating Performance by Incrementally Integrating Rust into Existing Codebase

Lily Mara explains how to avoid high-risk software rewrites through incremental FFI refactoring. She shares how engineering teams can replace Python bottlenecks with Rust via PyO3, demonstrating how to achieve dramatic function-level speedups, seamless integration testing, and meaningful infrastructure cost savings without microservice overhead.

All in development

Architecture & Design

Featured in Architecture & Design

Agentic AI Architecture: Current Challenges and Future Opportunities

Agentic AI architecture extends microservices and cloud-native architectures with AI agents that work with context, memory, and protocols such as MCP and A2A. The article covers bounded-context integration, task-dependent coordination strategies, new security risks, and observability needs. It expects gradual adoption alongside existing architectures, with platforms absorbing common components.

All in architecture-design

AI Infrastructure

Featured in AI, ML & Data Engineering

Batch Intelligence at Scale: Cost-Efficient Multi-Agent LLM Workflows with Built-In Resilience

Aditya Mulik explains how Walmart Global Tech deployed multi-agent LLM workflows across 4,700 stores for nightly batch inventory forecasting. He discusses tackling production hurdles - schema violations, cross-department data mismatches, and network drops - via MCP grounding and loop agents, while slashing cloud inference costs from $100k+ to $10k per month shifting from PTUs to pay-as-you-go.

All in ai-ml-data-eng

Culture & Methods

Featured in Culture & Methods

The Socio-Technical Staff+ Engineer: Architecture, Culture and Organisational Change

Vanessa Formicola shares how technical leaders can move beyond "wishful architecture" by practicing holistic socio-technical engineering. She discusses why engineering leaders must account for organizational maturity, reward structures, geopolitical climate, and team dynamics, framing the staff-plus role as designing the conditions for technology to thrive rather than just picking tools.

All in culture-methods

DevOps

Featured in DevOps

Wrangling Telemetry at Scale: a Guide to Self-Hosted Observability

Colin Douch shares battle-tested strategies for building and scaling self-hosted observability stacks. He explains how to effectively correlate metrics, structured logs, and distributed traces using open-source tooling like Prometheus, VictoriaMetrics, and Jaeger, while navigating the engineering trade-offs of avoiding SaaS vendor lock-in.

All in devops

Events

Helpful links

Choose your language

AI Engineering Certification

Work through your production AI decisions with senior engineers from other companies. Apply their perspectives to challenges from your own work.
Online. Register Now.

AI Engineering Certification

Building RAG pipelines or AI agents? Work through retrieval, context, evals, and reliability decisions with experienced practitioners.
Online. Register Now.

High-Performing Teams Certification

Compare team design, delivery flow, AI roles, and metrics with experienced practitioners.
Online. Register Now.

Architect Certification

Distributed systems, decentralized decisions, platform engineering, and AI architecture.
Online. Register Now.

QCon San Francisco

What's working across AI, architecture, and leadership, from the teams doing it.
Register. Early bird ends Nov 10.

QCon AI New York

Production AI across agents, context, evals, security, and infrastructure, from the senior engineers building it.
Register. Early bird ends Nov 10.

QCon London

What early-adopter teams have proven in production, across 15 engineering tracks.
Register. Early bird ends Nov 10.

QCon AI Boston

Learn from the decisions and trade-offs behind AI systems already running in production.
Registration open.

InfoQ Homepage Streaming Content on InfoQ

Presentations

RSS Feed

Newer Older

Samza in LinkedIn: How LinkedIn Processes Billions of Events Everyday in Real-time

Neha Narkhede of Kafka fame shares the experience of building LinkedIn's powerful and efficient data pipeline infrastructure around Apache Kafka and Samza to process billions of events every day.

Neha Narkhede
on Dec 05, 2014

Icon

49:40
Mantis: Netflix's Event Stream Processing System

The authors discuss Netflix's new stream processing system that supports a reactive programming model, allows auto scaling, and is capable of processing millions of messages per second.

Justin Becker Danny Yuan
on Dec 04, 2014

Icon

41:50
High Throughput Stream Processing with ACID Guarantees

Terence Yim from Continuuity showcases a transactional stream processing system that supports full ACID properties without compromising scalability and high throughput.

Terence Yim
on Dec 04, 2014

Icon

48:46
TSAR: How to Count Tens of Billions of Daily Events in Real Time Using Open Source Technologies

Gabriel Gonzalez introduces TSAR (TimeSeries AggregatoR), a service for real-time event aggregation designed to deal with tens of billions of events per day at Twitter.

Gabriel Gonzalez
on Nov 22, 2014

Icon

27:24
Samza: Real-time Stream Processing at LinkedIn

Chris Riccomini discusses: Samza's feature set, how Samza integrates with YARN and Kafka, how it's used at LinkedIn, and what's next on the roadmap.

Chris Riccomini
on Nov 28, 2013

Icon

44:23
Complex Event Processing: DSL for High Frequency Trading

Richard Tibbetts discusses Complex Event Processing in the context of High Frequency Trading and the advantages of using high level DSLs, followed by the case study of a system built with StreamBase.

Richard Tibbetts
on Jul 11, 2011

Icon

52:49
Event Patterns

Ian Cartwright presents some of his work (developed with Martin Fowler) on event patterns: Event Sourcing, Event Collaboration, Parallel Model, and Retroactive Event.

Ian Cartwright
on May 17, 2007

Icon

29:50

Newer Presentations

Older Presentations

BT