BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Register Sign in

Unlock the full InfoQ experience

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources.

Log In

or

Don't have an InfoQ account?

Stay updated on topics and peers that matter to youReceive instant alerts on the latest insights and trends.
Quickly access free resources for continuous learningMinibooks, videos with transcripts, and training materials.
Save articles and read at anytimeBookmark articles to read whenever youre ready.

Logo - Back to homepage

News Articles Presentations Podcasts Guides

Topics

Development

Featured in Development

How to Use Apache Spark to Craft a Multi-Year Data Regression Testing and Simulations Framework

Vivek Yadav, an engineering manager from Stripe, shares his experience in building a testing system based on multi-year worth of data. He shares insights into why Apache Spark was the choice for creating such a system and how it fits in the "traditional" engineering practices.

All in development

Architecture & Design

Featured in Architecture & Design

Where Architects Sit in the Era of AI

As AI evolves from tool to collaborator, architects must shift from manual design to meta-design. This article introduces the "Three Loops" framework (In, On, Out) to help navigate this transition. It explores how to balance oversight with delegation, mitigate risks like skill atrophy, and design the governance structures that keep AI-augmented systems safe and aligned with human intent.

All in architecture-design

AI Infrastructure

Featured in AI, ML & Data Engineering

Lessons Learned From Building LinkedIn’s First Agent: Hiring Assistant

Karthik Ramgopal and Daniel Hewlett discuss the evolution of AI at LinkedIn, from simple prompt chains to a sophisticated distributed agent platform. They explain the transition to a supervisor-sub-agent model that enables parallel development and modular quality evaluation. and share a blueprint for building resilient, high-scale agentic systems.

All in ai-ml-data-eng

Culture & Methods

Featured in Culture & Methods

Changing Power Dynamics: What Senior Engineers Can Learn from Junior Engineers

Beth Anderson discusses the "power distance index" and its critical role in communication. Using the Korean Air Flight 801 tragedy as a case study, she explains the dangers of hierarchy-driven silence. She shares actionable frameworks for building the four stages of psychological safety, implementing reverse mentoring, and using PRs as tools for knowledge sharing rather than gatekeeping.

All in culture-methods

DevOps

Featured in DevOps

Fix SLO Breaches before They Repeat: an SRE AI Agent for Application Workloads

Bruno Borges discusses a paradigm shift in performance management: moving from manual tuning to automated SRE agents. He explains how to leverage the USE and jPDM methodologies alongside LLMs to reduce MTTR from hours to seconds. By utilizing MCP tools for real-time diagnostics and memory dump analysis, he shares how engineering leaders can scale systems while meeting strict objectives.

All in devops

Events

Helpful links

Choose your language

QCon AI New York 2025

Go from AI demos to real engineering impact. Learn to embed LLMs, govern & scale securely.

SOLD OUT!

InfoQ Architect Certification

Join Luca Mezzalira for this 5-week online cohort. Master socio-technical architecture leadership.

Save your spot.

QCon London 2026

Learn what works in AI, architecture, data, security & FinTech.

Early Bird ends Jan 13.

Learn how leading engineering teams run AI in production—reliably, securely, and at scale.

Early Bird ends Jan 13.

InfoQ Homepage Interviews Eli Collins on Hadoop

Eli Collins on Hadoop

Download

15:06

Bio

Eli Collins is a lead on Cloudera's Platform team, an active contributor to Apache Hadoop and member of its project management committee (PMC) at the Apache Software Foundation. Before that he worked on the ESX Hypervisor at VMware. Eli holds Bachelor’s and Master’s degrees in Computer Science from New York University and the University of Wisconsin-Madison, respectively.

About the conference

Software is changing the world; QCon aims to empower software development by facilitating the spread of knowledge and innovation in the enterprise software development community; to achieve this, QCon is organized as a practitioner-driven conference designed for people influencing innovation in their teams: team leads, architects, project managers, engineering directors.

Aug 17, 2012

Interview with

Eli Collins

by

Ari Zilka

BT