InfoQ Homepage Incident Response Content on InfoQ
-
The Time it Wasn't DNS
Sean Klein explains how Azure handles massive outages using modern incident analysis. Moving past the "Five Whys," he shares how systemic factors—not operator error—caused the 2023 global WAN outage.
/presentations/incident-dns/en/mediumimage/sean-klein-medium-1781687984845.jpeg)
-
The Human Toll of Incidents & Ways to Mitigate it
Kyle Lexmond discusses the human side of major system failures. He shares psychological insights and architectural tactics for surviving high-pressure incident rooms.
/presentations/incident-response-mitigate/en/mediumimage/kyle-lexmond-medium-1779869148329.jpg)
-
The Ironies of A^2 I^2
J. Paul Reed explains the "ironies of automation" and AI in incident response. He discusses how reliance on AI can erode manual skills and camouflage system failures during high-stakes outages.
/presentations/automation-incidents-ai/en/mediumimage/medium-1778662652640.jpg)
-
AI-Powered SRE for Autonomous Incident Response
The presenters discuss incident response, how AI-enhanced SRE platforms connect signals from logs, metrics, traces, and historical incidents to enable autonomous decisions.
/presentations/ai-sre-incident-response/en/mediumimage/infoq-live-medium-1777368469045.jpg)
-
Week-Long Outage: Lifelong Lessons
Molly Struve shares a "murder mystery" outage story from a massive Elasticsearch upgrade. She explains why you need a rollback plan, how to check biases, and why leadership support is a stabilizer.
/presentations/outage-lessons/en/mediumimage/molly-struve-medium-1776864399990.jpeg)
-
An Incident Story: Tips for How Staff+ Engineers Can Impact Incidents
Erin Doyle discusses her experience with a critical 3-day-long incident and how she missed a key opportunity to help prevent the incident and how to build a culture to prevent similar situations.
/presentations/staff-engineer-incident/en/mediumimage/ErinDoyle-medium-1719919462925.jpg)
-
The Incident Lifecycle: How a Culture of Resilience Can Help You Accomplish Your Goals
Vanessa Huerta Granda describes how to apply resilience throughout the incident lifecycle in order to turn incidents into opportunities, looking at real-life examples.
/presentations/incident-lifecycle/en/mediumimage/VanessaHuertaGranda-medium-1718109200373.jpg)
-
Two Years of Incidents at Six Different Companies: How a Culture of Resilience Can Help You Accomplish Your Goals
Vanessa Huerta Granda looks at real-life examples of companies she has worked with who chose to invest in improving their incident programs and have seen it pay dividends.
/presentations/incidents-culture-resilience/en/mediumimage/VanessaHuertaGranda-mediu-1709639426940.jpg)
-
Comparing Apples and Volkswagens: the Problem with Aggregate Incident Metrics
Courtney Nash presents data from the Verica Open Incident Database (VOID) to demonstrate how aggregate incident metrics (MTTR) aren't representative of systems' resilience.
/presentations/mttr-resilience/en/mediumimage/Courtney-Nash-medium-1704801402530.jpg)
-
How Did It Make Sense at the Time? Understanding Incidents as They Occurred, Not as They are Remembered
Jacob Scott explores the basics of failure in complex systems, the theory and practice of how it made sense at the time, and actions to take.
/presentations/incidents-investigation/en/mediumimage/JacobScott-medium-1694080759044.jpg)
-
Rethinking Reliability: What You Can (and Can't) Learn from Incidents
Courtney Nash discusses research collected from the VOID, challenging standard industry practices for incident response and analysis, like tracking MMTR and using RCA methodology.
/presentations/incidents-void/en/mediumimage/Courtney-Nash-medium-1692694914223.jpg)
-
Incidents, PRRs, and Psychological Safety
Nora Jones discusses the context around PRRs and provides takeaways on how one can improve production reliability.
/presentations/production-reliability-incidents-prr/en/mediumimage/nora-jones-m-1658385386344.jpg)