BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Doing Safe-to-Fail Experiments

Doing Safe-to-Fail Experiments

This item in japanese

Bookmarks

Safe-to-fail experiments can be used in complex environments to probe, sense, and respond. You have to know what success and failure look like and need to be able to dampen or amplify the effect of probing to handle potential failures. Safe-to-fail experiments can help you to deal with risks and uncertainty, learn, and keep your options open.

Liz Keogh, an independent Lean and Agile consultant, spoke about safe-to-fail at the European Testing Conference 2017. InfoQ is covering the conference with Q&As, summaries and articles.

Keogh started her talk with a short introduction of Cynefin. She stated that most IT initiatives are in the complex domain, where you can use a safe-to-fail approach, something she described in her blog post Cynefin for developers:

In a complex environment, you probe, sense and respond. You do something that can fail, safely, and it tells you things about the environment which you respond to, changing the environment. This is the land of high-feedback, risk and innovation.

In this domain, because the outcomes we look for keep changing, we can’t merely apply our expert practices and expect success. Instead, we have to change the practices we use based on what we learn. In this domain, we have emergent practices.

The Agile manifesto came into being because of this domain. We couldn’t get everything right up-front, so we started creating feedback loops within our process instead.

In the InfoQ summary experiment using behavior driven development, Keogh explained how you can measure complexity on a scale from 1 to 5:

5. Nobody has ever done it before

4. Someone outside the org. has done it before (probably a competitor)

3. Someone in the company has done it before

2. Someone in the team has done it before

1. We all know how to do it

You can map these complexity levels on Cynefin. Level 1 belongs to the obvious domain, level 2 and 3 to the complicated domain, and level 4 and 5 to the complex domain.

Keogh suggested that we should take the risky newest stuff first. This approach helps you to build trust with your stakeholders, as usually they worry about the risks and want to see them addressed. If your stakeholders don’t trust you, Keogh recommends delivering a nicely complicated 3 in order to gain their trust, instead of going for 4s or 5s.

A safe-to-fail probe has to have a way of knowing that it’s succeeding or failing. As you don’t know what will happen, you must be able to dampen or amplify the effect of probing, said Keogh. Safe-to-fail is not about avoiding failure completely, but you need to be able to handle potential failures.

Earlier InfoQ interviewed Tiago Garcez and asked him how a safe-to-fail experiment should look like:

(...) make sure before you start any initiative where there are considerable risk, that you use controlled experiments where you know what success and failure look like, so that you can evaluate potential solutions or ways forward. Such an approach keeps failure from being expensive or mission critical, while still providing opportunities to learn (if you structured the experiments in a coherent way).

In high uncertainty scenarios, provide coherence, not tests, said Keogh. Under such circumstances, "tests" can’t be certain; they’re examples of what might happen instead. But testers are still really good at coming up with those examples, and that mindset remains essential.

Keogh referred to the work on real options from Olav Maassen and Chris Matts, described in their book Commitment. Experimenting is a way to keep your options open, said Keogh. For instance, a rollback is an option that you can use when things go wrong.

Keogh also mentioned the Pachinsky Principles:

  • Seek out new ideas and try new things
  • When trying something new, do it on a scale where failure is survivable
  • Seek out feedback and learn from your mistakes as you go along

Keogh concluded her talk by giving two suggestions in looking for ways to make it safe to fail:

  • Use ritual dissent from Cognitive Edge, a technique to test and enhance ideas by challenging them
  • "Ask a tester", if you want to know if something is safe-to-fail. Testers are very good at this

Rate this Article

Adoption
Style

BT