Amr Elssamadisy: Why Agile Works

1. Why does Agile work?

Does Agile work? Agile doesn’t work all the time. What I’ve come to believe is best illustrated by this hypothetical question I ask people in the class. Usually, if I’m in a class or a presentation or a group, there are a lot of people in the room. Many of them are new and many of them are very experienced gray hairs that have been in the business for a long time.

I pose this hypothetical question: "Team, I’m your customer. I come to you and I say: ‘I’d like you to build this software for me.’ We’re a new team, we come together, we decide what hardware we were going to use, what software languages and so on and so forth. I give you the requirements, you go build something and you bring it back and I say ‘That’s what I asked for, but it’s not exactly what I needed. Can I have something else?’ We go through the process of building the software and at the end of a year, we’re done. I take out my checkbook and I write my check and I say ‘Thank you, team. Here is your money for building the code.’ Then I take that code and throw it out."

Now, remember, this is hypothetical. Then I ask the team "I’d like you to build the exact same software again, but nothing is going to change. We are going to have the exact same people. We’re going to build the same features, we’re going to use the same hardware, and we’re going to use the same software. We’re not going to do something different. Hypothetically, again, we’re setting everything the same - nothing is changed. How long will it take you? Will it take you another 12 months to rebuild the system?"

There are outliers, but invariably we feel from our experience that it’s going to take significantly less. Some people will say it will take 60-70% amount of time. Some people will say it will take 30% amount of time. It’s a guess, but it’s all under 100% and significantly under 100%. That’s a pretty big chunk, whatever it is - 50, 60, 70% of the time. What’s changed? My hypothesis and what I generally get agreement with is what we’ve learned. We’ve learned about the requirements, we’ve learned about each other working together, we’ve learned about the hardware and the software. We’ve built this thing.

For any developer who’s written code and lost it and had to rewrite it, it doesn’t take the same amount of time to rewrite it. Learning is a huge part of software development. Part of the pains from Waterfall and traditional is we didn’t learn. When things are cycled out, when you do requirements first and then you do design and then you do coding, it could be six months - a year down the road until you say "Oh, there is a problem. It needs to come back!" The cycle is so large and we don’t learn. Your question was "Why does Agile work so well?"

Whether Avertently or inavertently everything we do is in loops. We do test driven development. We write a test and within a few minutes we try to get that test to pass. In iterations we decide what we’re going to do for the iterations for the next two weeks and we go try to do it. Then, we come back and say "Did we meet it? - Yes. Did we not meet it? - OK, let’s learn from that." The daily standup meetings, the retrospectives, so many things in Agile are cyclical in nature and that has a huge effect on learning.

We learn fast, we learn from failure and we learn from success, but because we cycle over everything that we do, we have many more opportunities to learn. Teams that actually take advantage of that cycle and learn become much more productive. But learning is not automatic, so you can still do the practices and not learn. And if you do the practices, you go through the motions and you don’t take the time to learn and don’t take the time to face and confront the problems that come up you can be "doing" Agile but you won’t be getting the benefits.

2. I talked to a project manager at breakfast this morning and she told me about her experience with adopting Agile into her company. Her estimate was that they were moving about at half speed that they were when they were doing Waterfall before, so they slowed down with Agile. It’s a pretty well known fact that some Agile teams are a lot better than others and by a lot better we’re talking orders of magnitude better. In this case, the Agile team has a lot of work. There are obviously different things that are making this happen. Why are some Agile teams so much better than others?

God only knows. I couldn’t tell you exactly why this particular lady has her problem. We all know in adoption there is a J curve. When you learn something, you slow down. If you stop doing anything new, stop providing value, you could go on in the learning curve, but I don’t think that’s what she was communicating. You are right, there are orders of magnitude differences between the most productive and the least productive of us, whether Agile or non-Agile.

When we talk about software development, I could tell you from the experiences that I’ve had that there are some things, some impediments to learning. Remember, if we go back, learning is the biggest part of software development and my guess is that we can go a lot faster than 70%, we can cut it much more if we really find a way to learn from everything that we do. What’s the difference between the Agile teams? Are they getting in their way? Are all these cycles that we’re talking about?

There are some obvious things that impede learning and there are some things that are not obvious. The short answer is I’d bet you that they are getting in their own way and there is a huge impendence to learning. If you do these cycles without learning, you just create a lot of waste a lot faster. There are a lot of quotes that say: "Agile is not about solving problems, it’s about really throwing problems in your face and rubbing them in your face".

You have a choice in that if you confront the issues and solve them, then you’ll get better. But if you don’t solve them, it will be much more painful and much slower because what you’ve done, everything is now in your face and all these problems are there and they are overwhelming many times.

3. Tell us more about that confrontation process, that learning process, the process of transforming pain into learning.

I have an image in my head. We have a cycle; we start out by setting the goal. Before we start anything we need clarity on the goal. Test Driven Development is a clear test: it passes or fails. The iterations? Many teams starting out don’t really have clarity on what does it mean to be done. If you are not clear on the goal, that really impedes your learning. There is a cycle and the first thing is "What are we going to do?" Then, the next thing down the cycle is "We’re going to do something about it."

If we are developing we are going to write code. If we’re iterating we’re going to go work as a team to solve the problem. If it’s a retrospective, we’re going to take an action to change the way we work over the next cycle. So, we’re doing something about it. Then, at the end of the cycle, however long it is, we come back and we say "All right, did we meet the goal or not?" That’s our test. If we did meet the goal, we succeeded, if we didn’t we cycle again. When this happens correctly, we learn at the test and we learn whether we succeed or fail.

"Hey, we succeeded! What did we do right?" - we’ve learned. Or we failed - what did we do wrong? How can we change it? You asked about confront. If something goes wrong, it’s not really immediately obvious or comfortable to solve the problem. Many companies have an "us vs. them". You hit a problem - "It was their fault. It’s was the manager’s fault, it was the tester’s fault. Whatever! But it’s not my fault." The problem is if it’s not your fault, you can’t do anything about it.

There are many things, but the issue of confront is very uncomfortable to all of us. There is an exercise that we run in our sessions where we ask our participants to just sit knee to knee and just look at each other in the face calmly for a minute. It’s extremely uncomfortable for many people. Confronting in uncomfortable without failure, but if you put failure on top of it, it’s very difficult and many of us will not confront the issue.

A really good example is continuous integration. Why does continuous integration work so well? When it works, it works because we have tests that fail. Whenever we break the test it’s thrown in our face and we go fix the test, we do something about it. But not everybody goes and does something about it. I’ve been on teams, I’ve worked with teams that just said, "You know what? Let’s just take the test out. Because that’s a really big problem we don’t want to deal with." First of all, they’ve just lost an opportunity to learn and they’ve gone around the issue. The issue is still there.

It won’t go away, it will probably grow because now it’s invisible. It’s difficult to confront and it’s a learnable skill, but this brings us there. I said there were obvious impediments and non-obvious impediments. The most important impediments, things that trip us up, nine times out of 10 are human factors, human dynamics that were not trained in - "We don’t know." They are learnable skills, they are easy. Well, they are not easy, they are simple, understandable, but they cause you and they are all about being aware, being clear with yourself, being clear with others and confronting problems.

It’s extremely difficult to do so. That’s really the difference between a high-performance team and a mediocre team: their ability to see things for what they are and react, respond correctly and go through the issue, learn from it.

4. This sounds very plausible sitting here in this nice comfortable room. Can you give us an example from your experience out in the real world where applying these ideas about increasing the rate of learning has actually helped a real company?

Without naming names, there is a common thing that we do with our clients. At Gemba we’re really focused on results, as we all should be. Agile is a means to an end, but we are not only focused on results, we feel focused on immediate results. There is an offering that we give regularly - we go to clients and we’ll work with them on their issue. We set them up.

We say "All right. We’ve got your executives, here they are in the beginning, and we’re going to give you this problem to solve. You are going to come up with the problem, by the way. I’m not going to assign it to you. The executives are coming back at the end of the week to hear your reports." We teach them a lot of ideas, human dynamic skills but 80% of the work is them choosing a problem that they work on and then taking it to solution.

At the end of the week, invariably, they’ve created and reported to their executives an order of magnitude at least more than they are paying us to teach them this. It’s a wonderful high to actually be forced to confront something and have no wiggle room. Careful! People could quit if you put too much pressure, they could fail. But we’ve got to be good at helping them not fail. It’s a wonderful experience at the end of the week to deliver so much values and hope.

"We did it! Why aren’t we doing this every week? What gives?" There are some simple skills and a set of steps that you can go to, to solve a problem and to really make it an asset instead of a liability. But unfortunately most of us go about it the wrong way. You asked for specifics so there is that specific for many clients. Something even more specific would be just any client starting iterative development.

When you start iterative development, one of the hardest things to face is on that first iteration saying we can do four stories and getting near the end of the iteration and not having any stories done or having one story done. And nine times out of 10 you say "Let’s change the iteration. We need more time." They can’t face that failure. You help a client face that failure and say "All right, we didn’t do what needed to be done. What can we do next time? Let’s learn from this. We failed, but that’s OK. Let’s face the fail. Now what can we do so that the next iteration we can do better?"

After three to four iterations, the confidence goes up; they can’t believe they are actually getting it done every single iteration. It’s a world of difference. That’s a common thing that many people at this conference have seen. But the underlying part is the ability to confront and, if they can’t confront their failure, you can’t learn from it and it will get worse.

The gentleman being interviewed right before me last hour was Ron Jeffries who wrote a wonderful article called "We tried baseball and it didn’t work." I don’t want to go off too much into it, but generally as well there is a story, people have heard of this baseball thing, but the rules didn’t really make sense so they kept changing the rules here and there because they didn’t make sense for them. At the end they said "Well, we tried baseball, but it didn’t work." "We tried Scrum," "We tried Agile." Well, no, you didn’t.

5. How have these ideas contrasted with some of the ideas that you’ve heard other folks applying to the problem of creating high performance teams?

I know that other people said it specifically, but last night at dinner I was part of a conversation where somebody attended a session and she had a problem person on her team, a real unpleasant individual. She kept telling us how bad he was and so on and so forth, that he was not willing to do this Agile stuff and she went to this particular session and what she came away from that session with was she should get rid of that uncollaborative person.

We were having dinner, it was a lot of fun and a lot of jokes but before we left, I made it a point to say "You know what? I’m not so sure I would recommend that particular action with this person." Of course, the first thing she said was "What would you do?" If we are talking about confronts, our human reaction is to go call somebody else on the mistakes that they’re making. Sometimes that’s effective, most times it’s not.

Even when they do what you want, you leave bad emotions behind it. We are talking about hyper-productive teams. Hyper-productive teams only happen when we are "clicking" together, we don’t have anything against each other. It’s really about: Number one, facing yourself first. "What have I done to create this?" Of course, all our answers are "It’s his fault!" Number two is really what do you want. I know what you don’t want. You don’t want him doing XYZ. What do you want?

For me, when we’re working through something I have my colleague Ashley say "Amr! What do you want?" I say "I know I don’t want this and I don’t want this." "Well, what do you want?" For many of us it’s not really clear what we want. It’s an opportunity to get clear on what we want, because we know what we don’t want, but until we do that we say "All right, I own it. Am I clear on what I want? Did I ask for it?" If you weren’t clear on what you wanted, did you ask for it? Nine times out of 10, if you weren’t clear you didn’t ask for it.

It’s kind of unfair to blame somebody for something that’s you haven’t asked for. Ask for it and then reach an agreement. Reaching an agreement is not mandating, reaching an agreement takes patience, takes time. If you’ve done all this and if you’ve owned it and you’ve got clear on what you wanted and then you reached an agreement and that person broke the agreement, then by all means call it.

But generally we start from the bottom. When we have a problem, we call it. If you want to be effective, if you want to be part of a high performance team, if you want to solve problems, start with yourself and go through that.

6. This sounds very idealistic. Personality conflicts happen in the real world. Are you saying that I have to love everyone on my team before my team can be a high performing team?

Absolutely not. What I am saying is don’t expect something from somebody that you haven’t agreed upon, that you don’t have an explicit agreement on. There are many high performing teams where people don’t necessarily like each other, but they have a common goal, they have a very clear common goal and a common task. This goal brings me to another thing. If you have a shared goal, it’s not enough to be clear on the shared goal, it’s important to a have a goal that’s big enough that none of us can succeed by ourselves.

High performance teams need a very clear shared goal that’s bigger than any of them. If you have that and you’ve got your agreements, you’ve agreed how you’re going to work together and you’ve asked for what you wanted, you are on your way. You are on your way to solving those problems and working together because the fact is if your goal is too small, that means I can succeed without you, when in reality we can’t.

But it seems that way and that’s one of the problems with the roles and the responsibilities. The easy problems are the problems that are straight within my role and responsibility. The hard ones are the in-between problems. Unless we have a shared goal, we won’t all have the correct incentive to go solve those in-between problems.

7. The shared goal has to be large enough. Is there anything else that the goal has to have to really make it a good goal or is that sufficient?

"Goal" is an overused term. What I don’t mean by goal is vision or something grand. I mean something very specific, task is a better word. It needs to be clear and it needs to be shared and shared clarity means you sit down and one of the best ways is "Dan and Amr are agreeing on something, so this is what we do together. Let me write this down".

By the act of writing it down, I’m forcing myself to be real clear. And a lot of times we will find out we weren’t on the same page. Shared, clear and big enough to include so that no one of us can succeed on his own; that’s it. I know there are a lot of more complex models out there, but I find that works really well.

8. Amr, thank you very much for speaking with me today.

My pleasure. Thanks, Dan.