Little and Spayd on Agile and Organizational Change

Recorded at:

1. Please introduce yourselves to our audience.

My name is Joe Little and I'm an Agile coach. I'm Michael Spayd and I'm an organizational change consultant, and I work mainly in Agile adoption in large enterprises.

2. You've been thinking about how organizational change unfolds. Why?

Michael: When you introduce Agile into organizations it tends to create the conditions for a different way of thinking, a different way of viewing the business - for being more honest with how our work goes and how it doesn't go. It tends to create pressure in the organization for change to happen, along lines of being more honest, more realistic. And that creates a lot of pressure in the organization to change. So it's important to think about that.

Joe: And when you are working in large organizations (we're both working on a Scrum project in a large organization) you'll find that the organizational impediments are the key factors that keep the teams from being productive as they would otherwise be.

3. Why are we hearing about organizational change now? Isn't it enough just to roll out the Agile practices with our software teams?

Joe: That in and of itself will certainly cause organizational changes, which you might want to think about managing in such a way that, for what you put into organizational change, you get more value out of it and the teams become more productive.

Michael: I think you need to start at the team level in terms of introducing Agile practices; clearly that's the place where it starts, and gives the rest of the organization a reason to change. But if you end there, if you don't think ahead to the end-game strategy, my experience is you're going to be in trouble, because something is going to change in the organization, there's going to be pressure to do things differently.

4. You've been observing and working with organizations that are introducing Agile: what have you seen? How does this change happen?

Michael: First it happens at the team level, and teams go through their own change curve with it. It's not easy for some teams to make these changes, and particularly if you don't have all the conditions that are right, you know: you don't have a good product owner or customer, you don't have a coach. There are things that are difficult just on the team level.

Joe: Absolutely. Related to what you're talking about, there's the general movement from ground up; you have got teams involved and their impact filters up in the organization. The other way things usually move is from the top down, and in the best scenarios both are happening in the same time. Usually the toughest changes are in the middle level; after a while middle management starts to look at this and go "oh, my job is starting to change. What does that mean for me?"

5. When middle management realises they need to change - what happens then?

Michael: The problem is that change at the team level in pretty clear: what has to be done, what needs to be true for a team to be successful using Agile practices: that's very clear; there are all kind of books about it. What does it mean to be an Agile manager in an Agile organization? There are pretty much no books that actually address the full subject matter of that, there are books that take a little piece of it, but most of them are dealing with project management, they are not actually dealing with management practices.

Joe: So, for example in Scrum the advice is "don't let the middle manager interfere with the team," that's what not to do, but it doesn't tell you much about what they should do.

Michael: There are not a lot of books about it or organized well articulated thinking about it, because it hasn't been done a lot. It's an emergent property, if you introduce Agile into the team and then it moves a certain kind of change upward in the organization, and that change hasn't been gone through by very many organizations... and so nobody has detailed or articulated what's going on there, what it is for a manager to do in an Agile organization.

6. I know that when Ken Schwaber teaches about Scrum he tells us "be prepared: as soon as you start solving one problem you're going to reveal another one". Can you give some examples of the ripple effect that happens when you start changing things?

Joe: One example: you've got a team and you want to drive them towards more "test driven" development. Perhaps you're not going to go all the way to fully automated TDD, but you at least want to drive that way. The person in the team reports to a manager that's totally outside the team. To get that person totally in line with what you want him to do on the team and with what they are responsible for their manager for doing, you often have to go talk to their manager and get them convinced. So, that's a kind of organizational change that affects usually not just one manager but also that manager's manager, the person who is totally responsible for the QS function. That's one example of the kind of change we are talking about.

Michael: I'll give you a different example: I worked with a software development team who was always the bottle neck for producing changes to the business and the business was always complaining about that. The team started adopting Agile and took a 3-4 month cycle time down to 3 weeks and suddenly the business started to freak out and said "Oh, uh, are you ready to deploy that? We're not ready to deploy that. We haven't developed the training for it, we haven't notified the users of this change, and we haven't scheduled the right staffing at the call centers to handle this change, so we are not ready for it". So you move the bottleneck (the theory of constraints idea) from the software development team to other places in the business. That can cause a backlash, like maybe the business doesn't want to be focused in that way, they don't want to have a spotlight shined on the training function perhaps. That's where you get into really complicated stuff happening from Agile.

7. We have this effect of change causing change and provoking reactions. Is that where change management comes in, as a practice?

Michael: Change management is a very interesting idea, it's what literature typically called, and comes from HR and organizational development background and technical approach. But what I seem to be learning is that "management" is asking a lot and it's really more like riding a wave successfully, more like guiding the direction of a river potentially by putting up some barricades, or facilitating, catalyzing, are better words for it than managing. I don't think you can manage change, you can help it go better, and you can do things that make it much worse.

Joe: I like to think about this in terms of, say, 1 or 2 characters: if you think about Scrum, then the Scrum Master, or perhaps there's someone in the organization who's responsible for the overall Agile program. From their points of view they need to help facilitate, and figure out how to organize and prioritize the changes that are happening, at least as much as they can affect them and facilitate them, influence and guide them.

Michael: A practice you have built right into Scrum, or any stand upm is what you did, what you are going to do and what is getting in your way. If you take the last question and you give in stand up, and you move it up from the team level to the manager level, to the vice president level, that becomes an "action list" for management. That's one of the main things that Agile management should be doing, is focusing on what are the patterns that are happening across my Agile team that are essentially organizational impediments because they are happening again and again: my configuration management process is broken in some way, my real estate provisioning process is broken.

Joe: So, you think of this as a very concrete process of, in relatively small ways, fixing one little problem at a time; you don't think of having organizational change so that in a month everybody understands fully what Agile means. If you conceptualize it that way you are going to be very frustrated, at least in my experience.

8. So you can't mandate Agile?

Joe: Sure, but your smile suggests that that's a rather provocative idea that I agree with. You are not going to mandate those changes, but you can influence those changes throughout the organization, through the small groups of people whose minds you want to change just a little bit at a time.

9. Have you seen examples that mandated Agile implementation?

Joe: You can put certain metrics into play that, in effect, mandate that Agile be adopted by certain teams. So let's say you have a metric of x% of projects that are going to be Agile by a certain date. That can lead to certain types of behavior that seem like a mandate to people who have to actually effect the changes that make that happen.

10. Mandated adoption: Is it effective?

Joe: It can often drive bad behaviours, particularly if the people are given that kind of guidance and nothing else.

Michael: I've seen it be mandated very overtly, and I've seen it be mandated quite subtly, and in both instances it doesn't work very well, because it's against the grain of Agile, as you know. And yet the people who are into Agile are very zealous; they tend to be true believers in Agile, because it's very cool, and at some point it's hard it not to give in to the desire to impose a mandate: "You should do Agile because it's so much better". And we know we are right. (laughter).

11. But if you don't push, is it just organically going to happen, and everybody will just embrace it and it'll be a big love-in?

Joe: One should look at it as a 3 phase process: you've got the early adoptors who'll be easy to convince, in many ways they will convince themselves and there will not be that much of an effort for the change agent, the Scrum master, or the overall manager of the adoption process. It won't be that hard for them to affect that change or have that change adopted within that organization for that group of people. Then you go to the next level of people who, with some effort, will adopt Agile principles, probably utilize the principles of Agile. Then you've got, further along the path, the third group, who are going to be rather resistant to it. They aren't in the first 2 groups so almost by definition they are resistant and they will push back and it will take a lot more time and effort on the part of the change agent to get them there. But, they are often convinced more, not by theory and talk, but by the experience of actually seeing the successes in Agile.

Michael: To me it's like being a coach: I'm a coach for teams but I'm also a coach for individuals. Sometimes you can push and sometimes you need to push and sometimes you need to back off. But always, if it's not the person's agenda or the team's agenda or the organization's agenda - ultimately if it's just your agenda you are going to fail. There will be a backlash on some level, or a revolt or a rebellion on some level, whether it's overt or covert. If you're helping push somebody for something that they actually want, but they are going through their own resistances, that's what good coaching is. If you're doing it for your agenda, that's what abuse is.

Joe: Yes, as long as you are respecting the other party you are much more likely to affect them in the long term, in terms of understanding Agile better.

12. Joe mentioned the term ‘change agent'. Is the 'change agent' typically the Scrum master or the team lead, the person who is leading a particular team?

Michael: It depends on what level you are talking about the change. Because there's a change that happens on an individual level, there's change that happens on a team level, there's change that happens on the organizational level. On a team, sure, the Scrum master could be a change agent, but you don't want them to be the only advocate for the change or that's going to be the mandate, the team is going to resist that, so you want other people on the team advocating that. Like Joe talked about, if you have a bunch of coaches and you have a whole movement happening within your organization, then maybe there are a lot of change agents, hopefully there's executive manager kind of people, some business people involved in that. And also I think you want to understand the geography of change, if you will, the architecture of change. You might need an architect, in a sense, somebody to view the whole picture, to articulate that for people: how this is going and test that: "are we going in the right direction? What parts of the organization are changing and what parts are not? What parts of the organization are not ready, are not digesting this?"

Joe: Let's come back to a practical example: let's say you've got one middle manager whose involvement in Agile is not where you want to be, you want to affect their thinking about it to become at least in some small way more Agile, in what they do or what they suggest their direct reports do. You might want to gather the forces of several people to talk to them in different ways about the kind of change that might happen, might have their manager or some senior executive talk to them: you might talk to them yourself if you're a Scrum Master or coach; you might get a third person who is in Scrum terms a Product Owner and very happy with Agile talk to them. They have several different conversations about the same subject, they start to see it, they can't resist it as a one on one resistance, they have to look at it from the bigger point of view and their change of thinking is more likely to happen, at least in my experience.

Michael: So, if you're going to have those kinds of conversations with people, my recommendation is to take one of Stephen Covey's 7 habits: "seek first to understand, not to be understood." One of my new favorite sayings is: "Resistance is to be understood, not to be overcome". What's the nature of the resistance, what's the cause? Because there's usually a very good reason for it. We label it with a rather negative term "resistance", which we think we have to crush somehow, or snuff out, and usually that just creates a lack of respect, or more resistance, rebellion. If we can instead seek to find out what's going on, what is the information that that person has (because resistance has information in it), can we find out more about it, something that can actually help the change go? The person who's resisting is our peer, in a collaborative sense.

Joe: And in some ways resistance has a real value that in fact Agile supports. So their resistance might be that they are concerned about a risk in this part of the whole process, or this kind of risks that occur to projects. It may be that the risk was addressed in the former process this way and in Agile it is addressed a different way, they just haven't quite understood that yet. So their concern about the risks is quite valid, they just haven't understood how Agile addresses that.

13. It sounds like the resistance might take different forms in different individuals, at different levels.

Michael: Yes, absolutely

14. It sounds like you have some strategies and some approaches to dealing with resistance to change. Does that suggest that in any organization is possible to overcome the resistance to the newness of Agile process? That it is possible that every organization successfully adopt these practices?

Michael: It's not always clear to me that every organization should, or certainly not on our time frame. Agile came in for a certain reason to an organization, because some developers really wanted to see it, or because the CEO thought it was the latest greatest thing, whatever. And it's important to uncover what that is about, to understand, to try and maximize the information there, the value that it's trying to create, but also to respect the parts that don't want it to be in a certain place. People say things like "Agile is appropriate over here but it's not appropriate over there". That might be true right now, and it might not be true later. I concentrate on people who want to adopt Agile, who want to learn more, even within an organization that's adopting Agile, because you are going to have higher leverag; it's essentially like a Lean principle of taking a thing, it's like a pull strategy: take the thing that is set up for the queue, that's ready to work on. So work on the people who want to adopt Agile not the people who don't want to adopt Agile.

Joe: I will try a different approach, use a water metaphor. As the water is rushing down the stream, some of the small impediments will just move out of the way, others will just go around and continue on, and not trying to change them because they are not going to change, the water is not going to break down that rock, at least not immediately, but in the long term, it likely to wear down the rocks in such a way that the flow is better than it was before. But expecting a complete and total adoption of Agile is like expecting a Nirvana, it's never going to happen.

Michael: That's a very good analogy. When you can you go around the big rocks, as the water, Agile is the water. If the system is dammed up, and you can't get around it, then you do need to do something and you may need to raise the question to the right level of decision making: "Do we want to do this? This block is in the way of Agile adoption or Agile working for this team." Let's just be honest about that; that is what's really going on, me as a consultant I can't say "you should choose Agile because that's right." I can say that and many of us have before, but that's not terribly useful to the organization, it's better to say "here's the truth as I see it and as you are all reflecting it to me, what do you want to do about that?" and then it becomes a matter of choice rather than being unaware.

15. So you are going into an organization, you are starting an engagement, you know that there's going to be resistance but you have no idea what kind of resistance it's going to be. Where do you start? What methodology do you use? How do you select?

Joe: We are talking about organizational change issues right now. In terms of dealing with that, I'd choose Scrum because it makes the impediments visible, all of them, and you have to prioritize all of them, in terms of who is working on them, they can't spend all their time in organizational change and no time on the technical issues, practices and things of that nature. It raises all those up, it makes it visible to a lot of people, not just yourself, and allows you to prioritize things much more easily than other approaches that I'm aware of.

Michael: I would agree with Joe that Scrum is the place to start, in a company, because it's the simplest to implement and therefore the most likely to have some early success. I don't think you want to end there, though, I think you want to move into XP or engineering practices because as an organization matures with it, and assuming that Agile really takes hold, it's going to be inadequate if mature engineering practices are not developed. But it's enough to start with Scrum.

16. Are you saying that XP is a more evolved form of Agile than Scrum?

Joe: Well, I'm not going to get into a fight between Scrum and XP! (laughter) I think they're both valuable. If it were up to me I would use Scrum and XP together. That's a simple answer to your question.

Michael: I think I would say that. I would say that Scrum is permissive and allows for anything within it, and a natural evolution (and I think Ken Schwaber would agree with this) is to develop into engineering practices. But the main game out there is XP. I mean, I like some of the language, projects management-wise of Scrum better than XP. But Scrum by design doesn't address engineering practices. It's not a particularly fault of it; it's not complete in that sense.

17. Can you share some advice with enterprises that may be looking at adopting Agile?

Joe: My thought about that, from recent experiences, is that you can have folks involved in Agile adoption who get very much enamored of the idea that are going to have some success in terms of organizational change. And there are a lot of great things that you can do, but you have to prioritize things. So the way I suggest to prioritize them is look at what you are really trying to accomplish. I would suggest that that is: making team successful. And only try to make those organizational changes that are likely to be successful right now, to make the teams that are working right now a little bit more successful. As opposed to having a goal of organizational change in and of itself, which I think is a tempting goal to have, but it's likely to cloud you and not actually make you successful in effecting the organizational changes in a gradual way.

Michael: My suggestion, and I think it is consistent with Joe's but in a rather different angle, is to run the adoption of Agile like you would in an Agile project and have a couple of points at which you can bail or keep going. And I would describe that as: the first phase "can you do it at all?" and that's like a single prototype. Create a perfect storm, all the right conditions, and find out if you can do it in this culture at all. Second phase: make a decision. "Should you go on or should you not?" Second phase is doing a series of pilots "Where can you do it and how well?". Test the limits of "can I do it with offshore, can I do it with non-collocated teams in the US or in the same country?" "Can I do it with a fractured product owner across a couple of groups?" Test the limits of how you can do it. Then you have a strategic decision to make and I think you should involve the highest level in the company, the CEO. Do you want to be an Agile organization or do you want to use Agile as a tactical IT strategy? And those are very different things to me. Using it as a tactical IT strategy - that's just methodology question. Using it as a means to transform the organization, if you want to put Agile everywhere - it's going to put pressure on transforming the entire organization. And then you need to have a change initiative, in my opinion, and the CEO needs to be the client to that.

Joe: I'll just add that at a bare minimum you want the projects to be business-oriented as opposed to IT-oriented. You want to spend some time with the Project Owners, to use Scrum terms, in the various lines of business, and get them actively involved. that's just adding to what Michael is concerned about, being strictly IT department focused. I think that might work in the long term, but the sooner you can get more business-focused the better.

18. What's this Agile organization that you are talking about? We don't have this now, or is it something different?

Michael: Yes, I think they are pretty different, I think you probably have a few of them, what Jeff Sutherland talks about with PatientKeeper sounds like an Agile organization, that's the most notable example that I'm aware of, and I'm sure that there are others out there but you don't hear about them a whole lot. I think they look like: where management is focused on impediments of Agile teams, and where management runs their business like Agile teams, short meetings, very quick moving, focused on things like portfolio management, how do you manage your entire portfolio from an Agile point of view, how do you do performance management and rewards and such from an Agile point of view? Probably from more of a team-based point of view. We've created a whole list of manager competencies that feed what it means to be in an Agile organization from work that I'm involved in right now but it's an emergent property, because like I said nobody has written a book on this.

Joe: At least for companies that aren't mainly software-oriented, aren't software development companies, nobody really knows exactly what an Agile organization looks like. But I think the more compelling thing to me is: "Is that the right question to ask?" The end goal is not to create an Agile organization,whatever you understand that to be, the end goal is to make the projects successful. We needed the organization to change in order for that to happen, but the end goal is not having an Agile organization in and of itself. So to me that's a question that can misdirect us into getting very focused on what's in the hearts and minds of middle management and upper management, very fuzzy stuff - I'd rather focus on something that's relatively concrete and easy to inspect, on how well the projects are actually doing.

19. So does this focus on organizational change signal a shift or a new stage pf growth in the Agile movement itself?

Michael: Good question. Yes, I think it does, because Agile is going more mainstream. And in Geoffrey Moore's technology adoption terms: to me it is crossed the chasm, and it's in the main stream now, and pragmatists and late adopters are the target of this change now. You don't have to convince early adopters of Agile to do a lot of changing; they wanted the change, they wanted to learn about that. People that are adopting it now: you do have to convince, you do have to bring a business argument to the table, a compelling business driver, for them to do this - because they don't care about Agile per se as a methodology.

Joe: I would say the same thing: it's an indicator that shows that Agile is being adopted by Fortune 500 kinds of companies, both in America and around the world.

Michael: Another thing is that the experiments that are being done around the world with Agile are starting to take seed and multiply across the organization, which is naturally forcing the conversation upward in the food chain at companies. It's one thing to have one or two renegade Agile teams that management looks aside at "We know they deliver well, so we'll let them get away with their weird stuff and violate methodology" and so forth, and it's another thing to have 20 Agile teams. Then you start to call into question everything about the way business is done, in a lot of instances. So that moves the conversation up to a higher level and it means other people in the system have to change.

20. Do you see further changes needed in the Agile movement right now?

Joe: In terms of organizational change, I think we're early in the evolutionary phase of understanding what organizational change means in terms of Agile adoption. Yes, I would recommend that folks talk to each other a lot, about how it's working in large organizations and how we‘ll make it more effective. But be careful about getting to abstract and getting into very fuzzy areas of "hearts and minds" and think about "how do we do this thing effectively and make a big project more successful?" There is more discussion to have on this subject.

21. Is there anything you would like to add?

Michael: I think that we need to do even more reflecting about what our new audience is, these pragmatist and late adopters that we are starting to work with, and that we make have to change some, not just them. That we may have to listen to them and really find out "what is it going to take to move Agile into the average organization?" We tend to be zealots, as a group, and those people don't care about it in the same way that we do. And it's probably good that they don't, that's very realistic, and then so: how can we get a little less serious and how can we open our minds and how can we help them open their minds at the same time. I think that what a couple of people teach, Alistair Cockburn being one of them, Bill Schneider being another, that if you are confronted with something that doesn't work in a certain culture or certain area, you can let go of the practices that you are used to engaging in and you go back to the principles to decide how to adopt or modify or make them work. So as long as you hold the principles, or even further back, the values - as long as you keep the values in mind, you can afford to let go of the practices and let them evolve and change in a different way for a new set of people that want to adopt it.