Mary and Tom Poppendieck on Lean Software

1. My name is Mike Bria and I'm here with Mary and Tom Poppendieck the authors of the popular Lean software books Lean Software Development, Implementing Lean Software Development and Leading Lean Software Development. Mary and Tom, thank you for joining us. Your most recent book, Leading Lean Software Development, its subtitle is "Results are not the point". That's a bit of a provocative title. Could you tell us a little bit about what's behind that, what that means?

Mary Poppendieck: (MP) In the area of leadership you have people that are going to get stuff done. If you think that leaders can get the work done, then they ought to be the workers. The job that leaders have to do is figure out how to make it possible for people to get stuff done, how to inspire them so that they are engaged in getting the work done, how to make sure that when there are problems, they get solved.

The whole idea of leadership is in developing the people so that they can produce the results that are necessary. From a leadership's perspective, it's not "Do I get the results I need?" it's do I develop the people and create the underlying systems environment so that the results that are needed happen? The book is much more about thinking about how you do those think, how you create an environment and lead people so that they get the things done that are important.

Tom Poppendieck: (TP) Agile software has always focused on delivering value, whatever that is. It hasn't thought much at all about what it takes to deliver the value in terms of understanding capacity, in terms of developing capabilities, in terms of invoking the passion that is so vital if people are going to really put in the energy and the time to do great things.

2. Your first book was published in 2003 and I'm sure obviously you were thinking about those concepts even prior to that. Very recently, in the past year or 2 even, there's been a resurgence of acceptance and popularity of Lean concepts. Why do you think that is?

MP: In 2003, Lean was an obsolete concept. It had been used in manufacturing from around 1991-1995 and it was the thing that was going to change manufacturing and it moved into some other areas and it became a silver bullet and didn't solve everybody's problems. In other areas beside software it fell out into some sort of disuse. In 2002 when that book was written the idea was to take some of those manufacturing ideas that were not even considered current to manufacturing that much any more (they kind of gotten into the mainstream) and see what they might do in software development.

Subsequently, in 2003, 2004 especially 2005 and 2006 we saw resurgence in the non-software area of Lean ideas. In manufacturing, in hospitals, in retail operations Lean became a big thing. As it became a big thing in the non-software areas it started moving over to the software area. Guess what? We finally tried these Lean ideas in our manufacturing operations and they work. So, we tried them in software and those same tools don't quite work. "How do you make these idea principles work in software?" was the question.

The idea of Lean moving into software to some extent had to do with it became big in operations. The other thing is the early on things that happened in software development were in Agile, about iterations and things like that and people adopted Scrum widely. In about 2007-2008, at that point, many people had heard of and tried various Agile concepts and they gotten some benefit out of them.

Almost never did then not, but they said "All right, we've been there, we've done that, we've got the benefit out of it, but it's not enough. How do we go to the next step, the next level now that the low-hanging fruit is gone? How can we get higher up the tree to some of the more difficult to do things?" Lean became "Well, maybe this is the way to look at that". TP: The reason I believe that many of the early Agile or Lean adoptions failed in manufacturing was that people were looking for the silver bullet and they did cargo-cult adoption. They believed that if they would imitate the rituals, the activities, and the practices of people that were successful with Lean that they would be successful, too and it rarely worked. The same thing has been true of Agile - if you simply imitate practices that have worked other places, and your place is not similar, it won't work for you either.

The dissolution in both areas led to a deeper understanding of what was really necessary. It wasn't imitating practices of domains that are just similar to yours, it was the approach of really understanding how they thought, how they decided, how they framed issues, what they paid attention to in devising the tools that are good for them so that you can devise approaches and tools that are good for you. That has I think been what we are seeing at this conference, that people are taking thinking approaches and applying them in a diverse set of situations and finding significant success.

The other reason why it is spreading is that the early adoption of Lean in manufacturing in factory, in the production line rapidly ran up against barriers. As soon as it reached the end of the production line, it got to the warehouse. They had to extend Lean to the warehouse. As soon as it got to the warehouse they had to extend it to the sales and marketing so that demand could be even, that the flow could be maintained.

They found that they had to extend the Lean to the suppliers so that the supply chain could be an even flow all the way through. We're seeing the same thing in software. As soon as Agile software bumps up against where they get their inputs from, the requirements and portfolio management and where they deliver their results to, all of a sudden there is a discontinuity and they unifying thinking that can be extended to the entire supply chain is not Agile, it's Lean.

3. There has also been a recent popularity in Kanban in software as a way of applying Lean concepts to your software process. Could you talk a little bit about that and how you think that fits into the software space and maybe even into the Lean software toolkit in general?

MP: The way I like to think about that is in the concept of flow. My favorite example, from the concept of flow is the building of the Empire State building, which happened in 1930 and almost only in 1930. They started actually tearing down the stuff that was on the site in September of 1929 and May 1, 1931 the building opened. They started laying the steel in late March of 1930 and by November, 8 months later, the exterior, all 85 stories of that building were completed.

It looked the way it looks today. That was the tallest building in the world for 40 years and the fastest building built. I don't know how many stories per day or per week, a story a day was what they did. This was back before there were computers and I think that's why they were able to build it so fast. They didn't use software to lay out this complexity of "How am I going to build this building?" They had to devise very simple techniques of figuring out how to do stuff fast.

Because their biggest problem was not "How am I going to build this room?" It was "How am I going to get all these materials on site?", think of all the stuff they had to move onto that 2 square block in the middle of a busy city. And they didn't work with lights at night, it was during the day time. Of course, it was summer so there were long days. 500 trucks a day they were doing with 3,000 people.

What they did was they orchestrated with very simple human and flow diagrams. For example, the structure steel - when they had to do the high level design and then within weeks they sent that high level design to the mills and then, within a couple of weeks after that the detailed design arrived at the mill and it was on a floor by floor basis. They started the building of the steel for the base long before they finished the design for the top. Then, they got to when it was delivered and when it was put in place.

These are just lines and if you look across those lines, everything goes single story at the same workflow. It was designed, it was ordered, the details designed right at the mill, it was built, it was delivered and it was put up within a week. That was the workflow for every single story the same workflow. What they is they didn't manage individual pieces of the building, they managed the flow of major substances. They managed the flow of the structural steel, they managed the flow of the building of the window metal spandrels they managed the flow of building the stone work and they managed the flow of putting the floors. That's it.

And for 8 months that's what they did. They managed those as independent flows. What enabled them to build the building so fast? Was to break it into a sequence of independent flows that interestingly enough did not have cross dependencies. They could put up the stones (the steel did have to go first) but other than that, the windows bend roll is different than the bricks and the floors could be independently. So, they had 4 independent workflows, each one was pulled as they got ready for the next one, it came.

They loaded the mill, keep shipping, "We need the next one", like that. It was a flow system. That's how they managed rapid work when that building was built. Today, even if you think about if you want to move something rapidly through your system, you don't think about it as a set of tasks to be managed and put on a pert chart or on a Gantt chart. You think about it as a series of flows to be managed. They managed 4 flows, that's what they managed, not every flows, but 4 flows. Those 4 that I said: structure steel, window metal spandrel, exterior stones and the floors. Then, in 8 months they had it built.

What we're trying to do in software is to think about it as managing flows rather than managing tasks. If you are managing a flow then you have to think about information flowing to a steady pace through the organization. So far, in an Agile environment there have been 2 mechanisms for managing flows. The first mechanism to become popular is the iterations. Say you'd have a 2-week iteration where the team commits and delivers, commits and delivers. The important thing there is to measure the velocity, which is the flow. A team will establish a velocity, which is the capacity, is how much work flows through that team every 2 weeks.

So, you've got a steady flow that you understand. This is the capacity of the team, this is how much they can do. Over time, you then understand how to manage work moving because you understand that flow. Similarly, Kanban is a mechanism to manage flow of work through a on organization. It's a different mechanism, but it does the same things: it makes the work visible, it makes it so that you understand the throughput. With Kanban, you don't measure velocity, you measure throughput. It's the same thing. It tells you your capacity, how rapidly stuff can flow. Then, you want to make it visible and you know the throughput and the amount of work that's in process.

What you can see in Kanban is that there is one prescription, you have limits on each column, each work flow column and similarly in an iteration, you accept only so much for the iteration, the amount your velocity dictates. Limiting work in process is a very important thing because that's what makes the flow work. And you want to have it go at a steady rate, always steady rate. That's what you manage.

You don't manage task. Kanban and iterations are 2 very good methods of managing the flow of work through a software development organization. In a manufacturing plant, Kanban was a manual paper mechanism for managing the flow of material through a plant. I look upon Kanban as a flow management mechanism and iterations as another flow management mechanism and there are probably other ones out there, too.

TP: The very early Agile methods were all about iterative development. Working iterations, put work in small batches rather than big batch and they told you quite a bit of detail about what to do, what practices to follow in order to be successful. Like all methodologies, the people that articulated them forgot about things that they did that they didn't think to include in their methodology. They were successful for the originators, but not always so successful for the followers, for the simple reason that there is a lot of tacit knowledge involved.

As Mary pointed out, one of the first mindset group was XP that was the one that became very widely popular and it was successful but extremely hard to do. People backed off a bit and said "Oh, let's just try Scrum. Scrum is less prescriptive and tells you less about what to do, gives you more freedom, but it still clung to the fundamental cycle of iterative development - iterations.

Kanban abandons iteration even and it doesn't really tell you much else about what to do, but most teams doing Kanban will do most of the things they did under Scrum if they were moderately successful. Most of the successful Scrum teams actually use many of the practices from XP, but the stranglehold of a prescription, of a recipe that tells you exactly what to do I think is very much broken. The thing I think Kanban is doing is it's making it OK to put together your own combination of fundamental building block disciplines in order to solve the problem that you are actually facing rather than imitating someone else's solution to their problem.

4. What do you think might most be at risk of being missed in the Lean philosophies or Lean software practices for people that adopt Kanban?

MP: What I worry about in anything that focuses only on flow is that there is another sort of leg here and that is the leg of making sure that you always make known good product, that you build quality in and you don't test it later. In software development we have this strange idea that it's OK to spend all of this time at the end of the development process running tests to find what's wrong (and that seems like a good thing to do) instead of saying "Wait a minute! I don't care what sort of development process you look at, even waterfall.

Every one of them was devised originally and its primary purpose was to not find defects late but to find them as early in the process as you possibly can. Somehow or other we've evolved to the idea that's OK to find defects at the end and it isn't OK to find defects at the end. When you're looking at workflow, workflow really doesn't work rapidly if you don't have this make sure that quality is built into the code mindset. I don't think you leave that up to the interested team to figure out. That's fundamental, so you got to be careful that that doesn't get lost.

TP: The thing that worries me most is that software continues to be an orphan. It is IT and the business - as long as people say "the business" rather than "our business", there are going to be problems, there is going to be a lack of integration, lack of cohesion. A team that is only software techy people is hardly a team, it's just a group of implementers. They aren't going to have the passion, they aren’t going to get the understanding to make the right trade-offs that they could get if they had direct contact with the customers whose problems they were solving, which is the only reason for their existence.

5. What would you say the 3 most fundamental aspects of a successful software process are?

MP: I'll start with what Tom just said. That software is always a subsystem of a bigger system, almost all the software is embedded. Even if you think you are doing software for software's sake, it's perhaps to support a process, if you are doing an IT set of software. So it's embedded in the business process or it's got a goal once you sell your software package to help somebody do their finances better or to help somebody have fun with the game. Software, all by itself is neither good nor bad, it's embedded in something bigger than the software, a bigger system. That is what people really want. Typically, people don't want software, they want something else. If they could get their objective without software, they would be delegated. All they want is something that works. Software isn't the thing, it's the bigger thing. You always have to remember that we don't do software for software's sake, we do software for something else and you have to understand that something else and get the development team in touch with the bigger picture so they make good decisions. Secondly, as I said, you have to know how to build known good stuff. Get out of the idea, habit of thinking, that we test software quality in at the end. You have to create testing mechanisms and integration mechanisms, which allow you to make sure that what you're doing yesterday, five minutes ago is good. It's going to not be 5-10 weeks later that you introduced an error. The third thing is flow. Not to create great big long lists of stuff that we'd like to do, but to keep short. Not to create all sorts of partially done work (everybody's work), but to move stuff through rapidly one thing at a time through the whole development process. Even if you can’t actually deploy it, you want to go from "Let's break this big idea into little tiny pieces.

Let's make it happen. Let's review it with customers, let's make sure it's what we really want in small chunks." As from big idea to implemented, tested, hopefully integrated and ready to deliver if not delivered as in this shorter flow as possible. I think those are the three things - the flow, the quality and the systems thinking.

TP: A critical aspect of the systems thinking is that you can't just have technical people doing the systems thinking, you need people of diverse skills, diverse responsibilities collaborating effectively together to solve the problem. Part of the solution might require software, but there is more to solve than just the software and trade-offs need to be made across all of the aspects. The separation of software from the rest of the business, once again, is a risk that a software process rather than a solution design process, a problem solving process that extends to the whole organization is something you have to watch out for. If you only do software, all you are going to get is software and you're not going to effectively address the real problems.

6. How do you go about influencing or making change in the organization?

MP: I just read a very interesting book by Chip and Dan Heath called Switch. How to change when change is hard. They talk about 3 what they call surprises. It's not a surprise that people have two personas - they have their rational self and their emotional self, that's for ever, for decades people have been talking about different metaphors for "Here is my rational self and here is my emotional self". They came up with a metaphor which is the rider and the elephant.

There is a rider sitting and an elephant. The rider would be rational self and the elephant would be the emotional self. The elephant will do what the rider wants unless he decide not to. If the elephant decides not to, there is no way the rider is going to change that, because when your emotions go this way, that's where you are going to go. There is 3 surprises - one is if you want change, the first thing you have to do is to make sure that that rider knows what to do, because the intelligence will go in circles if there are too many options.

What you want to do is look for stuff that works and make it work. You want to look for a clear direction for the rider, because what looks like resistance is often lack of clarity. For that elephant you have to get in touch with people's feelings, you have to connect the people to what they're doing and that's where getting our development teams and our customers connected can do a lot. It can match their mental model, so they’re think along the same line, but you might also can get the developers engaged in helping the customers be successful.

Typically, what they say you might think that people are lazy, but actually they might just be exhausted, because doing what's a habit, what you're used to is easy, but if you have to let your intellect say "Maybe this isn't what you want to do, but I direct you to do this" you can only do that for so long and you actually physically can only direct your emotions for so long and not. What you need to do is engage people so that the elephant and the rider are both heading in the same direction. You need to engage both feeling and intellect. The third thing is it's not necessarily either about the people, it's often the situation.

We have what they call fundamental attribution here. When something goes wrong, we usually attribute it to the people that are there, instead of attributing it to the situation that they are in. So look at the situation, the surrounding environment and see if you can change that to make the behavior that you're looking for, the change that you want easy. There are those 3 things: direct the rider, provide clear direction and make sure that the elephant, the feelings, the engagement of the people are there through their understanding and the last thing is make sure the situation is set up so that it's the correct situation.

7. Mary, Tom I want to thank you very much for taking time out of your day to sit down with us. This is been very enlightening. Is there anything else that you would like to offer?

MP: I think there is one thing that we haven't touched on. It is this concept of relentless improvement, constantly getting better. One of the things that I think an organization needs to think about is how to create a constant problem solving environment. An environment in which people are always looking to see what's bothering them, see what's getting in their way and they the guidance and the leadership and the expectation that they will solve their problems. Setting up an environment where constant improvement is an expected environment, where everybody is expected to do what I think is really important, and it's a very important leadership concept.

TP: Results are not the point, the point is growing your people, converting them into effective problem solvers who are relentlessly improving. If everybody in the organization is a problem solver, you'll get steadily better and better. You won't fall backwards.