Good morning, Robin. Tell us a little bit about yourself and what you are interested in these days.

I've been doing Agile since 2002. I have 18 years of experience in software development and I've done a lot of different things - DSP, embedded systems and a little bit of semiconductors and a lot web application work as well. These days, working with my partner David Douglas, we have a company called Inovelle, and we are doing Agile and Lean management consulting and team coaching and training - so, I am a certified trainer as well with the Scrum Alliance.

What are you interested in these days? What's keeping you busy, giving you a headache?

A lot of things are keeping me busy and giving me a headache, but I think that we are in a really interesting time for Agile these days. It's still quite small in terms of market share and mind share - a lot of interest. Some folks looking at it like it's going to be the next silver bullet for IT and of course, we know that it's not that, but it's a great time to be involved in the community because it's really growing quickly and we're seeing a lot of activity in the marketplace.

How do you see the community these days? What do you think the state of Agile in the Agile community is?

It seems to me like it's a little bit in a sophomore stage in that we figured out a lot of the basics. We have a lot of understanding of how to do the basic project management and software development techniques. The challenge is that Agile is not really like other methodologies. We are really changing the game in terms of how people approach organizing themselves to do work. The challenge for Agile in the next few years is how do we take more of an organizational focus to adopting these work styles and work methods. If you think about how we've organized in the past, we built companies and we organized them around the work that they do. If you are a logistics company, you're organized your supply team.

If you are a company like a software development company, you are going to organize around that. With a traditional waterfall method, you have your business analysts and you have your software developers and your testers and your release management folks, and all of those are organized as individual, functional groups. You had hierarchies developed in the business in terms of leadership in those different areas. Now with Agile, you have this different construct: you have team-based delivery.

Now you have groups of teams with all those roles within the teams. You have this difference because you have teams organized, which are groups of folks, and you have on top of it, this sort of matrix structure of different functional units. There is power structure here and there is an impedance mismatch. If you look at folks like BMC software, who have taken an organizational design approach to adopting Agile, Israel Gatts really decided to really define the organization around teams and structure the organization around teams, so everyone in the organization is part of a team. I think that's where Agile will succeed really phenomenally for companies, or be an enhancement to their current way of working, but not really deliver the grand breaking results that it can do. It's going to depend on how people can adopt it, from an organizational standpoint.

There is an impedance mismatch and this impedance mismatch you've seen cause problems already?

Yes, I think that there are about 3 or 4 big companies, big enterprises that have adopted Agile. Publicly, we know about those big British telecommunications company, big portal web company and company in financial services and each one of those early adopters was very focused on adopting Scrum at the team level. We had a lot of teams and we had a lot of training going on in those organizations and a lot work to build Scrum teams and get everyone working on that plane.

From that point of view, it's very successful. It improved their delivery timeframes, projects were getting done a lot quicker and people were happier working in those environments. But now, all of a sudden you have all this retrenchment happening in all those organizations where Agile is getting pushed the way side or is becoming less important to deliver things in this way. Part of that is because of the power structure and the current organization, which is still based on the waterfall method, and the Agile way of working, which is based on teams. This is a challenge to the Agile community to figure out how do we organize leadership. How does leadership interact most effectively with teams? How do we build an organization that is team-based, as opposed to functional-based? That's really an open question for the community.

Let me say back what you are telling me: you are telling me that Agile teams, within an organization that aren't Agile in and off themselves, is unsustainable. Either the organization changes or the team dies.

I don't know if it will die, but there is one anecdote that I can bring up: talking with folks at the federal reserve, one of the founders of Scrum went in and did some training in 2000 - they adopted Scrum at the time and they kept at it for 2 years and since then it's gone by the way side. I don't think that will happen with everyone, but we have some interesting data from the Lean community.

The lean.org, Jim Womack's organization, which is one of the big proponents of Lean in the industry, has been talking about how Lean has evolved during the last 10 years, just in time manufacturing came along and that changed how people did manufacturing, but it hasn't really changed those organizations. You have organizations like Toyota and Honda who've really organized around these team-based practices and have seen tremendous gains. Folks like GM and Ford have adopted some of the practices, but haven't changed their organization.

Because of that, you have this change with our current fuel - fuel crisis - not crisis, but extremely high prices. No one is buying Hummers but GM instead having flexible manufacturing, they can manufacture multiple vehicles on one assembly line, like Toyota does. They've just built the Hummer plant. Now no one wants to buy Hummers, so they are talking about moth-balling the plant or putting the plant on hold. That's highly costly for that organization. Making those kinds of decisions or understanding what Lean and Agile are and how that drives change in your organization is one of the keys.

Robin, you think one of the keys to the mismatch is possibly looking to the Lean community and learning from them how they have succeeded. Or is it more than that?

I think it's more than that, but Lean offers some great examples and ideas. In Agile, we are really focused on techniques to manage and do great software development, but if you look at that whole chain, from a customer need to need fulfilled, software development occupies a part of that, but it's not the whole thing. I was doing a value stream map, which is a mapping of the process of going from customer need to need fulfilled in an organization - big e-commerce organization - a couple of weeks ago. We discovered that the process efficiency in the software development area was 20%, which is pretty good. World class efficiency in value stream is around 50%, so they are doing pretty good: they are doing XP techniques and test driven development and then you look at the customer product management part, and the percentage was around 2% efficiency. A lot of going around, trying to build consensus and then discovering new features that needed to be added and then going back to those folks and finding new stakeholders that needed to be brought in.

The process of going from identifying that customer and even getting it delivered is really slow and the front end. At the back end they have a very long, painful release process, which is very - I wouldn't say rigorous, as much as - tedious and a lot of rework and retraining of people to do builds instead of automation. The process efficiency in there was 0.2% percent. When you look at that overall process from a customer perspective, you say "I need something" and then there is a black box. We have the customer and product management part, we have the software development part, we have the release management part. If we're optimizing on the Agile software development part, and forgetting about the rest of the process, we are still not going to be very happy customers. Lean gives us a perspective that we can look at from the very beginning of the process to the very end of the process delivery and look at optimizing that for speed. That will drive a lot of value for us.

Lean, because it takes a more global view, can help us, because in your example, no matter how good your software development group became, it wasn't going to really affect the end product for the customer? Not everybody is really very familiar with what exactly is Lean. We've all heard about it, but can you give us a quick primer? What are the important concepts of Lean?

Yes. I could do that. I'd actually like to take you back a little bit, with a bit of history. Lean is the generic name for the Toyota production system. What happened is that Toyota, in the '40s, decided to sell off their loom manufacturing and become a car manufacturer. They started that in post WW II Japan, scarce materials, hard to come by technical labor and an economy that had collapsed after the war. They started building cars and they were quite poor at it. They said "We need go and study the North Americans". They came over and looked at the North Americans, how they did work and they discovered the North American manufactures had these huge economies of scale. They could do mass manufacturing in a way that was just impossible for them to do in Japan. Japan had a small market that demanded a lot of variation and North Americans had a huge market, a lot of resources and a lot of materials to bring to bear - that was the essence of the problem.

They said "How can we compete in a marketplace where we are at this significant disadvantage?" What they focused on was how quickly they could deliver cars to their customers and how could they support variations in small batches. It drove them to a very different way of working and that focused on really conserving materials and improving the time to the marketplace.

If you fast-forward 20-30 years, they've actually adopted a lot of things: they adopted the supply chain from a grocery store. Lettuce coming from the field into the store has to be quick, otherwise it's going to go bad, so they adopted ideas from the that. They adopted ideas of the conveyer belt from Ford, but what they didn't do is they didn't dumb-down the production line. Coming from manufacturing complex machines like looms, they had a respect for engineering culture, so they said "How can we really leverage the intellectual capability for the people who are actually doing the work?"

When you bring these things together, Toyota developed this process of continually improving their production methods. They are not perfect, they have as many issues as we do in North America - maybe a little bit different ones -, but in the '80s people from North America went over and visited Japan, especially US auto manufactures and they didn't understand this way of working. They'd see people huddled in groups and talking at the line and they thought this was a Japanese culture thing, but in '82, GM and Toyota got together and did a joint venture - it was called the NUMI Plant. It was a plant in Fremont, California and GM volunteered it for this experiment. It was their worst plant and they had actually just shut it down a couple of months before. They had drug paraphernalia in the parking lots that had to be cleaned up every night, they had people in the lines joking about the cars. They also had over 100 job descriptions in the plant. Toyota came in, they ended up hiring 86% of the people back into the plant to do this as new Toyota production system plant.

They brought everyone to Japan and trained them for 3 weeks on Lean, on the Toyota production system and they reduced the number of job descriptions from 100 to 5: things like team member, team lead and a couple of others. They said to all these people who'd been in the plant where they really had no responsibility authority that "Anyone can stop the line. If you see a problem, stop the line and fix it". This was a huge change in terms of mentality and in terms of how these people were now being respected on the line and also asked to improve their working environment.

That year they implemented 8,000 improvements to their production facility. They went from being GM's worst performing plant to their best performing plant. It's the plant now that builds the Pontiac 5 and the Toyota Matrix. So, what is this? This is the same people, same plant in California. Lean is really a fundamentally better system of working than the traditional Taylor system.

Agile takes a lot of it's ideas in terms of team-based formations, self-organizing teams, organizing around a certain commitment to the work, the ability to stop and fix defects as soon as possible. A lot of these ideas between Lean and Agile are similar. Where Agile gets a little bit more different is that it's around knowledge work. The work is being done in people's heads, it's not being done in the assembly line were everything is visible. Agile is much more attuned to how people interact and how people work together and sets up environments that make people more successful.

Thank you first of all for that history of Lean. What is Lean?

Lean is a set of 4 principles: the first one is understanding what your value stream is, so how do you deliver value from customer need to need fulfilled; what does that process look like. One of the things that we do is we analyze that process. We go through and you can write it down using stickies, whatever, on your wall, go through and map out that process: how do you go from a customer need into your software development process to release into the marketplace.

That's a first step that gives you visibility. Then you can analyze all those things - you can say "What is customer value add, what is business value add and what is non-value add?" Those have some specific definitions. The thing about Lean is that it's a lens through which to look at our work. Customer value add are actually things that the customers will pay for - like software development. The actual act of coding. Business value add are things we need to do in the business to keep it running, like accounting or we need to do QA on the product. Non-value add are all of the other things that we need to do, but customers are not willing to pay for and the business doesn't need to do to run. That would be like creating documentation that no one actually consumes. That lens is what you apply in your value stream.

A world class operation will have 50% customer value add. Most IT organizations and most knowledge organizations when they haven't done any Lean work, their process is usually less than 5% customer value add. So 95% of the work done in a knowledge work process, from customer need to need fulfilled is actually non-value add to the customer. There is a tremendous chance to improve our ability to deliver to customers, just by looking at the things that are outside the customer value add business, outside the tasks that we do for customer value add.

Customers may only value software development, however we know we need to do testing. That's part of the process of delivering quality product to the customer. A short example of this is Toyota. When they think of Lean, if you are screwing a nut on a tire. In Toyota's terms, it's waste all of the motion of moving the nut down the bolt until it actually tightens to the wheel - that's waste. The only time you are actually adding value is time when it's actually tightening the tire to the car.

That's a very rigorous definition, but if you are parked on the side of the road and it's pouring rain, and you are changing a tire you can probably relate to that definition - it's your spinning these nuts on and spinning them off. If you think about it that way, it provides an opening for you to think about things differently - it's like "If we had a camming mechanism or you could just put the wheel on and you could just cam on these bolts" - that we would actually be able to change tires much more quickly. That's what Lean does: it allows you to think about your process differently and then come up with new ideas that are actually going to drive value.

Lean is primarily about seeing whole value stream and then seeing it through this lens of customer value and non-customer value?

Yes. It covers some other areas, too in terms of flow, ensuring that you are not trying to deliver too much or too little at any one time, keeping be a nice flow work through the system, understanding the capacity of your system and not going over or it trying to push too much work through the system. It also covers things like simple pull systems - how do you go from customer request to pull value through, as opposed to push value. In IT, we tend to push things through development processes. We've got this request from our business, we spend a lot of time doing requirements and analysis and then, by the time it has development slate we are trying to deal with a lot of work and we may be overloading our teams. Then you end up with quality issues. With Lean you are trying to pull work trough and continuously deliver features to your customers.

As a software developer, or as a member of a software team, or as a software manager, really why should I care? I understand, but what can I do? It seems that it's out of my control.

We all develop software for a reason: we develop software for customers. I think that there is lots of work that you can do, especially if you are new to Agile and new to Lean, there is a ton of work to do to get started. If you are already walking down this path and you've been doing it for a couple of years then you may want to look outside to see, from a Pareto analysis point of view, where the big wins? Where can I look at something and say "OK, I can tackle this and put a lot of time and effort into it" or "I can go after this and" - it's much more, in terms of value for effort, - "I can get some additional benefit from it."

The Pareto analysis - the 80 - 20 rule?

Yes, exactly. In terms of being a sophmore part of the industry we've got a lots of folks who are just starting. Folks have been doing it for a number of years and are good at both the software development practices and the project management practices. The question is "What's next for those folks?" I think that that's where Lean shows us some great things that we can adopt. The other thing about Lean is from a management point of view it provides a body of work that managers haven't tackled before that can actually add tremendous value to the business. So, there is a real need for leadership to be able to crosscut the organization from that beginning at the marketing phase to the delivery and release phase. How can we engage customers in conversations? A great book by Mike Moran out of IBM, called Do it wrong quickly, really focuses on the marketing part of being an Agile company and how do you engage your customers in conversations and continually use that dialog to fuel your development.

ou told us about the impedance mismatch. What have you seen as a solution or as a good direction or a step forward for Agile teams and organizations to move towards?

If you are moving to Agile and you want to get to the next level of performance, you need to move to a team-based organization. If you think about what we talked about a little bit earlier, with the Lean value streams, you have this concept of going from customer need to need fulfilled. Now you can organize your business along processes that deliver value to your customer.

If you take folks like, for example, the QA manager who's been busy managing 10-15 testers across different projects and figuring out who is going to be on what, there is a lot of that work that goes away because now you've got dedicated people in dedicated teams. We have a bunch of talented people who we want to retain in our organizations; what's the next step for them? I believe the next step is looking at Lean and looking at the process by which we do work from the customer need to need fulfilled. How can we continually improve that process?

Toyota has been working on Lean for 50 years. They continually deliver new ways of building cars. That's really what Toyota is - they consider it to be a lab in which they are continually refining and improving the process of building cars. Cars are actually a byproduct in their minds, which is interesting. If we are delivering software for example let's say we are delivering ERP software, we organize ourselves around teams and organize ourselves around products and deliver value to the customers, then, as management group, you want to say: "What are the needs of our process and what are the needs of our customers? How can we continually improve the speed at which we can get value to our customers?"

There is a ton of work to be done in our organizations to make that happen. I think there is a very rich field. It's different than what we were doing before - we are not doing as much HR management, we are not doing as much of that. There still is some of that, but there is much less of it. Now it's much more focused on more interesting work; it's around the process of delivering value.

The next step forward for an Agile team is to learn from Lean and work through the organization.

I believe that, for Agile to be successful in most organizations there needs to be a change in the organizational structure. How far organizations will go with that is an open question, but the best Lean organizations in the world, have really organized around this new way of working - around the process of delivering value, as opposed to the functional silos of "I do this, I do this, etc." As Agile is fairly new to the mainstream market, there is going to be this issue that businesses are going to struggle with.

If somebody is interested in learning more and taking a look at this, maybe somebody on an Agile team who is in that mismatch and is living it, what do you recommend they do ? Where do they start and where should they go?

There is a lot of great literature on Lean. Lean has been around for 50 years and so it's more mature than Agile in many respects, in terms of the thought processes and the amount of research and work that's gone into it. You can find great books by Jim Womack and Daniel Jones; lean.org is a great place to look; a great book by Mathew May on Lean product development, called The Elegant Solution. Then, take a look at the web, at our website - we have some content on Lean and Agile at inovelle.net and I'm sure infoq.com will have some great material as well.