1. Thank you for joining us. Can you tell us a little bit about yourself?
I started life as a programmer, actually I studied Math and my first job was programming. And at that time they were just starting to have computer science programs so I have a Math major and I worked at B labs doing programming and switching systems, and then I went back to school and got a Masters degree in Math and then I worked at the University of Wisconsin for quite a few years, doing programming and computers that run physics experiments. That brought me into the whole idea of using, at the time the big thing was minicomputers to run process control systems, I did an automated vehicle control for General Motors for a automated vehicle they were developing for airports or something like that. And then I got a job working at 3M doing process control systems. And I worked there for twenty years.
After a while I followed one of my process control systems to a manufacturing plant and I became the IT manager there, and we got into this Just-in-time thing, which eventually became Lean manufacturing. So our plant was one of the leaders in 3M in Lean manufacturing and because I was the systems manager, and that was a system so I had a lot to do with putting some of the really early Lean manufacturing stuff into 3M. Then I got a job doing product development at 3M and 3M does product development for a living, they really do it well and so I learnt a lot about product development. And so now I have software and I have Lean, and I have product development. So when I left 3M and decided that I was still interested in doing interesting stuff, I combined those three things to say how you can use Lean principles doing software development. And that's what I have been writing about ever since.
Yes, we have just finished sending over the final copy edited, approved edition to be typed at Addison-Wesley, so in November "Leading Lean Software Development" will become available.
3. Excellent, so can you tell us a little bit about the book?
Well, it looks like we write a book every three years. And the reason is because started by writing a book and then having written a book I said well maybe I ought to talk about it or something like that, I think going and giving classes and conference talks is interesting and after about three years of going around to companies and conferences and talking, we learned a whole bunch of stuff. We learned like another book worth of stuff. So we added the next book which was called "Implementing Lean Software Development" which was not repeating the first book but what are the details about how you would go by which is what we learned from what we teach we learn.
And then about a year ago, Tom said to me "Well, I think there is another book here, and that you are going to have to stop talking and schedule some free time so that you could write a new book and it's gonna be called "Leading Lean Software Development" and I said "No, I am not ready yet, it's a lot of work to write a book". So, we were talking to various department heads that were struggling with, they tried Agile and it always made things better but never quite got to where they had hoped it would be and so we saw a lot of people struggling and finally I said "Ok, I guess we do have another book worth stuff to say to the leaders who are trying to figure out how to make all that stuff work", so that is what the new book is about.
All of the above. I don't think you can get a really good Lean implementation without having the leadership team be on board. One of the problems with Lean, if you want to call it a problem, at least in western cultures, many of the ideas are counterintuitive. And so say we give classes to development group and they say "That was about the most frustrating class I have ever had because we can't do that, because our organization doesn't let us". We heard that a lot and so we have found that if we can't get the leadership team on board with at least the counter intuitive concepts of Lean, it's going to be very difficult for Lean and Agile to thrive in their organization.
I suppose. If you start at the bottom than I guess you have to scale it. But we have worked with a lot of big organizations and they don't use the word scale. They say "Here is who we are and we want to do this. How do you do that?" so for example in our book in chapter six there is a case study of IBM. IBM is a very big organization, I don't know maybe twenty five or thirty five thousand developers and they did Agile across the company and they didn't call it scaling Agile, they called it "Agile at IBM" and in fact they branded it in a little ad sign "Agile at IBM".
6. And so now it's new. So large companies don't call it scaling, it's just doing it.
Yes, and IBM developed their own thought trainers, they developed their own way of talking about it, something that fit in their culture and they understood that anything they did was going to be across the organization. But when they did it they basically defined Agile as short stable iterations with stakeholder feedback every iteration, and that's an interesting definition. Short – you figure out what short means, but tends to mean two to four weeks, stable - like how "done" can you make it? Something that you can get customer feedback from. And stakeholders – stakeholders to them are what I typically call customers people who pay for, or who use, or who derive value or who support your software.
And every iteration you get stakeholder feedback, how do you do that? Well it's up to the interested team to figure out how you do that but they have done some interesting things, one of the things they did was create an early access program where at the end of an iteration they post their software for download by customers who have signed a special agreement and then they set up a discussion group, between customers using the software and the developers, direct no intervening marketing people. And they actually get people saying "You know, you've almost got it right but you have to add these three features or it's useless" And then the developers as a group get to say "Oh, we'd better do that", and they did that with their trail run which was a WebSphere service, excuse me I can't remember.
One of the WebSphere products developed in England, in the UK, and they got this feedback and they changed what they had decided to do and it wasn't done, I mean at this point up until here the governance system said "Commit to what you are going to deliver" and they discovered they couldn't do that because if you got customer feedback you have to listen to it and if you are going to get it you have to have a stable iteration so they could look at it. So if you think about it short – stable – iterations – with customer feedback to every iteration, causes good behavior on the part of the development team because it has to be stable, it causes direct customer to developer interaction it causes the executives to have to allow space for change, so there are a lot of good thing that happen from that definition. So as a company they had training, they had this program, teams could use it if they wanted but guess what? When they used it they got better results. That's not scaling Agile, that's doing Agile in a big company.
Yes, this particular case study is in the book, but also the concept of listening to customers, allowing time for things to change. We have found that governance is an important issue when you try to implement Agile in any environment. Another important issue is architecture. One other place where we see Agile teams struggling a lot is in a monolithic architecture environment because Agile encourages feedback and change, adapting to what you hear from customers, and monolithic architectures do not help you with that at all, make it very difficult. So there is a big dissonance that is created if you don't have a good low dependency architecture.
And so you don't have one, so what do you do, say "Oh sorry I can't do Agile" or the thing you have to figure out how to do is to gradually evolve to a low dependency architecture. So that is fundamental. Another thing we found when we were talking with a lot of development managers is that they had lots of the project management practices of Agile but they didn't have the technical practices to go with them. And guess what? That doesn't actually work very well. If you don't have really good testing harnesses in place change is tough, if you have a bad architecture then it's tied a lot to huge dependencies, Agile is really tough, if you don't have the people that are skilled in figuring out what you need to know before you proceed, a lot of the good development patterns, those kinds of things you need to have in place, you need to have good solid technical practices in place for Agile to work.
And if you start up with "Let's see what we can do in a month" you can dig yourself into a hole pretty fast, you have to see what you can do in a month, alongside of the very important technical practices that put quality into the code at the same time.
If you try Agile without the technical practice. I have an interesting thing in the book it's like a study of history of software development, starting in '68 with the Neil conference … going on from there. And if you take a look at what went on, let's go back to forth generation languages. Tom has actually worked for a company called ForthGen. Forth generation languages kind of disappeared, so why is that?
They were very useful at first, but guess what? They created massive code basis that were basically unchangeable. Because it was lots of "Oh, now we can get users to do it" and users did it, lots of thrown together stuff, without good solid testing, without good solid technical practices. In fact many super high level languages are prone to create un-testable code. So even today we see people use code generators and they get fine for the first nine months, a year, and the all of a sudden they have got a mass of un-testable code that has become as complicated as any other code doesn't matter what level it is, you create the problems just as fast, whether it is low level or high level if you create un-testable code in a certain amount of time, nine months, six months, something like that, you create a mass of un-maintainable, unchangeable code and it's extremely difficult to go back and do it and it doesn't matter what level the language is.
It matters if you have a devisable architecture where you've got separation of concerns, you've put the stuff that is going to change together and the stuff that isn't going to change in different places, and if you have novices that are not skilled in understanding those concepts building code pretty soon you've got a mass of un-testable code and once you have a mass of un-testable code that's called legacy. And you can get it in any language really fast if you don't have good solid technical practices. Bob Martin's book "Clean Code" is a really good one because it talks about how you create code you can read, code you can do something with, code you can change over time. And you have to have that if you want to have something that is good no matter what language.
So if you throw in Agile practices all they do is get people hurrying up and developing code that is passing today's tests you'll do fine for a year and then you'll have another mass of legacy code. It doesn't last and it isn't sustainable over time. You can do the exact same time but you can say the only way we can get real true speed is with high quality. With quality built into the code and then you can go just as fast but you will end up having high quality changeable code. Then it works, so if you don't fall in the high quality changeable code into Agile after some amount of time you will find you have created a mess. Just like it happened in many other languages over the last few decades in software. Yes, you can create stuff fast but if you don't create stuff that's also got quality built in and has the right kind of architecture than it's not going to get you over the long term where you need to be.
9. So quality built-in is not just tested code, it's tested and well designed, loosely coupled.
That's much more than testing, good architecture, good coding practices these are known things but they are not known to somebody who just graduates out of school. We got this thing in the book called Deliberate Practice and Deliberate Practice comes from a guy named Andrews Erickson originally from Stockholm and he studied skilled talented people, violinists, those sorts of people, and he discovered, and this might not come as a surprise to you, but most talented ones are the ones who practice the most. But he actually put a name on it, and a kind of practice, deliberate practice is practice, think about an athlete trying to get better or a musician, you practice at what you are bad at, a skill that you need to improve, say a golfer has one problem with a swing so they practice and practice under the guidance of a teacher or a coach and with immediate feedback as to whether or not what they did was right.
So deliberate practice is all those things, practice at a skill you are bad at with somebody who knows how to tell you the right mechanism in the practice and improve it and with lots of feedback. So Andrews Erickson discovered that people who are highly talented had some ten thousand hours or like ten years of deliberate practice got them to an elite level. That original paper of his was published in 1993 and subsequently research has shown that this is kind of a magic number, ten thousand hours or ten years. You can't get three hours a day for ten years and you can't actually do deliberate practice for more than three or four hours a day. So deliberate practice over time means somebody gets better and better until they really are skilled and that doesn't just happen in sports, in medicine, it happens in engineering, and software development and just about anywhere else.
So if you think about that and you look at really good code, and you know there is a big difference in somebody who is writing code here can be ten times more productive or effective as somebody else writing code where does that come from? Talent. Where does that come from? Practice. Skill. And skill development, we've spent very little time talking about it in software development. But where I worked at 3M we had two kind of managers, we had what I decided to call the competency manager, who in our company had been the line manager, but the competency manager is the one that taught and develop the skills. I never as a technical person have worked for somebody who wasn't highly skilled, really good top master in what I was doing technically. So, if you have guidance and also you have to have challenging work assignments that constantly challenge you, and you have to have feedback. You have to know if what you do is good or not.
And people who do deliberate practice, think about musicians and athletes, they become obsessed of wanting to become better in their area. How do we do that in software? There is actually one spot where it works well and that is open source. So if I am doing open source then my code goes to a committer and the committer says "stinks" or "it's good" and I learn really fast whether or not I am doing the right thing and then once it gets past the committer, you know what, all the community criticizes it. And it works in small chunks, not big chunks, just the kind of things I'm trying to do, so there is constant feedback with more senior people critiquing constantly in open source.
And I have heard if you really want to get good in developing software try open source. Now where do we provide that deliberate practice in software development? Instead of routine stuff, well you are good at that so do more of it, well that is not deliberate practice. So real talent comes from having really good, if we go back to Bob's Clean Code book. He talks about how really skilled developers structure their code so that it is simple, it does just the right things. And whoever reads it later knows exactly what the person was thinking. That's what he calls clean code. Now if you have code like that, that you can come back to some years later say "Oh yes that's what I was thinking" then somebody else can look at it and know what it is saying then that's the kind of code that is changeable. And if you just throw stuff in and nobody can guess what you are thinking that is not good code. How do you know how to get there? Well you need to practice and you need to skilled guidance on how to get there.
10. So the technical practices are extremely important without them you crash and burn.
Well not right away. I have never seen an Agile implementation that hasn't improved on current practice. So Agile always helps a company to get better but does it get it where it wants to be? Not without good solid technical things. Can it create a massive code base that becomes legacy real fast? It could.
11. And then we will be right back where we are.
Yes, and it could if you don't have good technical underpinning. Just having a team and hoping they will figure it out is not equivalent to having a good competency leader, good skill in the team, that sort of thing.
No, ten years doesn't mean skill, it doesn't calculate like that. It means getting good at what it is you are good at. I don't consider software development a monolithic field, there should be generalists. There is nothing wrong with being generalist but I am looking for what lots of people call T-shape people, so you need somebody who has good deep depth in the particular area they are good at, it can be database, user interaction design, there is lots of different areas where you actually need specific skills.
You also need that person who work very closely with upstream and other people, so if it's user interaction design how do I do the transactions and stuff like that. So a T-shape person is someone who can work well upstream and downstream with other people but in one area they have a lot of depth and it's in that area where you have a lot of depth that the deliberate practice is the most important.
Yes, they are talking about the fact that you need to become a master and it takes time. And it's a concept that first you learn what that means and you have some sort of apprenticeship and you don't come out of school becoming really good at that. But think about the airplanes you fly, think about the co-pilot, they are basically in an apprentice ship role usually for decades before they actually become the lead pilot. And you don't necessarily want that to change.
And I think programming complex software is as complicated as flying a plane and in addition to that there is often times when there are many lives at stake.
14. Can you tell us about one more idea that's in your book that you would like to share?
Yes, there is another one we ran across, it's from John Seddon who writes about service organizations and Lean service, and he is from the UK. And he talked about failure demand and value demand. So demand is when you have an organization that is providing some sort of service like software development, and there is two kinds of demand coming into your organization: one is you want to provide customers more value, so that is value demand, but then there is this thing called failure demand. Failure demand is demand created in your organization by your failures. So you have written some software and it doesn't work.
It's got a defect so that's failure demand or it's hard to understand so people call up because they don't get it. Support calls are almost all failure demand, anything coming in where you haven't provided a clear understanding or a correct set of code is failure demand versus value demand. So the first thing to do is to sort those two out and to treat them differently. Failure demand is bad, should be gotten rid of, you shouldn't try to do it faster you should try not to have to do it in the first place. Often times people in our classes will try to do value stream maps. And one of the value stream maps that I have seen most times was "Let's do a value stream map of our change control process". And it has always strike me as "well why do you have it in the first place?" But now I find words to talk about it, so change requests are actually failure demand.
They are failure demand because either you decided too soon, so you have to change, or you decided the wrong thing. You didn't get a clear understanding of what was necessary. So what you really should do in change request is say "Why do we have change requests? What is it about our process that requires us to change? Why don't we figure out how to make decisions at the right time so we don't have to change?" Failure demands are defects coming in, so what is it about our process that lets us sent defects out on the field. So people start looking at their demand profile, failure demand versus value demand and guess what? They have huge percentages of their demand is failure demand, is not value demand.
And that is bad stuff that is stuff you should say to yourself "Why do I have failure demand? How do I get rid of it?" You should not say that it's impossible to get rid of it. Yes, it might be impossible to get rid of it in a week, or month or even year, but failure demand is something you try to figure out the cause and extinguish. Meanwhile of course you should respond as fast as possible. But you want to get rid of failure demand so you have more time for value demand. By making a clear division in your head as to when you have requests what kind of requests they are, helps you decide whether or not you should actually spend time on that request. And if you are spending time on that request then what do you do about the way you do work so that you spend less time on that kind of thing.
Yes, failure demand is an indication that your process can be improved. And it can be improved significantly so that it doesn't generate failure demand.
16. Excellent, that's a very different and enlightening way to look at that.
And you just think about it, you think "Wow there is some opportunity here if I just think about this a little bit differently".