This is Amr Elssamadisy here at Agile 2008 here with Alan Cooper. Excellent having you and thank you for taking the time. To start off with, since we are in the Agile community not everybody is familiar with Alan Cooper although they should be. Can you tell us a little bit about yourself please?

Thank you. That is a tough question. I think of myself as an inventor, an inventor of tools and processes and sometimes products. And I have been in the software business for over 35 years and the first half of that I spent inventing, designing, building and deploying software products. I was an early player in the micro computer software field, and I invented and sold software to some of the big players and a lot of people know me as the father of Visual Basic, I created the visual programming paradigm, that is behind Visual Basic and Visual Studio, and sometime around 1992 or so I really got focused on the issue of how is it that you design software as opposed to how is it that you build it, I was thinking about the process that I used to invent software.

And I began to really ponder and I began to consult, help other people design their products. And I learned enormous by my stopping programming. Because most people who can program at a releasable code level don't ever stop programming. And so it was a really interesting prospective that came into my head, and I learned quite a bit and my thesis was that the creation of good software that satisfies human users is a process that can be taught and learnt rather than just a neck you are born with. So I set out not to just do good design but to understand the process of design, and I think I made a lot of headway, I wrote a couple of books about it.

I wrote a book called "About Face" in 1995 and I wrote a book called "The Inmates Are Running the Asylum" in 1999 and both of these books were pluming this idea of how does software get constructed and how can it be designed well from the point of view of how it behaves in the world, as though it were a person with a social presence. And I have built a software design consulting company around that and we are kind of process walks, we are really, really, good at what we do and our clients tend to be large companies with big organizations and big products and they are kind of classical organizations, with methodical construction organizations, and they are stuck with this idea of how do you know what to build, how do you know what will delight your clientele? And we come in and we answer these questions.

And we are really, really, good at it. It is interesting, because of my background as a software guy the reason my design consulting company got off the ground is really because of my contacts and my reputation in the software construction business, other programmers knew who I was, and they were willing to hire me because they trusted me. And over the years my reputation in the world of programming has faded as it has waxed in the world of design, and so I have become this kind of design guru and I have very mixed feelings about that because I am very proud of my programmer roots. When Jeff Patton asked me to come and speak here I was reluctant and he finally came up with the kicker that synced the deal for me, which is he insisted that if I do come and speak here that I attend the entire conference for the entire week and that I attend the sessions.

Kurt Vonnegut says always accept strange travel invitation like dance lessons from God and I recognized that immediately as one of those celestial dance lessons. And I realized what he meant by that, that I would learn things here. And it is written on my badge, my button says "Student". And that is how I came here as a student to learn and I have learnt an enormous amount and one of the things that I learnt is that while extreme programming is Agile, Agile programming is not necessarily extreme programming. And I have also learnt that the fundamental goals of Agile programming are to let programmers who are knowledge workers get to the core of their motivation and the core motivation of all knowledge workers is to do good work. And I think there are a lot of people around the fringes of the Agile movement who don't get that yet and who think that Agile is about productivity, and it is not about productivity, I think that productivity is a byproduct, but if you set that up as your goal you will fail, but I think that the hard core of the people who are driving the Agile movement understand that it's an attempt to reconstitute the motivations behind, what lies behind the open source movement in a captive commercial purpose full market place.

And the thing about open source is that it is self directed, it is self motivated. And in order to be self motivated, you can't have roosters coming around you squawking and give you a tangential instructions. And that is very much what Agile is attempting to do only doing it in a business context. And this is exactly completely congruent with everything that I have been trying to do in my interaction design work for the last sixteen years. And so I am extremely happy with that. Like I say, there are still a lot of people at the sides going "Faster releases" and they are just as clueless as they have ever been and probably continue to be. I wrote a book, ten years ago calling "The Inmates Are Running the Asylum" and a lot of people have misunderstood the title, a lot of people who misunderstand the title are not the inmates, they are those people around those people talking about productivity.

I think that a lot of the inmates do kind of understand that the inmates running the asylum is that it's not a palace coup but a royal abdication. And that the inmates, the programmers, the technical people, are in fact running the asylum because nobody else is running it. There is this idea of being left over from the industrial management that you say "We are making refrigerators or automobiles" and then everything goes down in that pyramid in that classical manufacturing hierarchy and down to the lowest level, the individual tradesmen contributing their bolts to the mechanical device. And we are post industrial, programmers are knowledge workers, they not only don't think that way, they don't act that way, shouldn't have common values to that, and the organizations aren't structured that way. So it has created this huge vacuum and management is horribly hobbled by their industrial age beliefs.

And what that has done is that kept them out of the vacuum, that's the route of the abdication. And the thing about programmers is they didn't have the right skills, they didn't have the right talents, they didn't have the right training, they didn't have the right motivation, they didn't have any support at all and all they saw was the vacuum and they stepped into what they had, and that's what I wrote about. Now, I also wrote about how well the programmers stepped in as inmates surrounding the asylum, they were doing it without all those appropriate tools.

Mainly all of the things that cause software projects to have cost overruns and schedule overruns to be unpredictable and unlovable and often unusable and frequently not more than half the time the death march that produces pieces of software that just rolls over and dies upon roll up. They brought the same wonderful gifts to the center of the knowledge worker company, the post industrial company, where it's not just a cost centered service buried deep within the organization but now it's the database of the company that has been revealed through their website, directly to their customer base. I mean it is the toxic means of software construction that have become the toxic means of business. Wow, that's pretty bleak, I mean that's really, really, bleak and what's interesting is that's what I saw in the programmer community seven or eight years ago.

I saw that there was a deep on "we", there was a boredom and a frustration that went deep. I found the best programmers were on the benches, they were not building software, they were talking, they were writing, they were speaking but very little goodness was getting done. And in the .com bubble was this enormous distraction and Extreme Programming was a way for programmers to hunker down in their bunker in this hostile environment. And things have changed, they have changed dramatically in the last few years, and extreme has morphed into Agile, and I really think that there is something quite existential here, is that you have to hit bottom before you can change.

And I really think the programming community hit bottom. I mean again back in the old IT days, they just came in and punched the clock and did their work and some went home and did open source and then it went on and that was ok. But now that it really is the beating heart of business, the programming community has become aware not certainly by the things I have written, but they became aware that they are the inmates running the asylum and a few of them kind of said: "Damn that as hell, I am not going to take it anymore. I am going to do something about it". And they stepped out and said "What do I have to lose?" A miserable job, in a miserable company, with a miserable boss, with miserable schedules that are just fantasies, producing a piece of software that nobody likes and when it's done it's going to be ignored. And what are they going to do? Fire me? I will go to another company where I will get the same thing. So what they did, as I see it, maybe I am sort of post facto putting a dramatic story on to something that was more prosaic.

It's a great story.

Thank you. They finally said "Let's do what is right". And be damned to the industrial age managers, trying to survive in a post industrial world. And this is where the brilliance starts to come through. I just came from a workshop called "The beginner's mind" and this is a Zen concept, it's the way you achieve true understanding, is by being the student, is by being the beginner. And this is what programmers, and programmers who are the experts, they are the smartest people in the room, regardless of what room you walk in.

And what they did - I love this about programmers - is they said "I don't know". In fact I am unclear about anything I am doing here, I am unconvinced that anything is correct except of what I know is that the way I have been doing things doesn't work. It doesn't make me happy, it doesn't make the business happy and it doesn't make the customers happy. So what I am going to do is I am going to question everything. And one of the fundamental ways they were going to question everything, and this is one of the reasons why I know they are deadly serious, is because one of the core tenants of extreme and Agile programming is pair programming. If there is one atomic element of programming, it's code.

And what they did is they opened the beating heart of programming to inspection by others. This is a reflection of the sincerity, the honesty, the profound depth of their questioning, their Zen mind, to open the code up and then they begun to open up the idea of how long is it going to take us to do this, how do we know what this is? Wonderful, fundamental questions that have been never truly been asked or answered in the world of software, mostly because the world of software has its roots in academia, rather than having its roots in industry or commerce.

Industry or commerce may have primitive methodologies, but they are always about effectiveness, always about achieving goals. In Academia the goal is to learn, and to fail is to learn, therefore failure is ok. And that's one of the reasons why I think the software world has long been willing to accept failures as ok. But in a modern digitally interconnected world failure is much more noticeable. So I think there has been this fundamental questioning, and the wonderful thing about it is that when Agile is done right.

Bill Gates used to say "I know how to make user friendly software, get a big rubber stamp and stamp user friendly on the box". And there are a lot of people out there saying "I know how to make software development Agile, stickers on the wall and a 3 months turn around" And neither of them worked, neither of them were honest. So you really have to do it right. Now there are lots and lots of practices that compose Agile and I am certainly not going to stand here and tell you which of those practices are the right ones, in fact I will take that back, I will tell you which of those are the right ones, they are the ones that work for you, but not the ones that make you feel good, but the ones that allow you to create a success by external standards and by the internal standards.

Do I feel actualized in the work I do and does the product got there and pleased some constituency who was paying for it in a commercial environment? And that's the definition of what Agile is and it can be any collection of those things but at its core it's introspection, it's doing it and then make sure we did it right, and then pay close attention to what parts of it we didn't do right, and bringing some questioning analysis to that so to increase the proportion of rightness in what you do in the future. So the essence of Agile is reflective, it's transformative and it's on going, and it's self correcting.

When you say design, for our audience, we think architecture and design as in design patterns and I get the feeling that is not the type of design you are talking about.

There is an awful lot of design that goes into software. Software is not an industrial product. In an industrial product, design is done and then a thing is manufactured and lots of people labor executing the design. In the world of software that first step also exists, there is the design of what would the product be. And in manufacturing design there is also the design of how it will be built, but then the construction is not a repetitive thing, every single line of code is different, and so the design continues all the way down to the lowest levels, or furthest levels of the execution.

So, unlike industrial processes where design begins and ends and then construction begins, in the world of software design permeates construction. However, there are very different flavors of design in the world of software. One of them is the design of the problem. Most programmers know that to be a given, then there is the design of the solution, mainly we know what the problem is, now how do we solve it? Then there is a third design problem which is how do we build it? And there is a forth design problem which is building it. Now in the world of software, the problem is moot, generally the design of the problem is moot.

It is usually some business thing that is ended up from high that, by the way, doesn't work, but that's the status quo anti. Design problem number two of what would the product be and how will it behave, that's what I do that's what I have been focused on and that's what I call interaction design, the design of form and behavior: what does the product do, who does it do it for, how does it behave? The next stage of design is how do we construct it and I call that design engineering, and the forth stage of design is construction engineering.

Generally those last three phases are conflated and in the old battle days we are done just under pressure by some random group of people and usually the human facing side of it was given the leak of a promise and then everybody as slaved over the design and engineering aspects of it. Yet another wonderful thing about Agile is that by being relentlessly self introspective, it exposes all the weaknesses in the process. And one of the great weaknesses in the process is that there is nobody figuring out what the problem is and what the solution is. There is lots of good people figuring out how to build the solution, and there is lots of people figuring out actually building the solution.

But if you built the solution to the wrong problem?

Precisely. And there is an enormous amount, I mean it is considered normal in the software business, that building the solution to the wrong problem is normal, and you will go on from there. And this is why multiple releases, I am not talking about just builds but I am talking about real product releases. I mean Microsoft led us all down on the path, yes we know we are not all going to get it right until six years after the initial release, and when it's 3.4 which is just called 3.4 but is actually 7.9 and that's juts crazy.

That's the bondware talking, it's not the way it should be, it doesn't have to be that way, it's just toxic old industrial construction methods under pressure being pushed on a community of knowledge workers it's what gets that and it's heinously wasteful and it's incorrect and it crushes people's wills and happiness. So it is great that what Agile is doing is actually saying "Wait a minute, there is a hole here". Now this is where you are going to accuse me of being self serving, because I am saying right and into that gap in the process is where interaction designers go.

Actually what I was going to say is "Boy, if I didn't know any better, I would say you are one of us, and we are one of you, and why are we two different communities"?

I couldn't have put it better. That's exactly right and here is the thing: as an inventor and as a designer I have one objective: to make a great product that people love. As a community of technologists who are knowledge workers being managed by industrial age managers who are treating them like factory workers, the goal of programmers for the last fifty years has been to survive and most of what they have developed are coping strategies, how do I survive in a hostile environment? Agile is unique as in its goals are not to make it easier for me to deal with hostility. I mean, yes, it does that, but its number one goal is to create good products, successful products that people like and will use.

And you can only do that by being honest of what is not working and what's missing.

Yes absolutely and all processes feed into that. But that's why the goals of Al-The designer, and Kent-The programmer have become congruent in the last few years is because the programming community has said "We are tired of surviving, we wish to thrive", and that's where I have always been and I have suffered because I have a tiny little influence on the corner of the industry, but I really feel like as the industry adopts this introspective quality centered new paradigm, things are really going to change. I really think that the management community, they are going to be forced to abandon their industrial ways.

You pointed out that there is a hole and that hole is interaction designers. You also hear some instances, like Jeff Patton, are marvelously successful, others are maybe not because of the nature in which we do things; but the nature of generally there is a too different company, there is the interaction design company and there is the software development company, and it's a throw over the wall mentality.

Look, I would say some enormous percentage like 99.9% of the software in the world sucks really badly. I would not say that 99.9% of the programmers in the world suck really badly. I just think that 99.9% of the programmers in the world are in an hostile environment. I think a similar quality of an output product of the design community sucks really badly, not because they suck as designers but because they are also in a hostile environment, mainly the environment is not conducive to create good software nor is it conducive to create good design. So one of the problems you get from that is when programmers go Agile and come up for air and they look around and they see interaction designers who are curling little hand grenades of bad design over the waterfall walls and they are going "Geez, they are having bad management".

And it kind of looks like that. Now, yes of course, there is a percentage of incompetent interaction designers out there but just as there is a percentage of incompetent programmers out there, but I would submit that they are about the same and they both are pretty small. So there needs to be an organizational change, for Agile to thrive and there needs to be organizational change for interaction design to thrive. One of the chapters in my Inmates book was called "Skin in the game". But I love "chickens and pigs" so much better, it's just an wonderfully colorful metaphor and it drives the point so much better than "skin in the game", the one thing about programmers is they have been pigs from day one. And they will always be pigs. And interaction designers, the good ones or the majority, also want to be pigs. The organizational structure forces them to be chickens, and this is the thing that I see it's happening in Agile that is so intriguing to me it's that it's creating this opening in the pig barn and say "Come on in, do you dare"? And what I am saying is this is going to be an epiphany moment, for each individual interaction designer out there. Are they going to be a usability tester standing on the side line saying "You build it and then we will see if it is any good"? Or they are going to step in and say "Here is a ham"? Are they going to step in and occupy that gap in the Agile team that says "I will take responsibility for the quality of the behavior of the human facing side of this software from start to finish".

Because that's what the programmers do, they stand up and say "I will take responsibility for the quality of the execution of all of the software from start to finish". They have got the pigs, they've got skin in the game. For fifteen years I have been saying the definition of a good interaction designer is one who has skin in the game. And the organizational change that Agile is bringing is for the first time seriously allowing that and you can see this is my organization which is over sixteen years that we have been in operation, we have gone from delivering solutions to IT departments or to programming organizations we have gone to delivering solutions to management.

And I would say in parallel with that arch has been the lessening of authority within the organization of the programming community and what Agile is, it's an upsetting of that apple card, and it's programmers reasserting their authority and saying "Let's not follow bad leaders". So I very much see a shift from our design organization delivering solutions to management which are worshiped but not followed to feeding directly into the Agile cycle because the thing that motivates good designers is the same thing that motivates good programmers which is seeing their products get built and having them satisfy the users.

It sounds like we should be seeing a merging of these two worlds in best cases, of interaction designers and Agile

Yes, and one of the things about open source it lets programmers get into pure motivation, but the problem is, the thing about open source is you are struggling with the solution you can change the problem definition.

It's not industrial.

Yes, but it's also not commercial. So if you happen to be in the business of needing a language compiler or an operating system or some kind of a post processor, or some system software that's the kind of thing that programmers like to do for fun, well you'll probably find an open source version of it, but if you want an invoicing system or some calendaring program for human users, something boring, you are not going to get it, or if you do it's going to be a version that only a programmer can use or love.

This is Deborah Hartmann stepping in from Amr. Nice to be here with you. So we have been talking about the history of interaction design, the history of Agile and also the shared values that seem to be natural between interaction designers and Agilists in the engineering area. Can you talk to us a little more about how these two skill sets, how these two passionate groups can work together?

Well, it just so happens that this is one of the core competencies of a good interaction designer. All the reflectivity that Agile programmers do within their Agile programming organizations are "Let's look at the architecture of the code, let's look at the code, let's look at the models within the code, let's look at the objects, and let's do all that reflection", that's what brings quality. So where interaction designers come in, one of the places they come in, is they bring the same level of reflectivity, to the business problem. The number one thing, not the number one thing, the first thing that interaction designers do is they plum the brains of the stakeholders and the users and the customers. Users and customers are usually not the same and stakeholders typically have lots of interesting things to say, some of which is useful and most of which needs to be transformed to be useful.

Then what interaction designers can do is they can reflect the business problem back to the stakeholders and they can reflect the user definition back to the stakeholders, and this is a step that is missing from almost all conventional or Agile software construction projects, and is one of the reasons why so many projects, even if they are Agile, can go off into the weeds because the project definition itself is incorrect or inappropriate or misaligned somehow. The next step is, once the interaction designers who are experts at doing this kind of human research and understanding what the users are trying to achieve, then they can block out the behavioral solution and they use Agile methods within their own teams to create proposed solutions and we have simulation tools for narrowing down to a good sense of correctness, but then what we can do is take those proposed solutions in narrative form and in visual form but primarily in narrative form, let's think of it as an user story on steroids, is we can reflect those back to the stakeholders. Now again, this is a process that is typically missing from conventional software construction processes.

This is this sort of disconnecting, the hand off, the business people are cycling over here and they hand over one of their hand grenades in the tech group and the tech group cycles with it until they come up wit something. And they have user stories and they have got their stakeholders and people come in but there's never really this formal let's state the problem, let's state the solution.

It's a skill set that is missing in there.

Precisely and then this is the skill set that the interaction designers bring to the problem. That's all done before day zero. After that there is a whole bunch of user interface, that have to be done and there are a lot of programmers who are good at that too, and a lot of product managers who are good at that, interaction designers are good at user interface design too, but user interface design is not what makes your product a success or a failure, but it's really that deep and profound understanding of the problem and understanding of the solution and making sure that the business side is what they want. Exactly. Who are the users, what are they trying to accomplish, what motivates them, what tasks are they performing, but what is their in stake, what does success look like, the same way Agile programmers want to know what done looks like, interaction designers want to know what success looks like. And those two work together to create a successful product and that is the missing piece.

Tell me what's next for you?

I think what's next for me is more of the same. In 1992 I went out into the world of programmers after a long absence and begun to do some interviewing and some researching and I found the programming community morale to be the at the lowest ebb I have ever seen and programmers were unhappy, and disempowered and on the side lines and frustrated. And I came home to my little interaction design organization and I said "Let's turn our focus towards management. Instead of trying to get into the organization through the programming community let's try to go in through the management side.

We need to make in roads, we need to understand business managers and we need to understand how organizations work, and we have been remarkably successful at this and at my company we typically work at the highest levels of business management at some of the largest corporations. And they are very pleased with our work, and now as I go out into the Agile community, and I see the seed of this joyous behavior of introspective programmers trying to create good product, I can clearly see that where we go next is to re-establish those clean lines of communication with the programming community.

I kind of presage this several months ago in February '08. I gave a keynote talk, at the opening keynote at the IXDA interaction design conference in Savannah, Georgia. And I decided to do something really different and I gave a talk called "An Insurgency of Quality". And in that talk, what I did is I basically said to the community of interaction designers: forget management, they are a lost cause, they are not getting it, let's instead go directly to the programming community and go to them and make a bargain with them directly say "We can help you do your job if you will work with us". And that was the insurgency of quality is by passing those corporate path ways, and I find kind of to my surprise there was more presage than I thought because that's exactly what I'm finding today here at Agile 2008, that there is the other side of the insurgency already exists and it is called Agile.

Before we go, one last question: for those people who are watching this video, do you have any advice?

Yes, I do have some advice. And thank you for asking this question. The economics of software are qualitatively different than the economics of industry. In industry there were economics of scale, and what we tried to do is drive the cost of manufacturing down in order to drive the price down, in order to deliver value to the customers. In software there is no on going cost, there is no manufacturing cost there is no material costs, so driving cost down just reduces the desirability of the product. Instead what you want to do is not waste money, not throw money at the problem, but cost reduction is an ineffective tool. What you want to do is create your number one goal, to say what do we have to do to elevate the quality, the desirability, of the end product. And when you worry about costs, you hurt that. And one of the great things that I see in Agile is an understanding that says "Hey mister businessman, stop worrying about the costs and start worrying about the quality". And what I am saying is thank you, thank you, thank you, don't let that out of your side, don't start thinking about delivery times and don't start thinking about costs reduction and don't start thinking about ROI, think about the quality because that's why we are all playing this game.

This is the difference between efficiency and effectiveness?

How do you know that?