Ward Cunningham on Agile: 10 Years After

1. I am Michael Floyd with InfoQ. I’m the chief editor, and we are here at Agile 2011 in Salt Lake City. I am sitting here with Ward Cunningham, how are you doing Ward?

I am doing great. It’s good to be here.

2. Ward really needs no introduction, he is one of the original signatories on the Agile Manifesto, and also we know his work in design patterns, extreme programming, of course the wiki, but this is the ten-year anniversary of the signing of the Agile Manifesto.

That’s right and this is actually the first time that the bulk of the editors have actually gotten together again, we all know each other and trade emails every now and then, but to actually be together, and they put us on the stage together and we had kind of a group discussion about where we are at and where we are going and so forth. It was kind of funny because it was a lot of collective remembering. We didn’t take a lot of movies or a lot of notes and so forth it was "Well yes, I think we were really trying to do this", but there was this magical moment when those four values just got written on the board, and kind of in counter point this over that and we just looked at it and said "Oh, that is such a perfect expression of what we collectively believe, even though we are all very different people".

3. It was interesting hearing the recollection of how that inspiration came about.

Yes, and everybody is a little vague on it but when it was on the board it was like "Wow, that is so good, that is so much what we want to say". And of course a lot of people have read it. In the last year and a half it has been translated to almost forty languages that has been a real effort and it turns out to be very difficult to translate. The words have a lot of meaning but you sort of read yourself in those meanings and so what we’ve done to do the translation is we asked every translator to find the Agile community within their country and kind of work it out together. A lot of people have found that it kind of makes, you know that it’s a project, that the Agile community come dirty, really pull something together and then we get a lot of reach.

4. Talking with Jeff Sutherland yesterday he said that there has been a retrospective basically that took place in February.

Yes, it was on the date, we are here in Salt Lake and of course it was kind of invitational workshop ten years ago where people who were the proponents of a specific methodology that had the - at that time it was called lightweight methodology got together and talked about it, we say that lightweight has actually a negative connotation, kind of not really very strong. So we crafted that manifesto and then we said well but what are we going to call it? We are not going to call it the lightweight manifesto. We picked the word Agile and the rest is history.

5. So the names are really important.

Names are important. And of course I was a fan of extreme programming, which is actually a pretty comprehensive prescription for how to develop software. And Agile was kind of an umbrella term that allowed extreme programing at one end of the spectrum and really much simpler things at the other end of the spectrum. And there are a few extreme programing proponents that felt that it was a real step back to broaden the tent but I think in terms of having impact on the world that it’s better to be a little less dogmatic and a lot of the principles of extreme programming have found their home where they are welcomed.

6. So, you were probably trying to solve maybe perhaps a slightly different problem ten years ago, hopefully, than we are today. So, we are kind of setting the landscape for that period when you were sitting down with the other signatories. I myself was a C programmer, and back then the notion of quality assurance was actually kind of a new concept. We had this C programming language which was kind of a sharp set of Swiss army knives that you could cut yourself very easily with, we kludged together a lot of code back then. So Agile is about software craftsmanship and caring about the quality of the work you are doing.

You know I think that what Agile really brought to software, is an ability for practitioners to actually watch each other work, to agree upon what their goals are, and then just gain experience. There are plenty of people, you know for as long as there have been computers, there have been plenty of people who produced high quality work, they know what they are doing because they’ve done it a long time, they’ve had access to a computer they try a lot of things, what we wanted to do is make it easy for people to have that kind of sense of exploration, the ability to try things, if something is not going well to back up and try something else without undo pressure, from schedule. In other words to say if we want good there has to be time to do good. And we need to make the relationship with what the customer is expecting and what we are doing, line up.

I think a lot of bad software is written simply because developers thought that was what they were supposed to do. And somebody who doesn’t know software isn’t going to tell you to do otherwise because they don’t know software. So it was really kind of software developers stand up for what they know has to work. And that has worked well. There has been a little drift from that actually, as it becomes more popular and more people are drafted in as coaches and so forth and we get people who aren’t software developers, reading the books and understanding the practices and managing and facilitating those practices. Without actually having an empathy with what it is to design software. And it’s not as strong as it was.

But there is always people who will be devoted to software and the perfection at it and I think that’s what’s bond the craftsmanship movement that says "Gee, not only do we need Agile to manage a schedule, we need individuals to take responsibility for the quality of their code". I’m not saying that it wasn’t like that but I think that that’s recognized now as an important part of software development and career development.

7. Maybe playing off of this, I was reading in one of the other magazines which I forget right now, and in that interview you said, just a quote, "I have seen my ideas diluted as they diffused through the industry". You said "I’d much rather move to the next idea than struggle to keep the last idea pure."

Diluted is a little judgmental. It’s more like my ideas get plugged into bigger frameworks and to me it looks diluted because it was like aimed to what I was trying to do. And people do other things. On the other hand I think the industry keeps moving, I was interviewed once early on when extreme programming was catching just on and the interviewer asked me "Well, in your own work, do you use extreme programming?" I said "Of course not, not always". And they were floored and I said "Well keep in mind that if you don’t try different ways to do things, how would I ever come up with extreme programming?" The computer is so powerful that you have to keep trying ways that it can work and of course since then the Internet is really flourished and I think something that is really hot right now is the client side, the browsers, HTML 5, and JavaScript is getting fast and the approach to JavaScript is becoming uniform and that’s a computing environment that isn’t going back to desktop.

But it isn’t completely the document model, it’s something new, and finding out how to use that well and especially on the new platforms, on pads and stuff like that, it’s pretty exciting, there is a tremendous amount of potential there, the computers keep getting faster but finding the right way to approach that kind of problem. I think Test Driven Development will always be a part of it. I think learning from each other, and of course I’m a fan of Pair Programming, because there is so much to learn if you try to do it alone, it’s just overwhelming, but to learn by doing with others, you read one chapter, I read another chapter we see if we can put them together and get something to work.

8. We are better collectively than we are alone.

Absolutely and it makes the learning fun instead of drudgery.

9. Sure, when you get stuck you have someone to help you get unstuck.

Yes. Well the other thing sometimes when you get stuck you just look at this and say "I’m stuck" and your partner says "Well, I’m stuck too". And you say "Thank goodness, I thought it was just me" just being stuck and knowing that it’s ok to be stuck. It’s oh, now we have to think about how to get unstuck. That’s amazingly empowering. I don’t think any developer really wanted to work alone, but they got a reputation as being loners that was unjustified because they did so much in their head. What we’ve done is, computers are powerful enough that you can do it on the screen. And when it’s on the screen we can have a conversation.

So I think this needs to be like that and the finger pointing becomes the nouns, because the nouns are so complicated, if we tried to pronounce the nouns, some path expression or something, it’s just too much to say, but this needs to be there in the sentence, and you know what I mean, because you put it here and now it needs to be there, and we can have a conversation about something as complicated as code. It never seemed to have worked on a whiteboard by the way, because you draw the boxes and you listen to me and you say "Oh yes, but here is what I am thinking" and you erase my drawing and draw your boxes and arrows in there and so we are really not communicating at all.

10. So that’s where the tools of collaboration have really helped us as programmers. And that kind of leads me, segues into what you are working on right now.

I am lucky I’ve recently taken a one year appointment with Nike in their sustainable business and innovation group as their open data fellow. So I am very interested in what data, to make data as mobile as, well, originally code. We were all about making code move around in an organization, understanding code and what work for code, I kind of brought those concepts into documents with wiki. Wiki was modeled after what were doing in code. Agile code, wiki documents and now I am working to see if I can do the same thing with data. If you think about it the last real advance in humans using data is a spreadsheet. Most of my career has been focused on the kind of modeling that Alan Kay dreamed that we would do, with objects and SmallTalk. Truth be told, people don’t build SmallTalk programs, and make important decisions in their lives based on those models, but they do build models and spreadsheets and make very important decisions based on what those spreadsheets say.

So that’s a modeling environment that an average person can do, maybe not an average person, but a lot of people can do and then make very important decisions so I think it’s possible in an open data environment where the variety of data, the flow of data is much higher than you can really sustain with a spreadsheet and make that. Well I think it’s going to fuel, in an outcomes-oriented perspective, and everything that we do in our economic world, a world of ever diminishing resources, and how we decide to live in it.

11. The other evening when the signatories were all up there, there was kind of a spirited discussion about Lean. I am not sure if you were actually a part of that, but I would like to get your prospective and the question boiled down to "Is Agile Lean and then is Lean Agile?" Do you have any comment on that?

You know I think that again Agile is an umbrella term, and so it certainly was created to cast over those things, we didn’t have, we had people who understood Lean, at the workshop that drafted the manifesto, but I think Lean wasn’t seen as an important movement in software at that time. As it’s become more and more of interest, it turns out that in the conversation last night was that there is actually a wide variety of Leans. And people have borrowed some things from Lean and kind of made a softer Lean that isn’t really consistent with what the I don’t want to say the true, but let’s say maybe the most productive Lean that’s been applied in the most progressive manufacturing. And even both of those sort of fit under the umbrella.

Now it’s a real mistake as I was just saying to try to find the perfect methodology for all situations, which really want to do is understand what any practice what it really brings to the situation. And this is something that we kind of got early on with extreme programming, we felt that people were programming in a foolish way. And when we looked at any one thing they were doing there was good reason why they were doing it. But the whole was foolish and so what we did is we had to change like a dozen things at the same time. And we changed a dozen things, we sort of re-jiggered what influences what and why, the industry has sort of stabilized in this idea of way too much planning. Fortune telling, we call it, trying to tell the future and then trying to make that future happen, and when it wasn’t happening just working harder, delaying schedules, whatever, and it was foolish, and instead what we should do is say "Well what is, and what are we going to do tomorrow?"

12. So optimization was not really part of the extreme plan.

Optimization, well a lot of people want to know how can I get more software more cheaply, or how can we avoid these disasters where months go by and nothing happens. I sort of feel that the formulation of most Agile methods are a little plodding, you are coming in, you do the same amount of work every day, but you don’t have days go by where nothing gets done. Like the tortoise and the hair, and the tortoise wins the race, because the tortoise doesn’t get stuck. I just hate it when I am programing and I just stand there looking at the screen and I don’t know the next thing to do. I always want to know whatever situation I am in "Ah, I am in this situation, I’d better do this". What I like to do is program the computer to tell me what it wants to do. And test-driven is really a simple approach for that, you write a test and the test passes, you are allowed to go forward. The computer is telling you what it wants to do.

13. A couple of questions: first of all for an organization - our audience is the enterprise market - and they are not starting projects they are taking on legacy projects. And for those that are maybe perhaps in the early stages of adoption or in the middle of it, even at late stage of adoption, they may be dealing with other teams across the globe in different stages. We kind of see this is a misalignment, and do you have any advice kind of a general advice for pulling these teams together.

There are a couple of principles that fueled our creation of practices. One of the principles is that when you get to know a person you become motivated at many different levels. If you are looking at a document, and it’s got somebody’s name on it and you are going to read the document and guess what that document means. But when you get to know a person, then you understand what would write that person, and it might be only a small part of what that document is, especially if you get a habit of delivering and you find out what they like and how delivery changes their minds. So this idea of being closer to more people, as many people as you can. The other thing is don’t be afraid to move in small steps. It’s more important that you move in steps, regularly and then be opened to where you’ll be then. I know that for all manner of reasons we’ve produce a lot of software in the last few decades that is no fun to work on.

But if you say what is the business need and what can we do, and if we know what we don’t know and we make a plan for that the next week, and then just work away out and don’t apologies for what’s there, don’t promise things you don’t know you can do, in earnest, chip away at it. Just know more tomorrow than you know today, and that can be about the code what you have to work with or what the real needs are. And that is the basis of anything Agile, and that can be applied anywhere. Now there’s a lot of people who are doing that and kind of offering up techniques that work better in some situations than others. The one I like is if you don’t have any tests just try writing some tests, test your way into the code, don’t look for high levels of coverage but if you see something then write a test to see if that seems to be true. And then you grow that out until you have control over code, that you might not have control of otherwise. But maybe you are in a situation where a little more daring, maybe your plodding isn’t going to solve the problem, and then you just have to be daredevils, and of you have to be daredevils, well then be good at it.

It’s a crucial be you’re trading off the future for the present when you do that. I am actually proud of having coined the term "technical debt." And a lot of people say debt that is bad, you don’t want any technical debt but I actually think as anybody who has ever taken some venture money knows that you take some debt to go faster. And if you want to go fast you take some debt. But the misunderstanding was that people thought that you never have to pay it back. You are going to debt you have to pay it back or you go out of business. It’s called bankruptcy and there have been plenty of bankrupt projects so kind of managing, understanding the choices you are making and don’t be afraid to make some daring choices, but know the consequences of what you’ve done. And I think that the other thing is that computers are awfully powerful. If you stop for a moment to say here is what I am doing day in and day out, can I step aside from what I am doing? And write a program that helps me do what I am doing? One thing that I like to do is write programs that read my programs.

14. Strapping basically.

Well, not just to do something, like one thing I found very valuable to look at a huge program I just read every file and squeeze it to one line. The easy way to do that is to do everything except for some critical punctuation: like the curly brackets, the pluses, the assignment operators, semi colons, and you see a string of that, one per line and you can look through a large code base and see patterns. Now, you see some crazy "Oh, look at this one has all semi colons, I wonder what’s’ going on there", then you go look at that file "Oh, I see what’s going on there". But it’s sort of a signature, so you wrote a program that takes something that is too big to look at and makes it possible to look at from a thousand feet. And then you have to learn "Oh, I see these patterns, what kind of code is in these different patterns". So it’s making the computer help you, it’s not test driven development, but it’s something else like that, it’s putting using the computer to help you do your job. And we are all programmers, so let’s program.

15. Is it fair to say that you think of yourself as a developer, I mean we can call you many names, researcher, is that where you place yourself at this point?

I would say that if I look at my life that the kinds of skills that I really perfected would be straight ahead development skills. I also have found a way to turn a lot of problems into development problems, and then use my skills as development to do something that fits in the organization in a way that the organization wasn’t expecting. I like to tell people that the shortest path to exceeding expectations doesn’t necessarily go through meeting expectations. Now if you are a salesman and you are only being measured by sales volume, it probably does, before you can sell ten million, you have to sell eight million. But in something as complex and as multidimensional as product design, there is a lot of ways to approach it and sometimes if you just step back and say "This way is just a slug, I wonder if there is a shortcut going around this way", and like I say the computer is really powerful, how can you have that computer help you in a way that most people wouldn’t think that they are allowed to program.

16. Is this the kind of thought process that brought the wiki?

Oh, absolutely. It’s funny I did make what can be called a prototype, sort of a mini wiki that I did in HyperCard a few years before I made the wiki, and I was in a research organization and HyperCard had just come out and I thought "Well this is really a weird program" it’s a drawing editor and a database and a programming language. It’s just this hodgepodge of stuff and I had the liberty, I took the time to just play with it and try to figure out what I want it to be. And then I thought well I want it to be what I would call a ragged database, it’s not rows and columns but it’s really a database. And I thought "What can I put in that database that is sort of ragged?" I had a pet theory at the time that engineers are really conservative and they wouldn’t do anything on a project unless they’ve seen it work somewhere else before.

And it’s that seeing it with their own eyes that gave the confidence to put something in there. So I thought I work at a technical company - Techtronix at that time - and I thought well, let’s see if I can build a database of all the ideas that have ever moved from project to project in Techtronix. Well, I don’t even know how to do that but I said I will start; I’ll start with the people around me. So I made a thing where you could write down a project and mention who is on it and what ideas were in it, and where they went for their next project and I just made a web of people, projects.

17. Kind of Kanban-like, isn’t it?

Except it was historical. It’s like what were you doing before you did this and how did you get the idea to do this? So you got it from over there. But I made it so you can do this top down editing, so you can refer to a project with a hyperlink, when there wasn’t a card there and that’s not the way HyperCard wanted to work but I found a way to get it to be able to refer to pages that didn’t exist. And then people would just sit down and start reading, and then pretty soon they would say "Well that’s not how that project worked". Because there were a lot of senior people around me who knew a lot more about my company than I did, they’d say "That was really for this reason", and I say "Go ahead, type it in". And then they type it in and they type something else in and something else in and I couldn’t believe my desk.

Because they were pouring their hearts out, writing this history of ideas. When we started looking at patterns, substitute patterns for ideas and we said "What we really want to find out is what works, let’s not just take any ideas, let’s just take the ideas that are shown to work over and over, we need a database, and the web was new", and I said I wanted to make something that had that right about things that are on the edge of your ability to understand, be able to talk about things out there, that don’t exist yet and that’s become the red link for something, that’s the link that’s incomplete. If you think about it what it really does is how can we make a database useful when the database is incomplete? Well that’s exactly the same as how can we make a computer program useful when the computer program is incomplete.

Agile is all about first iteration, you make something that works, and then you add more to it, so you grow it. Well wiki is the same thing, how do I make a database of ideas in my company, that’s incomplete but still useful enough that people want to add to it? And the feel in working in that in text is kind of like the feel of doing Agile really well. And when I did it in HyperCard I feel it’s all I have to do to do wiki is achieve the same feel, and I knew it when I had it, it was that simple, it actually came together very quickly. So that’s it. Now I am trying to do the same thing for data, as I mentioned, and of course the tool set is completely different than it was fifteen years ago, it’s client side, so I have a wiki now that’s all in JavaScript, runs in your browser, the only thing it talks to the server is to move Jason documents back and forth, and in those Jason you can embed data in there, that could be rendered in a wide variety of ways. And what is neat about it is you can have a wiki full of data, and I could just grab data off of it and put it in my wiki.

In fact, when I grab the data and edit it, you can say "Oh, that’s an improvement" and grab it back so that it’s an environment for passing things around. And again I have to look at computer programing for advice. Just as what we are talking about with Agile, that the computer programing methodology came before the wiki methodology, well the best sharing that I know of right now is on github. Github has a good merging model, your work and you can fork my work, improve it and I can bring what you’ve done back into mine. And that federation is the key to casual collaboration so to be able to do the same thing. Now if you are looking we can share a document, and I can check out the document and whatever and I can watch you while I’m editing, and have one document, the definitive document, but I think the breakthrough is to say "No, there doesn’t have to be a definitive document, there is your version of the document and my version of the document, your version of the document", that is all versions and then we read all of them.

Now to make that work you have to have one way to say "Oh, how is this document different than this document?" and the way I do that is by keeping enough history, so if you move a paragraph up to here I know it’s a move. I don’t see it as a delete, you’ve just moved it and kind of watch each other. So I think it will be used a lot for anything you read, you just make a new version of the document that’s kind of the cliff notes, the part that you care about and it’s kind of like underlining except that you just make a new document.

And then when you read my cliff notes, you’ll make your own cliff notes so there will be cliff notes of cliff notes, of cliff notes. And maybe they will get bigger maybe they will get smaller, but that ideas will move through that environment. If you think about it it’s like me talking about the engineer, who wouldn’t do something unless it has seen it work before. So it’s this how do people know things. I’ve just been trying to accelerate it.

18. How is this going to manifest itself? Is this something you are working on privately now?

Of course I am doing it on github, you know it’s up there, I call it - I was actually in a workshop called "The Indi Web Camp." It was for people who created a lot of content independently. And it said don’t create it on other people’s site, make it your own site, and manage your own content. They had a simple rule: if you are not paying rent, you don’t own your server. The service provider should be working for you not for somebody else. And I think that’s a good model, and so I was at this camp, and it was two days and in day one I said "You know I think we ought to try to make the smallest federated wiki." Federated means that we each have our own pages, and I sort of thought it would be a day project and we just knocked it out the next day and it turns out we - most people had something else to do and I didn’t get the crowds around it that I thought I would, but we got started and it’s been about a month of development now and it’s up on github.

Of course because I like to write programs to teach me things, there is very little editing capability but the first editing command I put in was move. So that I could move things around and the first record I kept was journaling the moves and you can see that right in the page, what is the history of the page right on the page. And so I just sit and move paragraphs around. And the cool thing now is to have two different pages, and grab a paragraph here and drag it over here. And that’s how you make the cliff notes, I read your pages and grab paragraphs.

19. Something we can use it in InfoQ I guess.

Perhaps, it’s possible, I’m reinventing publishing. I hope to reinvent data publishing. Data publishing is - data is one of those things you say: "Well I am glad to have the data but I don’t even know what’s there". And there is no way to know what’s there" and I think "Well, if that data is embedded, in a conversation, and if that conversation is like the wiki which is like everything else I do, which is all about coming to grips with understanding something." I start with programing automatically difficult, and I look through words and I am trying to generalize that to data. And I think it’s promising, I wouldn’t say I’ve done it, but I do know that I have to be dramatically more powerful than a spreadsheet, where people just use spreadsheets. Spreadsheets are marvelous in the fact that somebody could build a model on a spreadsheet to make important decisions. I want them to build a model in a wiki, and make important decisions.

20. We are running pretty much at the end of our time here. Is there anything else that you would like to share with the InfoQ audience before you take off?

Well, hang in there for once, I’ve devoted my life to programming, I think programming is one of the most powerful ways you can exercise your mind. And I think that that power has led people to be afraid of programmers and there is a lot of "We don’t know what you are doing" basically controlling programmers but in my life I found that I can do a day job and serve whoever is paying my salary in a way that respects their goals, and still have energy left at the end of the day and do things just to please me. Or to please people who know me. And I would say that of Agile, is a pretty marvelous set of values, I say open source is a different but pretty marvelous set of values and both of them are based on understanding code. So I say don’t hesitate to learn more about code, don’t hesitate to do it in an Agile way, do a test driven, maybe work with friends, but don’t hesitate to open it up.

And the open source community is pretty amazing, github makes it easier than ever and whatever comes after github be sure it is going to be better than that. But I think that it’s easy to be overwhelmed you say "Gosh I just learnt this and now it’s that", but I tell you, you put two days into studying something and you are already half way to expert. Stuff is coming out that fast, take the time and there is great blog posts on stuff and learn something new every week.

21. I like to say that we are in a constant state of learning.

Yes, absolutely and when you don’t know it, and you are reading something and you say "Oh, my Gosh I am so overloaded", well then just read it until you can’t read anymore and then read it again tomorrow and then try writing a program. I stayed up all night, not all, I was up for an hour in the middle of the night I woke up and I was kind of awake so I had my pad, called up Coffee Script and I read Coffee Script.

22. You are doing Coffee Script?

I am doing Coffee Script, a little pitch for Coffee Script, JavaScript is amazingly powerful and inconvenient in almost every regard and it’s tedious to write and hard to do correctly. So all the little idioms that are hardened, JavaScript, have just been polished over in Coffee Script. Coffee Script and the environment will all the powerful browsers is the closest I felt to the power I had twenty years ago in SmallTalk.

But not it’s not in one machine for one person, it’s on the whole world for the whole world. And that’s night and day but the ability to do reasonable complicated things in have it code-up nicely to look clean, to have it fit on the screen to be able to read it and understand it. And I am excited to go down the stairs because like I said there is a lot more to Coffee Script than I even knew and I learned it last night and I am going to go apply it and in another week I will be an expert. And you can do it too.

23. Ward thank you for taking the time with InfoQ we really appreciate it.

I appreciate your attention and to you audience hang in there and learn something new.