Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Eric Steven Raymond on the Hacking Culture and Agile

Eric Steven Raymond on the Hacking Culture and Agile


1. I’m Olaf Lewitz from Berlin and I am very happy to meet Eric Raymond today who gave an amazing speech this morning at the Agile Culture Conference in Philadelphia and who just told me when we walked over from lunch about a fork test that he is doing for cheesecake. Could you just repeat what you told us about the fork test?

Oh, dear, I’m fussy about my cheesecake, I like good old fashion dense cheesecake and I found that a good way to test for this is you ask the waiter at the restaurant “if I took a metal fork and I stuck it vertically in a piece of your cheesecake and then I took my hand away, would the fork fall over?” and if the waiter doesn’t immediately answer “no”, then it’s probably a lousy cheesecake because it’s not dense enough.


2. Awesome, nice idea. I like this concept of doing a failing test before you start changing anything. As we’re talking about changing culture, hacking culture, what would be your failing test you would start with when you want to influence an organization to hack that culture, to improve their culture?

What happens to people who fail, is there room to experiment?

Olaf: That’s a good idea.

Is it allowed for people to take risks?

Olaf: Why is that important in your opinion?

Because if you can’t take risks you can’t try things that might work better than what you’re doing, you’d be paralyzed by the risk of failure all the time. So an organization that wants to be able to learn has to have a certain degree of tolerance for risk and failure. People have to know that they can try things without being punished for getting the first or second or third iterations wrong.

Olaf: Ok. So your understanding of a good culture would include learning?

Well, yes, because the world doesn’t stand still, cultures that are not capable of adapting to their context, whether that’s economic, political, ecological, cultures that are not capable of adapting stagnate and die.


3. Yes, that’s my experience too. So, when you continue after the first test, let’s say you identify something that fails, something that’s not allowed, what would you do, ask for permission?

Well, if it’s the kind of culture where you have to ask for permission to do anything, my future as a change agent there is probably sharply limited. But there are a couple of heuristics you can use. First, there is a law of bureaucracy that states that it’s easier to get forgiveness than permission. Whenever you ask permission you are inviting people to tell you “no”, even if they don’t actually have the authority to do that. So, I’m actually generally against asking for permission. It’s much better to do the experiment and then take your lumps if it fails.


4. Yes, very good point. And that law is true in Germany, too, and every other country that I’ve been in so I would go for it, yes. Apart from learning, what other things would you seek in the culture that you try to achieve or try to aspire?

You mean what do I want from cultural change?

Olaf: Yes.

Ok. What I want from the cultures that I try to change? The most important thing to me, ultimately, and I’ll talk about how this ties back to things that are more obviously relevant, the most important thing about any culture to me is the degree to which it supports and defends the liberty of the individual.

Olaf: Wow.

Does it help people become more free, does it break their chains or bind them in chains?


5. This is a beautiful introduction to hacking, where does that come from, can you shortly summarize what hacking means to you?

Let me tie liberty back to some more obvious things. Cultures that are hostile to learning, that require constant permissions checks before you do anything are not cultures under which liberty can flourish. When I say does it support and defend personal liberty, this is not some kind of floating abstraction that is disconnected from organizational behavior you can see, that is actually a value decision that has real consequences. I’m sorry, I got you off track, what were you trying to ask me?

Olaf: I was asking for a short introduction to hacking, not all of the audience might actually have the same understanding of what hacking means, what’s your definition?

I have to explain where I’m from. You can more or less think of me as an ambassador from the hacker culture, and the hacker culture is a tradition with roots that go back certainly as far as research labs and academia in the 1960s, arguably earlier, people at the early M.I.T. A.I. labs were hackers, people at Stanford doing some of the first robotic vision experiments and experimenting with early, what we now call CRTs. Hackers were people who were at the cutting edge of software and artificial intelligence and networking all along. Hackers built the World Wide Web, Tim Berners-Lee is one of us. And that culture is still very much alive today and it’s programming stuff that people don’t realize is important to them. As an example, every time that you use a browser or a smartphone you rely on software that I wrote and gave away as part of the hacker cultures work.

Olaf: Which was that specifically?

I have been a maintainer or major contributor for the reference implementations for two of the three major web graphics formats. I was and am still the maintainer of the reference library for GIFs, graphic interchange format, which is actually a project I stepped away from in 1994 and I recently retook the maintainership because the guy I handed it to didn’t want to do it anymore.

Olaf: So one of your former selves never leave you, right?

Yes. So I was and am the maintainer for reference library for GIFs and I was also a major contributor to the PNG library. So the result is every time anybody’s digital device throws pixels on a screen they’re using my code and that level of unsung ubiquity of being the plumbing and the masonry and the electrical wiring that everybody depends on, that’s what hackers create.


6. Ok, so as obviously the general society seems to take advantage of the impact of the hacking culture, what else could we learn, would the hacking culture be a good model for other organizations?

In some respects it already is a model. When I wrote the foundational papers on open source development, which has become a major way for profit entities to develop software, I was operating directly from my experience in how the hacker culture did things. We pioneered the themes of transparent process, rapid iteration and decentralized peer review that are important in open source and also of course in the Agile community. One thing that’s actually not commonly known is when the big meeting at Snowbird where they wrote the Agile Manifesto, I was supposed to be there, I was actually invited but couldn’t be there because of a schedule conflict.

Olaf: That’s sad to hear. Did they invite you to the reunion thing, two years ago or one year ago?

Sorry, I wasn’t there for the first one so I wasn’t eligible for the reunion.

Olaf: They invited more people than the first time.

But those of you who are involved in Agile development, we’re sort of your grand daddies over here in the hacker community.


7. [...]So how would you start after your initial fork test, to actually hack the culture of the corporation?

Olaf's full question: Welcome, happy to be your grandson actually. So, open source is obviously the place where hacking culture is still most alive. You’ve mentioned practices, like frequent iterations, peer reviewing, etc., that obviously work well in an organization at a corporate context, but from a cultural point of view like allowing people to be free and actually what we are talking about here at the conference is culture hacking, so how would you start after your initial fork test, to actually hack the culture of the corporation? Say you meet people who are frustrated and you see the potential of improvement.

Well, the way I have usually done it and this was one of my insights back in the late 90s when we were facing this problem, how do we sell the idea of open source development to corporate America, because what happened was that in early 1998 Netscape created the opening, Netscape fired the shot around the world when they said “we are going to open source Mozilla” and I realized this created a brief window during which we could get the message out to the rest of corporate America and actually get heard. Unfortunately you may not like it when I tell you how I did it.

Olaf: It might surprise you.

I developed a theory of how corporations actually work, with a three layer model in it and at the bottom of the layer are the people that do things and at the top of the pyramid are the people who are capable of steering the entire organization and there’s a middle layer composed of middle management and the middle manager’s job is to be a conservator of organizational stability, that is the middle manager’s job is to say “no”. And at the time I was thinking this through, the only evangelism method we had was to start at the bottom, persuade the engineers, motivate the engineers sufficiently that they could fight their way through middle management and reach the top management who would suddenly experience a flash of enlightenment and say “ok, we’re going to change our corporate practices”. What I concluded was that this does not work.

Olaf: Ok, why?

Because the middle management inertial layer is just too thick. There’s an easier way than trying to fight your way through middle management. The strategy I centered on was actually a media centered one where I concluded that if we could flood the technology and business press with enough positive memes, if we could get our narrative there where the CEOs and the planning executives could see it, what would happen is their heads would get infected with those memes and then they would then impose them on the organizations underneath them. So I said “screw middle management, I’m going to run right past them through the media that their bosses are reading”. This worked.

Olaf: Why settle for the middle if you can reach the top, right? Cool.

And so that may not be good news for you depending on how your evangelism is structured.


8. Well, in general, I find that strategy quite familiar actually. But what were some of these key messages that you sent to CEO management?

Well, at the time, in the late 90s there was a lot of agonizing about the “software complexity problem”, that is that software projects are complexifying beyond our ability to manage them. And basically one of the things I pointed out was you never had that control to begin with, look at the actual statistics, look at the failure rates. Now, if you are willing to give up the illusion of control, I’ve got something better, I’ve got a transparent process that will reduce your defect rate by orders of magnitude. And what we were able to do is we were able to point them at specific examples of open source projects that actually had astonishingly low defect rates and what I was able to say to them was “look, if you’re prepared to embrace an open process and the open source ethos, then your software can be this good too”.


9. Awesome. Do you have an example of a company that tried that, succeeded and whether it might be an interesting story to tell in how they actually implemented it?

Well, the first example obviously was Netscape, but they make a lousy example because they got crushed. And it’s not open source’s fault that they got crushed, it’s a long story but the most obvious example of a company that was able to internalize open source principles and remake itself as a service business in large part around that idea was actually IBM.

Olaf: Interesting.

It completely blew my mind because I go back far enough, I’ve been in the hacker culture long enough to remember when IBM was the great Satan, this was before Microsoft was the great Satan.

Olaf: Yes, I remember that too.

So, for IBM to turn around, and they had the courage internally to realize that the way they had been doing things was a dead end. And some of their planners got hold of the idea of open source and said “well, let’s reorganize ourselves as a service business and embrace open source as a way to blow open all the proprietary fortresses that are keeping our guys from getting consulting engagements. And again this actually worked for them, well enough that they ended up plowing something like a billion dollars into Linux related software development and messaging and stuff like that. So they must think it’s working and their stock price suggests that it did.


10. Do you know how specifically the CEO reading a message and wanting a transparent process, how he actually started to make that work in the organization? Because I could imagine that the CEO just planting a poster on a wall, like “now we are open, now we are transparent”, might not really work, right?

No. The way I could tell it was starting to work was, the typical process would go something like this: the CEO and the planning level executives would read about these ideas and they would get excited about them and they would say to some underling “hey, go investigate this” and the underling would then go find me and say “why, don’t you come to corporation X and give a talk about these ideas and we’ll invite in our executives and our senior product leads and our planning people” and I could tell how effectively open source ideas were penetrating the organization by who showed up for the talks. It’s usually possible to look at an audience and tell if you’re speaking to the real decision makers or to second stringers. And the companies that really got it were the ones that sent their first string people to these talks.


11. And what did these people do then? Do you know of any example how they started because my experience talking about lots of people who tried to influence or transform organizations in any positive way, the first step is the hardest.

Yes. Well, you know now that I think about it, I’m not sure that’s true. The first step sometimes is relatively easy because it involves people reacting to novelty and getting enthusiastic. The step that’s hardest is sustaining that momentum for change in the face of the institution’s inbuilt resistance and one of the things that I had to learn as a geek, as a techie, as a programmer, because that’s what I am at bottom, I had this reflex that this corporate resistance to change is a sort of innate evil. It isn’t evil, corporations really do have a need to conserve their stability and have a steady cash flow and have some predictability in their operations. That conservatism is not innately bad, it’s bad when it’s dysfunctional; it’s bad when it prevents them from adapting to the environment as it really is.

Olaf: In the end, all of these people who want to be free, still want to be paid, right?

Yes. And correspondingly, I ran a bunch of informal seminars for hackers who wanteded to learn how to do the PR thing properly and one of the things I told them is that you have to stop seeing business people as being stupid and the laugh line with that is if it helps try to think of them as being differently abled, because they are; “Dude, you couldn’t run a business, you’d go crazy if you tried, think how you react to office politics, you can’t run a business, you can’t meet a payroll. These people have valuable skills, don’t dismiss them, they have their place in the ecology”. And the thing is, I told a budding would-be evangelist, you have to treat managers and business people with respect and in order to treat them with respect you have to believe they’re worthy of respect, because if you don’t your contempt is going to leak through and it’s going to poison the whole presentation.

Olaf: Very very good.

So, that’s an advice I have for Agile people: you actually have to respect the people you are trying to convince, otherwise your lack of respect is going to leak through and it’s going to mess up the transaction and neither party is ever going to really understand why.


12. I know because I’ve been there. So, yes, very good point. Is there any other topic that we haven’t yet covered, like anything that is in your opinion crucial to understanding this thinking idea?

That’s a dangerous question to ask me because I can rant for an hour. Things we haven’t talked about, let me see. Another crucial topic. Don’t lose sight of fun.

Olaf: That’s a very good one.

Because one of the things I wrote in my original paper is enjoyment predicts efficiency. There’s a sort of optimal challenge for human beings, when they’re not challenged enough they’re bored out of their skulls, and that’s not fun. When they’re too challenged they get stressed out and don’t function well. People function best when they are dealing with a challenge that is just below the limit of their ability to handle it and it is in meeting and surmounting challenges like that that we have fun. So work to put the fun back in, it actually predicts efficiency.

Olaf: Yes, that’s a very good point. And coming back to your point of respect. I think if we create an environment or if we try to influence it in a way where we make sure that the managers have as much fun as the people who, as you said, do the work, then everybody will probably be willing to have more of that.

Yes, I think so too.

Olaf: Create a sustainable momentum of change which is to me the heart of Agile.

And I guess another good wrap up point is in order to sustain a momentum of change, you have to not forget what you are actually after. When you get involved in a change project it’s easy to get caught up in making goals that are too abstract. I hear for example a lot of talk here at this conference about freedom at work. I think the sentiment there is worthy but that goal is too abstract. It’s not connected enough to anything that you can actually measure, so I would prefer to see people have more concrete goals, such as “I’m smiling when I leave every day”.

Olaf: I’ve heard of a company where they had kind of two baskets next to the exit and a lot of balls and green balls and red balls, like good day, bad days balls, so when you left the company every day you needed to put the ball into the good day bin. Management’s sole purpose and metric was to make sure that most balls went into that bin.

That’s good, that’s actually the kind of thing I mean when I say choose a goal that you can measure your progress against.

Olaf:We need a measurable goal, we need to make sure we have fun and we need to make sure we are allowed to make mistakes, does that about cover it?

Yes, I think so.


13. Ok, how do you establish policies that don’t get to rigid, that don’t block the progress on improvement in the future?

I can’t answer that for every organization, I can only answer that for the culture I come from. In the culture I come from, there is a lot of tradition that tells us what quality looks like and I know that sounds kind of paradoxical but the tradition says things like “it’s elegant, it’s comprehensible, it’s properly modularized, it’s something that you could hand to a competent developer tomorrow and that person will be able to pick up the baton”. We have a lot of really strong cultural norms that emotionally feel to us like centuries old traditions even though they are not really centuries old they are at most decades old. And I think for my culture it’s that sort of accumulated cultural capital that makes the difference. I’m not sure how to create that kind of experience elsewhere.

Olaf: Ok. So in my experience what works is that you start by having the teams define their definition on quality and then as with every other thing we want to improve, work with feedback and frequent inspection and obviously you need to talk to the right people, you need to talk to the customers, you need to talk to everybody what is your understanding of quality and I very much resonate with your definition, you know it when you see it, and it’s different for every product and for every service so I think the important bit to focus on is to find your definition of culture in your organization.

Yes, probably. One of the things I am learning at this conferences is that Agile is different, because where I come from there is one big tribe, there’s one big open source community that has shared values across literally hundreds of thousands of developers. And there isn’t really anything like that here, there’s more like an agglomeration of sub tribes and individual cultures that are existing in tension with individual corporations. Maybe someday you’ll develop that kind of big tribe feel but it’s not here yet.

Olaf: That’s a very good point and I fully agree with you. The Agile community is something that still has to emerge and probably the idea that you gave us with, I think that was before we started the interview, with common projects that people work on and you kind of commit often enough to the project to show that you are actually part of the tribe, so that you have an evidence based membership, I like the idea.

Long ago, membership in the hacker culture was much more difficult to define, in fact when it was very small it was a significant part a matter of who you knew. Nowadays it’s not like that anymore because we have version control and because we have repositories and because our game is produce useful open source software, the membership test has been “do you regularly push commits to a public repository?” And the interesting thing is having that concrete a membership test is a tremendous relief to all of us, it makes things easier.

Olaf: I take that as an inspiration into the Agile community and try to create some sort of similar thing so that we can work on our identity.

But there is another level that you should think about. I talked about cultural capital and here is an example. There is an internet RFC which I think is 1149, “Transmission of IP packets over Avian Carriers”.

Olaf: I heard that a while ago.

It’s a joke RFC about using carrier pigeons for networking and I have said and written occasionally, is one of the ways that you can spot the hackers in the room is you read the first three paragraphs of RFC 1149 to them and see who laughs. Beyond the shared projects there is also another level of shared jokes, shared jokes and shared stories about the tribe’s history. And common projects is part of it, but the other thing that the Agile community needs and maybe doesn’t have yet is the shared jokes and the shared history.

Olaf: Wow, thank you, that’s a very inspirational closing of this interview. Thank you.

You’re welcome.

Dec 14, 2012