Bio Patrick Copeland is Senior Engineering Director at Google, he created and leads the Engineering Productivity Group. Patrick has over 15 years of experience in software development and testing. During his 11 years at Microsoft he led testing and engineering teams that created real-time micro-operating systems, implementations of XML and cloud search systems.
QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community.QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.
Sure. I manage a team called "Productivity" that does a bunch of things that enable teams to produce high levels of innovation. One of the key things that I try to do is improve Google’s ability to innovate, so we accelerate Google - this is what I call it. We have teams that do things like build all of the tools, the build system. We do things like run hundreds of millions of tests per day; I think we’re running something like 60,000 product builds every day. There are 2,000 active products, 6,000 developers, so it’s a very fast moving infrastructure. We do a lot of other things to help enable process, keep the code tree clean. A lot of it is very left-handed and Agile-focused.
2. This morning at QCon San Francisco 2010 you talked in your keynote about innovation. If people are only going to take away one or two things from that talk, what do you think are the most impactful things you’d like to see them take away?
One of the things that I talked about was the thing called "The Pretotyping Manifesto" and what it is, is really simple, is just really some ideas. One of them is that ideas are the by-product of innovation and a lot of people have a misconception that ideas are the most important thing. Google has something like a database of hundreds of thousands of ideas and a very democratic approach to innovation at times. One of things I’m trying to talk about is "How can you have a little bit more of an analytical approach?" One where you can use iteration - all the ideas of Agile can still fit in, but you are trying to make earlier decisions so you don’t fall in love with your ideas and end up spending too much time on the wrong path.
I think that there are a lot of checkpoints along the way and a lot of times we think of it as kind of a final checkpoint when really it’s a series of hundreds of checkpoints that we should have made and we continually adapt our products to meet the needs of our customer. When something is done, it’s not really done. I mean if you look back in history at the number of products, they’re constantly evolving, even if you didn’t do that innovation, somebody else takes the product further. So there is really no end to this decision.
So how do you continually iterate? The idea of pretotyping is the idea that we talked about today. Pretotyping is about trying to test your idea early and ensure that that idea really matches what customers want or that your own belief is high. But there are lots of problems with this and one of them is that you fall in love with your own ideas.
The other thing is that other people will be negative towards your idea unnecessarily, you end up with false negatives, like one of the examples was Twitter. A lot of people when they first heard about Twitter they thought "I’m not going to use that!". And it ended up being a huge hit. On the other hand, there are false positives where a lot of people will support an idea that really isn’t that good and until you actually do the testing, do the iteration, it’s hard to tell sometimes. You have to introduce and innovate, but also test, test, test and question those assumptions as you go.
4. What you talked about this morning reminded me a lot of some of the ideas from the Lean Startup movement that’s going on right now and their ideas about customer discovery. But they are talking to entrepreneurs, people who want to start up companies and it sounds to me like what you are talking about is, in a larger organization, sort of intrapreneurial effort as opposed to entrepreneurial. How do you foster that kind of intrapreneurial spirit at Google?
That’s a great question! Every organization or company or individual has some capacity to do innovation, but there is often a lossy conversion to being able to produce that and get it out into reality. So people use different models and a startup would use a very Agile and Lean - you have three people or you have two people and that’s a great place to start. You don’t want to have a committee making decisions. Having a small narrow group is a great place to start. Google has a lot of these microcosms where you have small teams that are basically startup teams inside of this Darwinian environment where they are competing for resources and idea time, with executives trying to pitch their idea, trying to get things out.
But one of the key things is data is a big driver. Can you show that your idea is successful and can we give people tools in order to prove that those ideas are working early in the lifecycle of the product? It works for a big company, but there are examples in the talk where I was talking about a lot of big companies where it’s at a macrocosm level where these are used and you can use them at a microcosm level, like in Google Labs. People are building small tools and a lot of times these small tools are complete waste of time. I know somebody needs to do such and such, like changing the space in C++ code, but when you actually release it, nobody uses it or a small segment of people use and then you end up with a lot of maintenance costs.
Could we give those people enough data to determine whether that idea is useful early and then they can change gears. Because a lot of times products would be fantastic with small tweaks, if they were made properly. But we fall in love and we don’t think about adjusting our strategy and we end up committing to an idea that maybe isn’t fully rounded out.
5. How do those visionaries transfer that passion in a way that they can trust it to the rest of the organization and how does the rest of the organization respond to the visionary who’s been making all the decisions?
Diana's full question:Many of these intrapreneurial efforts or even entrepreneurial startups, as you’ve mentioned, start with the idea from a visionary, somebody who really can almost see it, feel it, taste it, is really ready to go with it. They are the ones who kick it off and it’s that small group who really get it into the early implementation. But you also mentioned the idea of a culture of empowerment. How do those visionaries transfer that passion in a way that they can trust it to the rest of the organization and how does the rest of the organization respond to the visionary who’s been making all the decisions?
I think it really depends on the lifecycle of a product because at the beginning or the inception I think the founders really need to imprint and keep true to their vision. I think that it really depends on the lifecycle. As things begin to change and you want to grow, you have to bring in more people that you trust and it doesn’t need to be everybody, but it needs to be a few people that you can trust. But also you need to tell the greater company that they have accountability, that they are accountable. You want people to buy in and so I think of almost as a founder as somebody who’s collecting followers, somebody who can inspire other people but also give them accountability in that process. It’s not a free-for-all, there is some structure to decision making, but if the top is bottlenecking everything you end up in a gridlock situation where you can’t really move forward very quickly.
So that inflection point has to do a lot with the leader growing up. I think that's the long and the short of it. You have to feel comfortable with people owning pieces of your vision and moving those things forward. It happens in a microcosm too. Inside Google there are, like I was saying, around 2,000 projects and of those there are passionate leaders on top of them and there are followers around there, so it really talks about the empowerment. The role of a leader or manager is to collect followers as opposed to dictate and I think that’s a key thing. If people are turned off and don’t feel like they are part of your vision, or feel accountable to it, then you end up in a situation where you do have to drive from the top. That can’t go on forever, but it is effective in some phases of a project and you have to come in sometimes and make decisions.
You think of it as a team; there are all kinds of players on the team and you have to come up with a complementary structure, including the founder. So the founder is just another player, in the sense you can have all founders, but you can’t have all linemen in a football team or you can’t have all the same position in a football team. You have to have a complementary group. I think that’s how I answer that question.
6. You have all these people who are passionate. Inside Google you talk about how you may have thousands of projects going on at the same time, doing some of these pretotyping kinds of things. How do you know which wants to keep going and when it’s time to kill it and who makes that choice?
Killing something is very difficult decision and there are multiple forms of "death" (I would call it). I mean it’s not death, that’s a horrible way to put it, but there are multiple ways that things don’t continue on. One of them is atrophy and it’s very common where something is built and the core people who built it are no longer passionate about it and they move on and it ends up basically just atrophying. Different things can happen in that situation like it can turn into the kitchen sink getting thrown into those projects. You bring in a second tier of people and they don’t quite have the same vision and they morph it slightly and you end up with something that’s not right. One form is atrophy or inheritance atrophy where the teams kill it themselves.
Another thing is probably over-loving where you have lots of people who are collaborating on the same thing and you end up [like] in San Jose [where] they have this saying called "The Winchester Mystery House" which is this house that this person just kept on building and building to no end and that’s another form of eventual death. At first everybody is excited, but you end up building something that’s unmaintainable. But the traditional way is through looking at data. So the way that things would get killed is you would look at "Do customers like it or not?" "Are customers using it?" It really depends on the situation or the intent of that project, whether we want to continue it. For an internal tool, a lot of people would build the tool and they’ll say to me "Hey, I have 30% of engineering using it" and I say "I can get 30% of engineering to do crazy stuff".
What you really want is to have critical mass on these things, so it has to be something like the majority of engineering is going to be using this for us to commit to it internally. For external, I think that there is, that Google’s radar, is different than most companies. It needs to be in something like the tens of millions of users. The dollar figure is very large, they're different than most companies. If we handed these projects off to a startup they would be considered widely successful, but for a company that has such successful businesses in comparison, like ads and search. You are living in the shadows of these giants and you have to be able to come up with projects that can at least, from the innovation point of view, capture the imagination at the same level as these other things. That’s very difficult to do.
7. In your talk you quoted Thomas Edison with his assertion that invention is 1% inspiration and 99% perspiration and you asserted that the ideas aren't as important as the actual innovation. How do you distinguish between an idea and an innovation? I think a lot of people think those are synonyms.
I think that’s one of the big problems. A lot of the talk is exactly about that. It’s saying that ideas are an outcome of innovators. It’s a byproduct of innovation. As you are doing some kind of iterative development process or iterative thinking about a product, a lot of small ideas will pop up and that’s really what innovation is: taking those ideas and introducing them into the product as opposed to the kind of thinking in isolation, like "Here is the idea". One of the things I did today is I sold a billion dollar idea. I tried to sell it for 1,000 dollars, but nobody would buy it for 1,000 dollars. Why? - Because we live in thought-land. You have an idea, other people have opinions and those opinions are just as valid as my idea.
So for every idea I have you are going to have a counter idea. In that room there are probably thousands of people who believe in different things and they are going to disagree with me and that’s exactly where we are in idea-land. It’s hard to tell who’s right. One of the things you need to do is use data and to try to show people how you can iterate and make decisions earlier in the lifecycle. Thomas Edison is like a perfect example of this. We have a mythology about him that he invented the light bulb, but he didn’t really invent the light bulb, he didn’t invent incandescence, he didn’t invent electricity being used for light, he didn’t invent the vacuum tube, but he combined those ideas. One of the key things he created is an innovation factory where he was doing lots and lots of testing and lots of what we would call "Agility" today, it was Lean, it was basically all of these concepts.
You look at it back in time and, (I think that this is part of survival in the industry), you have to have that approach. We keep rediscovering these things and calling them "new things" but really it’s about common sense: you have to test out your ideas early and often and iterate on them because those big grandiose ideas are often built on the shoulders of lots of giants and they are not novel. That’s especially true in a very congested marketplace. If you look at the browser market space, there are lots of people competing in that. The idea is they are constantly cloned by others, so the industry is kind of evolving and pushing other people forward. Anyway, that’s the thinking there. Ideas aren’t really the important thing; it’s people who can take those ideas and make them into something customers want. That’s a totally different thing! And there are very few of those people.
8. I liked what you said about data over opinion. When Edison was conducting all those experiments he was collecting a lot of data that helped him move forward and I think that’s an important thing. You made some other contrasts in addition to the pretotyping over prototyping and data over opinion and one of them was commitment over committees. Can you talk a little bit about that one?
I think a sure way to kill anything is to throw a lot of people at it, especially decisions about products. I think you typically need a few key people who have a vision and who are driving. If you look at the success of some of the biggest companies, two is a company and three is a crowd. There is an expression: "The more, the messier". If you end up with too many people, you end up in a gridlock or in a group think situation. How can you keep the decision making much more narrow and also very focused, but also allow a lot of people to have freedom in that space. For example, the founder of a company can’t be a dictator; you have to create an environment where lots of people can have ideas, but at the end of the day, it can’t be a committee that makes these decisions, it needs to be a visionary person who’s saying "Here is the direction we’re going to go."
I think there is a yin and a yang - you need control and you need freedom and they can’t live in pure isolation of each other. In order to get it to work, I think, you need both of those to be in balance. Every time that I am on a committee where we are trying to decide some major thing, you can get into a gridlock very easily because you are in that thought land again. You are in "All opinions are equally valid".
9. It sounds to me like also the idea of commitment carries with it some passion for actually seeing the idea manifest in the real life as opposed to maybe interest, which is just what would happen on a committee.
That’s right. Commitment meaning you care about the outcome. Sometimes what can happen on a committee is there is a dispassionate ownership. By having everybody own something, nobody owns anything. How could you make a committee own something? It’s very difficult, it becomes a challenge. Typically you have a leader who people will follow and there is a vision and then those people can be used as sounding boards. You can have a committee system, but there still needs to be a visionary approach, I think.
10. There was one more contrast that you made which was "now" versus "later", "now" over "later". I’m seeing this in a lot of companies where they struggle with this idea. Could you talk a couple of minutes about that one?
A lot of times when we start thinking or envisioning a product we immediately go and start building a full product. What I’m really trying to get out is now is about trying to vet the ideas earlier and test them as opposed to trying to do everything perfectly and trying to think of the whole product because that is sometimes a later thing. "Now" meaning in the early stages, in the inception of product development are we building the right "it", as opposed to building "it" right? As you get more mature in the lifecycle, you start "now" meaning that "We need to start refining the way that we’re going to actually release this product." You have to think about the full product at that point. Then, at the end stages, you’re really thinking about fit and finish things, so "now" kind of means different things through the lifecycle of a product.
But it also is trying to put an urgency on these decisions, so having checkpoints all along the way, "now" meaning, don’t defer everything, try to make the best decisions that you can at the time. A lot of this is very left-handed Agile type stuff, because if you defer, you end up with a lot of baggage and that happens with bug fixing, cleanup, all kinds of things. "We’ll clean it up later" - you’ll never end up doing it and you end up with a lot of debt in your hands and that will kill you eventually. So "now" meaning do what we need to do at the right stage at the right time in the product cycle?