Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Kohsuke Kawaguchi, Creator of Jenkins Discusses its History, Present and Future

Kohsuke Kawaguchi, Creator of Jenkins Discusses its History, Present and Future


1. Hello and welcome, InfoQ audience. With me, I have Kohsuke Kawaguchi. Am I pronouncing it right? Please take a moment to introduce yourself.

I’m Kohsuke and I’m the creator of Jenkins, and I am the CTO of CloudBees, Inc.

Rags: Great. So let’s talk a little bit about the history of Jenkins. It goes back to your Sun Microsystems days, right? So take a little bit of time how a project, you know, it started off really small has pretty much taken over the world these days.

Right. Yes. So I was working as a software engineer and I really liked coding..So I had all sorts of hobby projects going on. And one of those was, so I began Jenkins. I used to be a guy who’s known to break builds. So I thought, “Well, let’s write a program that makes sure that that catches a program before my colleagues.” So I started writing those and I started using it at the workplace, and my colleagues started liking it.

They had all sorts of suggestions. They’re like, “Oh, what if this software does this or that?” They kept working on it over time. And then eventually, I discovered that this program that I had, the breaking builds is not just my program but in other people’s program. So it became more popular outside the company and then it kind of grew from there. The next thing you know all sorts of people are using it. So that was an amazing journey.


2. One of the things I think you talked about in your keynote in the Jenkins days is about once you automate the build, right? It makes your life a lot easier, right? So is that kind of one of the motivations for Jenkins? What is the motivation, I guess? What was the one main motivation?

Yes. I think the automation is not only a driving force in the whole industry. And I guess in my case, the part of the motivation was -- it’s not the mundane parts of all work that I feel it would be nice to be able to offload to somewhere or someone else. And I didn’t have a large area of -- I’ve got like my apprentices that could take care of me so I thought of the software, which is why this software has a human name, like I think of it as another person in the team.

So yes, it was partly being of myself. But I also thought, to make the junior engineers more productive, I thought it would be good to help more, machine help so that they don’t have to be -- even if they make mistakes, their program around them is going to catch them. And that piece of comfort, peace of mind is going to really help them take bigger risks without being dangerous. So that’s I think, I would say, another motivation.


3. I think you know, if you talk about some of the new buzzwords in the developer terminology or in the developer lexicon today, it's containers, cloud, and DevOps, right? And it seems like Jenkins is right in the middle of that. How does containers, cloud, and DevOps have an effect on Jenkins and vice versa?

Right, yes. I think part of this movement towards containers is to be able to take the same approach that the software developer is using, there is source code which goes through the build process that produces this artifact, the binary practices that is runnable and take that concept and then apply it, expand the domain more to the downstream because, say, ten years ago, the ops world was not working like that. There was no source code. Maybe there are some instructions in the Microsoft Word and then you’re going to type that command on the terminology, apply the changes to the actual production machine, or with container that’s sort of changing. You’re trying to avoid doing that and then use this immutable unit of deployment as a container. Therefore, it’s kind of nutshell, therefore, the tools that the software developers are using. It could be Jenkins and so on.

It’s therefore like becoming more applicable to this ops device. They are trying to be more like developer. So that’s how I think the Jenkins is expanding the domain in that space. But what’s kind of interesting for me is that the runtime through the container and cloud thing rather than the technologies. We’re acquiring this new capability of the runtime side, which is like being able to spin up the new instances more rapidly and tear them down. So now that we have it, the Jenkins could actually use that capability to really create some convincing values like being able to spin up the whole new environment for, say, a feature range that lasts for the duration of the feature range. Things like that's happening automatically without you really doing anything. It’s kind of amazing.

And then I think the runtime people in there, you look at their capabilities, they do work. They also want some shining use cases. So if people are still using that same runtime, like they are using this fixed set of environment that they’re running, they didn’t put the spotlight on the elasticity. So I think in some sense, once the runtime side ups their game like we, the Jenkins side, I think we can up our game duly to highlight that. And then there’s this interesting exchange of ideas and so on that happens to sort of rise up all that time, and I think that’s really why this space is interesting in Jenkins. It’s such a big part of it. Yes.

Rags: One of the things that I as a developer, what I use the Jenkins all the time is plugins. There are so many plugins, which is both a blessing and a curse but really in a sense that it becomes very hard to manage those plugins. There is really no easy way to use the command line to install the plugins. I know there is a command line but do you have any recommendations of how to deal with these plugins? And maybe you can give me a brief history of how plugins came about.

So there is definitely this -- yes, so right now we have about 1200 plugins in a committee. And I think if you ever get the large number of them, there are kind of really obvious plugins that integrate this specific technology. Say if you’re using like edition tracker or Mantis, you know, there are a lot of tools that’s not as widely known but still has a wide user base. So for those people, actually, finding those plugins then using it is actually not that hard. I’m using Mantis.

So, is there a Mantis plugin? Okay, there is Mantis. So great. The other part that is somewhat confusing is -- but there are some other parts of the plugin ecosystem that’s -- like I know that’s not so obvious. For the lack of better word, the plugin that are used to like choreograph automation. So I can do the entire class of plugin that people have recently put together lots of different jobs to do some interesting things. And those are often reflection of how different the organization does things. So like I wrote one of those very much modeled after how Sun Microsystems used to do automation.

And I explained that in terms of, like according to how they know, but it doesn’t translate very well to other people. Some other organizations come up with something similar. They don’t know that there is potential overlap. So we’ve been upgrading lots of, similarly, like a little thing in the single space. So we're trying to make -- knowing that -- as a project founder, the way I interpreted it is there’s this space of program like YAML that we end up taking the head-on responsibility in the core. So we have plugin developers solve the problem, but therefore, they end up solving them independently from different angles, which creates more confusion to the users.

We wanted to take that initiative back into the core because this could have been an important area. Otherwise, so many people won’t be recognized, so we wanted to take this program space and look at more, like take that more head-on. So that’s what we did. Part of the motivation of us doing these pipelines system in Jenkins. is to really own that space. That’s a primary program domain. So I think that basically, that could substantially reduce the need for this kind of choreographic plugins. And I think it would bring more clarity to plugins.


4. That’s great. I could like use git Jenkins as an indispensable CI/CD tool, right? What other tools would you recommend for CI/CD, maybe two or three tops?

Sure. The same program space at Jenkins is I think we have what the 70%, 80% of the market share. So I don’t know what else are out there. But I think this whole DevOps domain is still pretty big that they need a number of tools that’s working together to achieve the end vision. So I think of all the things I see in this space, I see the Jenkins they are using together is this space like something like Ansible, Chef and Puppet topic, which really controls the ops side, the runtime infrastructure side of things. And we have some integration story between them, like we need to make that game really better but we still have some story to be able to tell today, which is really good.

I think on the upstream side, there is -- so Jenkins is like I needed to do domain and the source code and then the binary and where that proceeds. The source code changes. They’re motivated behind to get the user stories and so on. There are holes to get management system or the project management system like I want to have a better story with Jenkins to be able to paint the bigger picture. So that’s I think maybe the three biggest tools in this ecosystem out today.

Rags: I think that may be a good time to kind of look at Jenkins 2.0. So tell me a little bit about Jenkins 2.0. I think you talked about how Jenkins is going to be ten years from now, or maybe two or three years down the road, how is Jenkins is going to look.

So the Jenkins 2.0 is that major new release of Jenkins that we’ve never done before, actually, in the history of projects. We shipped that a few weeks ago and there are a couple of pillars. They run these pipelines of subsystem at the front and [0:10:52] [Indiscernible]. I mentioned about this like the needs for our choreography. But the thing that the people are trying to do in Jenkins is becoming bigger and bigger and more sophisticated. So we needed a tool, we needed a language or a description mechanism to define how those activities happen collectively, in what order and at what figure.

We defined the way to do that by writing this textual, basically, as a configuration DSL. This allows people to put this definition into the version control system practicing this. We can also collocate that piece of source code. And then by doing that, you can make Jenkins happen more implicitly. So you don’t have to anymore come to Jenkins to explicitly create the new job. You just need to put this instructional file inside the source tree and then that’s enough to Jenkins. They notice that, “Oh, this branch has this Jenkins file which is the marker file.” And then that has this instruction on how to -- what’s the pipeline for this project.

And then you do automatically set things up or it becomes more transparent and more sensitive to what you’re doing without saying anything. In some sense, that’s what a good partner is. You don’t want to tell every step of the way expecting to look at to you to figure out what you’re wanting to do without saying anything. So that’s the kind of work that you're taking into. So that the first pillar is pipeline subsystem. The second one is 2.0 is more the curated out-of-the-box experience. We have richer set of functionality that we present the best plugins in the community. We’d add Jenkins more secure out of the box.

I think in some sense; we are trying to shift the mindset of the developer from Jenkins so the core like a castle on top but rather more product-like. Think about what the software together should be able to do. And then yes, there’s always this extensibility to plugins that allows people do different things beyond what we anticipated. But we need to be more responsible about what should be the central experience of the people and then help people use that and then document that and those sort of things.

And the third one is using interface experience improvement. So we’ve started out catching various different parts of the UI. That’s picked up people use every day. We haven’t really made any improvements in that for a long time. So we are working on those like a configuration page for a whole new group of things like that.


5. I feel like Jenkins is one tool which kind of spans from startup to enterprises development, and really choose across the board, you know, just like git is as well. Is there more to be done on the enterprise side, for instance, or are more to be done on any one particular segment?

Right, yes. You’re right that our audience is kind of on a wide spectrum and then there’s on the developer, individual heroes in the team that’s driving CI/CD. For them, the OSS is the great story. And on the other side that we are seeing more lately is like a large organization wants to think enterprises trying to take this practice across the org. And they need a different kind of health. And then as a CTO of CloudBees, that’s the primary audience that this CloudBees can help. So we have more like enterprise new products or people, the staff that help people run Jenkins at the service internally and at the large scale. So there’s a lot of work to be done in there as well. Yes.

Rags: Kohsuke, thanks a lot for doing this. I know we did it short notice but thanks, InfoQ audience, for listening.

Thank you.


6. You talked about Jenkins throughout the Jenkins days here in Boston, right, and I’d say series of events that you’re going at on the country, the world, and talking about where Jenkins is. How do you feel the community is reacting? What are your thoughts about, now that you are talking to the community one on one and you see them face to face?

Yes. So this is changing this year like it used to be. This year, we are doing like a one big Jenkins world event where we are trying to fly all the important contributors in the -- so that’s a one big party for everything in Jenkins. So to accompany that, we are doing this Jenkins Day. It’s, I think, in different parts of the world and then to take this miniature version of the experience. And as I just said, like I get opportunity to talk to lots of different users. And it’s always -- I think that people are generally happy with this program that we are solving. So in that sense, I feel very validated. That’s not the scale, the sides of the adoption that they are trying to drive is really amazing. So that’s for me the biggest takeaway. Just when I thought that we dominated, there’s actually more mainstream that open that’s coming more to this practice, this industrialization of like a CI/CD that’s going on. It’s cool, yes.

Jun 17, 2016