Today on The InfoQ Podcast, Wes Reisz talks with Rod Johnson. Johnson is famously responsible for the creation of the Spring Framework. The two talk about the early years of the framework and provide some of the history of its creation. After discussing Spring, Reisz and Johnson discuss languages Johonson he’s been involved with since Java (these include Scala and TypeScript). He talks a bit about what he liked (and didn’t like) about each. Finally, the two wrap up by discussing Atomist and how they’re trying to change the idea of software delivery from a statically defined pipeline (located in individual repositories) to an event hub that drives a series of actions for software delivery. He describes this as creating an API for your software.
Key Takeaways
- The initial origins of the Spring Framework really came about through a process of trying to write a really great book about J2EE in 2002. It was through that process that Rod Johnson found he felt there was a better way, which ultimately lead to the creation of the Spring Framework.
- What started as examples and references became the Spring Framework. By 2005 there were about 2 million downloads of the Spring Framework.
- After leaving VMWare in 2013, Johnson spent several years working with Scala. One of the elegant features that really attracted Rod to Scala was how everything is an expression. One of the things he didn’t like was an affinity to overly complex approaches to problem-solving.
- Today at Atomist, Johnson does a lot of work in Node. He really enjoys the robust extra layer of typing over a dynamic language and the ability to escape to JavaScript if needed (similar to escaping types with reflection in Java found in the internals of the Spring Framework).
- Atomist, the company he founded after leaving VMWare, is rethinking CI/CD from a static pipeline defined in every repository to an event-driven system that defines how to respond to specific events (such as a push from Git). For example, all pushes with Spring Boot can be configured to be scanned with SonarQube or because a push has kubespec it might get deployed to a K8 cluster. He describes this as creating an API for your software.
- One of the reasons Atomist integrates so tightly with Slack (and other similar messaging platforms) is because it allows developers to shape their own relevant messages. By joining (or leaving channels), people are able to subscribe to only the information they actually want. Meeting developers inside Slack is an important interface for Atomist.
Subscribe on:
Show Notes
Spring framework started with the book: Expert One-on-One J2EE Design and Development, is that right?
- 01:45 Well done on getting the full name of the book right!
- 01:55 It was the one with the strange camera position that the photographer took.
What came first, the book or the code?
- 02:10 There were 30k lines of book in the code - and I would say to everyone out there, don’t do what I did …
- 02:20 Writing a 700 page book is a lot of work, and if that wasn’t enough, you need to write 10s of thousands of lines of code.
- 02:30 What happened: I had a lot of experience as an enterprise Java architect before, and I had started off enthusiastically about J2EE and EJB.
- 02:45 In attempting to apply it to the real world, I discovered that it didn’t really work that well.
- 02:50 I set out writing that book in the hope of writing a really good book on J2EE and why it was great and how you should use it.
- 03:00 I discovered while writing it that I didn’t believe it any more.
- 03:10 When I was writing the sample application I realised that it was absurdly complex for the functionality that we were actually delivering.
- 03:15 The process took me about 13 months elapsed time with a couple of months consultancy interspersed.
- 03:30 During that process I arrived at a very different view of J2EE: that EJB was not a great technology.
- 03:45 I wrote a framework that introduced the core ideas of Spring: dependency injection, simplified JDBC, unchecked exceptions were in the first framework.
- 04:00 I didn’t think anyone would run this for real; I thought they would find it an interesting illustration.
- 04:05 Then Juergen Hoeller came along and wanted to use it, persuaded me to open-source it.
- 04:15 Juergen stepped up and is an amazing engineer - he continues to lead Spring to this day and his contribution was incredible.
In the book it was called the Interface 21 framework?
- 04:30 That was the original name for SpringSource - we changed our name three years into the history of the company.
- 04:40 I was thinking about the 21st century (having registered that domain in 1997) and everyone knows my love of interfaces.
- 05:00 The name didn’t stick but I thought it was fun at the time.
Spring became the winner of J2EE?
- 05:10 That name was suggested by a community member, Yann Caroff from Luxembourg.
- 05:30 Yann was really involved at the beginning and encouraging me to open-source it.
- 05:40 He also came up with the name, because as you can tell with Interface21, I suck at naming things.
- 05:45 It’s sad that people don’t know about Yann’s contribution, because he played a very important role.
By 2005 there were half a million downloads?
- 06:00 We got over a million in the first year; probably more like 2 million by 2005.
- 06:05 The framework itself became open-source in February 2003 and we would have passed the million download mark early 2004.
What was the next big addition to Spring - Aspects?
- 06:20 After the open-source, Juergen focused on adding more data and transaction features, such as the Hibernate integration.
- 06:30 I focused on OOP and declarative transaction management.
- 06:35 I spent a good part of 2003 trying to get that right, and then Adrian came along later and we integrated it with the AspectJ model and we really got it right.
- 06:50 We had already established the Spring triangle of dependency injection, OOP and portable service abstractions - that was well established by the end of 2003.
When 2008 SpringSource came along, your raised your Series B then?
- 07:30 We founded the company in mid-2004, with myself and the other core framework developers.
- 07:40 We were bootstrapped with a Series A in 2007, and we rebranded to SpringSource in 2008.
- 07:50 Ben Hale joined along the away - the aecgi security which evolved into Spring security.
- 07:55 Ben actually lives about a mile from where I live now.
SpringSource was acquired by VMWare in 2009 - what was that like?
- 08:05 Ultimately it was very good, but it was painful along the way.
- 08:10 I felt good about the acquisition, because VMWare were very generous to the engineers.
- 08:20 From that point of view, it was a very good outcome; I was concerned that it was wonderful for both our investors and everyone in the team.
- 08:30 I had tremendous admiration for Paul Maritz, the CEO of VMWare at that time.
- 08:40 Paul is one of the smartest people I’ve ever met, and a tremendous visionary.
- 08:50 Paul saw that putting the lower level deployment technology like VMWare with the way people wanted to build apps was a really powerful thing.
- 09:00 It is what powers Pivotal’s business today.
- 09:05 Paul was 100% right on the vision, and I really enjoyed working with him.
- 09:10 Unfortunately for the first couple of years, the execution was substantially affected by internal politics.
- 09:20 There were a couple of years where that promise didn’t come to fruition, due to execution and political struggles.
- 09:30 It’s been great to see that Pivotal have fully embraced Spring and they get how important is, so it has turned out to be a good acquisition.
What happened when you left in 2012?
- 09:55 I think it was just the politics that got to me.
- 10:00 There were specific frustrations at that time around individuals who subsequently left.
- 10:05 Those things don’t apply and won’t affect others now.
- 10:10 There’s a lot to like about VMWare and it’s a great company that has achieved some fantastic things.
- 10:20 It was two things: big company politics isn’t my thing; and secondly I’m more of an entrepreneurial start-up kind of guy.
What happened with you and Scala when you left?
- 10:40 By the time I left VMWare, I hadn’t written much code for the last three years - executives don’t tend to write much code.
- 10:50 I was carrying around a MacBook Air - you can’t write Scala on a MacBook Air!
- 10:55 I decided that I wanted to look what was out there, and what was interesting.
- 11:05 I wasn’t going to continue in the Spring world - I wanted to look at other things.
- 11:10 I found Scala really interesting, quickly fell in love with the language.
- 11:15 I spent two or three years predominantly writing in Scala.
Why Scala, and not JRuby or Groovy?
- 11:25 There are many things I love about Scala - and there are many things in Scala, which is a problem.
- 11:40 Over time I started to realise that some of the negatives that people say about Scala are actually real.
- 11:45 I love the elegance - for example, anything in Scala is an expression.
- 11:55 Once you’ve seen that and how glorious it is, it’s really hard to live without it.
- 12:00 Of course any other C language doesn’t have that.
- 12:05 There are things in Scala that I find absolutely beautiful.
- 12:10 The problem is that Scala tends to have a fairly academic crowd, and there tends to be a lot of people who want to take Scala in a very functional direction.
- 12:20 I’m not sure that sits well with the language.
- 12:30 There does seem to be a great joy in taking very complex approaches to things - I don’t think there’s a lot of pragmatism in that community.
- 12:45 One of the things that really bothered me is that people really hate Java libraries.
- 12:50 One of the beautiful things about Scala is that you can take just about any Java library and you can put a Scala wrapper over it which would make it much nicer to use than if it were in Java.
- 13:00 The problem is that is not what people do.
- 13:05 They assume because they are Scala developers, they are smarter than any other Java developer who ever lived.
- 13:10 So they write a hopelessly buggy library that has 1% of the usage of the Java library and they all converge on things like that.
- 13:20 For example, I saw a number of things that did transaction management, and of course nobody in Scala wants to use Spring, because that’s old-skool.
- 13:30 I would have to hold my head when I read some of that code - we had worked through things you haven’t yet found in that code back in 2005.
- 13:40 For example, there are crazy edge-cases with buggy products - that is what getting a transaction abstraction right involves, and implementing it.
- 13:50 I found that frustrating: if the Scala community had emphasised playing really nicely with Java, it would have been way better for everyone.
What led you to using TypeScript today?
- 14:05 We got into using TypeScript at Atomist realising that Node was the best runtime for a particular part of our product.
- 14:15 Our programming model that we were exposing to users was going to be based on Node.
- 14:20 For my entire career I have been convinced of the benefits of strongly-typed languages, so therefore TypeScript was very attractive over JavaScript.
- 14:35 I love TypeScript - of any language I have programmed in, it’s my favourite.
What makes you love it so much?
- 14:50 With TypeScript, an extra layer of typing over a dynamic language works remarkably well.
- 15:05 If you look at the type system in TypeScript, it’s significantly more advanced than the type system in Java.
- 15:10 To be fair to Java, everything tends to work predictably in Java - you can get some weirdness coming through in TypeScript.
- 15:20 Fundamentally it’s got a stronger type system than Java, but fundamentally you can just opt out of the type system.
- 15:25 If you want to opt out of the type system in Java or Scala - and in the guts of Spring you are doing that all over the place - you have to use reflection, which is fairly unreadable and clunky in the implementation.
- 15:45 Whereas within TypeScript you just drop to JavaScript, and that works really well.
- 15:55 There are a number of clever things in TypeScript - type guards work really well - it’s also a more modern language than Java with interpolated fields etc.
- 16:10 I also like the fact that functions are first class citizens in JavaScript and TypeScript.
- 16:20 There is a bit of a difference too in Scala, which is more capable as a functional language than TypeScript, but you have to associate the function with an object, it’s not native.
What was the idea with Atomist?
- 16:40 Atomist was based around trying to re-think delivery for the cloud native generation.
- 16:50 I think how we have traditionally done delivery of CI and CD doesn’t really handle modern cloud native projects that well.
- 16:55 Obviously the whole concept of CI was a tremendous advantage and an advance for the industry, but there hasn’t been a lot of innovation since it first emerged.
- 17:05 For every one of our own projects, it will have its own pipeline, typically defined in YAML or some kind of DSL.
- 17:20 That was fine when a line of business had 5 lots of 5 million line applications - that was fine.
- 17:30 You don’t think about duplication between those pipelines because all of those applications have unique requirements.
- 17:35 Roll the clock forward, and now you’re looking at hundreds of much smaller applications - this trend in the enterprise towards micro-services, people are fragmenting their applications.
- 18:00 Suddenly you don’t have 5 YAML pipelines any more - you have 500.
- 18:05 The problems with that are fairly obvious - how do you consistently apply change across all of those things?
- 18:15 People are drowning with duplication because these concepts don’t translate well to modern requirements.
- 18:20 That was one of the first motivations for Atomist: what should delivery look like if you are in a cloud native world?
What does cloud native look like - at a basic level are you packaging some code and deploying it?
- 18:45 We change from a model of having a statically defined pipeline in every repository to an event hub.
- 19:00 So a push, build, deployment all end up as events in the event hub.
- 19:05 Atomist allows you to have event-handling policies.
- 19:10 You can change how pushes can be handled across your entire organisation in one place.
- 19:15 That’s a slight over-simplification - sometimes people prefer to break out particular concerns like security.
- 19:20 But they don’t want 500 places to change their code - they only want a few places.
- 19:35 If you think about the model of a push as an event, we schedule a goal.
- 19:45 We dynamically compute based on push rules what should happen with this push.
- 19:50 We can look into the repository and discover that it’s a Spring Boot repo - and we know how to apply SonarQube to Java code and schedule a scan.
- 20:05 This repository looks like it has a kubespec in it, so we know how to deploy that to one of our kubernetes clusters.
- 20:15 We can find a Node project that has an AWS SAM file in it, so we can deploy it to Lambda.
- 20:30 The push rules can define a number of things that can happen for each push, and so we create a number of goals.
- 20:35 These goals then go into a workflow where the infrastructure will try to achieve those goals.
- 20:40 There are essentially three things that are different from traditional CI/CD.
- 20:45 Firstly, there is an event hub rather than multiple statically defined pipelines.
- 20:55 Secondly, push rules allow for a rich model - we can look inside the code, what files were changed, what repository was it, who made the push.
- 21:15 Thirdly, we don’t use YAML or an external DSL; we take the view that the best way to express this behaviour is in code.
- 21:30 This brings us back to TypeScript - we want to provide a framework that enables you to implement these goals in TypeScript.
- 21:45 If you want to set up your push rules, you use an internal DSL in TypeScript to do that.
You recently said, “We already have an API for hardware; we need an API for software, so we can develop our own development experience.” What does that mean?
- 22:00 One of the great things that has happened over the last 10-15 years is that we’ve got an API for just about everything.
- 22:05 We’ve got an API for your car, for your fridge - you’ve seen Silicon Valley.
- 22:20 There’s also increasingly an API for hardware.
- 22:25 Fifteen years ago, you wanted a new server, you had to get a forklift.
- 22:30 VMWare was just starting to change that and virtualisation was the first wave.
- 22:35 Now it’s inconceivable that your route to a new server is going to involve a forklift.
- 22:40 You’re going to spin up a cluster, put it on an existing cluster, use Docker or AWS or CloudFoundry - something that has an API.
- 22:50 We haven’t really internalised the incredible importance of that.
- 22:55 If everything has an API the best way of using it is through programming languages.
- 23:00 You want to extend the range of things you can do with your language, and you get awesome tooling support, modularity, loads of libraries …
- 23:20 For me, that was the lightbulb moment - if everything has an API, then wouldn’t you want to program your delivery?
- 23:30 If the business asks us to deliver a business application, we don’t go back and say we can do it with YAML and a few bash scripts.
- 23:40 We develop something that’s testable that uses computer science concepts that generations of people have developed.
- 23:50 I’ve always believed in the magic of code.
There’s also the idea of the software delivery machine - is that the collection of event handlers?
- 24:05 Originally it did look like a collection of event handlers, and it was pretty low level.
- 24:15 It was originally similar to the Servlet API in Java - you could implement this API that would enable you to handle events, but you don’t get a framework or any help with it.
- 24:30 What we did was discover that we needed the Spring Boot of that - and that is the software delivery machine.
- 24:35 The software delivery machine makes it really easy to add handlers with a little bit more structure.
- 24:45 The kind of handlers that you need 95% of the time have a specific API, strongly typed and really easy to work with.
- 24:55 If the Servlet API is the underlying automation client, the software delivery machine is like SpringMVC or SpringBoot.
- 25:05 It’s something that is really focused on the kind of things you probably want to do.
You’re involved with the Software Delivery Manifesto - how does this relate?
- 25:20 That was a group of people who had a shared vision of doing software delivery in code.
- 25:25 You want to apply modern engineering principles like event sourcing but also the power of modern languages to develop your delivery.
- 25:40 That document was produced in November 2018, and there are a number of people from Pivotal, a few individuals like Mik Kersten from Tasktop, and Kenny Bastani from Pivotal you can blame for the name.
- 25:55 We realised that we had a shared view around a take on delivery that means you can apply your core engineering skills to your daily engineering work.
- 26:25 We’ve got this marvellous ability to change the world, and we can do that not only to the software we deliver but also to our daily work.
- 26:35 For example, when we create new projects can we automate all the steps, like raising tickets in different systems, notifying people that they need to care for and feed this new app - can we automate all the things?
What’s the relationship with the work you’re doing with Atomist, compared to the work you did with Spring?
- 27:20 The most important is that we’re reflecting a lot of the lessons that we learnt from Spring.
- 27:30 In terms of Atomist, there are two fundamental parts of our our product.
- 27:35 There is the service, which sits in the cloud and ingests events.
- 27:40 That provides stable WebHook support for GitHub, GitHub Enterprise, GitLab, Bitbucket, various build tools, deployments etc.
- 27:50 It also maintains the database where we’re correlating incoming events on everything we know, which gives us a system of record for a particular commit through to production.
- 28:00 That service exposes the events in GraphQL and can support subscribers over secure WebSockets.
- 28:10 For the language geeks, that is written overwhelmingly in Clojure - our backend team are big Clojure fans.
- 28:20 The client side, which is where you define your push rules and how delivery happens and how events are handled are Node apps, hopefully written in TypeScript.
- 28:30 That is the area I personally work, and I’ve written a ton of code in the past couple of years - I enjoy that very much.
- 28:40 That is where I think we’ve been able to apply a lot of the lessons from Spring.
- 28:50 It’s the idea of building a framework that will be extensible but nevertheless fairly easy to use.
- 28:55 The core people designing those APIs and the experience have been myself, Christian Dupuis and Jessica Kerr.
- 29:05 We have applied a lot of the lessons learned from Spring.
You’ve got millions of events since 2015 on how developers deliver and push software - what insights do you have?
- 29:20 There’s definitely something that could be done there.
- 29:25 One of the obvious insights is that we can see trends in terms of build performance.
- 29:30 We can see if something is degrading over time, trends in terms of successful deployments and builds, time taken - those kind of things.
- 29:45 I think we’ve barely scratched the surface of what can be done with that kind of data.
- 29:50 One of the things we have to be really respectful of is ensuring that we’re not doing anything creepy with people’s data or that they wouldn’t intend.
- 30:00 I do think that there might be some things to be done with anonymised data that we could present our customers, such as comparing to the average for a particular metric.
- 30:20 Currently the majority of our paying customers are large enterprises, and with a large enterprise it’s really their data that’s interesting to them.
Why do you use Slack to interface with Slack?
- 30:45 We heard that a lot of developers live in Slack - we do ourselves.
- 30:55 There are some really nice things with Slack from this point of view - the biggest is that you have natural communities of interest.
- 31:00 If I’m interested in one of our internal projects, I’ll join that particular channel.
- 31:05 If I decide it's not interesting, I leave the channel.
- 31:10 It’s a perfect way of allowing people to shape what they need to follow and to ensure they only get relevant messages.
- 31:15 Slack has always been one of our primary interfaces; we also now support Microsoft Teams, which is becoming popular in enterprises - and we also have a web interface.
- 31:30 We do find that the Slack integration is a popular feature of our product.
- 31:35 It enables you to follow what is happening in your delivery in Slack, it allows you to implement Slack commands very easily, and you can implement them in a way so that you can access your code.
- 31:45 For example, if there is a repository associated with your channel, you can write a command that will perform an update or transformation in that repository.
- 32:00 One of the interesting and differentiated things is that we enable you not only to observe but also make changes to the world through Slack.
- 32:05 That’s where I think what we have is superior to combinations of various point integrations of integrations like GitHub or Jenkins - we give you one really elegant focused view of what’s happening.
Is there anything I can use for Atomist if I’m not an enterprise?
- 32:30 There are two things you can do: firstly, you can use our open-source Atomist CLI from your own machine.
- 32:45 It doesn’t have the persistent database in the sky, so it doesn’t benefit from all the relation features.
- 32:50 Most of the delivery features are fully available through that, and you can explore the programming model through that.
- 32:55 We are currently allowing people to set up their workspace - we’ll be announcing a freemium tier in the future.
- 33:05 It is not our goal to make money out of individual developers - currently our paying customer base is almost exclusively large customer enterprises.
- 33:15 We really see ourselves being able to provide something like GitHub, where for the vast majority of individual developers don’t need to pay us to access the core benefits.