BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Podcasts Asim Aslam on Microservices, go-micro, and PaaS 3.0

Asim Aslam on Microservices, go-micro, and PaaS 3.0

Bookmarks

In this podcast, Asim Aslam, founder and CEO of Micro, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: microservices vs functions; the go-micro and micro frameworks; and the evolution of PaaS and how the new M3O platform fits into the landscape.

Key Takeaways

  • Go Micro is a standard library for creating microservices using Go. It provides the core requirements for distributed systems development including RPC and event-driven communication. 
  • A related project, Micro, is a CLI-based cloud native development framework. The Micro philosophy is “sane defaults with a pluggable architecture”. 
  • The concept of cloud platforms and specifically “Platform as a Service (PaaS)” has gone through several iterations over the past two decades. Finding a “sweet spot” for developer productivity is challenging. Although both platforms provide a lot of value, it can be argued that “Heroku took too much away and AWS gives too much back.”
  • “PaaS 3.0” focuses on taking the modern approach of using containers, cloud, and (CNCF) open source delivery technologies, and making this easier to use for developers that want to focus on writing application code and delivering this to production.
  • M3O is a cloud platform for microservices development. It is a fully-managed service and has been built as “micro as a service.”

Transcript

00:04 Daniel Bryant: Hello, and welcome to the InfoQ podcast. I'm Daniel Bryant, news manager here at InfoQ and product architect at Ambassador Labs. I recently had the pleasure of sitting down with Asim Aslam, founder and CEO of Micro. I've followed Asim's work for a number of years via the London tech scene and I've recently been hearing increasing buzz about two projects he's been working on: first, the Micro and Go Micro projects for building Go-based microservices, and secondly, the M3O platform, new platform as a service offering that he's been working on that built on top of Micro.

00:34 Daniel Bryant: I was keen to pick his brains about the future of cloud infrastructure, continuous delivery, and what he's referring to as the rebirth of PaaS. I also wanted to discuss the topic of developer experience and learn more about what Asim thinks will be the optimal way for developers to interact with the platform. I believe that we should strive to make it easy for developers to do the right thing and I was keen to explore how Asim thinks developers should approach the implementation of cross-cutting concerns like scalability, resilience, and security across the platform.

01:00 Introductions 

01:00 Daniel Bryant: Hello, Asim. Welcome to the InfoQ podcast. Thanks for joining us today.

01:03 Asim Aslam: Thanks very much for having me.

01:04 Daniel Bryant: Could you briefly introduce yourself please and share a bit about your background with the listeners?

01:08 Asim Aslam: Sure. My name is Asim Aslam. As you mentioned, I'm working on a product and a company called Micro Services, and I'm the creator of an open source framework called Micro as well. And previously worked at a startup called Hailo and did a brief stint at Google as well.

01:24 Could you introduce go-micro and talk about the motivations for building such a framework, please?

01:24 Daniel Bryant: That's what I was going to mention. Listeners may know you from your work on the Micro and the Go Micro frameworks. Could you introduce this and talk about the motivations for building such a framework please?

01:33 Asim Aslam: Yes. Sure. So the framework is effectively in the space of Rails or Spring, Rails for Ruby and Spring for Java. And the assumptions I was making was back in 2015, there was really no development model for the cloud. And what I found was everyone was focusing on microservices. It was a hot topic back then. People wanted to write it in Go, and Go didn't really have a framework surrounding it in the way that Rails and Spring had done for those previous languages. So I set out to build something based on my experiences at Hailo and Google. You can see now that Google released GRPC based on their experiences writing Stubby. And at Hailo, we had also built a framework. So I took those ideas and I basically codified them into something I thought were primitives that developers needed for building distributed systems. And here we are five years later kind of getting to version three. These things take a long time to work, really.

02:28 Did Hailo pioneer a lot of interesting microservices technology and platforms?

02:28 Daniel Bryant: Yeah, yeah. I remember when I was working at a company called OpenCredo in London around the time. Hailo was just around the corner, funnily enough, actually, from the offices. And we were hearing lots of good stuff of what you were doing. It was kind of like a Uber type, Lyft type company, if I remember. And it was very sort of pioneering in that space, I think, wasn't it? Lot's of interesting tech?

02:44 Asim Aslam: Yeah, that's right. So Hailo was a company that was founded in 2011. It was a ride hailing company focused on taxis and the regulated market. And it was a very dominant service in London and Ireland in Europe, but had a harder time in the US competing against Uber and Lyft, but effectively in the same category. And Hailo had gone through this interesting journey where they had built a monolithic application to serve a mobile app, right? PHP API, Java backend, and they had to try to scale that out over the next few years and moved into SOA and everything started to become a bit fragmented. And as the company had scaled up, raised over a hundred million dollars in funding, they knew that they needed to replatform to scale to compete against Uber and go global.

03:26 Asim Aslam: And that's when some of us sort of banded together to build this platform. That's when I came in and we effectively built it on the Netflix OSS kind of model. Netflix back then in 2013 were blogging about microservices, the pioneers in some ways, we could say. That platform that they built ended up becoming Spring Cloud, evangelized by Pivotal. And we had built something similar based on those premise. So we said, hey, developers need to be super productive. We need to build a cloud native platform, which abstracts away all of the details of distributed systems and allows these teams, these product and feature teams, to kind of build and scale independently. So it was driven largely by an organizational need, and then secondly, a technical need. It was interesting technology, it sparked a lot of ideas for me, but unfortunately the company never really worked out, but a framework came out of it, basically.

04:21 Daniel Bryant: Yeah. That's good stuff, like technical innovation paid on someone else's dime, then you can go and do your own thing, right?

04:26 Asim Aslam: Oh yeah. I actually think a lot of these companies are the leapfrogging moment for a lot of things that come out. Right? It's not necessarily that those companies end up being the thing that dominates, but actually stuff comes out of it. And you can see all sorts of innovation coming out of different places, which it's almost like it was an R and D lab of sorts.

04:44 Can you explain the relationship between go-micro and the micro CLI toolkit?

04:44 Daniel Bryant: So there's Go Micro, which is the sort of underlying Spring-like or Rails-like framework. There's also Micro, which I understand is sort of like a CLI toolkit?

04:53 Asim Aslam: Yeah. So when I started out, I was looking at what is the most minimal tool that I can give a developer that they can just pick up and use? And the idea was like, how do you get many, many companies to adopt something? And I should scale back and say the thing that made us productive at Hailo was not really a framework, but it was a platform. We had a platform tied to a development model and that's what worked really well. But as you're trying to launch something, you have to start with the smallest piece. And I thought the thing that you put in the hands of the developer as a framework. And so I wrote this thing, I called it Go Micro. Wasn't really very inventive with the name. It was Go, prefix to Micro, which is supposed to stand for microservices.

05:31 Asim Aslam: And I just wrote the basic building blocks you would need to do something to effectively write microservices, which is service discovery, RPC communication, maybe some sort of pumps-up messaging and some form of key value storage. And I made it Go's interface models so that those underlying pieces could be pluggable because every company is adopting different types of infrastructure and that would kind of allow that to scale. Over time, you came across these next-level primitives, which were required, which is, hey, I'm going to build a platform and I need to access these services through a CLI, a web dashboard, an API gateway, all this kind of stuff. So that's what Micro became.

06:06 Asim Aslam: In version three, we actually decided to consolidate all of this. So we found that there was a lot of complexity between the two components and Go Micro had gotten to a level where it was very hard to imagine. This is something I hadn't really experienced before, because I'd never worked on software long enough to see such a problem being uncovered. But once you hit a level of a number of years and amount of complexity, you have to reframe how you're solving a problem. So Go Micro for us is now a standard library for microservices development. We effectively define some building blocks, we make them pluggable, but we've moved the framework to be Micro itself, the all encompassing thing.

06:44 Asim Aslam: It looks a lot like Rails. So you have a microserver, it spins up the kind of primitives that you need, like authentication, configuration, key value storage, service discovery, everything like that. And it also includes a service library, which is effectively part of that framework, which you can use to write your services and then you just type "Micro run, hello world," and it will effectively run your service. And that's a long-winded way of saying Micro is effectively now the framework.

07:12 How does go-micro interface with infrastructure, such as VMs, Kubernetes, or things like a service mesh?

07:12 Daniel Bryant So you mentioned a number of things there, which I'm definitely hearing are sort of hot topics at the moment, right? Service discovery, service mesh this, service mesh that. Or I'm guilty as charged. I'm banging that drum. How does this framework platform run in terms of things like Kubernetes and service mesh, VMs? How does it interface with those sort of computing primitives, if you like?

07:31 Asim Aslam: So I guess the idea is to look at this divide we have between development and operations and effectively business logic and infrastructure. And you can see that service mesh is the latest point at which we're trying to create that division in terms of what the infrastructure should do and what the developer should do. And so for us, what we really want to say is, hey, the developer wants to write business logic. They need a framework that enables them to write that business logic, but they need to be able to access the underlying infrastructure in a way that they can leverage it. Right?

08:03 Asim Aslam: So the example would be, Hey, I need to be able to call other services. And a lot of people in the kind of service mesh space, they're like, well, this is transparent service mesh. You can just do whatever you want and it works. But it doesn't actually inform the developer of the design pattern of microservices, right? It doesn't actually codify that in a framework that enables them to do it. The strongest equivalent I can give you is something like an elastic search, right? If you imagine standing up search infrastructure and then just telling the developer, "Hey, you can do search. Like, you can just figure it out." They'd be like, "Well, I don't really know what you mean by that. I don't understand this." But if you then give the developer a library and you put that into their hands and then go, "Hey, use this library. Here are the five calls that you need to make. Here are the methods that you need to use to store a record into the search index. Here's how to retrieve it or to query it," it starts to make more sense.

08:57 Asim Aslam: So for us, it's the same thing as saying, "Hey, okay, service mesh is great, but actually what the developer needs is this understanding of what service discovery is, how to call the services, how to encode these things." And if we can just provide it in a very simple interface for them that's what they need. And so, what you run beneath the covers, you choose an Envoy or something else. It doesn't matter to us. We would love the infrastructure to take care of those details. But from our standpoint, the developer needs certain parameters as well.

09:25 How popular do you think Platform-as-a-Service (PaaS) is now?

09:25 Daniel Bryant: I like it. I like it. And I think this is moving nicely into the topic that you and I discussed off-mic around the rebirth of PaaS, which I like that phrase. So I've used things like Heroku, Cloud Foundry over my career. Hugely productive. I could get stuff out really quickly. I definitely know since my consulting days that a lot of developers pushed back against the notion of PaaS, this one size fits all was the kind of common thing I heard against it. What's your thoughts on that now?

09:52 Asim Aslam: Yeah. I think having done this long enough now, probably as long as you and the listeners here, we've gone through many, many different iterations of it. I'm calling this sort of PaaS 3.0 now. That's what I think of it and I think it's down to the fact that we've had five years of complexity and containers and container orchestration, and we have CNCF with a landscape of over a thousand tools to figure out how to build for this new wave. The key thing for me is I think Heroku solved the problem for web 2.0 development back in the day, right? 2010 Heroku was the epitome of a PaaS that you needed. Then we started to kind of move on and realize, hey, actually what I need is multi-service development and a platform that provides me more than just the ability to run a Rails application.

10:39 Asim Aslam: Heroku tried to scale. The cost was pretty high. The performance was not great. And again, what had worked for them initially, they effectively let go of. So what worked was, "Hey, I have this great development model, Rails, tied to this platform as a service. And I'm also using Git to basically just version control everything. So it's great." Git push. I write my stuff with Rails. Everything works really great. Once we went multi-language, it stopped having opinions. And I think this is actually a bad thing, right? So I think you kind of came up with this platform where it's like, ah, well, I could run anything, but that's not really beneficial. What really would have been an advantage was if it had given you microservices development or multi-application development, which really didn't happen.

11:22 Asim Aslam: I think the term I use is Heroku took too much away and AWS gives too much back. And I think we didn't find a happy medium. And so we go through these kind of cycles of PaaS or PaaS and infrastructure, whatever you want to call it, where we get to a point where we find the opinionated platform works really well. Every company actually ends up giving their developers and opinionated platform. Then we say we need more, and we just go through these cycles of innovation. And I think, yeah, you've seen it, right? And now Docker came about, it redefined what it meant to build infrastructure, and now we're building all the building blocks again. And so I think we're at a phase where we're like, "Hey, I've pieced together enough things. I need to PaaS 3.0 for a container in Kubernetes world that actually builds a development model into whatever this is."

12:10 Asim Aslam: And the industry hasn't come up with that model yet. But I see us converging. If you look at every tech company, they're all using Go-based microservices development in the cloud on the backend and they're using the Jamstack for the front end. And so if we start to uncover that, you feel like, "Hey, there's something coming." We're going to extract away Docker, we're going to extract away Kubernetes. We're going to come up with the development model and we're going to start this thing that we thought was a hot topic in 2014, microservices. Actually, we're going to revisit this as the cloud services development model.

12:44 What should the developer experience look like in “PaaS 3.0”?

12:44 Daniel Bryant: Very interesting. So a couple of things popped out there. I'd love to pick your brains on more. As in the first is, what do you think the developer experience can look like in this new world? I've heard you sort of use similar terms there several times, which is great to hear, because I think it's so important and it's very tempting as an engineer to not think about that stuff. I'm just building the cool Docker staff. But the developer experience, the dev workflow, is fundamental, as you said, for getting value to customers. What do you think that's going to look like in PaaS 3.0?

13:08 Asim Aslam: I think the key thing to really understand is that this is an evolution of platforms, right? So PaaS 3.0 is going to serve cloud services development, which means I am building a service that exists in the cloud. It is consumed as an API from multiple clients. So the client might be a web app. It might be a mobile app. It might be your Alexa. It might be some AR device and that can only be consumed through an API. And so I think we actually moved to this world where PaaS 2.0 was, "Hey, I'm building the whole stack." It's NBC, it's Rails. I build the web front end as well. It's all monolithic. And I think we moved to, all right, well, it's the Jamstack for the front end. It's iOS or Android for the mobile side. And the backend is a cloud-based API which is headless. And I have a PaaS that allows me to deliver that in another monolithic way, but actually a multi-service way, in a Micro Service way.

14:07 Asim Aslam: And I think Kubernetes and containers all facilitate the desire to build distributed applications. That's another way to phrase it. And we're just kind of getting to the point where it's like, okay, we have Kubernetes, we can deploy distributed applications. We have these different ingress controllers and service meshes that let us break this stuff down so that we can use GRPC to call things and a single API can be broken down per path to different apps. And so it's just a case of saying, okay, the next PaaS will pull together all of the building blocks that I need, which is authentication, configuration, key value storage, API gateway, secrets, everything else into one form and allow me to focus on writing the APIs.

14:51 What’s the difference between a microservice and function (as-a-service)?

14:51 Daniel Bryant: One interesting thing I wanted to dive in a bit deeper there was, how I'm hearing you talk, there's almost not much difference between microservices and what we're calling function as a service. Would that be a correct interpretation?

15:02 Asim Aslam: I think there is actually a distinction between the two. So the way I see the breakdown is we had monolithic services, we had SOA, we have microservices, and then we have functions. I think microservices is just a domain, so it serves a specific domain. And I think through that, you have certain actions which you can operate on in that domain, which is it's not one thing, it's many different things. So an example would be a customer service. It probably has a CRUD interface along with some other functions. And I think functions really went to the extreme, which is to say, hey, it's one function. It does one thing. And while there are frameworks of grouping them together, the function as we know it as developers is one function. It does one thing and it usually acts on events.

15:47 Asim Aslam: And so I actually think the FaaS model is an event driven model that is for inputs, from external things, whatever it might be. And yeah, there are singular functions. And I think it's a complimentary thing to microservices. I actually think on the backend in the cloud, you want to build microservices. In fact, you want to start with a monolithic app, right? You don't want to start with microservices. You want to start with a monolithic app. But eventually you moved to SOA, then microservices. And then I think there are things that need to be event driven that you don't need a long-lived service for. And that's what FaaS is for.

16:19 Can you explain what the typical workflow would look like in a modern PaaS?

16:19 Daniel Bryant: What does a typical developer workflow look like in this new world? Would I be purely on the terminal, just doing my "git push heroku master" equivalent? Would I be perhaps using a web interface to do things or maybe combination? What do you think?

16:32 Asim Aslam: I think there will be no one way. There will be no one solution. But everyone will have opinions on what they want to use, right? So you are seeing these low-code, no-code movements that is a graphical user interface and you're pulling together things. I think it has its merits, but maybe it won't replace what is the traditional backend development. If we stick to what it is, which is the evolution of backend service development as we know it, there is an element of CLI. There is an element of UIs, right? So I think you're not going to get rid of the CLI. I actually think that is the most important bit, because you don't want the context switch, right? You're doing everything in the terminal. You're maybe using them something else. I mean, if you're using an ID or whatever else, that's fine, but mostly you're in the terminal. So you want to be able to write code there, push things from there, inspect problems from their logs, stats, everything else.

17:24 Asim Aslam: And then I think the UIs are a nice to have. I think the key thing to understand there is what problem is the UI solving that the CLI can not solve? Observability is probably the biggest one. Right? And so I think when you get down to observability, you need a graph somewhere. You need a chart somewhere that's kind of useful. I don't think they're mutually exclusive. In my case, I'm starting purely in a CLI-driven manner just based on some experiences of failing development for dashboards. But I think one thing to highlight there is the world as we know it, in terms of open source, is a bunch of unique snowflake dashboards and there's nothing that has really tied it all together. So I think as 3.0 brings together a single consolidated experience on the CLI and on the web.

18:06 Could you introduce your latest project, M3O, please?

18:06 Daniel Bryant: Ooh, that seems a very interesting concept, which is actually a perfect segue, too, to what you're working on now, M3O. Could you introduce their folks what M3O is and how it relates to Go Micro and Micro?

18:17 Asim Aslam: Sure. So M3O is a cloud platform for microservices development. It's a fully-managed service and it's built as micro as a service. So our open source tool is a Go-based development framework. And what we're doing is effectively taking it, hosting it as a service, and just providing you this fully-managed, vertically integrated experience for what we're calling PaaS 3.0.

18:43 Daniel Bryant: So again, as a developer, maybe I don't have to worry about the infrastructure here, but will I still build out like CI and CD pipelines? How do I get my idea to code to value?

18:52 Asim Aslam: You will not have to worry about CI or CD. So I think the best way to think about this problem is imagine you are a developer at your current company or a past company or any other kind of technology company. You likely have a team who manages your platform and your entire pipeline. So you, as a developer, you're writing code, you push to GitHub, it goes to some CI/CD system. You see the build succeed. You have the option of pushing that manually, or maybe you're using some form of GitOps that automatically deploys that. And then you go look at the logs or look at a dashboard and see the health of that thing and you start to play around with it.

19:27 Asim Aslam: And in the same framing, we're just saying, "Hey, we want to take away all of these things that you're doing so you can just write your code, push it to the platform, see that it's working and start the query it." On our homepage, we have 10 commands from install to deploying and querying your service, including signup, and that's it. And so from my experience, 10 years ago, I was absolutely tired of the state of the world. I looked at all this complexity coming my way and I thought, why can't I just write some code and deploy it? Where is this source to running experience that I'd been promised?

20:03 Asim Aslam: And the next wave of innovation didn't actually solve that problem. It just made it worse. Now I'm sort of expected to go to some AWS dashboard of a thousand services. I'd spin up my Kubernetes cluster and then download my config and my keys. And at what point do I get from running my code to shipping it? Because even then, they're asking me to build a container. And so I just wanted to remove all of that from the developer's experience and just say, "Hey, write this code. And not just any code. Write a microservice, defined with this framework, type this command 'Micro run,' your GitHub repository. We will pull, build and run it for you. And you can query it through a HeNB gateway or via GRPC. And that's it."

20:47 Daniel Bryant: Super interesting. So you do things like build packs in the backend, I guess?

20:50 Asim Aslam: To start with, we literally just took the source code and we ran it. And I took source to running quite literally, but we realized that's not a real product. So we're actually building containers transparently for you. So we pull your source code, we build a container for you. And most people have done this before. They use some sort of scratch image or something like that. They build a Go binary, they put it in there, and they run it. So we're doing the same thing. We push it to a private registry for you and we run that thing. And that's it.

21:16 How should engineers think about things like security and performance in M3O?

21:16 Daniel Bryant: Nice. How do folks think about things like security and performance? Because there's always that trade off of making it easy to do the right thing, but not hiding some of this stuff. So developers, we atrophy on our security skills, for example. What's your opinion around that stuff?

21:29 Asim Aslam: It's quite a nuanced discussion and problem there, which is you want the developer to understand that security is a concern and these things need to be secure, but at the same time, you don't want them to have to be responsible for everything. So our goal is to really say, "Hey, look, the framework and the underlying platform will take care of a lot of the security and isolation requirements for you." So we will define all the network policies, we will use service mesh, we will use namespaces for isolation, we will use some other form of container isolation for you, like Firecracker or something else. We will define the resource limits and then we will use Let's Encrypt for the API and provide you with JWT and stuff like this.

22:09 Asim Aslam: And all we have to do is explaine to the developer, "Hey, here's the feature set that you get. Here's the secure kind of aspects of it. Here are the things that you need to take into consideration." And they should just be able to get on with things. And so the hope is that the developer doesn't actually have to think about security beyond their own code, even to the point of the data storage. Right? So if the storage system that you're using actually understands multi-tenancy, then you as a developer just have to pass through the user's credentials to ensure that the data is actually secure. And so this is the kind of approach that we're taking. We're trying to codify in the development model and it's still early days, but hopefully that works.

22:50 Daniel Bryant: If folks want to get involved in any of this kind of stuff, what's the best way for them to do it? Pop it on the website? You've got a Slack channel or something?

22:56 Asim Aslam: Yeah. So go to m3o.com. There's a link to the Slack channel. Come talk to us about it. The project Micro is totally open source. The M3O stuff currently in private beta, but we're happy to invite anyone who wants to test it. Yeah. And that's how you can get involved.

23:11 How can folks contribute to M3O, and what is the future direction of the project?

23:11 Daniel Bryant: So if folks are listening, Asim, and they're thinking, maybe I can write docs or maybe I want to do some coding, maybe I want to do some infrastructure staff, what are you and the team, what's your vision? Where you headed at the moment?

23:21 Asim Aslam: Yeah, right now, our goal is to effectively build a production PaaS for people. And so the focus is on driving that forward. I think at the moment, we're in V3B on the open source framework side. And so what we're really looking for is people who want to, as you say, help out with documentation. That's one thing we've had a lot of comments on, people really want stellar docs, so we would love help there. I think the next thing is really kicking the tires and all of this stuff to understand, Hey, is the developer experience right? Are we headed in the right direction? And then the next thing is, play a role in that. If you have ideas about where this should go, we are totally open to that. Because I think when you have a community telling you, that's effectively like customer-driven development. And so happy to hear any of it.

24:08 What’s the best way to get in contact with you?

24:08 Daniel Bryant: Super. Super. So what's the best way to get in contact with you online, Asim?

24:11 Asim Aslam: You can join the Slack channel. You can ping me there. I'm no longer on Twitter. I don't frequent that channel anymore. But otherwise you can email me at asim@m3o.com as well.

24:20 Daniel Bryant: Awesome. Thanks for your time today.

24:22 Asim Aslam: No worries. Appreciate it.

More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article

Adoption
Style

BT