Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Podcasts Vaughn Vernon on Developing a Domain Driven Design First Actor-Based Microservices Framework

Vaughn Vernon on Developing a Domain Driven Design First Actor-Based Microservices Framework

Vaughn Vernon is thought-leader in the space of reactive software and Domain Driven Design (DDD). Vaughn has recently released a new open source project called vlingo. The platform is designed to support DDD at the framework and toolkit level. On today’s podcast, Vaughn discusses what the framework is all about, why he felt it was needed, and some of the design decisions made in developing the platform, including things like the architecture, actor model decisions, clustering algorithm, and how DDD is realized with the framework.

Key Takeaways

  • Vlingo is an open source system for building distributed, concurrent, event-driven, reactive microservices that supports (at the framework level) Domain Driven Design.
  • The platform is in the early stages. It runs on the JVM. There is a port to C#. All code is pushed up stream.
  • The platform uses the actor model and all messages are sent in a type-safe way.
  • Vlingo supports clustering and uses a bully algorithm to achieve consensus.

Show Notes

Vlingo is open-source, used to simplify the building of distributed, concurrent, reactive, event-driven micro-services architectures - it is buzzword compliant, isn’t it?

  • 1:20 I didn’t say containers or meshes, so it’s not all of them - but it’s where we are these days.
  • 1:25 That’s where computing has gone; it’s very distributed.
  • 1:30 Every organisation that I engage with is trying to get to micro-services; many have problems with ball-of-mud monoliths, and they’re itching to get out of that.

What is Vlingo?

  • 1:50 You know my interest in Domain Driven Design; I teach workshops and wrote books on DDD, so I’m doing a lot with that space.
  • 2:00 I feel there is a need for DDD support in framework and toolkits.
  • 2:15 Some in the DDD community have been pretty negative in terms of frameworks, and that you can write your own support code in a day or two.
  • 2:30 I haven’t found that to be the case - teams struggle to understand what they need to implement, let alone do it.
  • 2:45 I want to give teams a strong jump-start into DDD with a DDD-friendly toolkit.
  • 2:55 The toolkit is designed around multi-core concurrency, and addresses a lot of different areas, with more than 12 repositories on GitHub.
  • 3:10 We are bundling specific areas of what you would typically want to use in a reactive platform, and you can extend it with what you need.

How does the framework help you with DDD or bounded contexts?

  • 13:50 A lot will be learned from our examples; what does a bounded context boundary look like, where would I draw the boundary, and what kind of reactive components would I want to use.
  • 3:55 Whether it’s for supporting REST, or streaming, or events, or CQRS - any of those topics will be in the bounded context.
  • 4:10 We will also have components from outside the bounded context; and with Vlingo cluster, you can cluster the bounded context.
  • 4:30 The framework does not get in your way; I stay away from a lot of technical thinking.
  • 4:45 The REST interface has a very fluent feel to it, where your expressions about an asynchronous completion for some kind of response to your request is being handled in an expressive or fluent way.

Where did Vlingo come from?

  • 5:25 I was working on a Vlingo like vision seven years ago, and I was speaking at SpringOne, and I gave a talk on DDD along with talks on reactive.
  • 5:50 There were some reactive proponents in my talk, and they approached me afterwards and asked me to join their efforts instead of starting a new one.
  • 6:05 I put my vision on hold, so it’s not new.
  • 6:15 I recently decided (for various reasons) that there needs to be a reactive toolkit that is a better host and tools for a DDD environment.
  • 6:30 I think being lightweight is important, and so Vlingo is a light-weight set of tools - the whole platform is less than 1Mb in size.
  • 6:45 I could stop developing right now and consumers would have plenty of support that would take them a very long way - but I’m not stopping now.
  • 7:00 I grew up in programming when IBM PCs had at most 256k of RAM with dual floppies; I had to write code that worked very well in that environment.
  • 7:25 That’s been my mindset for a while - and it’s pretty amazing to think of what I used to do with 256k of RAM or even 1Mb of RAM - you can’t boot an OS with 1Mb of RAM now.

Is there an underlying philosophy with Vlingo?

  • 7:55 It’s light-weight and actor based - I have been in to the actor model approach for quite a while.
  • 8:05 I have a lot of respect for what has been accomplished with Erlang and those kind of always-on, can’t fail ecosystems.
  • 8:20 Actors are what I think are objects done right; Alan Kay observed how the Actor model retained the most important ideas behind object orientation.
  • 8:45 Message passing was the key function; we sent messages to Smalltalk objects, although they were all blocking at the time.
  • 9:00 I’ve had messages and eventing on my brain for over 20 years, and is the way I think about solving problems.
  • 9:10 To me, it’s not intimidating at all, but I want to introduce people to the idea that the Actor model isn’t that far off how messages are sent to windows in a GUI.

What does the system look like?

  • 10:00 Under the covers, there is a Vlingo-common library that is 10-15 classes used by other Vlingo components.
  • 10:20 On top of that is Vlingo-actors; the actor model implementation, which is the foundation on which everything else is built.
  • 10:35 There’s the abstraction of an actor’s world, and the abstraction within a world on a stage (actors play on a stage…).
  • 10:50 There can be multiple stages in a world - so you can partition your thread pooling or protect a certain stage like clustering or actors or http.
  • 11:20 Vlingo actors are type-safe; if you go to any other popular actor systems, type safety is not strongly supported - but it’s a baseline requirement in Vlingo.
  • 11:45 You have a virtual reference to an actor, but any method you invoke on the interface is type-safe, and the message created and sent is type-safe as well.
  • 12:00 I think that’s a huge win, and it has zero overhead.

The platform’s small size makes the API and implementation understandable.

  • 12:40 Contributors who have come to the JVM side of things (where the implementation is further along) can jump in and understand the Vlingo components quickly.
  • 12:55 We recently had a contribution ‘Vlingo telemetry’, so you can now have full metrics on actor-message throughput, which actors aren’t keeping up with their messages.
  • 13:15 It only took an hour for me to explain to this contributor were to make the changes.
  • 13:25 There’s another contributor for Vlingo HTTP to add a fluent API to resource handlers.

How does a stage or world relate to bounded contexts?

  • 13:50 You would have a Vlingo world per bounded context.
  • 14:00 Virtually speaking, that’s true - but if you’re on a multi-node cluster, you will have a world per VM.
  • 14:10 You can think of having a different stage for the distribution through a cluster; the stage is an entry on a hash ring.
  • 13:50 On top of actors, there are Vlingo clusters - a way of spreading nodes over a number of VMs.
  • 14:45 There’s Vlingo directory, which is service discovery, running in its own cluster.
  • 14:55 Services register with the directory service, and whomever is interested in that will know what it means will be informed over multicast.
  • 15:25 There’s Vlingo HTTP, which is a REST centric web-server (although it doesn’t support templating or HTML at the moment).
  • 15:45 There’s Vlingo Auth, which is an authentication and authorization tool; you don’t have to use it, but what it does is gives various Vlingo components a way to protect data.

Is all this implemented on the JVM?

  • 16:20 Yes, but there’s also a team working on Vlingo.Net, which is a .Net edition of Vlingo.
  • 16:25 I didn’t necessarily want to get into that on my own, but there were some folks who were interested in contributing it.

What’s the community look like?

  • 13:50 I’m starting to get volunteers on the JVM side - that may not sound significant, but I’ve actually not done a lot of work to try and attract contributors.
  • 13:50 There are more folks on the .Net side - the Microsoft world continues to amaze me, how they’ve re-invented themselves from a proprietary world to people who are heavily supporting open-source.

What stories have you heard from early adopters?

  • 18:00 I have a few early adopters, but I’m not going to name names - the product isn’t ready for primetime, and so it would be unfair to claim I am at that point.
  • 18:20 We should be primetime before QCon in San Francisco 2018.
  • 18:25 The reports that I’m getting - for example, for Vlingo HTTP - is amazing.
  • 18:40 We’re only four months old but already competing against Apache Tomcat, SpringBoot, and something HTTP that I don’t want to talk too much about.
  • 19:00 I am pretty happy - and I’m not even focussed on total throughput right now; I’m using JMeter to try and drive the bugs out.
  • 19:15 Other people are looking at it from the stand point of what kind of throughput we can get - and am seeing 3,000 requests per second a being a low estimate for a single node.

What are you supporting other than JVM and .Net?

  • 19:40 I could provide a polyglot interface for TCP to some of the components that we’re rolling out later.
  • 19:57 My target is the JVM - I see it as at least the second most popular language/platform in the world right now.
  • 20:05 Java (and JVM languages) are very popular, depending on the surveys that you read.
  • 20:15 For example, Kotlin has risen in popularity over the last few years.
  • 20:30 Depending on what you read, Kotlin has already overtaken Scala.

What version of the JVM are you running on?

  • 20:50 I started by limiting myself to Java 1.6 - that might sound nutty, but there are a lot of companies out there running on old JVMs, and I wanted to be able to support that.
  • 21:00 With the new cycles that Oracle are pumping out JVMs, already Java 8 is obsolete.
  • 21:20 When moving up to Java 8, I may be behind the curve for the most recent version; but I have all the facilities that I need and code to the lowest common denominator.

It’s open-source?

  • 21:40 If you go to GitHub then you’ll see all the source there - there’s no separate Enterprise edition.

What about support?

  • 22:10 I have contributors who are volunteering (as am I) - and the contributors I have are really strong.
  • 22:30 I’m proud of them and that they’re interested.
  • 22:40 Long term, I want to make this platform affordable - I don’t want there to be hesitation for companies to use this.
  • 23:15 When your licensing philosophy is that your entry to your platform $100,000 - in order to sell that support, you have to fly a couple of people on an airplane and have all the extra pre-sales stuff.
  • 23:50 If you spend $60,000 to make that sale, then you’re not going to clear more than $40,000.
  • 24:05 I think that’s unnecessary - it could be that few people can support the platform long term.
  • 24:20 My company is bootstrapped - anything I do is paid for out of my pocket.

Why is Vlingo a library and not a side-car model?

  • 25:00 I know that service meshes are solving a set of problems, with all encompassing view of understanding your distributed system is important.
  • 25:20 With a service mesh and a side-car model, you are literally doubling the number of nodes in your solution.
  • 25:40 You might say that it’s telling you everything that’s going on, but how many other things could go wrong in that situation?
  • 25:50 I’m not an expert - there are very few experts now anyway, because it’s so new.
  • 25:55 If you’re constantly sending messages between actors and going via a side-car, but I’m sending messages directly - say, 3m messages per second between Actors - that’s not going to happen over a TCP connectio.
  • 26:25 You an still take your bounded context, and deploy them in a container, or a service-mesh.
  • 26:40 Each of those nodes could be deployed with a side-car - the library is very lightweight.
  • 26:50 If you build a deployment and a JAR file that you’re deploying is a couple of megabytes instead of 40-50Mb, you might what else could go wrong.
  • 27:00 From that vantage point, Vlingo is going to be a very small overhead for that kind of deployment.

How do you achieve clustering or consensus?

  • 28:10 The boley algorithm is like Raft; you have a leader election comparison of what is the greatest node id.
  • 28:35 When a node knows that a new leader needs to be elected, it distributes its id to every node that it’s aware of, and delegates the leadership decision to those nodes.
  • 28:50 That election takes place very quickly, and the leader that is not aware of any other higher node or doesn’t hear back from others will assume leadership after a timeout.

What happens during partition?

  • 29:30 The cluster has to have a quorum, so you have to run with an odd number of nodes.
  • 29:40 If you have sufficient nodes, there is no way - let’s take an example.
  • 29:45 If you have five nodes, and there is a partition between two of those nodes, then the three aren’t talking together, there’s no way for the two to say it’s able to elect a leader because it doesn’t have quorum.
  • 30:00 What’s an interesting partition is where you have four nodes, and one nodes can see two of the nodes, and a different node can see the same two nodes, but they can’t all see each other.
  • 30:10 In that case, the boley algorithm works out well because the ultimate leader who declares that they have the highest node is seen by those nodes who can see the leader.
  • 30:30 Nodes can identify that a greater leader can be seen, so you can’t have it, even if that leader.
  • 30:50 It could keep sending out broadcasts periodically to claim leadership and the other followers can deny leadership based on the existence of another leader with a greater node id.
  • 31:00 In a repaired partition, it could eventually see the leader.

What kind of messages to they send?

  • 31:10 They are very lightweight TCP, but they could be UDP - the TCP is the initial MVP mindset.
  • 31:30 We can tune and make things faster later.

What workshop are you doing at QCon San Francisco?

  • 31:45 It’s called Event Oriented Domain Driven Design, so it’s an event-centric workshop where events are a first-class citizen and taking the front row of attention in the workshop.
  • 32:00 We are going through DDD from a very fast-paced introduction to DDD, including the things that you need to know, context mappings and 2-3 integration maps.
  • 32:30 We will cover the tactical patterns of DDD and it is a very reactive workshop, but it doesn’t have to use Actors - though if people want to use Vlingo then they can if they want.
  • 32:45 I have a very simple macro-services toolkit which can run on JVM, .Net or PHP and have a mocked-out micro-services platform in a single VM.
  • 33:10 If attendees want to use Vlingo, they will be able to and I hope to have the examples ready before that point.

More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article