Jonas Bonér on Akka, Actors and Shared State, STM, Typesafe

1. We are here at Erlang Factory 2011 in London and I am sitting here with Jonas Boner. Jonas, You have created a framework called Akka which brings actor programming to the JVM. What is so great about actors?

I think actors are great for many things. Actors implement what's called "message passing concurrency" also and the actor model enforces that there is nothing like shared state. Every actor has his own view of the world and the only way an actor can influence another actor because you can look at them like live components, they can run in different threads and the only way me as an actor can influence another actor is by sending him a message and have him update his state internally. I can’t go in and tamper with his state. And what this gives is this really nice isolation of state as a very explicit state management that lifts all of these problems that you normally have with mutable shared state in a concurrent setting.

I also think that mutable shared state is bad even in a non concurrent world, but even more so in a concurrent world, so it lifts all this plumbing that we normally are used to guard our state because anyone can touch my state at any point in time even while I am looking at it, things can get changed, then I need to guard it with locks to make sure that no one else can touch my state and that introduces a lot of low level plumbing that gets tangled with my code so instead of that the actor model will lift the logic of that into pure workflow so the concurrency is all about the workflow, which messages are flowing in my system and how my components are reacting when receiving these messages. So that makes it much easier to reason about concurrent code.

It also gives you a naturally nice model to write event based systems, actors they never do anything unless being told, they are completely reactive, they are also extremely lightweight. So actors only consume memory, they do not sit on a thread, so if you compare to using a thread based model, threads are extremely heavyweight, you can only run X number thousands threads on the JVM, while actors since they are only constrained by memory you can run millions of them easily, tens of millions of them. It’s all constrained by RAM. In Akka one actor is about 600 bytes so it’s almost nothing and this gives you a nicer model to model things that are almost impossible to model using threads.

So the way it works in short is that there is a scheduler that can wake up an actor up, resume it, have it being applied to the messages it has in the mailbox, because this actor has a mailbox, so while the actor is passive it can still receive messages that will be put in the mailbox and the actor can execute X number of messages before it’s suspended and another actor can use the same thread that the suspended actor was just using. So there is interleaving, they share the execution time on a thread pool, that is basically the actor model.

2. You have one scheduler or you can have arbitrary many of these?

You can have many schedulers. The default in Akka is to use a single global scheduler and that works great in many situations for most use cases is pretty generically tuned, so for performance and throughput reasons there aren’t really any need to use and yet another scheduler, but the reason you might want to do that is that you might have different sort of characteristics in how the system is used, in one place the system compared to another place in the system and then it can make sense to have two different dispatchers tuned differently. We have various numbers of dispatchers: we have for example a work stealing dispatcher that tries to steal from other actors if one actor runs out of jobs.

We have pinned dispatcher that pins an actor to a dedicated thread and they are all good for different kinds of use cases. That is usually where our users or costumers spend most of their time tuning the system that can make a huge difference, but can also make no difference all depending on your use case.

3. Unlike Erlang all your actors share an address space because they are on a JVM. So are there any models for avoiding that developers share state and just drag in pointers to something else?

That is a good question. Since we run on the JVM we have one single big heap. The way you do it in Akka now is all about convention. So Akka had both a Java and a Scala API. In Scala there are many nice idiomatic ways and you get a lot of support for from the language itself to use immutable state, so it makes you pretty easy to only send immutable state between messages, but you can still send mutable stuff between actors, but then you are back to shared state concurrency and you need to guard your state with locks. That is bad. However, we can talk about later, but Akka has very integration with Scala and now will get even tighter integration with Scala with the Typesafe launch and the Typesafe stack, we can talk about the company later.

But what this will give us is very likely that we can make the integration between Akka and Scala even tighter and currently there is in process a few people of Martin Odersky's Research department that are looking into ways of encoding these things into the type system, you can actually get compiler errors if you try to share mutable states or you can even enforce immutability into the type system. I think there is still a lot of work that needs to be done there, but I think we will see some progress within a year or so and that looks very promising that is something that we are going to embrace in Akka as soon as it is there.

There is also one way you can sort of enforce it at runtime that is by using deep copies. So we have an option that you can turn on and off, serialization messages. If you are running Akka on different machines then you naturally have that and then you do copy on send so then you don’t run into this problem. In process you can still also turn on serialization and that will avoid that problem.

4. I think Akka also has some support for software transactional memory. Does that mean you bring back shared state or do you bring back sort of well behaved shared state?

I think that a lot of focus is on actors in Akka, but I think there are some problems that are really hard to solve using actors, close to impossible to solve in a nice way using actors and that is when you need to have a global consensus, you need to have an agreement between two different actors or two different components, running in different threads or even on different machines. The classic way of solving that problem, for example the way to solve it in Erlang is by putting that state for whatever agreement they want to have into a database like in Erlang they have Mnesia so they can shovel that down into Mnesia you have that atomicity and then you can read out the state and you know you have the same view of the world.

The problem with that is that if those two actors can be in process, they just sit next to each other, it's really costly that if they only want agreement in memory you have to go through all the IO and all the latency with IO down to disk just to get agreement and then software transactional memory we tend to call transaction references and transactional data structures are really nice because they give you the same semantics that you have in ORACLE, in a SQL database, you get begin-commit and rollback in memory, so you can get ACIDness without the D, you get atomicity, consistency and isolation in memory and since everything is in memory it's extremely fast.

In Akka we can run millions of transactions per second, so transactions and this really costly agreement becomes very cheap. And I think it’s a really good complement to actors, I tend to perhaps use actors for 80% of the problem or more, but these 5 or 10% are really hard to solve in process. You basically need to implement a two-phase commit protocol yourself or put it down in a database. Instead you can use transactional memory for this. We also have unified actors and transactional memory, transactors. So we have two different use cases: the first one I was talking about is actually having two actors, actually share a common data structure, a transactional map, for example, so we can have both these actors have the same reference to this map, but that is now safe because now the map is transactional.

The other way is to have these two actors have their internal state both of them, but their communication can be transactional. So if you have 3 actors, for example, you have this classic bank account example, you have Bill and you have Bob and you have one guy who wanted to withdraw 10 $ from Bob and deposit that into Bill’s account. What if he withdraws 10 $ and the transaction fails before he deposits it? With transactors you can get atomic message flows, so you can ensure that either both of these transactions, even though they are based on asynchronous message passing, make it or none of them does.

So if the transaction fails after the deposit even though that happened asynchronously some time ago in another thread, that state inside that actor will be rolled back and the transaction will be retried. And you can use that for any number of messages sends, so it basically gives you atomicity at the workflow level between how the messages flow in the system. And that can also be very useful even perhaps for even tinier percentage, I tend to use that not very much, but when you need it you really need it, that is really a hard problem to solve without something like that.

We also have a couple of other paradigms for concurrency, we have agents that we borrow from Clojure, similar to actors but not quite and more functional style of using all the actor paradigm, and we also have dataflow concurrency that is heavily inspired by a language called Oz, it gives you a declarative and closely deterministic or fully deterministic model to concurrency.

5. What is the Oz model?

It has single assignment variables so that's a very simple concept. You almost have that, not quite but it is very close to Futures in java.util.concurrent, for example, so that means you have a reference that can be either unbound and then it can be bound and once it’s bound you can’t bind it again. So it’s one time only. But it has this thing on top of that that you can actually try to access, you can try to read a variable that is not yet bound and what is going to happen then is that you will not get an exception or anything, but you will actually block waiting there for that value to be set.

You get this on-demand kind of workflow, so threads wait until data is available and in Akka we have also one model based on delimited continuations, so you can get this waiting but the suspension does not block a thread, but it actually all runs in a single thread, because of using continuation passing style under the hood to block on the conceptual block. And that is a very useful technique. It gives you completely deterministic concurrency in the sense that if you have a program and you run it once and you get output "Hello World" or 5, you will get 5 every single time, nothing else can happen, but if you run it once and you get a deadlock you will get a deadlock one million times. It can never happen anything else than to get a deadlock. This can be code you fully can trust and that is also a nice addition to the actor model.

But we use actors for other things as well in Akka, is not only for concurrency. Akka is also about distributed computing, remoting and for that actor it’s a really amazing, very slick abstraction to use, making things much easier.

6. Is this distribution transparent with Akka, can we just put a JVM over here and then just send messages to an actor there, how is that?

I would rather say that instead of saying that distribution is transparent it is the other way around. We embraced distributed computing at its core even if you run it in-process, actors are really distributed components, because almost like if you run them in process that is just an optimization. So by writing a system with actors we are preparing it to be run on the cluster in a distributed environment. And if it does happen to run them locally, OK that is fine, but that is just an optimization. That is sort of the way I try to look at actors and since you already had prepared them as distributed components then in that context, sure everything becomes transparent because you have this separation: you have an actor here, but instead of getting a reference to the actor, the actual instance you get sort of a proxy, in Erlang is called "pid", the process id, that is like a pointer to the actor over here.

In Akka we have something called "actor ref", so even though you have this actor running right here, you don’t get straight access to it, you get sort of a proxy, a reference and this reference can actually point to something here but it can also point to something on another machine. You use it the same way, you send it the message, it doesn’t matter if it’s in-process or on another machine. Since you have this level of indirection it becomes transparent in the sense that is already prepared for being distributed in a way. And this also gives the runtime freedom to optimize things on the fly. It can actually move actors around and optimize things in the cluster and that is something that we work even harder on right now to make it easier, that is going to come out in Akka 2.0 after the summer, early Fall, or mid Fall latest.

7. Akka 2.0 - what are the big ticket items?

The biggest change is that Akka 1.1 and all the Akka up to now, we have had a distinction between remote actors and local actors and that sort of goes against like all I’ve said, like all actors are in a way distributed. And the problem with this approach is that it has been a choice that you need to make as a developer when you implement your system should an actor be remotely available or should it not be remotely available. Actually the way you write the actor is the same but the way you register it makes it remotely available so it is this extra step to remote-enable an actor.

That is something that we are completely removing now, there will not be a programmatic way of remote-enable some actors while others are not. Instead what we will allow you to do is that you write your system not thinking about which component should be where, that should be remote there and this one should be remote actor over there, this should be local here. Instead of doing that you configure the system. So if you write your system and you run it without a configuration everything will be local actors that will just run in process. But now if you write a configuration file and you feed your system or your cluster with that configuration file, in that configuration file you can declare the list which actor should be remotely available, which actors should have a proxy with a router; you can say: "I want my actors to be clustered, I want my actors to be proxied by a round-robin router, least number of messages, least CPU and that means that whoever uses that actor reference will then be routed to the node that has the least CPU right now.

And they are also declared to configure things like replication, is the actor stateful, should it be replicated if it’s stateful, how should it be replicated, how many replicas should we have, what should happen if it there is a failover and things like that. All that is declaratively configured, so it’s like separated from the implementation and the nice thing there is that then it becomes completely transparent and it becomes operations, an deployment artifact like something that even the operations can write and configure. This also makes it easier since there is nothing hard coded, this actor is there, you should access this by this IP address and this port and stuff like that we have in Akka 1.1. The system has all the possibilities now of optimizing things because there are no rules defined statically, so you can actually move actors around and optimize the cluster by re-balancing the cluster and things like that.

You can easily add more nodes, the cluster can start taking advantage of those, move actors over to the new nodes to spread the load and things like that. I am really excited about that; that is something that is coming in Akka 2.0 as sometimes it’s Fall.

8. The Akka 1.1 release also saw the release of a company that you founded together with Martin Odersky, Typesafe. Why did you do that?

Long story short, I started a company 3-4 years ago called Scalable Solutions and that was initially a consulting company, but I started also working on Akka during that time as an open source project. After a while I started getting a lot of interest in commercial services and also commercial products on top of Akka, like monitoring management and things like that. So I transformed my consultancy company into a product company focusing on Akka. At the same time Martin Odersky, the father of Scala, he got the same kind of questions, but for Scala, commercialization of Scala, the companies that use Scala wanted to have support and training. He started the company Scala Solutions.

About a year ago we started talking and we had common interests, Akka is built on top of Scala, it has almost no overlap or zero overlap is just like little one thing like the next layer in the stack and we also got a lot of requests for joint support on Scala. I did Scala trainings, Martin also had interest in doing Akka stuff, so we put our things together and said: "Let’s create a company together". We also raised the first round of investment from Greylock Partners, which has now enabled us to take the whole Scala and Akka stack to the next level.

We have the whole product series with developer tools, build system, SBT build system, we are working hard on IDE support now, we have a migration manager to solve the problems with bytecode, incompatibilities between releases and so on. It was working hard on some commercial products on top of this stack like monitoring management, provisioning and things like that. We’ve just been out a month now, but there is also already a lot of things happening, so I am really excited about that.

9. That is great. Let’s all look out for Typesafe and Akka and Scala and thank you very much, Jonas Boner.

Thank you.