Martin Thompson Discusses the Reactive Manifesto 2.0
Martin Thompson announced the second version of the Reactive Manifesto at Goto Aarhus 2014. This new edition offers subtle differences in both the naming and approaches prescribed by the original document. InfoQ was able to interview Martin shortly after the announcement and discussed the implications of those changes and the future of the reactive community. The React 2014 conference in San Francisco will delve further into the Reactive Manifesto and its implications as well as showcase the experiences of developers who are currently practice its tenets.
InfoQ: The Reactive Manifesto was well received and has gained acceptance in the development community. Why did you see the need for a rewrite?
Martin:After the original manifesto was released we received lots of feedback. Generally it has been really positive as many have begun to appreciate the changing environment of applications development. However some feedback rightly pointed out that the previous versions had been too verbose, and in places focused disproportionally on specifics. We were encouraged to make it more concise and focus on the properties reactive systems should exhibit. "If I had more time I would have written a shorter letter..." definitely applies in this case. Plus wider feedback really helps refine a focus.
InfoQ: Why now?
Martin: People like Kresten Krab Thorup made a strong case for "why now" in the manifesto. He pulled together what others are seeing with his own experience and highlighted that we are in a world where the Internet is the primary, or only, channel to market for many organisations, and he combined this with the confluence of the changes in platform in our multi-core and cloud world.
InfoQ: Did the structure of the manifesto shift? Is it written in a prescriptive instead of descriptive manner?
Martin: We wanted the manifesto to focus purely on why we now need a new mainstream approach to systems architecture, and the key properties these systems are required to offer.As part of the rewrite we wanted to be descriptive about the properties and qualities of a reactive system. The key properties being that a system should be responsive, resilient, and elastic which is fundamentally enabled by being message driven. We do not want to be prescriptive in the manifesto as to how this is achieved. The prescriptive ways in which this can be achieved can be illustrated in an appendix and via wider discussion.
InfoQ: Message Driven vs. Event Driven has subtle implications and expands the definition of what can be considered a reactive system. What drove this change in the name of this concept?
Martin: Yes it has subtle but also very important implications. The importance is in how components communicate. Events are an important domain concept. They can be passed directly by function call, or they can encoded and passed in a message. Events are only one of many concepts we might wish to communicate via messages. To achieve the other three properties we wanted to highlight that it is important to have an asynchronous binary boundary. A boundary that can provide decoupling of language, location, concurrency model, and temporal entanglement. This is also the fundamental level of isolation.Given this boundary as a means of isolation it is possible contain a failure. Exceptions should not cross an asynchronous binary boundary. Errors should be passed as messages between components and handled as first class concepts. If we do not get a response from a component then we must suspect it has failed.Memory should not be shared in a mutable fashion between components. Shared mutable state is a major source of contention which prevents scaling in the multi-core world. We need to move to design approaches where we communicate to share memory, rather than share memory to communicate - the old CSP mantra. By isolating components we can avoid contention, amortise expensive costs via batching, and take a shared nothing approach that scales in the multi-core and cloud world.In striving to achieve 100% uptime it is necessary to support hot upgrade. An essential ingredient of hot upgrade is to communicate with version encoded messages between redundant components.
Martin: Most people consider scalable to mean increasing scale. It can be just as important to scale back so a business can save costs. I've clients in the retail and holiday booking space that see 3 orders of magnitude in resource needs between quiet periods and peak periods. To always maintain the resource for peak usage would be wasteful to the point the business becomes uneconomic.I've found that when designing a system to both increase and decrease scale, I end up doing an even better job on the increasing scale as a result. This tends to happen because you learn to appreciate that simplicity is so important. Scaling back is also an exercise in ensuring a system can cope with failure because as components get removed, therefore the reminding system must share the load. Removing a component due to failure or scaling back is effectively the same thing.In a world where we can easily provision resources from cloud providers for scale out and in then elastic feels like a better description.
Martin: Responsive and Resilient seem to be terms that resonate well and are desirable properties. Resilience is well mentioned above. Just this last week I seen a demo of the Extempore programming language running a 3D wall that was interactive to human touch. The visualisation was amazing and real-time in its interactive response. If systems like this can be built then why do most websites take seconds to respond to simple page changes? The excuse of developers not having to care about performance is starting to wear real thin for many applications. We can and should do much better for our customers. I'm finding my work split very much between companies pushing the boundaries of performance and those who by becoming more efficient can make significant savings in operational costs. I can see this trend to operational efficiency continuing, especially with the advent of IoT.
Martin: The reactive properties can be considered quality of service requirements. Every system has its own unique requirements. These requirements need to be captured and factored into a design just like any other. The major difference with these requirements is that they are much harder to retrofit.
Martin: When I'm helping clients it goes one of two ways. Either it is new system development or piecemeal migration. Complete systems redevelopment is often much easier than many expect with a good team. However not everyone has the team or appetite for this.A good approach is to identify bounded contexts in a system and then isolate them with a asynchronous binary boundaries. This then enables these components to be evolved. These component may need an anti-corruption layer to integrate with the rest of the existing system. Once the first steps are taken this soon becomes fractal until the ideal size for components is arrived at.When reading about Reactive Systems many people think this implies the need for a message queuing system. This is not that case. Any communication mechanism that can provide an asynchronous binary boundary is sufficient. That could be a messaging product, but equally it maybe websockets or an asynchronous protocol built on top of REST.A lot can be done to build these types of systems yourself, and many people do that in finance for example, or you can use platforms like Erlang, Akka, Go lang, and various event processing products. Reactive systems are not new, the likes of Tandem exhibited these properties decades ago. I expect this to be an area of significant future growth as we come to realise that approaches like JEE and Rails are inadequate for today's quality of service needs.
Martin: This model is very much about building systems which are composed of components, or services, that are fundamentally message driven and offer high qualities of service by being responsive, resilient, and elastic.As we move to micro-service style architectures these properties become crucial. If we have contention on shared resources then things become coupled and brittle. Synchronous calls in this environment are the crystal meth of programming. At first you make good progress but then the sheer horror becomes evident when you realise the scalability limitations and how the brittleness holds back both performance and development flexibility. By then it is too late to save. We need the reactive properties and then apply protocols for the message interactions. Without considering the protocols of interaction this world of micro-services will become a coordination nightmare.So much of this is not new. Many domains have discovered, or rediscovered, these patterns long ago. Telcos, financial trading, betting, online gaming, and large social networks are all converging on similar design patterns that work for these requirements.Our industry now has far too many hipsters and not enough geeks. We need to look back and build on some of the great foundations from our past by those who pioneered concurrent and distributed systems. To this end we are running a small conference that focuses on the great work in this area. Earlier this year we promoted React Conf in London where, among others, Joe Armstrong shared his experience developing Erlang. We are continuing this in San Francisco in November with Leslie Lamport, Pat Helland, and other great speakers. These are the greats of our industry and we need to build on their experience. Where would physics be today if we ignored the work of Newton and Einstein, or chemistry without Curie or Mendeleyev?
Martin: That is an excellent question and one I often get asked. At its core the reactive manifesto is about bringing attention to the previous great work our industry has done in this area. We need it now more than ever. A few of us have been running the React conferences as one means to do this. Others like Roland Kuhn and Jamie Allen are writing a book covering some aspects of it. It is worth studying existing approaches like Erlang and Tandem; or the new approaches like Akka, Go lang, Rust, and Elixir. Fundamentally underlying all these approaches is a messaging passing style that provides the binary boundary necessary for isolation to enable the responsive, resilient, and elastic properties. There are many great papers providing guidance on this subject - unfortunately some are hard to digest and we need to make them more accessible.
Martin: Thanks for the questions Harry.