From Microliths to Microsystems: Jonas Bonér at QCon London
At QCon London, Jonas Bonér, CTO at Lightbend, presented 'From Microliths to Microsystems'. Bonér explored microservices from first principles, and discussed the architectural style in the context of distributed systems. Key takeaways included: avoid building mini-monoliths, or so-called 'microliths', and instead create systems that are resilient and elastic; minimise coupling and communication within system; 'embrace reality' by utilising reactive programming, reactive system design and eventual consistency; stateless behaviour should be separated from the stateful entity to enable easier scalability; and practice events-first Domain-Driven Design (DDD).
Bonér began the talk by stating that "we have been spoiled by the [design simplicity of the] almighty monolith", but the challenges of scaling development activities around a monolithic codebase mean that embracing the microservice architectural style is now 'becoming a necessary evil' for many organisations. Microservices are effectively distributed systems, and developers must design both individual microservices and the overall architecture accordingly.
Much like actor-based applications, which are naturally composed of multiple actors to accomplish any arbitrarily complex task, a microservice-based application should be designed as a distributed system composed of multiple services.
Each microservice needs to be designed as a distributed system: a microsystem. We need to move from microliths to microsystems.
Bonér proposed that many developers are building 'microliths': a single instance microservices that is not resilient or elastic. Referencing 'The Reactive Manifesto', the path towards resilience and elasticity includes:
- Decentralised architecture
- Failure detection
- Gossip protocols
- Location transparency
Bonér cautioned that as soon as a developer 'exits the service' or interacts with the system outside of a single microservice, a 'wild ocean of non-determinism' is encountered. Systems must exploit reality, such as the speed of light setting boundaries for speed of communication. Quoting Pat Helland, Bonér stated that all information has latency, and in effect this means that we are always looking into the past with distributed systems communication. Developers should strive to minimise coupling and communication, and embrace eventual consistency.
We have to rely on eventual consistency. But relax, it's how the world works.
Bonér proposed two helpful tools for designing microservice systems, which are reactive design and events-first Domain-Driven Design (DDD). Reactive programming, such as RxJava, can help make the individual service instance highly performant and efficient. Reactive systems based on asynchronous message passing can help build distributed systems that are elastic and resilient. Implementing microservices as asynchronous and non-blocking can lead to more efficient use of resources, and minimises contention on shared resources. Always apply back-pressure: a fast systems should not overload a slow system.
Each microservices should be designed as a distributed system, a 'microsystem', and stateless behaviour should be separated from the stateful entity, in order to allow individual service scalability. The entity can become a 'safe island of determinism and strong consistency'. However, it should be noted that scaling stateless behaviour is easy, but scaling stateful entities is hard.
Bonér suggested that developers should practice events-first DDD, and think in terms of 'consistency boundaries' with data on the inside representing the current present, data from the outside representing facts from the past, and commands used as future actions.
Don't focus on the things - the nouns. Focus on what happens - the events! Let the events define the bounded context.
A microservice should contain any mutable state and publish facts. Quoting Pat Helland again, Bonér discussed event logging, and stated that the event log should be a 'databased of the past' and the single immutable source of truth. Event logging can avoid the infamous object-relational impedance mismatch, and the read and write model (which often have differing requirements) can be untangled with Command Query Responsibility Segregation (CQRS) and Event Sourcing (ES). Developers should not implement large scalable applications assuming distributed transactions, and instead use a protocol of 'guess, apologise, compensate'. Bonér stated that this protocol is 'how the world works', and cited offline ATMs and flight overbooking as examples.
Bonér concluded the talk by briefly discussing the Lightbend Lagom microservices framework, which embodies many of the principles discussed in the talk, and also presented his free O'Reilly report 'Reactive Microservices Architectures: Design Principles for Distributed Systems'. The slides for 'From Microliths to Microsystems' can be found in Bonér's Slidehare account.