Microservice architectures and container-based virtualization have taken the software development community by storm in recent months. Adrian Cockcroft, one of the cloud’s most recognized visionaries called the combination of microservices and Docker a “disruptor”, citing that the benefits of microservice architectures can be amplified when used in combination with containers. Microservices encourage software developers to decouple software into smaller functional pieces which are expected to fail. Containers extend this decoupling, separating software from the underlying hardware. The result is applications that are faster to build and easier to maintain while having overall higher quality.
Largely absent from discussions of container-based microservices, however, is the heart of most applications: storage. This article will describe the advantages of container-based microservices and why failing to containerize stateful services like databases too, greatly reduces the benefits of microservices architectures.
Better, faster, cheaper: Microservices in modern software development
Although ‘microservices’ is a term which resists precise definition, if one looks at how and why these architectures are being used today, there are two main benefits that software teams receive: agility and resilience. Let’s look at each in turn.
Agility
The agility of a software solution refers to its aptitude to be changed quickly. Adrian Cockcroft, former Cloud Architect and proponent of microservices at Netflix, describes the need to move quickly when building software like this: “...everything basically is subservient to the need to be able to make decisions, and build things, faster than anyone else...”.
Agility is one of the most heralded properties of microservice architectures. According to Martin Fowler, a prominent microservices thinker and practitioner, this agility comes from the decoupling of the running system into “a suite of small services.”.
By limiting dependences on other parts of the system, this property allows microservice architectures to be changed more quickly in response to a bug or a new feature request. This is in contrast to traditional monolithic architectures where often “...a change made to a small part of the application, requires the entire monolith be rebuilt and deployed.”
Resilience
Martin Fowler points out in his aforementioned article, that due to the distributed nature of microservice architectures, the individual services “need to be designed so that they can tolerate failure of [other] services.” For companies like Netflix, with infrastructure spread across the globe, dealing with service failures is a constant reality. To make sure these challenges are met, Netflix famously tests their systems with their Simian Army, a set of tools that deliberately kill or degrade parts of their running software to test that the system still functions adequately under these conditions.
It is the distributed nature of microservice architectures that allows this to happen. The services in the system expect:
- That their dependencies will be remote - This enables risk mitigation by scaling across different infrastructure. A service does not care if it is talking to a process in the next VM, the next rack, or on another continent.
- That their dependencies will fail - This forces developers to think of resilience from the ground up instead of adding it as an after-thought.
While a powerful method for building resilient software, microservices are “no free lunch” as Benjamin Wooton, CTO of Contino recently reported. They can be implemented badly, just as with any other other sort of architecture. Also, because services are communicating over a network connection, latency becomes an important consideration. Despite this, microservices are getting a lot of attention right now with companies like Netflix, Spotify, Twitter and Gilt already onboard. As Martin Fowler himself states “...for many of our (ThoughtWorks) colleagues this is becoming the default style for building enterprise applications.”
Enter Containers
Microservices encourage software developers to decouple their software into smaller functional pieces which are expected to fail. The advantage, when done well, provides increased agility and resilience.
While already an emerging trend with sophisticated engineering organizations, microservices got a shot in the arm in 2013 when Docker Inc. (formerly DotCloud) released Docker, an open-source tool which wrapped existing container-based virtualization technologies in such a way as to make them more accessible to the wider engineering community.
The result was an explosion of interest and adoption of containers which had formerly been a tool used only by the most sophisticated organizations, such as Google which famously runs everything in containers. Adrian Cockroft summed up this newfound interest in his keynote at DockerCon Europe 2014: “Docker was on nobody’s radar for 2014. It’s on everybody’s radar for 2015.”
Docker’s appeal is twofold: it’s fast and it’s portable.
Fast
Unlike virtual machines that must boot an entirely new operating system instance each time they start, docker containers ‘piggy back’ on top of the host operating system by sharing its kernel. This means that instead of taking minutes to start, Docker containers can start, and stop, in hundreds of milliseconds.
This extra speed means that software systems built using Docker containers can achieve higher levels of agility than previously seen with VM-based solutions, even when those VM-based solutions were arranged in microservice-based architectures. Moreover, “containerized” applications can achieve comparable performance to virtual machines and bare metal with a 2014 IBM research reporting that “containers result in equal or better performance than VMs in almost all cases”.
Portable
With VM-based solutions, portability is normally restricted within the the available regions of one cloud provider, or in the case of an enterprise running their application in-house, within their own data centers. This is because the different cloud providers often use different virtual machine formats. It is possible to build virtual machine images to run on different clouds with tools like Packer, but it’s a lot of extra work. Effectively then, this binds users to one platform. More on why that might be an issue below.
Docker containers with their “write once, run anywhere” design allow developers to remove this restriction. Engineering and operations teams can spread their infrastructure across multiple cloud providers and as long as the Docker daemon is running, they’ll be able to run their application there too. This decoupling from the underlying cloud provider gives IT greater freedom and can be used to boost the resiliency of their software solutions by hedging bets across providers.
Microservices + Docker Containers: The sum greater than its parts
At DockerCon Europe 2014 Adrian Cockroft described the combination of microservice architectures with Docker containers as a “disruptor”, and it’s clear to see why: the reported benefits of microservice architectures can be amplified when used in combination with containers. The former encourages us to decouple our software into smaller functional pieces which are expected to fail, bringing us agility and resilience. The latter decouples our software from the underlying hardware, bringing us portability and speed not seen before in VM-based solutions.
What about state?
And yet while there has been an explosion of interest in microservices and Docker containers in 2014, to understand a significant barrier to these trends, we need to examine what is commonly referred to as stateless application design.
Stateless application design
Builders of microservice architectures prefer stateless services over stateful services wherever possible. The primary benefit of stateless application design is the ability to respond to events by adding or removing instances of a service without needing to significantly change or reconfigure the application. For example, if load suddenly increases, more stateless web-servers can be added, or if an existing stateless service dies, it can simply be replaced with another. Agility and resilience, then, are easier to achieve with stateless services.
Stateful services
Despite the trend toward stateless application design, there are many systems where state is simply unavoidable. For example, any system that saves data has to manage state and the number and variety of stateful tools is growing. MongoDB. PostgreSQL. RabbitMQ. Redis. Cassandra. Riak. MySQL. ElasticSearch. The list of data services used by modern developers goes on and on. As microservices architectures become the norm, developers and architects are picking different data services to use with different parts of their application. Billions of log messages to parse? ElasticSearch. Jobs queue? Redis or RabbitMQ. Customer signups? MySQL or MongoDB. all this can happen in the same application.
The point is that in the modern stateless application architecture, state is actually everywhere--and this state needs to be managed.
State? What to do?
Due to Docker’s explosive growth, some fundamental questions have been left either unanswered or partially answered; none more so than the question of state, also often referred to as persistence or storage.
From the very beginning some voices were claiming that Docker would never achieve wide-spread adoption due of the state question. Although their predictions turned out to be false, the question is no nearer to being answered. Andrew Phillips, VP of Product Management at Xebia Labs, has made ‘Storage’ number 1 in his ‘8 Questions You Need to Ask About Microservices, Containers & Docker in 2015’.
With little information about, or solutions for, managing stateful services like databases from Docker itself, the software development community has largely ignored the problem of state when it comes to containers. In most cases, containers are only used for the stateless parts of the application; any stateful services like databases are treated as ‘Backing Services’ managed outside of the normal application lifecycle. This suggestion, while popular, makes data--the heart of an application--somebody else’s problem. It severely limits the benefits to be gained from fully embracing container-based microservices for the entire application.
Benefits of running stateful services in containers
A chain is only as strong as its weakest link and if engineering and operations teams want to achieve agility and resilience at scale with microservice architectures and Docker, they need their stateful services to be as fast and portable as their stateless services, not managed outside of the application where there is less control. With this is mind, what are the specific benefits to be gained by bringing stateful services into a container-based micro-services architecture?
Benefit 1 - Dev/Prod Parity
Barry Boehm and Philip N. Papaccio first showed that bugs caught later in the development process cost exponentially more to fix1. In that first paper in 1988, the authors suggested that a bug caught “in the field” would cost 50-200 times more to fix than the same bug caught in development. Barry Boehm and Victor R. Basili followed up in 2001 and arrived at a less severe figure, stating that bugs caught in production would cost more 5 times as much2, but others have reported similar trends.
Later studies showed that engineers spent on average 40 times more effort fixing bugs discovered in production than those found in code review or unit testing, and 5 times more than those found in system testing.
Whichever study is reviewed, the point is clear, catching bugs earlier is desirable, but how can developers do that? Assuming that engineering already maintains an automated suite of tests that can be run by developers, there may still be one thing tripping IT up: ‘dev/prod parity’.
The idea behind dev/prod parity, which gained wider acknowledgment after its inclusion in Heroku’s Twelve-Factor App manifesto, is all about reducing the differences between the environment where code is created, ‘dev’, and the environment where code is run and used by your customers, ‘prod’.
The rationale is simple; if code is tested in one environment, and then run in another dissimilar environment, the test results may be invalid, and teams may end up paying more to fix any bugs which might arise later in production.
Since containers require far less memory to run than comparable VM’s, developers can run more of them on their development machines. Imagine having 10 VM’s running in production; it’s going to be pretty much impossible to run 10 VM’s, no matter how small they are made, on a normal development machine.
So in order for developers to do any local testing, they have to start taking shortcuts like running SQLite in development and hoping that everything will be alright in production. However, as Joe Kutner, JVM Languages Owner at Heroku, clearly states, “SQLite ≠ MySQL ≠ PostGresQL.” Each of these shortcuts reduces dev/prod parity and increases the chance of bugs slipping through to the later, more expensive stages of the delivery pipeline.
Running 10 Docker containers on a development machine, however, is easy. This enables developers to increase dev/prod parity by mimicking production in a development environment.
This general principle, i.e., that the likelihood of finding bugs early is increased by creating parity between the development and production environments, suggests that ALL parts of an application, including stateful services like databases, should be tested locally, just as they will be run in production.
Benefit 2 - Avoiding vendor lock-in
With many companies augmenting or entirely replacing their internal infrastructure with cloud based alternatives, vendor lock-in is a growing concern. In a 2014 global survey of more than 650 IT professionals, Dimensional Research found that 91% of respondents were planning to deploy “new cloud based offerings” over the next 12 months and that 77% “were planning to deploy to multiple clouds” within the same time period.
Before containers, running on multiple clouds was tricky. Competing cloud providers have different virtual machine formats which often makes running on multiple clouds prohibitively expensive in terms of the configuration management effort required.
Since Docker containers contain both the runtime environment and just enough OS for the application to run, the only requirement on the underlying infrastructure is that the Docker daemon is installed. This way software companies need to only maintain Docker-enabled VM images for each cloud provider on which they want to run.
The write once, run anywhere mantra of Docker promises an end to vendor lock-in. But if you are only using containers for the stateless part of your application, this argument breaks down.
Databases work best when located close to their app servers, so the ability to move stateless app servers between cloud hosts is of limited value if they can never move more than a negligible distance from the data upon which they act. Once databases, and their data volumes have been containerized, this picture changes. If operational concerns demand a full data center migration, an entire containerized application can be moved between data centers.
Due to the challenges of moving large data sets long distances, this is likely not an operation that an ops team will want to conduct on a regular basis. However, the ability to do it without rearchitecting the application as part of the migration, has great appeal for teams wishing to minimize their reliance on any single cloud provider.
Reason 3 - Reduced management surface for ops
In a 2012 presentation, Randy Bias, the CTO and co-founder of Cloudscaling Inc., made famous Bill Baker’s “pets vs. cattle” analogy for servers. “You name them and when they get sick, you nurse them back to health,” said Bias of the “pets” approach to systems administration; a situation in which operators have intimate knowledge of not only the servers they are administering, but also the software that gets deployed upon them. The result of this tender loving care is a proliferation of Snowflake servers, which are characteristically unique, delicate, and painful to change.
One result of a datacenter full of snowflakes is an adversarial relationship between developers and operators because their incentives are opposed. Developers want to maximize change because they are incentivised to release new software, while operators want to minimize change because they are incentivised to maintain the stability of existing software.
The answer to this impasse has been two-fold. Organisationally, the Devops movement has attempted to bring development and operations teams closer together in order to align their incentives. Meanwhile, technically, operators have had to change the way that they look at their servers. “You number them and when they get sick, you shoot them [like cattle]”, explained Bias.
This realization first lead to shorter-lived servers, or ‘Phoenix Servers’, and inevitably lead to servers that were redeployed every time the software changed, also known as ‘Immutable Servers’. Docker containers, it turns out, make perfect immutable servers.
Treating servers like cattle has enabled companies like Netflix to achieve agility and resilience at a huge scale. “Standardized portable container deployment saves time and effort,” explains Adrian Cockroft. The key then, is a small and well defined management surface for ops, meaning that maintaining different toolsets for stateless and stateful applications would be a step in the wrong direction.
As Mike Ward, Google Cloud Platform’s Global Head of Solutions, recently explained, “If you’re building applications today you have access to ‘bare-metal’ and virtualized on-premises infrastructure, public and private clouds, as well as a multitude of available PaaS options. You might have as many as six different ways to package and deploy software! By standardizing on a container format, providers in each of these models can deliver a uniform experience...”
From this vantage point, if standardizing on the container format is only for stateless application components, then the benefits of standardization are reduced, since there are still two systems to manage: stateless and stateful.
By providing a narrow and well defined surface area between all application components, both stateful and stateless, as well as the underlying infrastructure, we have the opportunity to continue to reduce the adversarial relationship between developers and operations. With a single set of operational tools for the entire applications, the wall of confusion becomes a healthy separation of concerns.
Reason 4 - Bare-metal performance without sacrificing multi-tenancy
“What do you guys think is driving business decisions about technology?” asked IBM’s Boden Russel at DockerCon 2014? His answer: “Revenue, revenue, revenue.”
One of the staple selling points of virtual machines has always been that they make more efficient use of the underlying hardware, a metric often referred to as ‘density’. The idea is easy to understand; operations departments like to isolate applications inside separate servers, but with bare metal servers this means that the server often spends a lot of time doing nothing but costing money. Virtual machines allow the underlying hardware to be shared which results in higher density and lower costs, or, higher revenue.
Virtualization is not without its problems, notably poor performance with databases which tend to be i/o intensive. This is why many service providers with a database-as-a-service offering, often use containers instead of virtualization. For example when Rackspace designed its cloud based relational database service, ‘Rackspace Cloud Database’, it opted for Linux containers over virtual machines. Virtual machines “imposed performance penalties on the very same hardware components that databases require to operate properly, because a guest is effectively not aware that is being virtualized.” explains J.R. Arredondo in a 2013 whitepaper.
In a white paper outlining the reasons for using containers and their benefits for databases, Arredondo explains that the two traditional virtualization options available, hardware virtualization and paravirtualization, presented Rackspace with a “Catch 22” situation. Hardware virtualization imposed performance hit on the guest OS and required more overall resources to run. Paravirtualization can achieve better performance than hardware virtualization, but at the expense of portability.
“The overhead cost in resources used by container-based virtualization tends to be small, typically around 2%, whereas traditional hardware virtualization tends to utilize 10-30% of the available CPU resources,” explains Arredondo.
By opting for a container based solution for its storage offering, Rackspace was able to choose for revenue on two fronts: by increasing density they reduced underlying hardware utilization; and by increasing performance, they made their offering more competitive in the market.
The Rackspace example is not an argument simply for why service providers should opt for container-based virtualization. As more and more businesses rely on software to deliver services to their customers, the need to optimize their infrastructure cost AND increase performance starts to resemble the same demands at service providers. Running databases in containers helps put the focus back on revenue.
Conclusion
This article has examined how the explosive growth of container technologies is changing the way that cutting edge companies build, deliver and maintain their software. When combined into microservices, container technologies can deliver agility and scale at an unprecedented scale.
In the early days of this huge shift, however, some challenges are being left under-examined, notably storage. As more companies start to realize the benefits of container technologies and micro-service architectures there will be mounting pressure to examine these challenges more deeply since the benefits of the microservice revolution are under-developed unless storage solutions evolve to work well with containers.
About the Author
Luke Marsden is co-founder and CTO of ClusterHQ, The Container Data People. He has 12 years of experience building web apps, and using containers and distributed storage. Inspired by the practical operational problems faced running web apps at scale, he started ClusterHQ. Luke holds a Computer Science degree from Oxford University.
References
1 Boehm, Barry W. and Philip N. Papaccio. 'Understanding and Controlling Software Costs,' IEEE Transactions on Software Engineering, v. 14, no. 10, October 1988, pp. 1462-1477
2 Beohm, Barry and Victor R. Basili. 'Software Defect Reduction Top 10 List,' Computer, v. 34, no. 1, January 2001, pp 135-137.