The Container Landscape: Docker Alternatives, Orchestration, and Implications for Microservices
I attended a great event organized by Microservices Meetup Berlin: “Orchestrating Microservices”. They invited experts from various companies (Google, Microsoft, Amazon, Mesosphere, CoreOS, DigitalOcean) to present container technologies respectively cloud services and how they orchestrate container-based microservices. This article extends the content of the event and gives an overview about many different alternatives, their differences and the potential future of container tooling. The container war has just begun!
Definition – Container Technologies and Orchestration Engines
If you do not live behind the moon you have probably heard of Docker and all the hype around it. However, we’ll begin with a short definition of two key container-based application concepts in order to establish a shared understanding:
Containers run as a group of namespaced processes within a Linux operating system, avoiding the overhead of starting and maintaining virtual machines. Key differentiators of containers compared to VMs are the packaging format and portability being created as fit for purpose for modern infrastructure, and therefore lowering footprint and startup times, and providing repeatability, better resource utilization of servers and better integration into the whole development ecosystem (such as Continuous Integration/Delivery lifecycle). See Microservices, Containers and Clout-Native Architectures” for more details.
Orchestration includes especially the following tasks: Scheduling (placement, replication / scaling, resurrection, rescheduling,, upgrades, downgrades), resource management (memory, CPU, volumes, ports, IPs, images) and service management (i.e. orchestrating several containers together using labels, groups, namespaces, load balancing, readiness checking). I quoted Mesophere for this definition.
If you are still unsure about container orchestration, please watch this fascinating 8 min video: “The Illustrated Children's Guide to Kubernetes”. This is one of the best technology videos I have ever seen! Worth watching even if you understand the concepts already.
Container Alternatives and Cloud Services for Orchestration
Plenty of container alternatives and corresponding cloud services are available on the market that orchestrate microservices (and therefore the underlying containers). Container technologies and orchestration engines are usually used closely together. Often, they are built into the same tooling. Cloud offerings, where “users pay only for the resources - such as compute instances, load balancing and scheduling capabilities - that they use” are called CaaS (Container as a Service).
The following list contains the differentiating container platform feature sets with different pros and cons. Also note that the following is – of course – not a complete list of container and orchestration offerings (but hopefully shows most of the currently relevant options):
Docker is the container technology with most public traction and is “almost” a de facto container standard right now. Docker is open source with several vendors behind it. Though, Docker Inc. is the one “official” company controlling the project. They also recently added the orchestration engine Docker Swarm to the main container project. Others like Red Hat and IBM contribute to the open source code. Various vendors offer support, consulting and cloud services (such as a public or private Docker Registry).
Core OS’ rkt (pronounced “rocket”) is another container technology emerging together with its container orchestration engine Fleet. It is a low-level framework built directly on systemd, often used as “foundation layer” for higher-level solutions. rkt focuses on security, composability (e.g. native Unix integration), and standards / compatibility – as differentiator to Docker. rkt also can run Docker images natively and has native Kubernetes integration via “rktnetes”. CoreOS therefore also offers an “Enterprise Kubernetes Solution” called Tectonic. This might be very important for future adoption in more projects.
Cloud Foundry’s Garden Garden is used under the hood of the open source PaaS CloudFoundry. As many relevant software vendors like IBM, SAP or Pivotal base their PaaS strategy on CloudFoundry, Garden containers get used by many enterprises “under the hood”. In contrary to Docker and rkt, there is no real market outside of CloudFoundry for Garden containers.
Kubernetes is an orchestration engine for containers with a huge community behind it. This project was released as open source by Google earlier in the year; with many other contributors including software vendors like Red Hat, CoreOS or Mesosphere. Kubernetes is open to run different container technologies such as Docker (mostly used today) or CoreOS’ rkt (pronounced “rocket”). The two most well-known offerings of Kubernetes are: Google Container Engine (public Kubernetes service) and Red Hat’s open source PaaS OpenShift (based on Kubernetes, for hybrid cloud deployments). The latter adds some useful features on top of Kubernetes like an enhanced web user interface and an automated ‘source-to-deployment’ system that does not require knowledge about the underlying container and Docker subsystems.
Amazon AWS ECS: This is a public CaaS to manage Docker images (that can be stored in the accompanying ECS Registry), run Docker containers (ECS Runtime) and schedule / orchestrate / monitor these container instances (AWS CloudWatch). These can also be combined with other AWS services like Elastic Load Balancer (AWS ELB), Virtual Private Connection (AWS VPC) or Identity and Access Management (AWS IAM). AWS Simplified Workflow is also tightly integrated with AWS ECS to use Docker CLI commands (e.g. push, pull, list, tag).
(Docker) CaaS Cloud Offerings are available by other cloud vendors, too. Microsoft Azure Container Service (ACS) works together with Docker Swarm or Apache Mesos’ based DC/OS as container orchestration engine. Rancher Labs Rancher also supports Docker Swarm, Kubernetes and Apache Mesos. Note that most CaaS have in common that you still have to create server instances (e.g. AWS EC2) first. You pay for the server instances where your CaaS container instances run on, instead of just paying for the container instances itself. If you want to pay by container instance, you need to leverage a “serverless architecture” (discussed later). Docker Inc also offers Docker cloud services, including Docker Cloud to deploy and manage Dockerized applications and Docker Datacenter to integrate Docker into the enterprise software supply chain.
Apache Mesos’ based DC/OS is a “distributed operation system” running on private and public cloud infrastructure that abstracts the resources of a cluster of machines and provides common services, i.e. it handles as cluster resource negotiator. It runs below the orchestration layer (Swarm, Kubernetes, et al), and is a complementary tool, therefore. Apache Mesos is intended for large scale and multi-use of different clusters on top of it. You “just” need to implement Mesos’ framework interface. Several frameworks are supported already, e.g. Docker or rkt containers via Marathon, batch processing via Chronos, but also big data solutions like Apache Hadoop, Apache Spark, Apache Kafka. Mesosphere is the company behind Apache Mesos. They offer an open source distribution called DC/OS, which leverages Apache Mesos under the hood; this is comparable to Apache Hadoop and its distributions like Cloudera or Hortonworks.
Flockport is a startup focused on building an App store based on LXC containers that users can deploy in seconds on any server, any cloud and any provider. Flockport is focused on simplicity, making things just work, giving users a cloud-like flexibility of portables instances and workloads that can be moved across servers easily. There is a great blog post explaining “the key differences between LXC and Docker”.
DigitalOcean is a cloud infrastructure provider which allows developers to build and deploy microservices by creating so-called Droplets (i.e. units of work) on their global cloud data centers, plus leveraging block storage and networking features. A droplet can be an instance of an operating system image, but also a Docker container application. DigitalOcean takes care of provisioning, monitoring and other platform requirements for the Droplets. It can be combined with different orchestration tools such as Docker Swarm, Kubernetes, Apache Mesos or Dokku (a Docker-powered mini-Heroku PaaS). Therefore, DigitalOcean is more like a IaaS - comparable to AWS EC2 - but focusing on ease-of-use to deploy and run your microservices clusters.
Microsoft Azure Service Fabric is a microservices framework and container orchestration engine. It is not dependent on Microsoft Azure, but also usable on premise and in other clouds (therefore, the term “Azure” is a little bit misleading in the product name). Service Fabric leverages Docker for container management on both Linux and Windows containers. It allows to use different programming languages (e.g. C#, Java, Powershell) and is supposed to support more container technologies in addition to Docker and also other programming languages in the future.
Serverless Container Architectures: An emerging concept providing a service to deploy “real” functional-style cloud-native microservices. The main idea is to reach 100% utilization. Thus, you just pay by function call (see e.g. AWS Lambda, Google Cloud Functions, Microsoft Azure Functions), which differs to CaaS offerings where you still have to manage the underlying operating system instances yourself (i.e. run, scale, utilize and pay). However, serverless offerings usually support only function calls of a few programming languages like Java or Python. IBM OpenWhisk and funktion (sponsored by Red Hat / JBoss) are two serverless vendor-agnostic open source offerings which also support Docker containers to realize serverless container architectures. While OpenWhisk is becoming a “real product offering”, I see funktion more as small framework without much commitment these days. Hopefully, the big cloud vendors will also provide Docker containers in the foreseeable future.
Container Wars with Various Technologies
As you can see, there is so many different technologies, frameworks and cloud services available on the market for container packaging and orchestration. The above is not even a complete list, and there are still new ones emerging.
Therefore, the key lessons learned from this event (from developer’s perspective): Do not focus on developing code for the container under the hood. Care instead about the business logic. Implement your microservices in a vendor agnostic way.
Do not make the same fault as we all did with J2EE / Java EE where all vendors used the same standard specifications, but still offered many vendor-dependent features and “added value” in their specific “standard implementation”. Migration, i.e. deployment to another Java EE application server was a lot of efforts (re-development, testing, …); sometimes a complete re-write was easier and faster.
Docker has huge momentum today. However, there exist some doubts about what the future will bring. Several software vendors are not happy with the power of Docker Inc. as company behind Docker. For example, putting Docker Swarm Mode into the main Docker project made other orchestration vendors like Red Hat or Google unhappy, because they focus on Kubernetes as container orchestration tool. Therefore, these vendors created a new open source project “cri-o” to run containers in Kubernetes without Docker. Read the following InfoWorld article for more about the latest discussion of forking the current Docker project into an independent open source project: “New Red Hat project looks a lot like a Docker fork”.
Combining Differing Container Technologies and Tools
Note, that the discussed technologies and tools can also work together very well. Often, they are really complementary. There is not always a need for war!
For example, a Kubernetes cluster can manage pods with different container technologies such as Docker or rkt in parallel. Another example is Apache Mesos, which can manage different clusters – including plain Docker Swarm clusters, Kubernetes clusters, but also big data clusters using frameworks like Apache Hadoop or Apache Spark. Though, note that this is not a trivial configuration! Just as side note: Even Apache Hadoop will offer Docker support soon to deploy independent components such as Apache Kafka or Apache Spark in their own container instances (I saw the roadmap at Hortonwork’s roadshow last week where they presented their future Hadoop strategy).
The Future of Containers: To Standardise or Not?
Let’s see what the future brings regarding a potential Docker fork, and regarding all the discussions about standardization of container technologies. The following three options exist for next years:
- Docker gets the de facto standard
- Different technologies spread out in parallel; maybe including a Docker fork
- Container technologies become (at least partly) standardized) and different technologies respectively vendors adopt the standard
Let’s hope that #3 will happen… Several initiatives and discussions are going on these days, including appc (App Container specification), CNI (Container Network Interface), CNCF (Cloud Native Computing Foundation) or OCI (Open Container Initiative. For instance, the OCI tries to standardize container image definitions. Docker, CoreOS, Google, Red Hat, Facebook, Amazon and others work together here.
Conclusion: Develop Container-Agnostic Microservices
This article discussed plenty of different fantastic container technologies, orchestration platforms and cloud services. All of them have their pros and cons. In addition, the market is evolving quickly.
The key conclusion for now: Develop the business logic of your microservices in a vendor-agnostic approach to be future-safe and have fun leveraging all the great advantages and features of microservices and container technologies in opposite to monoliths and heavyweight virtual machines. The article “Can we avoid cloud-vendor lock in?” elaborates this in more detail.
In summary, no matter if you develop your business logic within a microservice with source code (using technologies such as Java, .Net or Go) or visual coding (such as middleware technologies), you should be able to develop it once and be able to deploy it in different containers, test environments or cloud providers without re-developing it or even having to change the technology you chose before.
About the Author
Kai Wähner is Technology Evangelist and Community Director for TIBCO Software - a leading provider of integration and analytics middleware. His main area of expertise lies within the fields of Big Data, Advanced Analytics, Machine Learning, Integration, SOA, Microservices, BPM, Cloud, Internet of Things and Programming Languages such as Java EE, Groovy or Golang. He regularly write about new technologies, articles and conference talks on his blog.
Looking at flockport.com it seems a bit outdated. Do you know of current developement in that area and of that company? LXC seems to be actively maintained, but the flockport website is not very active...
Re: Nice overview
Canonical are doing interesting things with LXD, which builds upon LXC: linuxcontainers.org/lxd/