Luke Marsden stated at QCon London 2015, that although Docker containers provide a very useful deployment mechanism for development and test, the absence of host mobility for containers with state may provide problems in production deployments. The open source Flocker volume management tool provides a mechanism to overcome this by allowing stateful containers to be moved between hosts.
Marsden, CTO at ClusterHQ, began by proposing that Docker has won the ‘hearts and minds’ of developers, and has fundamentally changed the approach to packaging applications when developing and testing. However, one of the core tasks missing in the current Docker toolset is the ability to move combined application and state between hosts. This restricts the mobility of services and associated data around a cluster, and can be a problem when, for example, the underlying host hardware must undergo maintenance.
ClusterHQ have created an open source tool named Flocker, which is a volume and container management system for Docker based on the ZFS file system. ZFS is a combined file system and logical volume manager with features that include protection against data corruption, snapshots and copy-on-write clones, continuous integrity checking, and automatic repair. Flocker utilises ZFS based volumes at the back-end in combination with a network proxy at the front-end to enable stateful containers, such as databases, to be moved between virtual or physical hosts.
The Flocker front-end proxy can route requests for services or data stores to whichever host is running the active container. When a container is migrated between two hosts a snapshot of the data volume on the initial host is taken and replicated to the new location. Once this is complete, the original container is shut down, and a final (small) delta snapshot is taken and also replicated in order to fully synchronise the state between the two hosts. Marsden stated that this final replication typically occurs ‘within seconds’, and once complete the new container is started.
Diagram showing example host migration of container state for a MongoDB data store
Marsden suggested that Docker has 'grown up' around developers, and accordingly support for certain operational concerns of software development have regressed in comparison with other mechanisms for packaging and deploying applications. The operations team within a software development organisation typically cares about robustness, eliminating single points of failure, scalability, and reducing complexity.
Marsden proposed that servers should be treated as ‘cattle, not pets’ - if one member of a herd of cattle becomes sick, it can simply be gotten rid of without concern. This approach is problematic with servers that are hosting Docker containers with non-transient state, as data is typically persisted to volumes mounted into the container from the host. Accordingly, a stateful container such as that used by a database would have affinity to its host. Marsden stated that this is a problem.
You’ve turned one of your herd of cattle into a pet. You’ve named one of your machines "that’s my postgres database, and I really don’t want it to die". And when it does, then you get sad
This functionality offered by Flocker is an approximation to container live migration, and can potentially overcome the current limitation of mobility of container-based state that was suggested by Marsden.
Finally Marsden discussed planned upcoming work on Flocker to include a new API architecture that replaces the current YAML configuration files with distributed network nodes that contain the desired state of cluster, multi-tenancy, and pluggable storage backends.
Additional details about Luke Marsden’s “Docker, Data and Extensions” talk can be found on the QCon London schedule webpage.