At QCon New York 2015, Paul Payne discussed a project at Nordstrom that required modifying and re-deploying a live application service within twenty minutes, which was made possible due to the use of Go-based microservices, Docker container technology, and a continuous delivery methodology.
Paul Payne, software engineer at Ivy Softworks, began the talk by discussing that during his time working at Nordstrom an important client requested that a change to a production service be made within twenty minutes in order to support a demo that had been arranged. The service accepted GTIN stock identification numbers via a RESTful interface and returned details of the stock item. The GTIN service had been designed to accept GTIN-12 numbers, but the client had created their system to supply only GTIN-13 numbers, which when provided to the Nordstrom service, resulted in no matched stock items.
Only a small code modification was required within the GTIN service code, and due to the fact that the service had been built as a Go-based microservice deployed into Docker using a zero-downtime continuous delivery approach, the team was able to push this change to production within the required time frame.
Payne stated that developing applications as microservices had enabled his team to ‘code with confidence’. The use of the single responsibility principle, creation of modular and composable designs, and the enforcement of encapsulated state within each service allowed developers to easily understand and rapidly modify code. These properties lowered coupling between services and allowed independently deployable functionality. Additional recommendations for building this style of microservices included the robustness principle (be liberal in what you accept, and conservative in what you send), enable fast build iterations through tooling, and utilise Test-Driven Development (TDD).
The packaging and deployment benefits provided by application container technology also contributed heavily to the ability for Payne and his team to rapidly deploy a modified service into production. Application containers, such as Docker, may allow better hardware utilisation when compared with virtual machines (VMs), but Payne suggested that the primary advantage is that they provide the ‘right abstraction for software’. Payne argued that container technology promotes a ‘clean organisational interface’, and allow developers and operators to focus on their respective areas of specialism.
Payne discussed several patterns for zero-downtime continuous deployment, and began this section of the talk by referencing Google’s ‘The Datacenter as a Computer’ paper. Google began investing in what would become the precursor to container technology in 2003, with the creation of their ‘Borg’ container cluster manager. Payne joked that Google “didn’t get the memo about virtual machines”, and that their focus on developing container technology has allowed them to innovate faster than many of their competitors in respects to the utilisation of containers to package and deploy applications.
Payne suggested that the current breed of cluster managers and schedulers provide the foundations for microservice platforms. Technologies such as Apache Mesos, Mesosphere Marathon, CoreOS’s container platform, Kubernetes, and the Docker technology suite all promote rapid deployment of applications if used correctly.
Referencing Jez Humble and Dave Farley’s classic ‘Continuous Delivery’ book, Payne discussed deployment approaches such as blue-green deploys, canary deploys and rolling deploys. A live demo of a zero-downtime application deployment was provided utilising a locally run CoreOS cluster managed via Hashicorp’s Vagrant and controlled via fleetctl. A public/private proxy gateway was implemented using Vulcand, a programmatic load-balancer powered by etcd, and Go-based application services were packaged within Docker images.
Payne concluded the talk by warning of potential organisational culture issues that must be managed in respect to the speed and flexibility provided by continuous delivery (“wait, you’re not deploying to production now are you?”), and suggested that 'infrastructure as code' tooling such as Terraform provide a strong foundation to enable the rapid provisioning and configuration of microservice platforms. Payne also provided examples of where domain-driven design (DDD) inspired anti-corruption layers are essential to maintain high cohesion and low coupling within the application system.
Payne’s final suggestion to the audience was ‘build to learn’. Tooling such as Hashicorp’s Vagrant allow small VM clusters to be created on a local machine, and experimentation with cluster management software and microservice tooling can be conducted with low expense and low risk.