Mitchell Hashimoto, founder of HashiCorp and creator of Vagrant, presented the first Config Management Camp keynote, describing the status of the datacenter in the past and today, the existing problems and outlining solutions, such as distributed systems, failure tolerance or usage of containers.
For Mitchell, the year 2006 was an inflection point, when the datacenter was characterized by having:
- Very few APIs, lack of automation to interact with infrastructure providers.
- No elasticity, hard to provision new machines.
- Monolithic machines, multiple services installed in the same machine.
- Very little IaaS and SaaS.
The problems tackled then were:
- Uniformity of servers, in order to avoid "snowflakes".
- Scalable change management, to simplify change across multiple machines.
- Auditing server state, gathering data from packages and configurations in servers.
- Early service discovery.
The problems were closely tied to the solution, using configuration management tools, such as Puppet or Chef, that were built on:
- Manual node (de)registration.
- Single master servers.
- Checking and correcting divergence.
- Following an agent model, usually in Ruby.
In early 2011, organizations began to push the boundaries of configuration management. Moving towards immutable and ephemeral infrastructure, features like service discovery and provisioning quickly became more present than infrastructure-as-code features such as rendering templates and installing packages.
Nowadays, the datacenter is very different:
- API driven, everything is done primarily through the API.
- Highly elastic, with the ability of provision big numbers of machines in little time.
- Small, single purpose servers.
- SaaS for core infrastructure: DNS, mail,... something that organizations saw as unimaginable a few years before.
- Using virtualization and containers.
Containers are more hype-y today than virtualization. You can not do anything in the datacenter today without virtualization, and containers are becoming the same way.
In Mitchell's opinion, today's datacenter problems are:
- Infrastructure creation and deletion.
- Service discovery, in today's fast moving infrastructure, service discovery needs to be immediate.
-
SaaS management, more and more services are being used from external providers.
Luke Kanies (Puppet founder) said once "There is no future with less servers". There is no future with less servers but there is a future where you are running less of them. Specially with PaaS. The bottom layer is the API.
-
Layer cake management, where multiple layers are used together, containers running on top of virtualization. e.g. OpenStack is another layer in the system.
- Faster, more realtime change.
And the solutions from his point of view are:
- Distributed systems.
- Failure expectation and tolerance, e.g. cloud instances may disappear. The type of events that tools like Chaos Monkey exercise.
- API driven and managed, e.g. containers are managed by an API, it is a feature given out to users.
- Low resource usage for scale. When using containers a higher server density needs to be handled, resources need to be optimized.
- Interaction with SaaS, managing the interaction between the internal systems and SaaS providers.
- Virtual machines and containers, knowing if the software will work in them both.
The design of solutions follows two trends, build out and build up, used by an equal number of people. Build out involves creating new tools, assuming the foundation laid before is no longer valid and more tools are needed. Build up leverages existing foundation, assuming that what already exists is correct enough. HashiCorp follows the build out approach:
What is the best technical design that is realistically possible? We are following the build out approach, building new tools to solve the problems. You can not take a project that is inherently not parallel and make it work.