Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage News Service Discovery with Consul

Service Discovery with Consul

This item in japanese

Lire ce contenu en français

HashiCorp Consul is a solution for service discovery and configuration, designed to run distributed, highly available and scalable to thousands of nodes.

Consul is used for service discovery, key/value configurations and health checking. It works on a similar way than Zookeeper, doozerd or etcd, but adds a DNS interface for service discovery, useful for existing or legacy services, and claims better scaling by using a gossip protocol. Consul provides monitoring services in a fully distributed architecture unlike other tools. Nagios uses central servers to query services and Sensu a central broker to manage service notifications, which can cause scaling issues.

There are also differences with more modern service discovery tools. SkyDNS provides a DNS interface to perform queries too, but the monitoring checks use heartbeating and TTLs, while Consul allows defining custom checks, as well as support for multiple datacenters out of the box. SmartStack, a project started by Airbnb, is a service discovery and registration framework composed by ZooKeeper, HAProxy and two Airbnb open source projects, Synapse and Nerve, used for service discovery and service registration. Consul is the only tool using a gossip protocol, claiming better scalability and fully dynamic tracking of the nodes in the cluster. A comparison with other software explains in more detail the differences with the tools mentioned and other ones typically used in the same scenarios.

Consul clusters are composed by clients and servers, the main difference being that only the latter store data, and use auto discovery. Reads and writes can be done to any member of the cluster. Services are registered using configuration files or via an HTTP API against any agent (client or server) in the cluster. Registered services and nodes can be queried using both a an HTTP interface and a DNS interface that makes it easy to use it without modifying existing services.

The gossip protocol used to manage membership and broadcast messages to the cluster is provided by the Serf library, another Hashicorp project, based on the "SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol" paper published by the Department of Computer Science at Cornell University.

Consul can be provisioned with Puppet, as described by Ben Schwartz, who included examples using Vagrant boxes, showing how to enable a Spring Boot demo application using a Puppet module to configure Consul and the Puppet Dnsmasq module that routes Consul DNS lookups to the Consul agent for anything ending in .consul.

Rate this Article