Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage News HashiCorp Consul Improves Envoy Integration, Adds Debugging Tool

HashiCorp Consul Improves Envoy Integration, Adds Debugging Tool

This item in japanese

HashiCorp has released Consul 1.15, adding new features that improve interacting with Envoy and troubleshooting issues within the service mesh platform. The release introduces improvements to Envoy access logging as well as adding Consul Envoy extensions. To improve the troubleshooting experience, a new service-to-service troubleshooting tool has been added.

Consul provides support for both a built-in L4 proxy, Connect, as well as first-class support for Envoy. Proxies permit applications to connect to other services within the service mesh. In previous releases, escape-hatch overrides had to be used to modify the existing Envoy configuration to rewrite Envoy resources to be compatible with Consul. This was difficult to use as it required understanding how Consul named Envoy resources.

This release introduces a new extension system to Consul that allows operators to modify Consul-generated Envoy resources without having to customize the Consul binary. Extensions can add, delete, or modify Envoy listeners, routes, clusters, and endpoints. At the time of release, the Lua and AWS Lambda extensions are supported.

Envoy extensions can be configured to be used using the EnvoyExtensions field. It is definable in both the proxy-defaults and service-defaults configuration entries. However, it is recommended to enable EnvoyExtensions with service-defaults.

Within Consul, Envoy access logging provides details to help understand the incoming traffic patterns to the proxy. In previous versions, adjusting the bootstrapping configuration for Envoy to enable access logs required using escape hatches.

In Consul 1.15, logs are now centrally managed via config entries and CRDs. This simplifies enabling and disabling access logs for all proxies within the service mesh. Logs can be configured to output different request properties and to output to a stdout pipe or a file. For example, enabling the logs within the proxy-defaults configuration entry can be done as follows:

Kind      = "proxy-defaults"
Name      = "global"
AccessLogs {
  Enabled = true
  Type = "stdout"

This release introduces a new built-in tool for service-to-service troubleshooting. The tool will validate the Envoy configurations on both the upstream and downstream proxies for both VM and Kubernetes setups. When run, the tool performs a number of checks to help detect issues including:

  • Validating the existence of the upstream service
  • Checking if one or both hosts are unhealthy
  • Checking if a filter is affecting the upstream service
  • Validating if the certificate authority or any services have expired mTLS certificates

For example, to troubleshoot between services on Kubernetes the upstream IP address is used:

consul-k8s troubleshoot proxy -pod frontend-767ccfc8f9-6f6gx -upstream-ip

Version 1.14 of Consul introduced a new Consul Dataplane that removed the need for deploying the Consul client agent when using Kubernetes. However, this had the side effect of removing the rate-limiting support provided by Consul clients. With version 1.15 the Consul server has added support for rate limiting.

This includes a set of global limits for read and write operations for each Consul server as well as a mode to apply when that limit is reached. This mode can be one of enforcing, permissive, or disabled. When set to enforcing, the rate limiter will deny any request to the server that exceeds the configured rate. Permissive mode will continue to allow requests but will produce metrics and logs to review the traffic patterns. This mode is intended to be used during configuration and troubleshooting. Setting the mode to disabled will disable the rate limiter.

Consul 1.15 is now generally available. More information about the release can be found on the HashiCorp blog or within the Consul documentation.

About the Author

Rate this Article