We are excited to release HashiCorp Consul 0.8. Consul is a critical infrastructure service that organizations rely on for service discovery, key/value storage, and health checks.
The focus for Consul 0.8 is operational stability with Autopilot features and federation improvements. Consul is highly available with three or more servers in a cluster, but it is complex to operate amidst failures, newly introduced servers, and upgrades. The goal of Autopilot is to automate Consul cluster operations so that it can be run in an auto-scaling group or scheduler without worry. Consul Enterprise has enhanced Autopilot functionality to automate the complete server cluster upgrade process and safely increase cluster size for read scalability. Federation improvements aim to simplify joining and operating Consul clusters across global infrastructure. With this release, Consul open source gains join flooding and soft-fail features to ensure that clusters are properly connected across datacenters. Consul Enterprise introduces network areas to enable advanced networking topologies such as hub-and-spoke.
Read on to learn more about the features in this release. You can also read the 0.8 changelog for details on complete ACL support, a more uniform CLI interface, and much more.
Introducing Consul Autopilot
Autopilot simplifies Consul operations by automating complex workflows, such as:
Dead Server Cleanup: Remove a server that died unexpectedly and was replaced with a new server
Server Health Checking: Determine the health of the Consul cluster
Stable Server Introduction: Safely add new servers and remove old servers
Consul Enterprise has features for companies seeking higher reliability and additional operational simplicity:
Server Read Scaling: Non-voting servers add more read scaling for stale queries
Redundancy Zones: Promote hot standby servers in situations where it's hard to have all active servers (eg. running >3 servers with only 3 availability zones)
- Upgrade Migrations: Orchestrate in-place Consul upgrades
$ consul operator autopilot set-config -cleanup-dead-servers=false Configuration updated!
For more information on Autopilot, please see the Autopilot Guide.
Join Flooding and Soft-fail
Consul has always had support for multiple datacenters, and Consul 0.8 builds upon this with new federation features to allow operations to more easily manage complex federations of Consul clusters across multiple datacenters.
Join flooding uses information about the Consul servers within each datacenter to ensure that all servers are properly joined onto the WAN, or to any network areas which are discussed below. Prior to join flooding, it was easy to forget to join a new server onto the WAN, which would mean that not all servers would be able to properly route requests to remote Consul datacenters. Now as long as at least one server is joined, all others will join the WAN automatically too.
Soft-fail takes the same approach to server connection management added in Consul 0.7.0 for clients connecting to Consul servers within a datacenter and applies it to server-to-server communication to remote Consul datacenters. If there are issues between some subset of remote datacenters, Consul will still route requests as long as requests are still flowing. Prior to soft fail, any pair of datacenters having connectivity problems could cause all Consul servers to stop sending requests to those datacenters. Soft-fail makes large federations of Consul datacenters much more robust to connectivity issues.
(dc2) $ consul operator area create -peer-datacenter=dc1 Created area "2aea3145-f1e3-cb1d-a775-67d15ddd89bf" with peer datacenter "dc1"!
(dc1) $ consul operator area join -peer-datacenter=dc2 127.0.0.2 Address Joined Error 127.0.0.2 true (none)
(dc1) $ consul operator area members Area Node Address Status Build Protocol DC RTT cbd364ae-3710-1770-911b-7214e98016c0 node-1.dc1 127.0.0.1:8300 alive 0.8.0 2 dc1 0s cbd364ae-3710-1770-911b-7214e98016c0 node-2.dc2 127.0.0.2:8300 alive 0.8.0 2 dc2 581.649µs
(dc1) $ consul kv put -datacenter=dc2 hello world Success! Data written to: hello
Unlike the WAN, traffic between network areas is all performed via server RPC (8300/tcp) so it can be secured with just TLS. This is much easier to manage across organizations, and eliminates the need to manage gossip keyrings in addition to TLS. For more information on network areas, please see the Advanced Federation Guide.
There are several important considerations to read about before upgrading to Consul 0.8, and some steps are required post-upgrade to fully enable the new features described in this post. Please see the Upgrade Guide for more details.
As Consul adoption continues to increase, we're committed to making Consul easier to operate and implement in complex environments. The Autopilot and federation features in open source make it safer to operate Consul clusters in one or many datacenters. Download Consul 0.8