Announcing Consul 1.17 beta and HCP Consul Central

Today at HashiConf, we are introducing a number of significant enhancements for HashiCorp Consul, our service networking solution that helps users discover and securely connect any application. We're also formally introducing HCP Consul Central, previously known as the management plane for HCP Consul. These new capabilities help organizations enhance workflow management, increase reliability and scale, and bolster security for operators as they leverage a cloud operating model for service networking.

Some of the notable updates include:

• Multi-port support (beta): a new, simplified way to onboard modern distributed applications that require different ports for various traffic types for intricate client-server communication
• Locality-aware service mesh routing within a Consul datacenter: optimizes traffic routing within datacenters, prioritizing local instances for lower latency and reduced costs.
• Sameness groups (GA): simplifies multi-cluster operations, enhancing service reliability for enterprises.
• HCP Consul Central: introduces observability features for HashiCorp-managed and linked self-managed clusters, enhancing cluster health monitoring. Additionally, a global API simplifies integration with HCP Consul Central, allowing platform operators to streamline workflows and access cluster details.

Here is a closer look at all the new enhancements in Consul 1.17:

»Simplified workflow management

The new APIs in Consul 1.17 simplify workflow management by enabling multi-port services and flexible routing. The specific enhancements include Consul catalog v2 API and multi-port services, both in beta.

»Consul catalog v2 API (beta)

Consul 1.17 includes a new set of APIs for interacting with the catalog, authenticating traffic using workload identities, and for managing Consul service mesh. These collective sets of APIs are now exposed under the new v2 API paths and are enabled using a v2 API feature flag. These APIs are the foundation for future versions of Consul and introduce a new catalog model that decouples services, service instances, and service identity by introducing new Consul resources called service, workload, and workload identity. (Note: These APIs are still in beta and under active development, so we don’t recommend using them in production.)

For more information, refer to the Catalog API v2 section in the documentation.

»Multi-port services in Consul (beta)

Starting with Consul 1.17, Consul users can register services with multiple ports per service using the v2 catalog API. Consul service mesh now allows services on the v2 API to call one another when utilizing transparent proxy over multiple ports and allows operators to authorize traffic between services over specific ports.

A key improvement to support multi-port services is that workloads (i.e. pods) can now use only a single sidecar proxy for all ports exposed by the workload. The reduction of resources and the ability to call services using transparent proxy ensures ease of use and reduces the operational overhead for managing Consul service mesh.

In addition, the v2 API allows routing to workloads selected by multiple services, similar to how Kubernetes allows routing to the same pod using multiple services. This unlocks additional workflows, such as hostname-based canary routing and routing via headless services in Kubernetes native deployments, which were difficult to do in the v1 API. We’re excited to see how users will use the increased flexibility to deploy an even broader range of applications onto Consul service mesh.

For more information, refer to the documentation for Multi-port services in Consul service mesh. Support for other runtimes outside of Kubernetes is planned for future releases of Consul.

»Policy and security

Consul 1.17 enhances policy and security by introducing locality-aware routing for low-latency connections and JWT-based authentication and authorization support for API Gateway.

»Locality-aware service mesh routing within a Consul datacenter

Consul is designed to be deployed in private networking environments that have low latency and high bandwidth. Consul refers to these environments as datacenters. In the public cloud, a datacenter is equivalent to a region.

In Consul service mesh, requests to upstream services are routed by the downstream proxy to any healthy and available upstream instance within the datacenter.

This type of random connection distribution is sufficient for most environments as it allows the load on an upstream service to be spread across all available instances. However, in some situations it may be desirable to ensure that traffic stays within the same availability zone in order to decrease latency or reduce inter-zone data transfer costs.

To address this issue, Consul 1.17 introduces support for locality-aware service mesh routing. Locality-aware routing lets operators prioritize routing to upstream instances located in the same zone over instances in alternate zones.

By keeping traffic within a zone, operators can reduce service-to-service latency, which helps improve overall service performance and decrease infrastructure costs. If all of the instances of an upstream service within a zone are unavailable, the service mesh will automatically fail over to healthy instances in adjacent zones, ensuring service connectivity and availability within the datacenter.

For more information, refer to the documentation for locality-aware routing.

»API Gateway supports JWT-based authentication and authorization

In Consul 1.17, the API Gateway can now be configured with policies that control access to services based on JSON Web Tokens (JWT) embedded in the network traffic sent by external clients.

These policies can control access to services and even specific URLs based on the claims contained in JWTs. This capability lets administrators control access to services from outside the service mesh without having to modify services that don’t support JWT-based authentication/authorization.

»Reliability and scale

Consul 1.17 bolsters reliability with sameness groups and rate limiting for services. Consul is multi-cloud and platform-agnostic, and simplified Amazon ECS deployments help operators scale.

»Sameness groups (Consul Enterprise, GA)

In June, Consul Enterprise 1.16 introduced sameness groups as a beta feature. Sameness groups allow operators to logically group Consul peers and partitions that share common administration, and create policies to export services, configure service failover, or authorize service traffic in a uniform fashion for members of the group. In Consul 1.17, sameness groups have graduated to general availability.

Enterprises can use sameness groups to simplify operations and increase service availability for multi-cluster or multi-region deployments.

Refer to the documentation for creating sameness groups or creating sameness groups on Kubernetes for more information.

»Traffic rate limiting for services

Consul service mesh now supports HTTP request rate limiting of network traffic to services. The rate limiting is configured per service and is applied per service instance. Operators can set HTTP request rate limits for the service instance or separate rate limits for specific URL paths. The rate limiting configuration includes settings for requests per second (RPS) as well as maximum request burst size.

These rate limiting capabilities help operators protect service instances from being overloaded, prefer traffic meeting certain criteria, or ensure service capacity is shared fairly.

For more information, refer to the product documentation for rate limiting.

»Simplified service mesh deployments on Amazon ECS with Consul dataplane

As part of Consul 1.17 we are also announcing Consul on ECS 0.7, which introduces a simplified service mesh deployment architecture that eliminates the need to deploy Consul clients per task on Amazon ECS. The new architecture deploys a new Consul dataplane component that is injected as a sidecar in the ECS task. This dataplane container image packages both an Envoy container and Consul dataplane binary.

For more information, refer to the product documentation for Consul ECS.

»HCP Consul Central

In addition to Consul 1.17, HashiConf marks the official introduction of HCP Consul Central, previously known as the HCP Consul management plane. HCP Consul Central is a cloud-based service that centralizes global visibility and management of Consul deployments, whether they’re self-managed or managed by HashiCorp. Global catalog and cluster details are available to all Consul clusters, including Community edition clusters.

HCP Consul Central’s observability and workflow features are available to Consul Enterprise and HCP Consul customers at no additional cost, and a 90-day free trial of these features is available to Consul Community users. Start your trial today by linking your Community edition clusters to HCP Consul Central.

»Observability enhancements (GA)

HCP Consul Central’s observability features deliver an out-of-the-box observability experience for HashiCorp-managed Consul clusters as well as self-managed clusters linked to HCP Consul Central. The server health dashboard offers a list of key indicators to help operators quickly understand the health and availability of their Consul clusters over time, which supports faster troubleshooting and proactively addressing potential issues.

We’re also introducing service-level observability metrics such as request rate, error rate, request duration, connection count, HTTP response status, and data transfer rate. These metrics can help service owners better monitor service-level health and identify performance issues within the service mesh. The charts below display metrics for request rate and duration and HTTP response breakdown:

For more information, check out the Consul observability documentation.

»HCP Consul Central API (beta)

The new Consul Central API simplifies integration with HCP Consul Central and all Consul clusters under management. This global API offers a unified interface to enable platform operators and ecosystem vendors to incorporate Consul workflows, ecosystem integrations, and network automation across multi-cloud and hybrid deployments.

With this release, users can programmatically access the metadata and health of their Consul clusters, nodes, and services. By integrating with this API, users can build dashboards and monitors to make informed decisions about scaling and managing Consul clusters across different environments.

Get started with the HCP Consul Central API by exploring the API documentation.

»Read-only mode for linking self-managed clusters (GA)

HCP Consul Central also provides additional flexibility for operators who want to link their self-managed Consul clusters to HCP Consul Central. Self-managed Consul clusters can now be linked in “read-only” mode in addition to the previously available “read-write” mode. As the name implies, the new mode grants HCP Consul Central read-only access to linked clusters to populate its global service catalog and cluster level details.

Clusters linked in read-only mode can utilize observability features without any limitations. Operators also have the flexibility to update to read-write mode at any time to enable HCP Consul Central management capabilities.

For more information, check out the benefits of HCP Consul Central in the documentation.

»Next steps for HashiCorp Consul

Our goal is for Consul to enable a consistent, enterprise-ready control plane to discover and securely connect any application. Consul 1.17 includes enhanced workflow management, reliability, and security for service networking. With improved failover, security measures, user experience, and extensibility, Consul empowers organizations to build resilient and efficient applications in distributed environments.

We are excited for users to try these new Consul updates and further expand their service mesh implementations. Here’s how to get started:

• Learn more in the Consul documentation and the 1.17 release notes.
• Get started with Consul 1.17 by installing the latest Helm chart, provided in the Consul Kubernetes documentation.
• Try Consul Enterprise by starting a free trial.
• Get started with HCP Consul

Finally, if you are attending HashiConf in person, we encourage you to visit the HashiCorp Zone to meet with Consul product designers, engineers, and product managers to share your feedback and tell us about your networking use cases. If you’re joining us virtually or unable to make it to HashiConf this year, please take this short survey on your service mesh experience to help us prioritize features your team cares about. To see what the Consul team has been up to, check out the HashiConf schedule and filter by Product:Consul. We look forward to seeing you there!