3 Surprising Things Consul Service Mesh Can Do

Most infrastructure engineers have a good idea what HashiCorp Terraform does, and those who care about security likely know about HashiCorp Vault, but what about HashiCorp Consul? Some engineers see it as a service discovery solution. Others recognize it as a service mesh. And some might only know Consul from its earliest use case — as a key-value (KV) store.

Yes, Consul does all those things. But since bursting on the scene as a popular open source networking tool back in 2014, it has grown into a much more comprehensive networking platform. So let’s take a look at three things about Consul that you may not be taking full advantage of.

»Consul Bolsters Zero-Trust Networking

Security professionals want to apply zero-trust principles across their whole infrastructure, especially throughout their network. Using traditional firewall-based security to protect the network perimeter or control access between internal networks, VPCs, and VNETs is an increasingly insufficient approach.

Traditional security approaches are based on IP addresses as the unit of access control between machines and services across different networks. But authorizing (allowing or denying) communication between individual services based on IP addresses requires a significant amount of effort at scale, especially in a modern world where machines and IP addresses are ephemeral.

Consul can relieve these pain points by offloading traffic between network services to be handled by Consul’s service mesh. The key is the service identity that is applied to every service registered to Consul. In this situation, a service identity replaces the IP address as a unit of control with which all future policy enforcements are made.

Consul service mesh service-to-service communication flow.

Administrators can allow or deny communication between services based on the service name rather than IP addresses. This makes it much easier to manage communication between services, particularly across microservice environments where the service names are constant while the IP addresses are dynamic. For example, administrators can impose a secure-by-default “deny all” policy, which is the first step toward zero-trust networking. After starting with secure defaults, you can build up service-to-service authorization policies in line with the requirements of your applications and threat models.

Once authorized, communication between services is authenticated and encrypted with mTLS. TLS certificates are automatically generated by Consul and tied directly to each service’s identity. The full authentication process and certificate exchange is handled by Envoy, Consul’s sidecar proxy, and will also ensure traffic is encrypted over the network. Put it all together, and Consul relieves developers from having to add separate authentication and encryption code to their services, which ultimately increases productivity. If it's built into the developer workflow and platform well, the developers will barely even notice it’s there.

»Consul Thrives on Kubernetes

Another misconception is that Consul is mainly targeted for virtual machine environments. Consul doesn’t just work well with non-containerized workloads, it works well with everything — including Kubernetes, Amazon ECS, and HashiCorp Nomad, and also across multiple clouds. It’s the perfect service mesh for heterogeneous environments.

Consul has powerful features for Kubernetes. Features like Admin Partitions and Transparent Proxy were driven by Kubernetes requirements. More recently, several new features were developed specifically to optimize the user experience for Consul on Kubernetes. For instance, Consul 1.11 introduced the Consul K8s CLI tool, intended to simplify the installation and lifecycle management experience of Consul on Kubernetes without the requirement of Helm charts or kubectl. Helm is still fully supported, but now users have the option to use a different approach.

Consul 1.11 also provided a tighter integration between Consul on Kubernetes and Vault. This made it easier to use Vault to automatically generate, store, and manage TLS certificates on both Consul’s control plane and data plane. This was part of a broader initiative for closer integration between Consul on Kubernetes and Vault to store all secrets.

This initiative continued with the 1.12 release, which allows Consul to automatically rotate TLS certificates on both the control plane and data plane without any downtime. Administrators no longer have to manually rotate certificates and risk outages due to human error. It also improves security, since certificates can be rotated more often without burdening the administrator.

Lastly, the 1.12 release gave users the option to store secrets like ACL tokens and license keys on Vault rather than as Kubernetes secrets. This also improves security since Vault encrypts all secrets by default and provides many additional auditing and reporting features for centralized secrets management and governance.

»Consul Is Easier Than You Think

Have you ever been to a casino or watched a movie where people are cheering at a craps table? If you don’t know the rules, the game can seem confusing and intimidating because there’s so much going on. However, craps is actually composed of many simple wagers, and players can choose to bet only on the parts that interest them.

Similarly, if you’re new to Consul, it can be easy to get overwhelmed by its many options and capabilities. But you don’t need to use every Consul capability. Every customer has a different use case, from service discovery to securing network services to automating network devices, and they don’t need to take advantage of everything right away.

The four pillars of networking on which HashiCorp Consul focuses.

So where do you start? Well, whatever your use case, the first thing to do after deploying Consul is to register your services. Once your services are registered, they have an identity that can be used for service discovery or service mesh capabilities. Consul has matured over years to become simple to deploy, manage and use. And most of Consul’s capabilities are transparent to the application developer. In fact, it is almost effortless with Consul on Kubernetes to automatically register your services. Just set the connectInject.enabled and connectInject.default parameters to true in your Consul Helm values file. This configures Consul to automatically inject an Envoy sidecar proxy onto every service in your Kubernetes cluster. In addition, Transparent Proxy is enabled by default and will ensure traffic between services are automatically redirected through the Envoy proxy. This frees developers from having to manually edit their application’s Kubernetes manifest to account for any upstream services.

Want something even easier? If you don’t want to bother installing and managing Consul servers, HCP Consul is a fully managed service that takes care of all that for you. The HCP Consul servers are installed in a dedicated, single-tenant cloud infrastructure environment called HashiCorp Virtual Network (HVN). Just install the Consul clients in your environment and join them to the Consul server cluster. We make it easy to connect the HCP Consul servers from our HVN to clients in your own organization’s VPC. All connections within Consul’s service mesh are secured and encrypted by default without any additional management overhead. All the best practices to update server software and ensure server resiliency are managed for you by HashiCorp expert engineers. Want to get started? Check out this HCP Consul tutorial.

Put it all together, and it's clear that HashiCorp Consul does a lot more — and does it more easily – than many people are aware of. Consul helps you implement zero-trust security where all the application traffic is authorized, authenticated, and encrypted. It provides service identity-based security across all platforms, runtimes, and clouds. Consul on Kubernetes has native integrations with Vault and provides a specialized CLI for simplifying Kubernetes workflows. And overall, the project’s goal continues to be giving users the ability to run a number of modern networking capabilities for any application across any infrastructure.

»Resources and Use Cases

• Getting Started - Learn Consul fundamentals
• Getting Started with Consul Service Mesh for Kubernetes
• How Stripe Uses Consul to Build Service Mesh
• Workday's Multi-Cloud Network Fabric With Consul & Vault

A version of this blog post was originally published on The New Stack.