HashiCorp Consul support for AWS Lambda is now generally available, enabling services in the mesh to invoke Lambda serverless functions.
We’re pleased to announce the general availability of HashiCorp Consul support for mesh services invoking AWS Lambda functions. In addition, we’ve released a public beta for AWS Lambda functions invoking mesh services. This release means service mesh users can now have consistent workflows for encrypted communications flowing between mesh services and Lambda functions.
Lambda support is enabled in the newly released Consul 1.14. In this blog post, we’ll examine the intersection of serverless functions and service mesh from a broad perspective and then show you how this integration works in practice with some code examples.
Commercial functions as a service runtimes like AWS Lambda offer serverless functions that enable users to run code in response to specific events. In this event-driven model, applications are broken up into modular functions that can be easily invoked and scaled up and down. Serverless functions eliminate the need for developers to manage the backend infrastructure and configure the server software needed to run application code.
Because all of this management is handled by the cloud provider, developers can focus on writing the business logic for their applications. According to Datadog's 2022 The State of Serverless report, more than 50% of companies operating in the cloud are using serverless technologies, with Lambda being the frontrunner in popularity.
The advantages of serverless functions include:
Despite these advantages, there are challenges that come with onboarding serverless workloads into production environments. This is especially true when dealing with distributed systems deployed across a variety of platforms, providers, and runtimes:
HashiCorp Consul is an enterprise-ready networking solution that provides a service mesh built to address these challenges. Consul service mesh:
Consul now has first-class runtime support for AWS Lambda. AWS Lambda functions can be fully integrated with your service mesh and benefit from the same reliability, observability, and security features that Consul provides. For example, you can use Consul intentions and traffic management on Lambda functions the same way you would use these features with other services.
Registering Lambda functions into Consul is automated using the Consul Lambda service registrator. This integration provides a simple way to register AWS Lambda functions into the service mesh.
First, you use the Consul Lambda registrator Terraform module to deploy Lambda registrator. Lambda registrator automatically registers all Lambda functions that you have tagged with serverless.consul.hashicorp.com/v1alpha1/lambda/enabled: true
into the service mesh. This happens in four steps:
Applications that call AWS Lambda functions can do so through a terminating gateway or directly through their Envoy sidecar proxy:
Once registered, you need to set the Lambda service as an upstream for any services that will use it. Then you can call it like any other mesh service:
Lambda functions that call into the mesh use the new Consul Lambda extension to make mTLS requests to services through a mesh gateway. The extension runs within the execution environment of your Lambda function and performs three main steps:
CONSUL_EXTENSION_DATA_PREFIX
. The extension periodically updates the mTLS data to ensure that it stays in sync with Consul.localhost
for each upstream port defined in CONSUL_SERVICE_UPSTREAMS
.CONSUL_MESH_GATEWAY_URI
:To use the Consul Lambda extension, you add it to your Lambda function as a layer. Then you set environment variables on your function to configure it. Your function doesn’t need to know anything about the extension, so there should be no impact to your existing code.
You can download the Consul Lambda extension from the HashiCorp releases website at the Consul Lambda releases page.
The following Terraform configuration creates a Lambda layer from the consul-lambda-extension
zip file and a Lambda function that uses that layer:
To call the example-service
upstream, the function makes requests to localhost:1234
.
Our goal with HashiCorp Consul is to provide a consistent, enterprise-ready control plane to discover and securely connect any application. Users can now further broaden their service mesh implementations by utilizing these new Consul improvements for AWS Lambda functions.
To learn more about how to use Consul and Lambda together, please visit the Consul and AWS Lambda documentation.
Use Minikube to create multiple Kubernetes clusters with Consul and test cluster peering configurations in your local development environment.
Consul 1.16 adds new reliability, scalability, security, service mesh UX, extensibility, and service discovery capabilities.
The HCP Consul management plane now offers deeper insights to your Consul deployments via cloud-based observability and seamlessly links new and existing self-managed Consul clusters.