The latest version of HashiCorp Consul on Amazon ECS adds support for AWS IAM authentication and mesh gateways.
We are pleased to announce that HashiCorp Consul on Amazon Elastic Container Service (ECS) 0.5 is now generally available. This release adds support for authenticating services and clients using AWS Identity and Access Management (IAM) identities. The new release also adds support for mesh gateways, which enable services to communicate across multiple runtimes and clouds and reduces risk for organizations by enforcing consistent end-to-end security for service communication.
With Consul on ECS, each ECS task includes sidecar containers that run alongside your application, including a Consul client container and an Envoy proxy container. These sidecars require two Consul ACL (access control list) tokens that authorize the task to join the Consul cluster and register as a Consul service:
In previous versions of Consul on ECS, the client and service tokens were created by a service called an ACL controller and stored in AWS Secrets Manager. This model was slower due to its reliance on polling to discover new services to target for token creation, and it required the use of AWS Secrets Manager as part of the process.
A new Consul AWS IAM auth method was added in Consul 1.12, which introduced the option of using AWS IAM identities for authenticating to Consul to receive Consul ACL tokens. Consul on ECS 0.5 adds support for this authentication method for ECS workloads. This new authentication option eliminates the need to periodically poll services for token creation, as authentication takes place when services start up. And instead of relying on AWS Secrets Manager for token storage, ACL tokens are now stored on a shared Consul volume.
The ability for service mesh traffic to communicate across Consul datacenters and Consul Admin Partitions (as well as the ability for Consul datacenters to be federated across wide area networks, or WANs) was introduced as part of Consul 1.6's mesh gateway feature, and we’ve now added this functionality to Consul on ECS 0.5. You can enable mesh gateways for Consul on ECS by adding the gateway-task
module to your Terraform configuration, which creates an Amazon ECS service to run one or more instances of the gateway.
TLS encryption on your Consul server cluster is required for Consul datacenter federation over a WAN, and as with prior versions of Consul on ECS, the ACL controller needs to be deployed so that it can provision the necessary tokens for tasks on the service mesh as well as the mesh gateway.
If you use the Terraform modules for Consul on ECS, the ACL controller will configure the AWS IAM auth method on the Consul servers with sensible defaults. Here is an example of how to launch the ACL controller with Terraform:
When the ACL controller starts up, it configures the IAM auth method on the Consul servers. The code snippet below shows the default value for the iam_role_path
variable. This tells the ACL controller to configure the auth method to allow IAM roles at the path /consul-ecs/
to login.
Now you can deploy an application to the service mesh using the mesh-task
Terraform module. The mesh-task
module creates the ECS task definition and the associated IAM role. The IAM role is placed at the /consul-ecs/
path. This role path matches the ACL controller so the role is permitted to login to the IAM auth method on the Consul servers.
When the ECS task starts up, it runs the consul login
command. This command presents the task’s IAM role to the auth method on the Consul servers. The auth method validates the role with AWS. If the auth method determines the IAM role is valid and the role is permitted to log in, then an ACL token is returned to the task.
You can use the gateway-task
Terraform module to add mesh gateways to the Consul service mesh to allow cross-datacenter or cross-partition communication over a WAN. This section presents an example of how to use Terraform to deploy mesh gateways on ECS to enable WAN federation of two Consul datacenters via mesh gateways.
First, the primary datacenter must have a mesh gateway running to allow the secondary datacenter to communicate with the primary datacenter. The snippet below deploys the mesh gateway for the primary datacenter (“dc1”) using the gateway-task
Terraform module. The mesh gateway must be reachable over the WAN. The example uses the optional lb_enabled = true
setting to automatically create and configure a network load balancer to provide ingress to the mesh gateway over the WAN:
The Consul servers in the secondary datacenter must be configured with the primary_gateways
field to enable WAN-federation via mesh gateways. The example below shows how to retrieve the wan_address
and wan_port
outputs from the dc1_mesh_gateway
module to configure the primary_gateways
field in the secondary datacenter:
After the Consul servers in the secondary datacenter have started, we can deploy a mesh gateway in the secondary datacenter. The following example shows the Terraform configuration for the mesh gateway in the secondary datacenter (“dc2”). This is the same as the configuration for the primary mesh gateway except for the family
, ecs_cluster_arn,
and consul_datacenter
fields.
The final mesh-task
code snippet below configures the client application task in the secondary datacenter to call the server application in the primary datacenter. The meshGateway
stanza in the upstreams
definition configures the Envoy proxy to route the requests through the mesh gateway in the local datacenter.
Once this configuration is applied, the mesh gateways will start and register in their respective datacenters, and service mesh traffic will flow through the mesh gateways. All network traffic that passes through the mesh gateway is encrypted using mTLS. Mesh gateways do not decrypt the data within mTLS sessions so all service mesh traffic stays encrypted between the source and the destination.
To learn more, please see HashiCorp Consul’s AWS ECS documentation, which includes step-by-step instructions for moving existing applications to the service mesh, along with detailed architectural information.
Finally, this webinar on Securely Modernize Application Development with Consul on AWS ECS, now available on demand, is a great resource to learn about existing features and the 2022 roadmap from the team behind HashiCorp Consul on ECS.
Learn how to use GitOps to deploy and synchronize a Consul cluster on Kubernetes with Argo CD.
Use Minikube to create multiple Kubernetes clusters with Consul and test cluster peering configurations in your local development environment.
Consul 1.16 adds new reliability, scalability, security, service mesh UX, extensibility, and service discovery capabilities.