Bridgewater: Securing their AWS Infrastructure with Vault

Bridgewater: Securing their AWS Infrastructure with Vault

May 16 2017 Joel Thompson

This is a guest post by Joel Thompson, Systems Engineer at Bridgewater Associates. Joel is a user of Vault at Bridgewater Associates and a contributor to the HashiCorp Vault project, specifically for the AWS IAM Authentication method discussed in this post.

HashiCorp released Vault 0.7.1 which ships with a major enhancement to the AWS-EC2 authentication backend, now renamed to the AWS authentication backend, making it easy for many different AWS resource types to securely authenticate with Vault and get a Vault token. Lambda functions, ECS jobs, EC2 instances, or any other client with access to AWS IAM credentials can use those credentials to securely authenticate to Vault to retrieve their secrets. I'm really excited about the feature, and I think it'll be a game changer for all the security-conscious AWS customers out there.

First, though, some introductions. Bridgewater Associates is focused on understanding how the world works. By having the deepest possible understanding of the global economy and financial markets, and translating that understanding into great portfolios and strategic partnerships with institutional clients, we've built a distinct track record of success. Today, we manage about $160 billion for approximately 350 of the largest and most sophisticated global institutional clients including public and corporate pension funds, university endowments, charitable foundations, supranational agencies, sovereign wealth funds, and central banks.

A few years ago, Bridgewater started moving to Amazon Web Service's cloud offering, and I was one of the first engineers involved with that effort. We loved many of the advantages offered by AWS, but there was one problem that had consistently caused us pain. How do we take advantage of the dynamic compute capabilities offered by a cloud provider without sacrificing security? Or, to put it concretely, how do we securely grant new instances in an AWS autoscaling group access to secrets they need?

This was clearly a need that others in the community shared. There were some suggestions from the community about how to achieve it, including two from one of my coworkers, Dan Peebles.

In response to this need, HashiCorp released the AWS-EC2 authentication backend. The AWS-EC2 authentication backend was great for what was feasible at the time, but it had limitations -- in particular, it only worked for EC2 instances and not AWS Lambda functions or ECS jobs or other AWS scenarios. And then things changed when AWS added the sts:GetCallerIdentity method. Suddenly, the proposal in GH-948 felt like it was within reach. As someone who loves AWS, loves security, and loves HashiCorp, implementing that proposal and making it real was a great opportunity for me to contribute something meaningful back to the community. After collaborating closely with the HashiCorp Vault engineering team, and getting help from Dan (who originally suggested GH-948, the AWS authentication backend was born.

The AWS authentication backend now has the concept of authentication types and currently supports two. The EC2 authentication type corresponds to the prior behavior of the AWS-EC2 authentication backend, while the IAM authentication type is the new method and it allows you to authenticate using AWS IAM credentials, mapping an IAM user or role to a Vault role. In many cases, AWS already does the hard work of securely providing your compute resources with IAM credentials, such as EC2 instances in an instance profile, AWS Lambda functions, ECS jobs, and AWS CodeBuild steps. AWS has done the hard work of providing your resources with IAM credentials, and now your applications can just consume these credentials to authenticate to Vault in order to access their secrets. Vault authentication is performed by having clients sign an AWS API request, sts:GetCallerIdentity. API request and then send the signed request to Vault. The actual AWS secret key is never sent to Vault, helping you keep your sensitive credentials safe. Vault forwards the signed request on to AWS and observes the result; if the result matches the entity configured in a Vault role, then the request is authenticated and a Vault token is returned to the client.

This also solves another common problem: how to make the developer workflow match the production workflow as much as possible. Many developers want to test locally on their own laptops, which aren't able to readily authenticate to Vault using the EC2 authentication method. However, with the new IAM authentication method, clients can consume developers' AWS API credentials to authenticate to Vault, providing the same workflow as in your production environments.

The IAM authentication method also supports a feature called "inferencing." The EC2 authentication method allows you to place restrictions on EC2 instances that can authenticate to a role, such as which AMI an instance was launched from, or which subnet the instance was in. Because these are attributes about an EC2 instance and not an IAM principal, similar sorts of restrictions aren't generally possible with the IAM authentication method. However, EC2 instances in an IAM instance profile authenticate in a way that makes their instance IDs visible to Vault. Inferencing tells Vault to look for the instance ID, look it up in EC2, and treat the client as an EC2 instance, allowing you to apply many of the EC2-specific binds as when using the EC2 auth method. (Note that there are some very important security caveats you should be aware of before just blindly trusting inferencing which are described in the documentation.)

The vault command-line tool makes it easy to authenticate to Vault using the AWS authentication backend. Just run:

$ vault auth -method=aws role=vault_role_name

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×