Explore the pros and cons of five different ways to manage credentials and other secrets in Terraform Cloud & Enterprise.
There are several ways to manage credentials or other secret types in Terraform Cloud and Terraform Enterprise, either natively, or with purpose-built secrets management utilities like HashiCorp Vault, so this is a somewhat opinionated article that lists what I believe are currently the best options. Please note this is not a replacement for some other best practices, such as keeping your Terraform CLI and code up to date.
The described patterns follow some common principles. Credentials should be:
There are also some nuances between Terraform Cloud and Terraform Enterprise that I will call out in each section.
In terms of security, both Cloud and Enterprise products encrypt their sensitive variables using the Vault Transit secrets engine and do not allow any external API call to decrypt these values. For more details please see the Data Security page.
Using only Terraform workspaces, your cloud vendor's Terraform provider, and the Terraform Cloud/Enterprise provider, you can set up a “Credentials” workspace that is able to generate new credentials and rotate the ones used by other workspaces. To avoid the secret zero problem, when setting up the credentials workspaces, you can make use of the Terraform Agents pattern described in the next section.
Note: In Terraform Cloud all workspaces live in the same Organization.
Both Terraform Cloud Business tier and Terraform Enterprise support running your code using external agents. This feature is called Terraform Agents. Any cloud provider declared in your Terraform code is able to take advantage of the credentials set in the Terraform Agent environment, which means the credentials do not need to be set at the workspace level.
It’s no secret HashiCorp Vault is able to generate dynamic credentials for all major cloud vendors, databases, etc. For those who know about Vault, this integration with Terraform is the first solution they ask about, since secrets management is Vault’s primary use case. Here are a few patterns to make that integration work.
There isn’t much to explain in this workflow. Most of the complexity is in setting up Vault authentication and some Terraform template code.
At the heart of this integration pattern, is the ability to confidently authenticate to Vault with an identity that is unique and combines workflow, repository, and branch/environment.
This is not the case with all CI/CD implementations, so I’m only mentioning two implementations where I know it’s possible.
Vault tokens should be very short-lived and linked to an entity, to restrict access in case any token gets leaked and allow for better auditing. Avoid storing Vault tokens as secrets!
Regarding the workflow, the CI/CD runner will need to authenticate to Vault and retrieve:
These pros and cons apply to the two subsections below.
GitLab runners are able to authenticate to Vault using the JWT auth backend, where you can configure separate roles for staging, dev, or production.
Once authentication has been solved, GitLab CI/CD will be able to retrieve the necessary secrets and interact with the Terraform Cloud/Terraform Enterprise API. Here’s an example.
For private repositories, you’ll need to have GitLab Premium.
For GitHub Actions to authenticate to Vault there are two options:
If you use AppRole, please make sure you also have a cron job set up to rotate the environment secrets, and use the “Repository Environments” to distinguish between production and non-production.
If you use the custom Vault GitHub Action authentication backend, right now it’s not possible to distinguish which branch is being executed.
There is no right answer or one-size-fits-all solution to managing your credentials and secrets within Terraform Cloud and Terraform Enterprise. This is highly dependent on your current requirements, environment, tools, etc, so it’s up to you to select the pattern that is best for you and your team, taking into account the pros and cons of each solution.
Please bear in mind that all of these solutions require you to plan and execute an efficient team/application onboarding process to be successful. To learn more about onboarding applications into Vault check out our other blog post Onboarding Applications to Vault Using Terraform: A Practical Guide or book a 1:1 session with one of our experts.
AWS and HashiCorp are collaborating to develop Terraform modules.
With a live demo, two engineers from Mozilla’s Pocket show how they use CDK for Terraform to define and provision infrastructure in a TypeScript application.
Read this curated list of HashiCorp learning resources to help practitioners and organizations better understand the cloud operating model.