The Vault Secrets Operator implements a first-class Kubernetes Operator pattern for HashiCorp Vault along with a set of CRDs responsible for synchronizing Vault secrets to Kubernetes Secrets natively.
The beta version of the Vault Secrets Operator is now available as a final addition to the HashiCorp Vault 1.13 release. The Vault Secrets Operator is the newest method for Vault and Kubernetes integration, implementing a first-class Kubernetes Operator along with a set of custom resource definitions (CRDs) responsible for synchronizing Vault secrets to Kubernetes Secrets natively. This post will take a dive deep into this Operator and its place in the Vault ecosystem.
Kubernetes natively supports the management and distribution of secrets within a cluster. Applications running on Kubernetes can consume secrets natively from the Secrets API. Kubernetes Secrets can be consumed directly from files in a volume mounted by a container, through container environment variables, or directly from the API by an application that supports that retrieval mechanism.
In general, Kubernetes Secrets can be thought of as a static configuration object containing confidential information. Secrets are typically created during the deployment of an application and remain unchanged throughout the lifecycle of the application’s container. Kubernetes does not support dynamic secrets or rotation, leaving that responsibility to users.
Vault fills the gaps left in Kubernetes by providing a complete solution for secrets management. It consolidates identity brokering, secrets management (including dynamic secrets), rotation, and security policy compliance in one platform. By coupling Vault’s secret management feature set with a Kubernetes Secrets cache, the developer only has to consume the secret natively from Kubernetes.
The current approaches to Vault/Kubernetes integration require some level of application modification if you want the ability to read from a file. Furthermore, applications need to be aware of when credentials have been modified so that they can be re-read from the file. In response to the user interest in a new, Operator-based method of Vault/Kubernetes integration, Vault 1.13 introduced the Vault Secrets Operator, which will allow users to natively sync secrets from Vault to Kubernetes clusters.
Kubernetes Operators allow the customization of automatable tasks within a Kubernetes cluster. This operator pattern removes some service management burdens from human operators, such as managing secrets for the services orchestrated by Kubernetes.
Today we’ve released the Vault Secrets Operator (beta), which implements a first-class Kubernetes Operator along with a set of CRDs responsible for synchronizing Vault secrets to Kubernetes Secrets natively. The Operator synchronizes static, dynamic, and PKI-based secrets from one or more Vault server instances. The Operator is also responsible for secret rotation and provides controllers for handling the various secret-specific custom resources (CRs) s. The Operator can also perform post-rotation actions, like notifying an application directly by triggering a rolling update of a deployment.
The Vault Secrets Operator implements a controller that uses Kubernetes custom resources defined by the Vault Secrets Operator’s custom resource definitions to manage the secrets used by the services. The secrets are then managed by Vault, orchestrated in Kubernetes using custom resources, and consumed by the applications in a Kubernetes-native manner, cleanly separating concerns. Using standard Kubernetes declarative patterns, the Vault Secrets Operator reconciles the current state to the desired state specified in the CRs.
To check out the code, visit the Kubernetes Vault Secrets Operator GitHub repo. The project SDK is based on the Operator SDK for Go and provides the tools necessary for building, testing, and deploying a Kubernetes Operator.
During the beta period, we are supporting only the Kubernetes authentication method. This lets us focus on the core aspects around secret synchronization. Over time, we anticipate adding support for additional authentication methods.
Vault authentication configuration can be applied from the VaultAuth CR. The Secret CR includes a reference to a VaultAuth. The VaultAuth resource provides the parameters necessary to authenticate to Vault and perform the sync.
apiVersion: secrets.hashicorp.com/v1alpha1 kind: VaultAuth metadata: labels: app.kubernetes.io/name: vaultauth app.kubernetes.io/instance: vaultauth-sample app.kubernetes.io/part-of: vault-secrets-operator app.kubernetes.io/managed-by: kustomize app.kubernetes.io/created-by: vault-secrets-operator name: vaultauth-sample namespace: tenant-1 spec: vaultConnectionRef: vaultconnection-sample method: kubernetes mount: kubernetes kubernetes: role: demo serviceAccount: default
The necessary Vault login credentials come from a ServiceAccount configured in the CR’s namespace. The Vault Secrets Operator requires both read and create access to any ServiceAccount configured in the related VaultAuth CR. The create permission is required to create short-lived ServiceAccount tokens.
By separating the authentication configuration from the Secret CR, we are able to provide future support for other authentication methods, and potentially other architectures or deployment strategies.
In upcoming versions, we plan to give the Vault Secrets Operator the ability to limit the supported auth methods via the command line. The code will resemble this snippet:
These options add an extra layer of security by preventing unsupported and unauthorized auth methods from being used.
The Vault Secrets Operator will be able to constrain sync activities to specific Kubernetes namespaces. By default, all Kubernetes namespaces are allowed. The Vault Secrets Operator beta currently supports only the Kubernetes auth method, but more auth methods are planned for the GA release.
The Vault Secrets Operator requires various Kubernetes permissions to function correctly. These include:
|Secret||create, read, update, delete, watch||Sync operations, Vault auth|
|ServiceAccount||read, create||Vault auth|
|Deployment||read, update, watch||Post secret rotation actions|
If granting these permissions for the entire cluster is problematic for your team from a security management or compliance standpoint, consider restricting the Vault Secrets Operator’s access to objects in specific namespaces.
The Vault Secrets Operator provides a Prometheus metrics endpoint, along with support for the Prometheus Operator. This allows users to be alerted to any potential issues with the system. The Prometheus output can be integrated with Grafana during the beta period. However, we anticipate providing a Visual Studio Online-specific Grafana dashboard for the GA release.
The Vault Secrets Operator gives users meaningful signals for determining the overall health of its secrets management processes, as well as analytics to measure security policy adoption.
Vault Secrets Operator metrics include:
To learn more about other Vault/Kubernetes integration methods, read our blog post on Kubernetes Vault Integration via Sidecar Agent Injector vs. CSI Provider and visit each method’s GitHub repository:
As you pursue your Kubernetes secrets management journey with Vault Secrets Operator, we’re interested in any feedback and recommendations you may have. To provide feedback on our beta, please log a GitHub Issue.
You can learn more about the Vault Secrets Operator by visiting the vault-secrets-operator GitHub repository. Further documentation on using Vault with Kubernetes can be found in our HashiCorp Developer guide to Kubernetes and Vault. To learn more about HashiCorp Vault please visit the Vault product page.
Learn how to use the Prometheus Operator with the new Vault Secrets Operator for Kubernetes to monitor secrets in a Grafana dashboard.
Learn how to build a secure infrastructure as code workflow with Terraform Cloud dynamic provider credentials, Microsoft Defender for Cloud, and HCP Vault.
The HashiCorp Vault partner ecosystem continues to show strong growth with the addition of more than a dozen new Vault integrations.