In today's digital landscape, enterprise organizations face significant challenges in managing cryptographic keys. The shift towards multi-cloud environments and hybrid infrastructures has introduced complexities in key management, often leading to fragmented security practices. This fragmentation is further exacerbated by the need to adapt to existing workflows to accommodate cloud adoption. The National Institute of Standards and Technology (NIST) says that the management of cryptographic keys becomes increasingly complex in cloud environments due to differences in ownership and control between cloud consumers and providers.

To address these challenges, there is a pressing need for centralized security platforms capable of operating seamlessly across multi-cloud and hybrid cloud environments. Such platforms would streamline key management processes, reduce operational inefficiencies, and enhance the overall security posture. By consolidating key management, organizations can better protect sensitive data, ensure compliance with regulatory requirements, and respond more effectively to emerging threats. Implementing a centralized approach simplifies management and fortifies the organization's defense against the ever-evolving landscape of cyber threats.

»Challenges

Generally cryptographic key management is a centralized task undertaken by a common security team in an organization to align with org-wide compliance and security standards. They maintain the necessary tools and services to manage the lifecycle of keys, which are used by the different applications in the organization, thereby creating a central platform for key management.

If we look at this problem through the lens of this security team, these are some technical challenges that they would face while creating this platform:

1. There is a strategic directive to use a specific cloud key management service (KMS) for storing, handling, and interacting with private key material. However, the application teams would also want to use a key lifecycle process that is fully automated. This means reusing existing auth and policy systems for authentication and authorization. How can existing authentication and policy enforcement mechanisms be leveraged while still using my cloud KMS for private key storage?

2. Application Teams have to use the central platform to generate key material to ensure compliance. They would also like to store it in the cloud KMS from where the cloud apps can directly consume these keys for their cryptographic operations (known as Bring Your Own Key - BYOK). The main goal is to have the auth and policy enforcements happen in the security platform while the keys reside in the Cloud KMS.

Even in a single cloud environment, these challenges are difficult enough. With a multi-cloud setting, the pain is multiplied.

Let’s see how we can solve these challenges with a cloud-agnostic solution like HashiCorp Vault.

»Pre-Requisites

Both the challenges described above can be solved with Vault key management secrets engine (KMSE) and 3rd party key management.

»HCP Vault and Vault Enterprise

You can follow along in this tutorial in two ways. You can use a Vault Enterprise license, and for KMSE, you will also need the additional Advanced Data Protection (ADP) module. Alternatively, you can also try this out in an HCP Vault cluster. You get a 500 USD trial credit to spend in the next 6 months when you sign up the first time.

»Azure subscription and Azure key vault

This demo uses Microsoft Azure, but the same general workflow could be used for AWS KMS or Google Cloud KMS. You will need a valid Azure subscription with an Azure Key Vault initialized.

To enable Vault to manage Azure Key Vault, we recommend creating an application in Azure and giving it the necessary permissions. A sample list of permissions are mentioned in this Terraform example. Once the application is created, you can generate a client secret which you will use for authentication with Azure from Vault.

In both the examples below, static credentials are used to authenticate between Vault and Azure. It is always a best practice to rotate these credentials periodically. You may leverage the Azure Secrets Engine to obtain time-limited Azure credentials, which can be used for this purpose.

»Vault managed keys

The 3rd party key management feature, also called ‘managed keys’, allows you to leverage cloud KMSes or external hardware security modules (HSMs) to store and generate private keys for use in Vault workflows. The key material is stored outside of Vault but will be used by Vault for various operations like signing a PKI certificate or data encryption.

With Azure KMS as your external KMS, follow these steps to use the managed keys.

1. Configure the managed-keys endpoint inside Vault.

Caveats:

• sign and verify are supported for Cloud KMSes. Other commands like encrypt or decrypt are supported only for PKCS#11 backends (like HSM).
• A full list of API parameters can be found here.

2. Verify the configuration.

3. Validate that you can access the managed key.

This will also result in Vault generating the key in Azure if that key doesn’t exist and if allow_generate_key is set to true in step 1. At this point, you can see the key has been created if you go into your Azure Key Vault.

4. Now you have a managed key within Vault even though the actual key material resides in Azure. From here you can use the key for various operations in Vault secret engines. For example, you can use the managed key for signing and verifying data in Vault’s transit secrets engine.

At this point the managed key is added to the transit secrets engine, which you can now use for signing and verifying data.

The full length of options that you can provide to sign and verify API calls in Vault transit is described here.

5. In step 1, the option any_mount=true was set while configuring the managed key. This means that the managed key could be used by any secrets engine created in the same namespace. You could restrict this by setting this flag to false and then configuring the usage of managed keys explicitly from the respective mounts using the option allowed_managed_keys.

Other than the transit secrets engine, the PKI secrets engine also supports managed keys.

To summarize, the managed key feature can be used to leverage the cryptographic capabilities of Vault’s secret engines while at the same time ensuring regulatory compliance by storing the private key material outside of Vault (in an approved HSM or a Cloud KMS). Additionally, managed keys make use of the auth and policy engines in Vault to streamline access to these functions centrally in an organization.

»Key management secret engine (KMSE)

While managed keys are suited for using keys residing in external locations and various Vault secret engines, the lifecycle management of these keys still requires manual intervention and fiddling around with various cloud provider APIs. KMSE was designed to solve this issue. It provides the following features:

1. Lifecycle operations like creation, updates, rotation, and deletion of keys
2. Duplicating keys residing in one cloud provider to another, thereby creating a standardized workflow and a shared root of trust in a multi-cloud setup

Let's take a look at how you can set this up:

1. The full length of options that you can provide to sign and verify API calls in Vault transit is described here.

Multiple KMS providers can be configured in Vault KMSE.

2.  Create a key.

3. Distribute key to the KMS provider.

4. At this point, the key would be uploaded to Azure Key Vault.

This can either be verified using the Azure Portal or using the az CLI utility.

5. Rotate the key.

The rotate command resulted in the creation of a new version of the same key in Vault. For duplicating the key to a second cloud provider, all you have to do is create a new KMS provider in Vault and associate this key to it.

6. Remove the key from the provider.

For Azure as of writing, only RSA keys are allowed. The compatibility of different key types with different cloud provider KMSes is listed here.

»Takeaways

Combining both these approaches, you have a powerful tool for creating an end-to-end workflow for managing the cryptographic material in your organization:

• KMSE for managing the lifecycle of keys residing in one or many cloud key vaults
• Managed keys for using these keys seamlessly for various cryptographic functions inside Vault

Here is a sample diagram of how this workflow would look:

»Learn more

Cryptographic key management plays a vital role in strengthening the security posture of an organization. HashiCorp Vault can help you achieve your security goals not just by centralizing key management but also creating end to end workflows around secrets, certificates and other security-relevant infrastructure.

• Key management secrets engine (KMSE)
• Managed keys
• Security Lifecycle Management