Learn our best practices and get customer-tested templates that help HashiCorp Vault users adopt efficient producer-consumer models.
Setting up a central service for HashiCorp Vault can be challenging due to the shared ownership of the secrets and data between the functional team that is using the secret (whether it’s operations, infrastructure, data, engineering, etc.) and the team that is operating Vault and securing it. Requiring that all of these functional teams acquire and maintain expertise in cryptography, secure secret use and management, and machine identity can slow down progress towards important strategic initiatives like building new applications. That’s why many organizations opt for a producer-consumer pattern, which shifts those specializations to the team operating Vault.
Having worked with hundreds of our customers who have adopted this producer-consumer model, we have identified the most common challenges organizations run into. In this blog, I will identify these challenges and share the best practices for solving them at scale. I will also link to several tools for implementing these best practices, including a use case matrix, intake template, and questionnaire.
Each team has its own set of objectives, metrics, and challenges when dealing with secrets management. At times, their metrics and objectives can be at odds, which can result in slow progress, poor security practices, and a failure to create a scalable, efficient Vault workflow that works for everyone.
To make sure this result doesn’t happen, it is important to understand and empathize with the various consumer-side stakeholders’ perspectives:
Overall, the producers must view themselves as providing a service on behalf of the consumers. Consumers hold the keys to their respective use cases and can choose whether to participate or not. It is incumbent on the producer team to make it easier to accept and adopt the central service.
Starting off on the right track requires a partnership between all of the stakeholders. In many organizations, top-down mandates can struggle to find success without buy-in at all levels. Greater success can be achieved by placing the needs of all stakeholders at the forefront.
When Vault is an essential part of all planning, Vault becomes a critical component of your zero trust strategy or security initiative. It’s incredibly difficult to achieve a robust producer-consumer model otherwise. As a critical central service, Vault requires adequate engineering resources to ensure the success of the deployment, operations, and ongoing adoption. Success requires continued focus on training and driving adoption across your organization — the job is not complete when the first use case is implemented.
It may be that there are too many teams for it to be feasible to meet with each. If this is the case, pick a representative sample of teams to work with on requirements gathering. This can include the most stringent use cases in terms of security or compliance, which may naturally allow you to build the service out in a way that represents the largest number of teams or services possible. If you choose a sample of representative teams to collect requirements for, this will map to the stakeholders section of the service-level agreement (SLA).
It can be extremely valuable to have a complete understanding of what the consumers need and then define what the producer team needs to build into Vault. In order to serve many internal constituents, documenting every use case and the applicable needs is required.
Using the use case matrix, intake template, and questionnaire linked below in this article can help you accelerate progress while the sample questions and guidance from this post will accelerate adoption.
When considering setting up the producer-consumer model, there can be a lot to think about - resiliency requirements, use cases, the geography of workloads, rotation requirements, and approved authentication methods. Here is a link to a questionnaire for defining operating principles for the Vault central service producers and consumers. This will help teams define the service and document requirements so that the Vault service is set up correctly, ensure that knowledge is not lost over time, and help with consumer onboarding.
It includes questions such as:
Sample Consumer Question:
Sample Producer Questions:
Here is a link to the full operating principles questionnaire.
As new consumer teams are ready to be onboarded to the central service, baseline information must be collected. Oftentimes, consumer teams are not experts in secrets management, encryption, or machine identity so they may not know what information to share with the producer team. Some of the important categories of information that must be collected are who the people are who own the use case, what functional requirements they may have, performance requirements, or any unique circumstances of their use case.
Here is a sample intake form that can be used by teams that are currently using Vault or new teams that need to start being served by the Vault Central Service. This form can be edited to take any unique requirements into account.
Here is a link to an intake form template.
The Use Case Matrix is a project management framework that will enable you to split the requirements of the producer team and the various consumer teams that will subscribe. By breaking out the requirements in this way, it is easier for the producer team to manage their existing consumers and onboard new consumer use cases. Over time, this will serve as program documentation as business needs and personnel change.
Here is a link to a use case matrix template you can use to document your program.
The producer team must reach out to consumer services owners across their organization to understand their needs for identity management, encryption, and secrets utilization. Three best practices for engaging with current and future consumers of Vault are:
Often the producer team that owns and manages Vault is focused on security and operational objectives. One such objective is getting as close as possible to 100% of all secrets being dynamically created, served, timed out, and destroyed on demand. Another objective should be to give consumers a customer-level experience since their adoption of your process will be key to reaching that 100% level of secure secrets management. Producers need to give consumer teams the easiest possible workflow so they can be focused on developing new application functionality. Producers also need to provide a reliable, highly-available deployment of Vault so that consumers can trust that their secrets, authentication, and encryption services will be available when they need them.
If you would like to learn more about building these programs at scale or would like our help, please reach out to our sales team or your HashiCorp representative.
The HashiCorp Releases API is now available. This API is your one-stop shop for finding and viewing extended metadata about HashiCorp product releases.
This release strengthens zero trust security architecture by leveraging HashiCorp Vault to reduce secrets sprawl and automate server TLS certificate rotation.
HashiCorp has partnered with AWS to enhance security on Amazon Elastic Kubernetes Service with Vault.