Cloud Foundry Vault Service Broker

Cloud Foundry Vault Service Broker

Feb 14 2017 Seth Vargo

We are pleased to announce the release of the official Cloud Foundry HashiCorp Vault Service Broker. This service broker connects to an existing Vault cluster and can be used by multiple tenants within Cloud Foundry to securely store, access, and encrypt using Vault.

Why Vault & Cloud Foundry Integration

One of the core components of HashiCorp is our commitment to heterogeneous environments. The Cloud Foundry HashiCorp Vault Service Broker is an example of our continued commitment to work with our ecosystem partners to deliver the best experience to our users regardless of their infrastructure or application type. In addition, the Tao of HashiCorp showcases simplicity and modularity combined with communication of sequential processes. These key pillars drive us to create pluggable components. Simply put:

You don't have to use all the HashiCorp tools to get the benefits of one.

You should not have to re-architect your entire infrastructure just to use a single tool. Vault is a tool for managing secrets, and while we would love you to deploy it with Terraform and run it with Nomad, you don't have to. Our tools function in isolation to build workflows that enable you to provision, secure, and run any infrastructure for any application.


Cloud Foundry provides a framework for deploying and managing applications. Those applications have the ability to communicate with each other, or with pre-configured services. These services expose APIs via service brokers, and can provide data to an application including credentials, endpoints, or application-specific data.

When a new service instance is provisioned against the Cloud Foundry HashiCorp Vault Service Broker, it will perform the following operations in the Vault server:

  1. Mount the generic backend at /cf/<organization_id>/secret/

  2. Mount the generic backend at /cf/<space_id>/secret/

  3. Mount the generic backend at /cf/<instance_id>/secret/

  4. Mount the transit backend at /cf/<instance_id>/transit/

The mount operation is idempotent, so service instances in the same organization or space will not re-create the mount. These mount points are returned to the application as part of the credential, so there is no need for applications to guess or interpolate these strings.

The read-only organization mount allows for sharing secrets organization wide, and the read-write mount permits sharing secrets between applications in the same Cloud Foundry Space via the generic backend. The transit backend provides "encryption-as-a-service" on a per-application basis.

We believe this architecture provides the best solution for applications to share secrets between instances, with other applications in the same space, and retrieve read-only information from the organization in Cloud Foundry. The beauty of this design is that applications do not need to be Vault-aware to gain the benefits of modern secrets management.

Getting Started

The Cloud Foundry HashiCorp Vault Service Broker assumes a Vault server is already-configured. This blog post denotes the address and token using the following variables on the local workstation:

$ export VAULT_ADDR="" $ export VAULT_TOKEN="abcdef-134255-..."

The Cloud Foundry HashiCorp Vault Service Broker does not run or configure a Vault server. This Vault server above does not need to be running under Cloud Foundry, but it must be accessible from within Cloud Foundry or wherever the broker is deployed.

Additionally the broker is configured to use basic authentication. Those variables will be denoted as follows in this post:

$ export USERNAME="vault" $ export PASSWORD="vault"

Additionally, this post assumes a Cloud Foundry cluster is already up and running. To start a cluster locally, we recommend using PCF Dev.

Deploying the Broker

The first step is deploying the broker. This broker can run anywhere including Cloud Foundry, Heroku, HashiCorp Nomad, or your local laptop. This example shows running the broker under Cloud Foundry.

First, create a space in which to run the broker:

$ cf create-space vault-broker

Switch over to that space:

$ cf target -s vault-broker

Deploy the cf-vault-service-broker by cloning the repository from GitHub:

$ git clone $ cd cf-vault-broker

And push to Cloud Foundry:

$ cf push --random-route --no-start

  • The --random-route flag is optional, but it allows us to easily run more than one Vault broker if needed, instead of relying on "predictable names".

  • The --no-start flag is important because we have not supplied the required environment variables to our application yet. It will fail to start now.

To configure the broker, provide the following environment variables:

name requested state instances memory disk urls vault-broker started 1/1 256M 512M

Grab the URL and save it in a variable or copy it to your clipboard - we will need this later.

BROKER_URL=$(cf app vault-broker | grep urls: | awk '{print $2}')

Again, there is no requirement that our broker run under Cloud Foundry - this could be a URL pointing to any service that hosts the broker, which is just a Golang HTTP server.

To verify the broker is working as expected, query its catalog:

$ curl -s "${USERNAME}:${PASSWORD}@${BROKER_URL}/v2/catalog"

The result will be JSON that includes the list of plans for the broker:

{ "services": [ { "id": "0654695e-0760-a1d4-1cad-5dd87b75ed99", "name": "vault", "description": "HashiCorp Vault Service Broker", "bindable": true, "plan_updateable": false, "plans": [ { "id": "0654695e-0760-a1d4-1cad-5dd87b75ed99.default", "name": "default", "description": "Secure access to a multi-tenant HashiCorp Vault cluster", "free": true } ] } ] }

The HashiCorp Vault Service Broker is now running under Cloud Foundry and ready to receive requests.

Register the Vault Broker

Before it can bind to services, the broker must be registered with Cloud Foundry. Remember that there is no requirement that the broker be running under Cloud Foundry, so we will need to provide the broker registration service the BROKER_URL from above. Again, this is important because at HashiCorp we believe in a vision of heterogeneity; you should not have to fit all your services into the perfect container to absorb modern best practices.

To register the broker, an application developer first creates a new space where they will request access to the broker:

$ cf create-space example $ cf target -s example

Next, register the broker in this space:

$ cf create-service-broker vault-broker vault vault "https://${BROKER_URL}" --space-scoped

To verify the command worked, query the marketplace. You should see the broker in addition to the built-ins:

$ cf marketplace # ... vault default HashiCorp Vault Service Broker

Generate Credentials through the Vault Broker

After registering the service in the marketplace, it is now possible to create a service instance and bind to it.

$ cf create-service vault default my-vault

Next, create a service key:

$ cf create-service-key my-vault my-key

And finally retrieve credentials for this instance:

$ cf service-key my-vault my-key

The response will look like:

{ "address":"", "auth":{ "accessor":"b1074bb8-4d15-36cf-54dd-2716fb8ac91d", "token":"dff95895-6a03-0b29-6458-dc8602dc9df8" }, "backends":{ "generic":"cf/203f2469-04e4-47b8-bc17-f3af56df8019/secret", "transit":"cf/203f2469-04e4-47b8-bc17-f3af56df8019/transit" }, "backends_shared":{ "organization":"cf/3c88c61e-875b-4530-b269-970f926340c4/secret", "space":"cf/0348d384-f7d4-462b-9bd9-5a4c05b21b6c/secret" } }

The keys are as follows:

  • address - address to the Vault server to make requests against

  • auth.accessor - token accessor which can be used for logging

  • auth.token - token to supply with requests to Vault

  • backends.generic - namespace in Vault where this token has full CRUD access to the static secret storage ("generic") backend

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now