Products & Technology

Dynamic Secrets for Waypoint with Vault

Learn how to source dynamic secrets from HashiCorp Vault in your Waypoint deployments using dynamic configuration.

Dec 08 2022Joe Rajewski

HashiCorp Waypoint is an application release orchestrator that simplifies how application developers build, release, deploy, and monitor their apps in cloud environments. Developers can use Waypoint to configure applications quickly and save time managing deployments. This can be done with static application configuration, where static environment variables or files can be applied to your running deployment. Alternatively, configurations can be sourced dynamically from external sources using a config sourcer plugin.

The HashiCorp Vault config sourcer plugin can be used to source static values from the Vault KV secrets engine as well as dynamic secrets. Dynamic secrets can significantly upgrade to the security of your application by allowing you to set secrets to expire after a set time-to-live (TTL). That means leaked secrets automatically expire after a pre-configured amount of time, limiting how long a potential attacker has access to your systems.

With Waypoint’s app deployment workflow, you can easily integrate dynamic configurations with Vault, regardless of whether you are deploying with HashiCorp Nomad, Kubernetes, or any other deployment target. This blog post will use an example application to walk through how to configure this workflow in Waypoint.

»Sourcing Dynamic Secrets from Vault in a Waypoint Deployment

The example application is located in our waypoint-examples GitHub repository. The application will eventually need read-only access from the Postgres database, so you will need to provide dynamic secrets to access the data. To make this happen, create a role in a Postgres database with read-only access using the psql command line tool. The IP, port, and user below should be filled in based on the configuration and environment in which your Postgres instance is running.

$ psql -h <POSTGRES_IP> -p <POSTGRES_PORT> -U <POSTGRES_USER> -c "CREATE ROLE \"ro\" NOINHERIT;" $ psql -h <POSTGRESIP> -p <POSTGRES_PORT> -U postgres -c "GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"ro\";"

Once you have a Vault cluster up and running, enable and configure the database secrets engine so that Vault can connect to the database and generate credentials dynamically:

$ vault secrets enable databaseSuccess! Enabled the database secrets engine at: database/ $ vault write database/config/postgresql \     plugin_name=postgresql-database-plugin \     connection_url="postgresql://{{username}}:{{password}}@<POSTGRES_ADDR>/postgres?sslmode=disable" \      allowed_roles=readonly \     username="<POSTGRES_USER>" \     password="<POSTGRES_PASSWORD>" Success! Data written to: database/config/postgresql

In the database secrets engine, configure a connection as well as a role in the secrets engine. With this, authorized entities in Vault will be able to generate dynamic credentials in the secrets engine.

$ tee readonly.sql <<EOFCREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}' INHERIT;GRANT ro TO "{{name}}";EOF $ vault write database/roles/readonly \     db_name=postgresql \     creation_statements=@readonly.sql \     default_ttl=1h \     max_ttl=24hSuccess! Data written to: database/roles/readonly

The final steps in Vault are to create a Vault policy, which will allow an entity to access the database secrets engine and generate a dynamic database credential, and to create a token with this policy attached to it:

$ tee waypoint-policy.vault.hcl <<EOFpath "database/creds/readonly" {  capabilities = [ "read" ]}EOF $ vault policy write waypoint waypoint-policy.vault.hclSuccess! Uploaded policy: waypoint-policy $ vault token create -policy=waypoint -ttl=24hKey                  Value---                  -----token                <THE_TOKEN>token_accessor       <THE_TOKEN_ACCESSOR>token_duration       24htoken_renewable      truetoken_policies       ["default" "waypoint"]identity_policies    []policies             ["default" "waypoint"]

Now that we have things configured in Vault, the next step is to configure Waypoint to connect to Vault. Add a configuration source for the Vault plugin like this:

$ waypoint config source-set -type=vault -config=addr=https://vault.service.consul:8200 -config=token=<VAULT_TOKEN>Configuration set for dynamic source "vault"!

Double check that the token and address look correct:

$ waypoint config source-get -type=vault   KEY  |           VALUE            --------+----------------------------  addr  | https://vault.service.consul:8200     token | <VAULT_TOKEN>

At this point, you can get the config source ready by adding it to waypoint.hcl and committing it to the Git repo for the project:

project = “waypoint-dynamic-config-vault” app “my-app” {  build {    use “pack” {}      registry {      use “docker” {        image = "devopspaladin/hello-database"        tag = "latest"        auth {          username = var.username          password = var.password         }      }    }  }   deploy {    use “nomad” {}  }   config {    env = {      USERNAME = dynamic(“vault”, {        path = “database/creds/my-role”        key = “username”      })       PASSWORD = dynamic(“vault”, {        path = “database/creds/my-role”        key = “password”      })       HOST      = var.postgres_ip      PORT      = var.postgres_port      DBNAME = var.postgres_dbname    }  }} variable "postgres_ip" { type = string} variable "postgres_port" { type = number} variable "postgres_dbname" {  type     = string  default = “postgres”} variable "registry_username" { type      = string sensitive = true} variable "registry_password" { type      = string sensitive = true}

In the configuration above, you’re informing Waypoint that you want the Waypoint entrypoint — which will wrap the entrypoint to the Docker container that the ‘pack’ plugin is building — to get a secret from Vault at the path “database/creds/my-role”. The credentials to access the secret at that path are set by using dynamic variables, using the keys “username” and “password”, and then configuring the USERNAME and PASSWORD environment variables in the deployment of your app. You can confirm that the secret was successfully templated after running waypoint up by checking the environment variables using waypoint exec:

// To run this operation, I exported my input variables to my environment, prefixes with “WP_VAR_”// The output below is abbreviated for brevity$ waypoint up ✓ Running build v2✓ Building Buildpack with kaniko...✓ Repository is available and ready: devopspaladin/hello-database:latest✓ Executing kaniko...…✓ Image pushed to 'devopspaladin/hello-database:latest'✓ Running push build v1✓ Running deploy v2✓ Job registration successful✓ Allocation "4a0788ae-a6c0-bf55-4a84-46a4f0ff4a81" created: node "b9a637ea-6098-4b22-6f52-9d5b94581a75", group"dynamic-go-01gj1s1sqj31r72qn38t09xj7p"✓ Evaluation status changed: "pending" -> "complete"✓ Evaluation "d1c03461-fddd-2e68-bb7b-d6e7e718f121" finished with status "complete"✓ Deployment successfully rolled out!✓ Job "dynamic-go-01gj1s1sqj31r72qn38t09xj7p" is reporting ready!✓ Finished building report for Nomad platform $ waypoint exec env…USERNAME=v-token-readonly-wOdEtyXQ5MktQ5SwTnXo-1668702202PASSWORD=e-4QVqEpUhGfmpTncKTy… $ waypoint logs2022-11-17T16:32:57.940Z D5KVPR: [INFO]  entrypoint: entrypoint starting: deployment_id=01GJ36WR2WP3MAVFVH47SR4EHT instance_id=01GJ36ZCACB1PFFVZ1C4D5KVPR args=["/cnb/lifecycle/launcher"]2022-11-17T16:32:57.941Z D5KVPR: [INFO]  entrypoint: entrypoint version: full_string="v0.10.0-575-gad4b66798 (ad4b66798+CHANGES)" version=v0.10.0-575-gad4b66798 prerelease="" metadata=""revision=ad4b66798+CHANGES2022-11-17T16:32:57.941Z D5KVPR: [INFO]  entrypoint: server version info: version=v0.10.0 api_min=1 api_current=1 entrypoint_min=1 entrypoint_current=1 auth_method=token2022-11-17T16:32:58.613Z D5KVPR: [INFO]  entrypoint.config.watcher: env vars changed, sending new child command2022-11-17T16:32:58.614Z D5KVPR: [INFO]  entrypoint.child: starting child process: args=["/cnb/lifecycle/launcher"] cmd=/cnb/lifecycle/launcher2022-11-17T16:39:18.009Z D60TJX: [INFO]  entrypoint: entrypoint starting: deployment_id=01GJ37ASH915E5QZ810KPVDW7P instance_id=01GJ37AZE227NJSFJQ5JD60TJX args=["/cnb/lifecycle/launcher"]2022-11-17T16:39:18.009Z D60TJX: [INFO]  entrypoint: entrypoint version: full_string="v0.10.0-575-gad4b66798 (ad4b66798+CHANGES)" version=v0.10.0-575-gad4b66798 prerelease="" metadata=""revision=ad4b66798+CHANGES2022-11-17T16:39:18.009Z D60TJX: [INFO]  entrypoint: server version info: version=v0.10.0 api_min=1 api_current=1 entrypoint_min=1 entrypoint_current=1 auth_method=token2022-11-17T16:39:18.707Z D60TJX: [INFO]  entrypoint.config.watcher: env vars changed, sending new child command2022-11-17T16:39:18.710Z D60TJX: [INFO]  entrypoint.child: starting child process: args=["/cnb/lifecycle/launcher"] cmd=/cnb/lifecycle/launcher2022-11-17T16:41:20.180Z D60TJX: [INFO]  entrypoint.exec: starting exec stream: index=4 args=["env"]2022-11-17T16:41:20.185Z D60TJX: [INFO]  entrypoint.exec: pty requested, allocating a pty: index=42022-11-17T16:41:20.210Z D60TJX: [INFO]  entrypoint.exec: exec stream exited: index=4 code=02022-11-17T16:41:20.213Z D60TJX: [INFO]  entrypoint.exec: exec stream ended by client: index=42022-11-17T16:41:36.491Z D60TJX: 2022/11/17 16:41:36 Successfully connected to the database! :)

Now that the application is up and running and connected to the database, you can iterate on the application confidently knowing that it will benefit from the enhanced security provided by Vault dynamic credentials. This will be the case regardless of the platform the app is deployed to, whether it is Nomad, Kubernetes, Amazon Elastic Container Service (ECS), or any other platform. Give this a try using Waypoint 0.10.4 with your own app and platform.

»More Things to Do With Waypoint

HCP Waypoint, running on the HashiCorp Cloud Platform, is currently in its public beta, and you can sign up for free to give this and other Waypoint features a try, without needing to set up your own server. To learn more about how Waypoint uses runners to connect your application’s infrastructure to the HCP Waypoint server, see the Using HCP Waypoint with Your Infrastructure developer guide.

Additional reading:

HCP