HashiCorp Vault 0.5

HashiCorp Vault 0.5

Feb 10 2016 Jeff Mitchell

We are proud to announce the release of Vault 0.5. Vault is a tool for managing secrets. From API keys and encrypting sensitive data to being a complete internal CA, Vault is meant to be a solution for all secret management needs.

This release focuses on enhancing Vault's core significantly, adding some long-awaited features while also making some huge improvements to existing systems. Feature highlights include:

Please see the full Vault 0.5 CHANGELOG for more details and the huge list of improvements. There are more breaking changes than usual in this version, so please be sure to read the upgrade information at the end of this post.

As always, a big thanks to our community for their ideas, bug reports, and pull requests.

Read on to learn more about the major new features in Vault 0.5.

Listing

Vault now supports list operations via the API and the new vault list command. This is implemented in generic and cubbyhole and a few other places (noted in the API documentation where supported).

For generic and cubbyhole, listing is available for directories only, e.g. vault list secret/ (when using the CLI, a trailing slash will be automatically appended for you, but users of the API will need to ensure that they are formatting their paths correctly.) A successful list command will display both keys and directories under the given path; directories will end with /.

API users can trigger a list operation either by using LIST as the HTTP verb or by appending the query parameter list=true to your request.

All HashiCorp-maintained API libraries have already added support for listing.

Fine-Grained Access Control

In past releases of Vault, policies consisted of paths mapped to one of a set of four values (somewhat confusingly also called the "policy" of each path statement): deny (the default), read, write, and sudo. In this system, write included read and sudo included read and write.

The common problem encountered was users having too much privilege. For example, imagine that you have an application ingressing customer data and writing it to a location in a generic backend mount. In order to write the data to the appropriate path, the application would need write permission:

path "custdata/*" { policy = "write" }

However, this also implied read permission, so a hijack of this Internet-facing application and access to its Vault token could allow extraction of customer data.

In Vault 0.5, these four values are replaced by a set of capabilities that are distinct and can be assigned independently: create, read, update, delete, list, sudo, and deny

ha_backend "consul" { ... }

In this scenario, encrypted Vault data will be stored in S3, but the HA features required for active/standby coordination will make use of Consul's native lock support.

Rekey Nonces, Unseal Key Archiving, and Keybase Support

Rekey attempts now generate nonces when initiated. The value of the nonce is chosen by Vault and communicated to the initiator of the rekey attempt, who then shares the nonce value with the other unseal key holders to provide along with their unseal key. This ensures that the attempt cannot be reset by a third party without knowledge of the unseal key holders; although the utility of such an attack is minimal, it could lead to a denial of service. (Our thanks to Josh Snyder for bringing this to our attention!)

Additionally, you can now have the generated keys archived in your backend storage for disaster recovery purposes. This archive value is outside Vault's cryptographic barrier; as such, this option is only available when the generated keys themselves are covered via PGP.

Since we believe all users should use PGP to protect the rekey process, PGP keys themselves just got a lot easier to use: instead of a file name in the list of PGP keys, you can now pass in keybase:<user> to have the user's public PGP key fetched from Keybase on-the-fly.

A demonstration of all of these features follows.

Key 1 fingerprint: 0f801f518ec853daff611e836528efcac6caa3db; value: c1c04c037eebc06b7a152d5501080092361a5396cfb4f7463dcb4a8322b4ff07dc50d1c81b4b587e71a2165d8d2124cc38ac63a80b9e5502c050df352c04981ca7dc47210150815a27ef3ea37c456375d6a4df78e6a4596e1e9f12abed52897555c1f9f3799cee85163a7f421c5899a4d6f772c9f990b86dc67e031de7192e7f4446dc5b05690b7630d98dd5d4b340d5efd9a202426c53460df677cf1507f662c785b2736d09942aebe507218d249e6a5bec2030ecd28b57103b101c883089f87f19fc8ee14d549eb5be5cbfc5217ebfaa671a096fee9110b7a67b298c310c88642e4d74e7a28fc0e3fd8685f5fe1d55f04339cdd7a33458442d0257557dcd99753a1b711b11b97dce0042b73a4e3ad2e001e4619bbb943b1f51b3e38248a92aec1a96e1c5fee0cce056e1663ae0e5e2dfbb4160e068e6fee01ca6301975d04c453657587d51e262ac18142fabb3eec73b1439f742e12f471a66b8cfceb04363fbba5ecc699e1030ad34cb761b567bba781802e6da111de096e4187cd3cf89cb95c2deeb3490916262c0e28f4cacf3e1859400

Operation nonce: fb2b9109-6d1f-f115-db66-39c3a4dcc705

The encrypted unseal keys have been backed up to "core/unseal-keys-backup" in your physical backend. It is your responsibility to remove these if and when desired.

At this point the encrypted key is displayed for us (along with the key fingerprint, for identification), but the command also notes that the encrypted unseal key has been stored in the physical backend at core/unseal-keys-backup. This can serve as a break-glass method of retrieving the key if it is lost, and even with the PGP encryption, you should ensure that you have proper protections in place around your data store.

For those with correct permissions within Vault, the value can be retrieved and deleted via the API:

$ vault rekey -delete Stored keys deleted.

$ vault rekey -retrieve Error retrieving stored keys: Error making API request.

URL: GET http://127.0.0.1:8200/v1/sys/rekey/backup Code: 400. Errors:

  • no backed-up keys found

    Root Token Generation

    Another new feature in Vault 0.5 is the ability to generate new root tokens via unseal keys. This helps with disaster recovery, for instance if the original root token was lost and other tokens do not have enough privileges to perform needed operations. It also allows existing root tokens to be revoked for security, only generating a new one as needed and with full agreement from unseal key holders.

    There are two ways that the new root token value can be protected. One is using a PGP key that can encrypt to any user's identity, similar to rekeying (and also similar to rekeying, root token generation uses a nonce to ensure that unseal key holders are providing keys to the same attempt). The other, which will be demonstrated below, uses a One Time Pad (OTP) of a specific length to protect the token. In this scenario, since only the attempt initiator knows the OTP, the attempt initiator is the receiver of the root token and should be given the encoded value from the last unseal key provider.

    The OTP is a set of 16 bytes encoded as base 64. Vault contains a helper method to generate a suitable, random OTP:

Encoded root token: 66jYpbXVwS4kKiTsa02b5g==

To get the final token back, we can use the -decode flag to generate-root:

$ vault generate-root -decode="66jYpbXVwS4kKiTsa02b5g==" -otp="o0HqCGchJ0AGL9RmIx2Fnw==" Root token: 48e932ad-d2f4-e66e-2205-f08a48501e79

Finally, we can verify that this is a real root token:

$ VAULT_TOKEN=48e932ad-d2f4-e66e-2205-f08a48501e79 vault token-lookup Key Value creation_time 1.454615413e+09 creation_ttl 0 display_name root id 48e932ad-d2f4-e66e-2205-f08a48501e79 meta num_uses 0 orphan true path auth/token/root policies [root] ttl 0

Other Highlights

There are too many new features and improvements in this release to describe all of them in depth, so a few more are covered below in brief:

  • Lease Duration Handling Across Backends, and Grace Periods: The various backends now honor the default/max lease TTLs set on the mount or system if not overridden by a configuration within the backend itself. This provides for much more consistent management of durations, but also introduces behavioral changes. In addition, grace periods have been removed in order to reduce confusion and remove discrepancies across backends in the true duration of a lease. Please see the upgrade details for information.

  • DynamoDB Physical Backend: a new community-supported backend that supports High Availability using Amazon's DynamoDB for storage

  • PostgreSQL Physical Backend: a new community-supported backend using PostgreSQL for storage

  • STS Support in the AWS Secret Backend: The aws secret backend can now issue STS tokens for the configured IAM role, rather than creating and deleting new IAM roles for requesting clients

  • Speedups in the Transit Backend: The transit backend has gained a cache, and now loads only the current set of keys (e.g. from the min_decryption_version to the current key version) into its working set. This provides large speedups and potential memory savings when the rotate feature of the backend is used heavily.

Important Build Note

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×