Products & Technology

Managing OpenAI API keys with HashiCorp Vault's dynamic secrets plugin

Secure AI credentials using dynamic, short-lived tokens that automatically expire.

Jul 03 2025Ricardo Oliveira

Every CIO is pushing their organization to "do something with AI," which means developers across companies are experimenting with OpenAI integrations. However, there's no scalable way to manage these API keys, and security practices are suffering as a result.

If you're integrating OpenAI APIs into your applications, you're probably managing API keys the same way most teams do: creating static keys, copying them across environments, and hoping they don't get leaked. This approach creates real security and operational problems that only get worse as you scale.

Static API keys mean:

Security risk: Leaked credentials can be used indefinitely
Operational overhead: Manual key rotation across multiple services (diverting engineering away from goals)
Compliance gaps: No audit trail of credential usage
Development friction: Teams avoid AI features because key management is complex

The solution is to stop using permanent API keys. Instead, you generate fresh, temporary credentials on demand that automatically expire after a set time. By using dynamic secrets management, your applications get new API keys whenever they need them, use them for their work, and then the keys disappear automatically.

This blog post will explain the challenges with traditional API key management, show how Vault dynamic secrets solve these problems, and take you through a proof-of-concept workflow for dynamically generating and managing OpenAI API keys.

Note: This tutorial uses an unofficial, experimental Vault plugin for OpenAI that I developed myself. This plugin is not recommended for production use.

»Challenge: Static API key management doesn't scale

Most organizations handle OpenAI API keys like they handled database passwords a decade ago. Here's the typical pattern:

Generate a static API key in the OpenAI dashboard
Copy it to multiple applications and environments
Store it in config files or environment variables
Hope nothing goes wrong for months at a time

This creates several problems:

Security issues:

Keys live indefinitely until manually rotated
Same key often shared across multiple applications
No way to quickly revoke access if a compromise is suspected
Difficult to track which applications use which keys

Operational complexity:

Rotating keys requires coordination across teams and deployments
No automated lifecycle management
Manual processes are error-prone and often skipped
Incident response is slow when you can't quickly identify credential usage

Compliance concerns:

No guarantee that API keys are regularly rotated
Inability to enforce maximum credential lifetimes
Difficulty proving to auditors that keys don't live indefinitely

These challenges become critical as organizations scale their AI initiatives and need to manage credentials across several services.

»Solution: Dynamic secrets with HashiCorp Vault

Vault's OpenAI dynamic secrets plugin changes the fundamental approach to credential management. Instead of static, long-lived keys, it generates fresh credentials on demand with automatic expiration.

»How dynamic secrets work

Application requests access: Makes an API call to Vault
Vault generates fresh OpenAI API key: Creates new service account with defined time-to-live (TTL)
Application uses temporary credentials: Normal OpenAI API operations
Credentials automatically expire: No manual cleanup required

»Key benefits

Improved security:

Credentials are short-lived (configurable from seconds to hours)
Each request gets a unique API key
Automatic revocation eliminates "forgotten" credentials
Complete audit trail of all credential generation and usage

Operational simplification:

No manual key rotation processes, saving significant time throughout engineering
Centralized access control through Vault policies
Automated credential lifecycle management
Clear visibility into which services are accessing AI APIs

Compliance enablement:

Consistently enforced credential rotation through automatic expiration
Configurable maximum TTL ensures keys never exceed policy limits
Built-in audit trail proves credentials are properly managed

»Why HashiCorp Vault?

Vault is an industry-standard secrets management platform with proven scalability and security in enterprise environments. The OpenAI plugin extends Vault's dynamic secrets capabilities specifically for AI workloads.

Vault provides

Battle-tested secrets management platform capabilities
Rich policy and authentication systems
High availability and disaster recovery options
Integration with existing infrastructure (Kubernetes, cloud platforms, CI/CD)

»Implementation guide

This tutorial shows how to implement dynamic OpenAI secrets management with Vault, using the Dockerised version of the the experimental Vault plugin for OpenAI.

»Prerequisites

HashiCorp Vault cluster (or Vault in development mode)
OpenAI organization with admin API key access
OpenAI project(s) where you want to manage service accounts

»Step 1: Set up the Vault environment

For development testing:

# Start Vault in dev mode
vault server -dev -dev-plugin-dir=./bin
export VAULT_ADDR='http://127.0.0.1:8200'
export VAULT_TOKEN='root'

»Step 2: Install and enable the plugin

# Download the plugin
docker pull ghcr.io/gitrgoliveira/vault-plugin-secrets-openai:0.0.2
 
# Register the plugin with Vault
vault plugin register -sha256=$(docker images --no-trunc --format="{{ .ID }}" vault-plugin-secrets-openai:0.0.2 | cut -d: -f2) \
  secret vault-plugin-secrets-openai
 
# Enable the secrets engine
vault secrets enable -path=openai vault-plugin-secrets-openai

»Step 3: Configure the plugin

# Configure with your OpenAI admin API key
vault write openai/config admin_api_key="$OPENAI_ADMIN_API_KEY" \
admin_api_key_id="$ADMIN_API_KEY_ID" \
organization_id="$OPENAI_ORG_ID" \
rotation_period="720h"
 
vault read openai/config
Key                           Value
---                           -----
admin_api_key_id              key_OInm3Qed3kNn4BUQ
api_endpoint                  https://api.openai.com/v1
disable_automated_rotation    false
last_rotated                  2025-06-09T22:58:19+01:00
organization_id               org-gAZ0NbaPX8FD2YcdLsHiKx8v
rotation_period               720h
rotation_schedule             n/a
rotation_window               0

The admin API key is used by Vault to create and manage service accounts in your OpenAI organization. The rotation period determines how often this root credential is automatically rotated. See all supported parameters here.

You can also use vault write -force openai/config/rotate to force the rotation.

»Step 4: Create roles

Roles define the permissions and TTL for credentials generated for specific applications:

# Create a role for your application
vault write openai/roles/my-application project_id="$OPENAI_TEST_PROJECT_ID" \
service_account_name_template="vault-{{.RoleName}}-{{.RandomSuffix}}" \
      ttl=”1h” max_ttl=”24h”

This creates a role that:

Generates service accounts in the specified OpenAI project
Issues credentials with a 1-hour default TTL
Allows credentials to be renewed up to 24 hours maximum

»Step 5: Generate dynamic credentials

Your applications can now request temporary credentials:

# Request credentials for the role
vault read openai/creds/my-application

Response:

Key                   Value
---                   -----
lease_id              openai/creds/my-application/h8iFG6HcoB5tmoXBOyqaz1bN
lease_duration        5s
lease_renewable       false
api_key               sk-svcacct-Syp4IfuKMz_**********__IOT6KycA
api_key_id            key_aA6FHEzhf6PRLBNm
service_account       vault-my-application-ggrj6h8i
service_account_id    user-uMA0WQaDxfXAvNF9psw5PCQW

»Step 6: Application integration

Modify your applications to request credentials from Vault instead of using static keys. The exact implementation depends on your programming language and infrastructure setup.

Integration resources:

Kubernetes integration using Vault Agent for pod-based credential injection or Vault Secrets Operator
CI/CD and other application integration patterns for Jenkins, GitHub Actions, Java, C# .NET, and several others
Vault client libraries, including official SDKs for Python, Go, Java, and more
Integration tools for no-code or low-touch integrations

»Next steps

Now that you understand how dynamic secrets work for OpenAI, here's what to do next:

Start with a pilot: Experiment with this blog’s plugin in a development environment with one application
Measure the impact: Track credential lifecycle, operational overhead, and application behavior
Scale gradually: Do your own security reviews of the experimental workflow, then make your own modifications and roll out to additional applications and environments if it passes review.
Consider automation: Integrate your OpenAI application onboarding workflow with custom Vault configuration

Dynamic secrets management transforms how you handle AI credentials. The initial setup investment pays dividends in improved security, reduced operational overhead, and better compliance posture as you scale your AI initiatives.

Artificial Intelligence Solution Engineering