Managing OpenAI API keys with HashiCorp Vault's dynamic secrets plugin

Every CIO is pushing their organization to "do something with AI," which means developers across companies are experimenting with OpenAI integrations. However, there's no scalable way to manage these API keys, and security practices are suffering as a result.

If you're integrating OpenAI APIs into your applications, you're probably managing API keys the same way most teams do: creating static keys, copying them across environments, and hoping they don't get leaked. This approach creates real security and operational problems that only get worse as you scale.

Static API keys mean:

• Security risk: Leaked credentials can be used indefinitely
• Operational overhead: Manual key rotation across multiple services (diverting engineering away from goals)
• Compliance gaps: No audit trail of credential usage
• Development friction: Teams avoid AI features because key management is complex

The solution is to stop using permanent API keys. Instead, you generate fresh, temporary credentials on demand that automatically expire after a set time. By using dynamic secrets management, your applications get new API keys whenever they need them, use them for their work, and then the keys disappear automatically.

This blog post will explain the challenges with traditional API key management, show how Vault dynamic secrets solve these problems, and take you through a proof-of-concept workflow for dynamically generating and managing OpenAI API keys.

Note: This tutorial uses an unofficial, experimental Vault plugin for OpenAI that I developed myself. This plugin is not recommended for production use.

»Challenge: Static API key management doesn't scale

Most organizations handle OpenAI API keys like they handled database passwords a decade ago. Here's the typical pattern:

1. Generate a static API key in the OpenAI dashboard
2. Copy it to multiple applications and environments
3. Store it in config files or environment variables
4. Hope nothing goes wrong for months at a time

This creates several problems:

Security issues:

• Keys live indefinitely until manually rotated
• Same key often shared across multiple applications
• No way to quickly revoke access if a compromise is suspected
• Difficult to track which applications use which keys

Operational complexity:

• Rotating keys requires coordination across teams and deployments
• No automated lifecycle management
• Manual processes are error-prone and often skipped
• Incident response is slow when you can't quickly identify credential usage

Compliance concerns:

• No guarantee that API keys are regularly rotated
• Inability to enforce maximum credential lifetimes
• Difficulty proving to auditors that keys don't live indefinitely

These challenges become critical as organizations scale their AI initiatives and need to manage credentials across several services.

»Solution: Dynamic secrets with HashiCorp Vault

Vault's OpenAI dynamic secrets plugin changes the fundamental approach to credential management. Instead of static, long-lived keys, it generates fresh credentials on demand with automatic expiration.

»How dynamic secrets work

1. Application requests access: Makes an API call to Vault
2. Vault generates fresh OpenAI API key: Creates new service account with defined time-to-live (TTL)
3. Application uses temporary credentials: Normal OpenAI API operations
4. Credentials automatically expire: No manual cleanup required

»Key benefits

Improved security:

• Credentials are short-lived (configurable from seconds to hours)
• Each request gets a unique API key
• Automatic revocation eliminates "forgotten" credentials
• Complete audit trail of all credential generation and usage

Operational simplification:

• No manual key rotation processes, saving significant time throughout engineering
• Centralized access control through Vault policies
• Automated credential lifecycle management
• Clear visibility into which services are accessing AI APIs

Compliance enablement:

• Consistently enforced credential rotation through automatic expiration
• Configurable maximum TTL ensures keys never exceed policy limits
• Built-in audit trail proves credentials are properly managed

»Why HashiCorp Vault?

Vault is an industry-standard secrets management platform with proven scalability and security in enterprise environments. The OpenAI plugin extends Vault's dynamic secrets capabilities specifically for AI workloads.

Vault provides

• Battle-tested secrets management platform capabilities
• Rich policy and authentication systems
• High availability and disaster recovery options
• Integration with existing infrastructure (Kubernetes, cloud platforms, CI/CD)

»Implementation guide

This tutorial shows how to implement dynamic OpenAI secrets management with Vault, using the Dockerised version of the the experimental Vault plugin for OpenAI.

»Prerequisites

• HashiCorp Vault cluster (or Vault in development mode)
• OpenAI organization with admin API key access
• OpenAI project(s) where you want to manage service accounts

»Step 1: Set up the Vault environment

For development testing:

»Step 2: Install and enable the plugin

»Step 3: Configure the plugin

The admin API key is used by Vault to create and manage service accounts in your OpenAI organization. The rotation period determines how often this root credential is automatically rotated. See all supported parameters here.

You can also use vault write -force openai/config/rotate to force the rotation.

»Step 4: Create roles

Roles define the permissions and TTL for credentials generated for specific applications:

This creates a role that:

• Generates service accounts in the specified OpenAI project
• Issues credentials with a 1-hour default TTL
• Allows credentials to be renewed up to 24 hours maximum

»Step 5: Generate dynamic credentials

Your applications can now request temporary credentials:

Response:

»Step 6: Application integration

Modify your applications to request credentials from Vault instead of using static keys. The exact implementation depends on your programming language and infrastructure setup.

Integration resources:

• Kubernetes integration using Vault Agent for pod-based credential injection or Vault Secrets Operator
• CI/CD and other application integration patterns for Jenkins, GitHub Actions, Java, C# .NET, and several others
• Vault client libraries, including official SDKs for Python, Go, Java, and more
• Integration tools for no-code or low-touch integrations

»Next steps

Now that you understand how dynamic secrets work for OpenAI, here's what to do next:

1. Start with a pilot: Experiment with this blog’s plugin in a development environment with one application
2. Measure the impact: Track credential lifecycle, operational overhead, and application behavior
3. Scale gradually: Do your own security reviews of the experimental workflow, then make your own modifications and roll out to additional applications and environments if it passes review.
4. Consider automation: Integrate your OpenAI application onboarding workflow with custom Vault configuration

Dynamic secrets management transforms how you handle AI credentials. The initial setup investment pays dividends in improved security, reduced operational overhead, and better compliance posture as you scale your AI initiatives.