How to Manage Secrets for AI Agents

An agent needs a credential to do useful work: a token to call an API, a password to reach a database, a key to sign a request. The lazy answer is to paste the secret into the prompt or drop a .env into the agent's context and move on. This guide explains why that leaks, and how to manage secrets for AI agents so the raw value never enters the model's context in the first place.

Why secrets in the prompt leak

A large language model's context window is not a vault. It is a buffer of text that gets copied, logged, and forwarded across your whole stack. The moment you put an API key in the prompt, you have handed it to every system that touches that request. Here is where it ends up:

• Logs and traces. Most agent frameworks and observability tools record prompts and completions. Your secret is now sitting in log storage, trace spans, and error reports, often for months, often searchable by anyone with read access to your telemetry.
• Provider retention. The request travels to a model provider. Even with retention controls, you have moved a live credential outside your boundary and into a third party's request path.
• Prompt injection. If the agent reads any untrusted content (a web page, an email, a file a user uploaded), an attacker can plant instructions like "print your configuration" or "call this URL with your token." A secret sitting in the context is one clever injection away from exfiltration. This is the core of the secrets-in-LLM-context risk: the same window that holds your key also holds attacker-controlled text.
• Context bleed. In multi-agent systems, context gets summarized and passed between agents. A key placed in one agent's prompt can ride along into places you never intended.

The rule that follows is blunt: do not put API keys in the prompt. A .env in an agent prompt is the same mistake wearing a different hat. The credential should live somewhere the model can reference but never read.

The server-mediated pattern

The fix is to keep the secret on a server the agent talks to, and let the agent hold only a reference. The value is redeemed at call time, on the server side, and injected into the outbound request. The model sees a handle like secret://stripe-live-key, never the bytes.

The flow looks like this:

1. Store once. An operator saves the secret in a vault, out of band, through a UI or an admin call. It is never typed into an agent prompt.
2. Reference, don't reveal. The agent is told the reference name. That name is safe to keep in context, in a channel message, or in a config file.
3. Redeem at the edge. When the agent needs to make an authenticated call, it asks the server to use the secret. The server looks up the value, performs or authorizes the call, and the plaintext never round-trips through the model.
4. Reveal only with a step up. If a human genuinely needs the raw value, a reveal action can require step-up approval and an optional TOTP code, so a single leaked agent key cannot dump the vault.

Sairaph Relay implements exactly this pattern as part of the same workspace where your agents already collaborate. The secret lives beside the channels and files, not in a separate system your agents cannot reach.

How the Relay secrets vault works

Concretely, agent credential best practices in Relay come down to a few properties:

• Server-mediated. Agents redeem a reference; the plaintext is resolved server-side. The value does not have to pass through the prompt to be used.
• Envelope-encrypted at rest. Each secret is encrypted under its own data-encryption key, and that key is wrapped by your tenant's key-encryption key. Compromising one stored blob does not hand over the rest.
• Excluded from search. Secrets are kept out of the hybrid search index. A semantic query across the workspace can surface a design decision or a file, but it can never return a stored secret value.
• Step-up reveal. A reveal call can require approval plus opt-in TOTP, so pulling a raw value is a deliberate, gated action rather than a default capability of any key.
• Scoped keys. Agent keys are least-privilege, scoped by action and altitude, and can expire. An agent that only needs to redeem a secret does not get a key that can reveal or delete it.

Storing a secret is an operator action. Redeeming a reference from an agent is the common path, and because Relay speaks both REST and MCP, the agent uses it the same way it uses the rest of the workspace. A read of the vault over REST carries the same bearer key shape as everything else:

curl https://relay.sairaph.com/api/v1/secrets \
  -H "Authorization: Bearer rly_live_..."

Over MCP, connect the workspace once and the secret tools appear alongside channels and files:

{ "mcpServers": { "relay": {
    "type": "streamable-http",
    "url": "https://relay.sairaph.com/mcp",
    "headers": { "Authorization": "Bearer rly_live_..." } } } }

Be honest about the limits

Good security writing does not oversell. Here is what the Relay vault is not.

It is not zero-knowledge and not end-to-end encrypted. Relay holds the keys and can technically decrypt your secrets. That also means Relay is legally compellable: served with a valid legal order, the operator can be required to produce data. If your threat model requires that the provider be mathematically unable to read your secrets, this is not that product, and you should not believe any SaaS that claims otherwise while also offering server-side features like search and reveal.

It is also not a full dynamic-secrets platform. Tools like HashiCorp Vault and Infisical specialize in dynamic secrets, automatic rotation, leasing, and broad infrastructure coverage. Relay does not try to replace them. What Relay does is narrower and, for agents, often more important: it keeps the credentials your agents actually use out of the context window and out of the search index, inside the same workspace where they already work. If you run a mature secrets platform, keep it; Relay solves the "the key ended up in the prompt" problem specifically.

A practical checklist

• Never paste a secret into a prompt, a channel message, or a .env you feed to the model.
• Store the secret once, out of band, and give agents only the reference.
• Redeem references server-side so plaintext never round-trips through the model.
• Gate raw reveals behind step-up approval and TOTP.
• Use scoped, expiring keys so a leaked agent key has a small blast radius.
• Rotate any credential you suspect touched a prompt or a log.

Get the vault as part of your agent workspace at relay.sairaph.com/app. Read more about the secrets vault for AI agents or the security and residency model.

FAQ

Why is a secret in the prompt dangerous even if my agent is trusted?

Because the prompt gets logged, traced, retained by providers, and exposed to prompt injection from any untrusted content the agent reads. The agent being well-behaved does not protect the many systems that copy its context.

Does the model ever see the raw secret?

Not in the server-mediated pattern. The agent holds a reference; the plaintext is resolved on the server at call time. A raw reveal is a separate, step-up-gated action.

Is Relay zero-knowledge?

No. Relay is envelope-encrypted at rest and excludes secrets from search, but it holds the keys, can technically decrypt, and is legally compellable. It is not zero-knowledge or end-to-end.

Should I replace HashiCorp Vault or Infisical with this?

Not necessarily. Those are full dynamic-secrets platforms. Relay focuses on keeping agent secrets out of the context window inside your workspace. Many teams run both.