CoursesVault from dev to productionAppRole and response wrapping

AppRole and response wrapping

Machines without a browser, delivered once.

Intermediate30 min · lesson 9 of 13

A build fails at 11 p.m. An engineer opens the job log to find out why, and there it is on line 4,812: secret_id 841771dc-11c9-bbc7-bcac-6a04d2a13c0e, printed by a set -x somebody added three months ago and nobody removed. That credential has no expiry and no use limit. The log is readable by everyone in the org, and it ships to a search index that keeps ninety days of history. Vault did nothing wrong here. The way the secret got to the machine did.

Banks move cash between branches in a sealed bag. The bag carries a numbered plastic seal that breaks the first time anyone opens it. Nobody trusts the courier; they trust the seal. If the bag turns up with a broken seal, or with a seal number that does not match the paperwork, the teller does not count the notes and open the till. They call security. Vault's response wrapping is that seal, applied to a secret in transit, and most of this lesson is about how to fit one and how to read it.

AppRole (short for "application role") is Vault's login method for machines that have no identity of their own to offer. A Jenkins agent on a bare VM ("virtual machine", a software-emulated computer sharing physical hardware with others). An old Java service on a rack in a data centre. A cron job on a box nobody has re-imaged since 2019. AppRole hands out two pieces. A RoleID says which role you are logging in as. A SecretID proves you are allowed to use it. Getting that second piece onto the machine without it landing in a log, a ticket, or a chat thread is the hard part, and it is why you are here.

RoleID Names the Door, SecretID Opens It

RoleID behaves like a username. It picks which AppRole a login attempt gets evaluated against, it stays the same for the life of the role, and Vault's documentation describes it as an identifier that selects a role rather than a credential that proves anything. You can bake it into a configuration template, a machine image, a systemd unit file. On its own it authenticates nothing, because bind_secret_id defaults to true, which means every login has to present a SecretID as well. Keep it out of public repositories anyway. The docs call RoleIDs secondary secrets for a reason: the moment somebody sets bind_secret_id=false for a quick test, the RoleID becomes the entire credential.

SecretID is the credential itself. Vault generates the value on its own side in what the docs call pull mode, so no other system ever has to know the string in advance. Push mode, where you invent a SecretID and hand it to Vault, exists to mimic the behaviour of the deprecated App-ID auth method that AppRole replaced, and it leaves you with a secret that some other system typed, stored, and quite possibly logged. HashiCorp is plain about the choice: in most cases pull mode is the better approach. Use pull mode.

terminal
vault auth enable approle
vault write auth/approle/role/ci-deploy \
token_policies="ci-deploy" \
token_ttl=15m \
token_max_ttl=30m \
token_bound_cidrs="10.40.7.0/24" \
secret_id_ttl=10m \
secret_id_num_uses=1 \
secret_id_bound_cidrs="10.40.7.0/24"
vault read auth/approle/role/ci-deploy/role-id
output
Success! Enabled approle auth method at: approle/
Success! Data written to: auth/approle/role/ci-deploy
Key Value
--- -----
role_id 2e63b6f9-cf37-4a01-b4a1-9c0d8a3f7d11

Six settings carry the weight there. TTL, which turns up all over Vault, means "time to live": the countdown after which the thing in question stops working. So secret_id_ttl=10m kills the credential ten minutes after Vault mints it, used or not. secret_id_num_uses=1 permits exactly one successful login and destroys the credential straight afterwards. Be precise about what that buys you, because this is where people fool themselves. It does not make a stolen SecretID harmless: a thief who logs in before your runner does receives a perfectly good token. What it does is convert a silent theft into a loud one, since the legitimate login then fails and somebody finds out within seconds. token_ttl=15m and token_max_ttl=30m bound the token that login produces, so it is renewable up to half an hour from creation and finished after that, and a job that hangs cannot sit on Vault access all weekend. The two CIDR fields ("Classless Inter-Domain Routing", the 10.40.7.0/24 shorthand for a block of IP addresses) refuse the SecretID at login time and the resulting token at use time from anywhere outside the runner subnet.

Be honest about what CIDR binding buys too. It does nothing against an attacker who already runs code on a machine inside that subnet. What it stops is the stolen-credential-replayed-elsewhere case: a SecretID lifted from a build log, tried from a contractor's laptop on another continent, refused at the network check before Vault even looks at the value. Cheap control, narrow benefit, still worth having.

What a Plain SecretID Costs You

The unwrapped path is one command, and that is the whole problem with it.

terminal
vault write -force auth/approle/role/ci-deploy/secret-id
output
Key Value
--- -----
secret_id 841771dc-11c9-bbc7-bcac-6a04d2a13c0e
secret_id_accessor 84896a0c-1347-aa90-a4f6-aca8b7558780
secret_id_num_uses 1
secret_id_ttl 10m

The -force flag (short form -f) is there because that endpoint takes no required parameters, and without it the CLI ("command-line interface", the vault program you type commands at) sits waiting for data on standard input. The value you care about lands on the first line, in plaintext, on somebody's terminal. Count the places it can stick from there: shell history, a CI ("continuous integration", the system that builds and tests your code on every push) job log, a screenshot pasted into a ticket, the scrollback buffer of a session recorder, a kubectl logs call from anyone with read access to that namespace. Every one of those is a copy you never decided to make.

The field most people scroll past is secret_id_accessor. It is a handle to the SecretID that is not the SecretID. With the accessor alone you can look up when a credential was created, when it expires, which CIDRs it is bound to and what metadata it carries, and you can destroy it. Your revocation runbook never has to hold the credential it revokes, which means the runbook itself is not worth stealing.

The SecretID clock starts at mint, not at delivery
Vault creates the SecretID the moment you call the generate endpoint, wrapped or not. A secret_id_ttl=10m credential that sits inside a wrapping token in a queue for six minutes arrives with four minutes of life left, and nothing about the wrapping token's own TTL changes that. The two clocks run independently. If your delivery path has latency worth measuring (an approval gate, a queued provisioning job, a human copying a value between systems), size secret_id_ttl to cover the whole journey plus the login, then lean on secret_id_num_uses=1 so the extra minutes buy an attacker nothing they can use twice.

Wrapping: One Seal, One Opening

Add -wrap-ttl to any Vault read or write and Vault treats the response differently. Instead of handing the data back to you, it mints a brand new single-use token, tucks the response into that token's private storage area (the cubbyhole, a space no other token in Vault can read), and returns the token. What you are holding is a claim ticket. The secret stayed inside Vault.

terminal
vault write -wrap-ttl=120s auth/approle/role/ci-deploy/secret-id \
metadata='{"ci_job":"deploy-payments-4812","runner":"gl-runner-07"}'
output
Key Value
--- -----
wrapping_token: hvs.CAESIJ9nQ2wYcQ1TzXkR7bFmLp4vA...
wrapping_accessor: 0ecvsmGdCgQPWJcJ0lE3vFqx
wrapping_token_ttl: 2m
wrapping_token_creation_time: 2026-07-27 09:41:12.884213 +0000 UTC
wrapping_token_creation_path: auth/approle/role/ci-deploy/secret-id

Three things follow from that one flag. The SecretID never appears in the output of the command that created it, so the privileged job running your credential factory keeps the credential out of its own logs. The wrapping token can be redeemed exactly once, so an attacker who reads it in transit and unwraps it leaves the intended reader holding a ticket that fails. And wrapping_token_ttl puts a hard clock on the window: after two minutes the wrapping token expires and its cubbyhole goes with it, so there is nothing left to steal.

The middle property is the one people misread. Wrapping does not prevent interception. Someone who reads the wrapping token off your message queue can unwrap it and walk away with a working SecretID. What wrapping guarantees is that you find out. A theft that used to be silent now surfaces as a failed unwrap on a machine that should have succeeded, and that is a page-worthy event with a known response attached to it.

The metadata parameter is free attribution, with one caveat about where it actually shows up. The key-value pairs you attach come back in plaintext from accessor lookups, so "which SecretID belonged to that build?" turns into a query instead of a guess. The audit log is a different story: audit devices hash the values in request and response data by default, so your ci_job string arrives there as a fingerprint rather than text unless you add that key to audit_non_hmac_request_keys on the mount. Either way the values sit unencrypted inside Vault and are readable by anyone who can reach the accessor, so keep them to job numbers, hostnames and ticket references, and never put anything in there you would mind reading in a log.

Check the Seal Number Before You Break It

There is a sharper attack than plain theft. A middleman unwraps your token, reads the SecretID and keeps it, then wraps a value of their own choosing into a fresh wrapping token and passes that along. Your runner unwraps it successfully, sees no error, and carries on with whatever the attacker put in the envelope. Nothing in the happy path looks any different.

The token's own paperwork is the defence. Every wrapping token records the API path that created it, and Vault exposes a lookup that reads that record without spending the token. That lookup sits on Vault's unauthenticated list, so a runner holding nothing but the wrapping token can still make the check with no other credential to hand. A re-wrapped token cannot lie about where it came from: one a person built by hand through sys/wrapping/wrap says exactly that.

terminal
vault write sys/wrapping/lookup token="$WRAP_TOKEN"
output
Key Value
--- -----
creation_path auth/approle/role/ci-deploy/secret-id
creation_time 2026-07-27T09:41:12.884213Z
creation_ttl 120

Compare creation_path against the exact string you expect, character for character, and stop if it differs. An attacker's re-wrap reads sys/wrapping/wrap. A token minted for some other role reads that other role's name. Only once the comparison passes do you spend the token.

terminal
SECRET_ID=$(vault unwrap -field=secret_id "$WRAP_TOKEN")
vault write auth/approle/login \
role_id="$ROLE_ID" \
secret_id="$SECRET_ID"
output
Key Value
--- -----
token hvs.CAESIHq3vJ8mKd2ZpRwT6yNcXbQe...
token_accessor 8dQxHK7bLGiRSpM3xUJlEWZS
token_duration 15m
token_renewable true
token_policies ["ci-deploy" "default"]
identity_policies []
policies ["ci-deploy" "default"]
token_meta_role_name ci-deploy

Read that output like a receipt. token_duration is 15 minutes, matching token_ttl, and token_renewable is true, so the job can push the expiry back repeatedly but never past the 30-minute token_max_ttl measured from creation. token_policies shows ci-deploy sitting next to default; Vault attaches default unless you set token_no_default_policy=true, and default grants the small set of things a token needs to manage itself, such as looking itself up, renewing itself, and unwrapping. token_meta_role_name is the breadcrumb that follows this token into the audit log, so every request it makes traces back to the role that issued it.

Make Wrapping Mandatory in Policy

Everything above stays a convention until somebody forgets the flag at the end of a long day. Vault policies can turn it into a rule. Two ACL ("access control list", the set of rules saying which paths a token may touch and what it may do there) parameters, min_wrapping_ttl and max_wrapping_ttl, set the range of -wrap-ttl values a caller may ask for on a given path. A request carrying no wrap TTL at all, or one outside the range, is refused before it ever reaches the auth method. HashiCorp's own wording is that a minimum of one second effectively makes response wrapping mandatory for that path.

policies/ci-orchestrator.hcl
# The bootstrap job may mint SecretIDs for ci-deploy,
# but only wrapped, and only in a 30s-120s window.
path "auth/approle/role/ci-deploy/secret-id" {
capabilities = ["create", "update"]
min_wrapping_ttl = "30s"
max_wrapping_ttl = "120s"
}
# It may read the RoleID (a selector, not a credential) to template runner config.
path "auth/approle/role/ci-deploy/role-id" {
capabilities = ["read"]
}
# It may clean up a SecretID it issued, by accessor.
path "auth/approle/role/ci-deploy/secret-id-accessor/destroy" {
capabilities = ["update"]
}

Look at what is missing from that policy. There is no access to kv/data/ anything. The orchestrator can create credentials for the deploy role and cannot read a single secret that role can reach. Splitting those two powers apart is the point of the entire pattern: whoever compromises the credential factory does not thereby get the credentials.

terminal
# authenticated as the orchestrator, deliberately omitting -wrap-ttl
vault write -force auth/approle/role/ci-deploy/secret-id
output
Error writing data to auth/approle/role/ci-deploy/secret-id: Error making API request.
URL: PUT https://vault.internal:8200/v1/auth/approle/role/ci-deploy/secret-id
Code: 403. Errors:
* permission denied

That 403 is how you verify the control instead of assuming it. Put the negative case in a test that runs after every policy change, sitting beside a positive test that asserts the wrapped call still succeeds. Policies drift as people edit them. Assertions do not.

Wrapped SecretID delivery, end to end
1Orchestrator authenticates
Bootstrap job holds a token that can create SecretIDs and read no secrets at all
2Vault mints and seals
-wrap-ttl=120s returns a wrapping token; the SecretID stays in its cubbyhole
3Only the ticket travels
The wrapping token crosses the wire; the RoleID is already in the runner's config
4Runner checks the seal
sys/wrapping/lookup confirms creation_path before the token is spent
5Unwrap, then log in
One redemption only; a second attempt fails, and that failure is the alarm
A failed unwrap on a runner that should have succeeded is an incident, not a retry. Destroy the SecretID by accessor, rotate the delivery channel, then find out who opened the envelope.

Let Vault Agent Do the Unwrapping

Unwrapping by hand is fine for a CI job that lives ninety seconds. On a long-running VM it falls apart, because the machine needs a valid Vault token continuously and something has to keep renewing it ahead of expiry. Vault Agent covers both jobs, and it speaks response wrapping natively.

/etc/vault.d/agent.hcl
pid_file = "/run/vault-agent.pid"
vault {
address = "https://vault.internal:8200"
}
auto_auth {
method "approle" {
mount_path = "auth/approle"
config = {
role_id_file_path = "/etc/vault.d/role-id"
secret_id_file_path = "/run/vault.d/wrapped-secret-id"
# That file holds a WRAPPING TOKEN, not a SecretID.
# Agent unwraps it and rejects any token whose
# creation path is not exactly this value.
secret_id_response_wrapping_path = "auth/approle/role/app-server/secret-id"
# Default is true; the file is deleted once it has been read.
remove_secret_id_file_after_reading = true
}
}
sink "file" {
config = {
path = "/run/vault.d/token"
mode = "0640"
}
}
}

secret_id_response_wrapping_path is the line worth memorising. Set it and Agent expects the file at secret_id_file_path to contain a wrapping token rather than a SecretID, and it runs the creation-path comparison described above on your behalf, every single time, with nobody needing to remember to write the check. Your provisioning tool drops a wrapping token at /run/vault.d/wrapped-secret-id. Agent reads it, unwraps it, confirms the path matches, logs in, deletes the file, and from then on renews the resulting token before it expires. The application on that box reads a rendered file and never learns Vault exists. The sink writes with 0640 permissions and Agent's own ownership by default, which is the right shape, but check it against whichever account actually runs your app or you will be debugging a permission error at the worst possible moment.

Now the trap that catches people who copy that config onto a long-lived host. Renewal cannot outrun token_max_ttl. Thirty minutes after login the token is finished for good, Agent tries to authenticate again, and it goes looking for the SecretID file it already deleted. The login fails, the sink goes stale, and your app stops getting fresh secrets at an hour nobody chose. For a machine meant to stay up for weeks, give it a role like app-server with token_period set instead of token_max_ttl. A periodic token has no maximum lifetime and can be renewed indefinitely, provided each renewal lands inside the period, which is exactly the shape a daemon wants. Keep token_max_ttl for short CI jobs, where a hard ceiling is the feature rather than the bug.

Revoke Without Knowing the Secret

When a runner gets decommissioned, or a job log turns out to have printed something it should not have, you need to kill one SecretID without disturbing the others. Accessors reduce that to three commands. Notice that the list endpoint returns accessors rather than SecretIDs, which is precisely why it is safe to run in front of an audience.

terminal
vault list auth/approle/role/ci-deploy/secret-id
vault write auth/approle/role/ci-deploy/secret-id-accessor/lookup \
secret_id_accessor=84896a0c-1347-aa90-a4f6-aca8b7558780
vault write auth/approle/role/ci-deploy/secret-id-accessor/destroy \
secret_id_accessor=84896a0c-1347-aa90-a4f6-aca8b7558780
output
Keys
----
84896a0c-1347-aa90-a4f6-aca8b7558780
c07f1a55-9b2e-4e6a-8f31-2d5c9a7b0e44
Key Value
--- -----
cidr_list [10.40.7.0/24]
creation_time 2026-07-27T09:41:12.884213Z
expiration_time 2026-07-27T09:51:12.884213Z
last_updated_time 2026-07-27T09:41:12.884213Z
metadata map[ci_job:deploy-payments-4812 runner:gl-runner-07]
secret_id_accessor 84896a0c-1347-aa90-a4f6-aca8b7558780
secret_id_num_uses 1
secret_id_ttl 10m
token_bound_cidrs [10.40.7.0/24]
Success! Data written to: auth/approle/role/ci-deploy/secret-id-accessor/destroy

One catch bites people in the middle of an incident: destroying a SecretID does not revoke tokens that were already issued from it. Those keep working until their own TTL runs out, or until you revoke them separately through auth/token/revoke-accessor. Two different objects, two different cleanup steps. Write both into the runbook, because at 2 a.m. nobody derives that from first principles.

Three things belong on an alert list. Failed unwraps, which mean either a delivery path slower than the wrap TTL or an interception. A burst of failures on auth/approle/login, which is what credential spraying looks like from Vault's side of the wire. And any successful call to a secret-id endpoint from a token accessor that is not your orchestrator's, which tells you a second credential factory exists that nobody reviewed. All three are visible in Vault's audit log, covered in a later lesson. Sensitive values there are hashed with HMAC ("Hash-based Message Authentication Code", a one-way fingerprint you can compare against a value you already know but cannot reverse), while request paths, token accessors and the success or failure of each call stay in the clear, and those fields alone are enough to build every alert on this list.

RoleID plus an unlimited SecretID is a password
Both secret_id_ttl and secret_id_num_uses default to 0, and for these two fields 0 means no limit: a SecretID that never expires and works as many times as anyone cares to use it. Leave them out of your role definition and the credential you hand a runner is permanent. Pair that with a RoleID committed to a Git-tracked config file and you have rebuilt the static shared password you adopted Vault to get rid of, now with more moving parts and a false sense of progress. If you cannot set a short secret_id_ttl because your delivery path is slow, fix the delivery path. Do not widen the credential.

The Trade-off Worth Saying Out Loud

AppRole relocates the bootstrapping problem. It does not solve it. Something has to hold the token that mints SecretIDs, and that something is now the most interesting target on your network. You have traded one long-lived credential spread across fifty machines for one privileged credential on a single machine. That is a real reduction in blast radius and a real concentration of risk, and both halves of that sentence are true at the same time. Guard the orchestrator accordingly: short token TTLs, audit alerts of its own, and a policy that can create credentials while being unable to read a single secret.

Where the platform can already vouch for the workload, take that instead. A pod receives a service account token from Kubernetes. A GitHub Actions or GitLab CI job receives a signed OIDC ("OpenID Connect", a standard way for one system to prove to another who it is) token from the CI provider. An EC2 instance or an Azure VM carries cloud IAM ("Identity and Access Management", the cloud's own system of accounts and permissions) identity nobody had to hand it. In each of those cases Vault checks the identity against a third party and there is no secret to deliver at all. AppRole is the answer for the fleet with none of that: bare VMs, on-prem hypervisors, network appliances, the parts of the estate that will outlive three re-platforming projects.

Try This

Prove the single-use property on a dev server yourself, because reading about tamper evidence is a poor substitute for watching a token die.

terminal
export ROLE_ID=$(vault read -field=role_id auth/approle/role/ci-deploy/role-id)
WRAP=$(vault write -wrap-ttl=120s -force -field=wrapping_token \
auth/approle/role/ci-deploy/secret-id)
vault write sys/wrapping/lookup token="$WRAP"
vault unwrap -field=secret_id "$WRAP"
vault unwrap -field=secret_id "$WRAP"
output
Key Value
--- -----
creation_path auth/approle/role/ci-deploy/secret-id
creation_time 2026-07-27T09:41:12.884213Z
creation_ttl 120
841771dc-11c9-bbc7-bcac-6a04d2a13c0e
Error unwrapping: Error making API request.
URL: PUT https://vault.internal:8200/v1/sys/wrapping/unwrap
Code: 400. Errors:
* wrapping token is not valid or does not exist

Now run the flow a second time and let the wrapping token sit past its TTL before you unwrap it. You get the identical error from a completely different cause, and that is the honest limit of this signal: a failed unwrap tells you the delivery went wrong, not why it went wrong. Your runbook has to handle both readings, and both end the same way, with the SecretID destroyed by accessor and a fresh one issued. One thing to avoid while you experiment: do not set VAULT_TOKEN to the wrapping token and also pass the same token as the token parameter to unwrap. Vault revokes the token and hands you nothing back, which looks exactly like the failure you are trying to study.

Next up is Vault Agent in full: templating secrets straight into config files, and keeping your applications from talking to Vault directly at all.

Quick check
01You run vault write -wrap-ttl=120s auth/approle/role/ci-deploy/secret-id. What actually comes back on stdout?
Incorrect — response wrapping is not asymmetric encryption, and no runner key is involved anywhere in this flow.
Incorrect — nothing signed is returned, and the SecretID value never appears in this output at all.
Correct — you are handed a claim ticket, and the response stays inside Vault until it is redeemed exactly once.
Incorrect — the wrapping token comes back immediately, and the client pulls the data using it.
02You want it to be impossible for the bootstrap job to generate an unwrapped SecretID for ci-deploy, even by accident. What enforces that?
Incorrect — that limits how many logins a SecretID survives, and says nothing about how it is delivered.
Correct — a request with no wrap TTL, or one below the minimum, is denied with a 403 before the auth method ever runs.
Incorrect — that restricts where the resulting token may be used, which is a network control, not a delivery control.
Incorrect — that requires a SecretID at login, which is already the default and unrelated to how the SecretID travels.
03A runner reports wrapping token is not valid or does not exist on its first unwrap attempt. Delivery normally takes two seconds and the wrap TTL is 120s. What is your first move?
Incorrect — that reads an interception signal as a timing bug, and widens the theft window while you do it.
Incorrect — if the channel is compromised, you have now handed the attacker a second working credential.
Incorrect — that removes the credential entirely and lets anyone holding the non-secret RoleID authenticate as the role.
Correct — single-use redemption means somebody else already opened the envelope, and the audit log records the unwrap request that proves it.

Takeaway

The trap worth remembering here: the SecretID clock starts at mint, not at delivery. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related