Detecting & responding to leaked secrets

Org-wide scanning, honeytokens, and a revoke-first runbook.

Prevention is never perfect, so a mature program assumes secrets will leak and invests in finding them fast and responding without hesitation. Two secrets in a private repo, a token pasted into a support ticket, an access key in a Docker layer — these happen constantly. The metric that matters is time-to-revoke: how long between a secret leaving your control and it being useless. Detection at scale plus a revoke-first runbook is what keeps that number small.

Scanning at scale: pre-commit, CI, and the whole org

Secret scanning belongs at three layers. A pre-commit hook (gitleaks, trufflehog) stops most secrets before they are ever committed — the cheapest possible catch. A CI job scans every push and, critically, the full history of new branches, failing the build on a hit. And an org-wide scheduled scan sweeps every repository (and often issues, wikis, and container images) because the leak you care about most is the one that predates your controls. Layer all three: the hook catches the careless, CI catches the hook-bypassers, and the org sweep catches the past.

scan-secrets:
  image: zricethezav/gitleaks:latest
  script:
    - gitleaks detect --source . --redact --exit-code 1 --report-path gl.json
  # --redact keeps the secret itself out of CI logs (do not leak it twice).
 
# org-wide, on a schedule, across every repo + history:
$ trufflehog github --org=acme --only-verified --json | tee findings.json
# --only-verified actually tests each candidate against its provider = few false positives.

Honeytokens: detection that has no false positives

Plant credentials that are never used legitimately — a fake AWS key in an S3 bucket, a decoy database user, a canary token in a config file — and wire an alert to fire the instant anything touches them. Because no real workload ever uses a honeytoken, any use is an intruder, with essentially zero false positives. They are cheap to deploy and devastating for attackers, who cannot tell a canary from a live credential and betray their presence the moment they try to use one. Salt your repos, buckets, and internal wikis with them.

The revoke-first runbook

When a real secret leaks, the order of operations is fixed and non-negotiable: revoke before you investigate. Kill the credential first — revoke the Vault lease/token (which cascades to everything it minted), disable the cloud access key, rotate the static secret — so the attacker’s window closes immediately, and only then trace the blast radius and root cause. Reversing this order to "understand it first" is how a contained leak becomes a breach. Because dynamic secrets and lease trees make revocation a single command with a wide sweep, they turn this runbook from hours of manual rotation into seconds.