Data bags & encrypted secrets

Shared data and secrets, encrypted.

A data bag is a store of JSON data on the Chef server, global and independent of any cookbook — users, application settings, lists of things recipes need. Recipes look up data-bag items at converge time, which keeps shared data out of cookbook code. But plain data bags are stored and transmitted readable, so for secrets you use encrypted data bags (or the Chef Vault pattern) rather than a plain bag.

# create an encrypted data bag item with a shared secret key
$ knife data bag create secrets db --secret-file /etc/chef/encrypted_data_bag_secret
{ "id": "db", "password": "S3cr3t!" }   # stored encrypted on the server

# decrypt at converge time using the node's secret key
db = data_bag_item('secrets', 'db', IO.read('/etc/chef/encrypted_data_bag_secret'))
template '/etc/app/db.conf' do
  variables(password: db['password'])
  sensitive true            # keep the value out of the run log
end

Encrypted bags vs Chef Vault

Encrypted data bags use one shared secret key that every node needing the data must have — simple, but distributing and rotating that shared key is the weak point. Chef Vault improves on this by encrypting each item to the specific nodes’ client keys, so only authorized nodes can decrypt and there is no shared secret file to distribute. For real secret management at scale, many shops instead integrate an external store like HashiCorp Vault.