Rotate keys and secrets

Use this page to rotate the keys an instance uses and to store the secrets its connectors need. Minting, reading and revoking an admin key, rotating or retiring a signing key, and creating, rotating or deleting a connector secret each write a row to the audit log. A change you make by restarting the server writes no row. That covers a new PBAC_ADMIN_API_KEY, a new pepper and a new credential key. The server log records the first two.

Set two values. Run the blocks below in order in one shell. Each one checks the answer it gets.

export POLICYARC_URL="https://your-environment.example"
export POLICYARC_ADMIN_KEY="paste the Admin API key"

Admin keys

An Admin API call sends an admin key in the X-Admin-API-Key header. An instance holds two kinds.

The key in PBAC_ADMIN_API_KEY. At start, the authorization server stores it as the key labelled env-bootstrap.
Keys you mint through the Admin API or the Admin Portal.

Mint a key and use it

Give each caller its own key, so you can revoke one without the others. A label must be unique.

mint=$(curl -fsS -X POST "$POLICYARC_URL/admin/api/admin-keys" \
  -H "X-Admin-API-Key: $POLICYARC_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -d "{\"label\": \"ci-pipeline-$(date +%s)\"}")
echo "$mint" | jq '{view, plaintext: "(hidden)"}'
export NEW_KEY_ID=$(jq -er .view.id <<<"$mint")
export NEW_KEY=$(jq -er .plaintext <<<"$mint")
code=$(curl -sS -o /dev/null -w '%{http_code}' "$POLICYARC_URL/admin/api/admin-keys" -H "X-Admin-API-Key: $NEW_KEY")
[ "$code" = 200 ] || { echo "Expected the new key to work, got HTTP $code" >&2; false; }

The call returns HTTP 201 with the key's plaintext. A label that is already in use returns HTTP 400.

Anyone with an admin key can read every admin key

GET /admin/api/admin-keys/{id}/value returns a key's plaintext to any caller that holds a valid admin key. The authorization server stores each key encrypted, not only hashed, so that it can return it. Treat every admin key as able to read all the others. The call writes an audit row with the outcome reveal, and the row does not hold the plaintext.

curl -fsS "$POLICYARC_URL/admin/api/admin-keys/$NEW_KEY_ID/value" -H "X-Admin-API-Key: $POLICYARC_ADMIN_KEY" |
  jq -e --arg key "$NEW_KEY" '.plaintext == $key' >/dev/null

Revoke a key

Revoking takes effect on the next call. The key then gets HTTP 401.

curl -fsS -X POST "$POLICYARC_URL/admin/api/admin-keys/$NEW_KEY_ID/revoke" -H "X-Admin-API-Key: $POLICYARC_ADMIN_KEY"
code=$(curl -sS -o /dev/null -w '%{http_code}' "$POLICYARC_URL/admin/api/admin-keys" -H "X-Admin-API-Key: $NEW_KEY")
[ "$code" = 401 ] || { echo "Expected HTTP 401 for the revoked key, got $code" >&2; false; }

The revoke call returns HTTP 204. It takes an optional body {"at": "<ISO 8601 time>"} to revoke at a later time. GET /admin/api/admin-keys lists every key with its revokedAt, and never the plaintext.

Rotate the key in PBAC_ADMIN_API_KEY

Set PBAC_ADMIN_API_KEY to a new value and restart the authorization server. At start, it replaces the active env-bootstrap key with the new value. The old value then gets HTTP 401. Keys you minted are not changed. We changed the value on a self-hosted container and restarted it, and the old key got 401 and the new one got 200.

To rotate without a gap, mint a key for each caller first, move the callers to it, and then change PBAC_ADMIN_API_KEY.

A blank PBAC_ADMIN_API_KEY stops the authorization server from starting. See Harden an instance.

OIDC signing keys

The authorization server signs ID tokens with an ID_TOKEN key and software statements with a separate SOFTWARE_STATEMENT key. It publishes the public half of both at /.well-known/jwks.json. Access tokens are opaque and are not signed.

Rotate the ID token key

A rotation creates a new key and retires the old one. The retired key stays in the JWKS until PBAC_OIDC_KEY_RETIREMENT_SECONDS has passed, 86400 seconds by default, so tokens it signed still verify.

out=$(curl -fsS -X POST "$POLICYARC_URL/admin/api/oidc-signing-keys/rotate" \
  -H "X-Admin-API-Key: $POLICYARC_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"purpose": "ID_TOKEN"}')
echo "$out"
export NEW_KID=$(jq -er .newKid <<<"$out")
export OLD_KID=$(jq -er .retiredKid <<<"$out")
curl -fsS "$POLICYARC_URL/.well-known/jwks.json" |
  jq -e --arg new "$NEW_KID" --arg old "$OLD_KID" '[.keys[].kid] | index($new) != null and index($old) != null' >/dev/null

During the overlap, both key ids are in the JWKS. The response's retiredExpiresAt says when the old key leaves it. The JWKS response carries Cache-Control: max-age=300, set by PBAC_OIDC_JWKS_CACHE_SECONDS, so a client can hold the old set for that long.

The body takes purpose, ID_TOKEN or SOFTWARE_STATEMENT, and defaults to ID_TOKEN. A retired software statement key stays published for PBAC_OIDC_STATEMENT_KEY_RETIREMENT_SECONDS, 400 days by default.

End the overlap early

Retire the old key by its id once you know no client still holds a token it signed. It leaves the JWKS at once.

curl -fsS -X POST "$POLICYARC_URL/admin/api/oidc-signing-keys/$OLD_KID/retire" -H "X-Admin-API-Key: $POLICYARC_ADMIN_KEY"
curl -fsS "$POLICYARC_URL/.well-known/jwks.json" | jq -e --arg old "$OLD_KID" '[.keys[].kid] | index($old) == null' >/dev/null

The call returns HTTP 204. Retiring the only active key of a purpose returns HTTP 409, because nothing would be left to sign with. Rotate it instead.

For a suspected key compromise, send {"purpose": "ID_TOKEN", "immediate": true} to the rotate call. The old key leaves the JWKS at once, and ID tokens it signed stop verifying.

If the instance sets PBAC_OIDC_JWKS_PATH, it reads its signing key from that file, and the rotate and retire calls return HTTP 409. Replace the file instead.

Connector secrets

The secrets API stores values that connectors and managed resources send upstream, such as an API token. It encrypts each value and never returns it. Register a managed resource shows how a resource refers to a secret by name.

H=(-H "X-Admin-API-Key: $POLICYARC_ADMIN_KEY" -H 'Content-Type: application/json')
curl -fsS -X POST "$POLICYARC_URL/admin/api/secrets" "${H[@]}" -d '{"name": "docs_demo_token", "value": "first-value"}'
curl -fsS "$POLICYARC_URL/admin/api/secrets" "${H[@]}" |
  jq -e 'any(.[]; .name == "docs_demo_token") and all(.[]; has("value") | not)' >/dev/null
curl -fsS -X POST "$POLICYARC_URL/admin/api/secrets/docs_demo_token/rotate" "${H[@]}" -d '{"value": "second-value"}' |
  jq -e '.rotatedAt != null' >/dev/null
code=$(curl -sS -o /dev/null -w '%{http_code}' -X DELETE "$POLICYARC_URL/admin/api/secrets/docs_demo_token" "${H[@]}")
[ "$code" = 204 ] || { echo "Expected HTTP 204 from delete, got $code" >&2; false; }

Call	Result
`POST /admin/api/secrets`	HTTP 201. A name that exists returns HTTP 409.
`GET /admin/api/secrets`	Each secret's `name`, `createdAt` and `rotatedAt`. Never the value.
`POST /admin/api/secrets/{name}/rotate`	HTTP 200 with the new `rotatedAt`. An unknown name returns HTTP 400.
`DELETE /admin/api/secrets/{name}`	HTTP 204, or HTTP 404 for an unknown name

A name uses lowercase letters, digits and underscores, starts with a letter, and has at most 64 characters. A name such as jira-token returns HTTP 400.

The pepper and the credential key

These two values are set once per instance and are not part of a routine rotation. A blank value for either stops the authorization server from starting. See Harden an instance.

PBAC_ADMIN_PEPPER is mixed into the hash the authorization server uses to look up an admin key. When it changes, the authorization server rehashes every admin key at start. We changed it on a restart, and a key minted before the change still worked.

PBAC_CREDENTIAL_KEY encrypts what the instance stores at rest. GET /admin/api/credential-key/status lists these stores and counts the rows in each.

managed_resource.rs_client_secret_enc
connector_secret.value_enc
identity_provider.client_secret
admin_api_key.value_enc
oidc_signing_key.key_jwk_encrypted

It also encrypts the upstream tokens and credentials the gateway holds for each user.

Do not change PBAC_CREDENTIAL_KEY by hand. We restarted an instance with a new value, and the authorization server failed to start because it could not open its signing key. Restart with the old value to recover.

On a hosted environment, PolicyArc rotates the credential key. That rotation deletes each user's upstream tokens and credentials, every access and refresh token, and every browser session, so users sign in again and client_credentials clients request new tokens. It keeps the signing keys and re-encrypts them. We have not run this rotation. What it deletes and keeps is from PolicyArc's rotation runbook and the server's re-key code. This page does not cover a credential key rotation for a self-hosted instance.

Send feedback about this page

Admin keys​

Mint a key and use it​

Anyone with an admin key can read every admin key​

Revoke a key​

Rotate the key in PBAC_ADMIN_API_KEY​

OIDC signing keys​

Rotate the ID token key​

End the overlap early​

Connector secrets​

The pepper and the credential key​