werkrevisor/docs/prs/2026-10-20-PR#128-rotate-secrets.md

PR#128: werkrevisor Rotates the Secrets of the Review Builds without a Stop

Version: minor

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

Problem

On 2026-10-20, the values of GITEA_BOT_TOKEN, OPENROUTER_API_KEY, TYPESAFE_API_KEY and WERKREVISOR_STATS_TOKEN appeared in a Claude Code transcript, since Werkator passed them to werkdock run on the command line. All four must get new values. The machine configs of eight repositories on mih09 and ~/.config/ai-coding-agents/secrets.env hold them. The rotation is done by hand so far, without a check that each copy has the new value, and 137 backups of the machine configs keep the old values.

Non-Goals

• No change of Werkator or werkdock; the session "werkator-env-secrets" takes the secrets off the command line.
• No automatic creation or revocation of keys at the services; this PR documents what an automation needs.
• No tool for the environment of the statistics service on mih09; the human changes its list of source tokens.

Specification

Feature: rotate one secret of the review builds

Scenario#128.01: set replaces the value in each file that has it

… so that each review build and the local tools get the new value without a restart and without a stop.

• Given machine configs on the host and a secrets file that hold the same old value of a key
• When the human runs tools/rotate-secret set <host> <KEY> with the new value on stdin
• Then each file that has the key holds the new value, with its quotes and its mode
• And a backup of each changed file holds the old one
• And the output names the files and the SHA-256 of the old value, never a value

Verified by

• test_rotate_secret.RotateSecretScriptTest.test_set_then_check_then_purge
• test_rotate_secret.HostSecretTest.test_set_replaces_only_the_value_and_keeps_quotes_and_mode
• test_rotate_secret.HostSecretTest.test_set_adds_a_missing_key_to_an_env_file_only
• test_rotate_secret.RotateSecretScriptTest.test_set_takes_the_old_value_from_the_host_if_the_secrets_file_lacks_the_key

Scenario#128.02: set refuses an unclear old value

… so that a rotation that stopped half-way or a drifted file does not lose a value that is still in use.

• Given a file that holds a value other than the old one of the secrets file, or files with two old values
• When the human runs set
• Then no file changes, and the error names the cause without a value

Verified by

• test_rotate_secret.HostSecretTest.test_set_refuses_a_third_value
• test_rotate_secret.RotateSecretScriptTest.test_set_refuses_if_the_host_holds_another_value_than_the_secrets_file
• test_rotate_secret.HostSecretTest.test_a_file_with_two_lines_for_the_key_stops_set_before_any_change
• test_rotate_secret.HostSecretTest.test_set_continues_a_rotation_that_stopped_half_way
• test_rotate_secret.RotateSecretScriptTest.test_set_stops_before_the_host_if_the_secrets_file_cannot_be_parsed

Scenario#128.03: probe tells an accepted value from a refused one

… so that the human knows a new value works before a review depends on it, and an old one is revoked.

• Given a value of one of the four keys
• When the human runs tools/rotate-secret probe <KEY>
• Then the output says whether the service accepts it, and an outage differs from a refusal by its exit code
• And the probe costs nothing and starts no review

Verified by

• test_secretprobe

Scenario#128.04: the statistics service takes two source tokens at once

… so that no usage record is lost while the runs change from the old to the new token.

• Given the service knows two source tokens
• When a run sends an empty record with either of them, or with an unknown token
• Then the service refuses the record with HTTP 400 for both known tokens, with HTTP 401 for the unknown one, and stores nothing

Verified by

• UsageApiIntegrationTest "an empty record tells each of two source tokens from an unknown one and stores nothing"

Scenario#128.05: purge removes only the backups with the old value

… so that the revoked value leaves the host, and backups with other values and live files stay.

• Given a revoked old value, backups with it, backups with other values, and a live file with the same name prefix
• When the human runs tools/rotate-secret purge <host> <KEY> <sha256>
• Then only the backups with the old value are gone, on the host and next to the secrets file

Verified by

• test_rotate_secret.HostSecretTest.test_purge_removes_the_backups_with_the_old_value_only
• test_rotate_secret.HostSecretTest.test_purge_keeps_a_file_that_only_shares_the_prefix
• test_rotate_secret.HostSecretTest.test_a_backup_it_cannot_parse_may_hold_the_value

Scenario#128.06: check compares each copy with a hash

… so that the human sees which copy still has another value, without seeing a value.

• Given machine configs and a secrets file
• When the human runs tools/rotate-secret check <host> <KEY> [<sha256>]
• Then each file with the key is named as equal or differing, with the number of backups that hold that value

Verified by

• test_rotate_secret.HostSecretTest.test_check_names_each_repository_with_the_key
• test_rotate_secret.RotateSecretScriptTest.test_check_reports_another_value

Scenario#128.07: purge waits until no machine config holds the old value

… so that a backup is never the last copy of a value that a review build still uses.

• Given a machine config that still holds the old value
• When the human runs purge for that value
• Then purge refuses and removes nothing

Verified by

• test_rotate_secret.HostSecretTest.test_purge_removes_the_backups_with_the_old_value_only

Scenario#128.08: a machine config that cannot be read is an error

… so that check does not report success while a file keeps the old value.

• Given a machine config that cannot be read or decoded
• When the human runs check, set or purge
• Then the tool names the file and exits with 2, and changes nothing

Verified by

• test_rotate_secret.HostSecretTest.test_an_unreadable_machine_config_is_an_error

Scenario#128.09: probe refuses a malformed value without showing it

… so that a value pasted with a line break does not reach a header and an error message.

• Given a value with a line break, a blank or too few characters
• When the human runs tools/rotate-secret probe <KEY> -
• Then the probe exits with 2 and names the allowed characters, but no part of the value

Verified by

• test_secretprobe.MalformedValueTest

Scenario#128.10: fetch copies the value of the host into a secrets file without it

… so that the backup of the secrets file holds the old value for the probe after the revocation.

• Given machine configs that hold one value of a key, and a secrets file without that key
• When the human runs tools/rotate-secret fetch <host> <KEY>
• Then the secrets file holds that value, and the output names the file and its backup, never the value

Verified by

• test_rotate_secret.RotateSecretScriptTest.test_fetch_adds_the_value_of_the_host_to_the_secrets_file
• test_rotate_secret.HostSecretTest.test_export_writes_the_one_value_as_json_only_into_a_pipe
• test_rotate_secret.RotateSecretScriptTest.test_fetch_stops_if_the_host_holds_no_single_value

Solution

• tools/rotate-secret follows tools/sync-host-config: one ssh connection per command, retries of a refused one, and a host part tools/host_secret.py that it sends to python3 -c. It handles every ~/<repo>/.git/werkator/.werkator.yml that sets the key, and the local secrets file.
• A value goes only through stdin, variables and pipes; printf is a builtin, so no process list shows it. On a terminal, set and probe - ask for it without showing it.
• set writes each file through a renamed temporary file, so Werkator never reads half a file. It checks every file before it changes one. It accepts only values of A-Z a-z 0-9 . _ ~ + / = -, which need no quotes in YAML or in the env file.
• werkrevisor/secretprobe.py asks each service without costs: Gitea /api/v1/user and the write access of ai-reviewer to mi/werkprobe, OpenRouter /api/v1/key without its label, TypeSafe /v1/models, and the statistics service with an empty record.
• The write access of the account says nothing about the scopes of the token (found by the Codex review). So the Gitea probe also writes a review to pull request 0 and a comment to issue 0: Gitea checks the scope first and answers 403 without it, else 404 or 422, and creates nothing.
• purge removes only backups by their names on the hosts, <file>.bak…, .pre… and .before…; secrets.env.production would be a live file (found by the Codex review).
• A file that cannot be read is an error, not a file without the key (found by the Codex review).
• set reads the secrets file before it changes the host; only a file without the key lets the host name the old value. Before, a file it could not read or parse let the host change and failed only after it (found by the Codex review).
• A backup that check or purge cannot read or parse may hold the value: check names it and exits with 2, purge refuses (found by the review of werkrevisor).
• fetch copies the value that all machine configs hold into a secrets.env without the key, through pipes only; the host part export refuses a terminal. A failed export stops fetch with its exit code (found by the Codex review). So the backup of secrets.env holds the old WERKREVISOR_STATS_TOKEN for the probe after the revocation (found by the review of werkrevisor).
• Six refused connections exit with 2, not with 1, since no file was read (found by the Codex review).
• An error in the secrets file exits with 2, not with 1, which means a refused or differing value (found by the review of werkrevisor).
• probe accepts only the characters that set accepts (found by the Codex review). Before, http.client refused a value with a line break in a header and quoted it in a traceback; MalformedValueTest failed so without the fix and passes with it.
• The statistics service took a comma-separated list of source tokens already; a test now fixes it.
• docs/betrieb-git.javagil.de.md has one procedure for all four secrets, the missing row of WERKREVISOR_STATS_TOKEN, and what an automation needs per service.

Findings on the way:

• ~/.config/ai-coding-agents/secrets.env lacks WERKREVISOR_STATS_TOKEN; set adds it.
• On 2026-10-20, all eight machine configs and secrets.env hold the same values, and 137 backups on mih09 hold the current values.
• OpenRouter has management keys: POST /api/v1/keys and DELETE /api/v1/keys/{hash}.
• Gitea creates and deletes a token of a user only with the password of that user (basic auth), or with gitea admin user generate-access-token on the Gitea host.
• The public API of TypeSafe has /v1/models and /v1/systemone only. Its console at https://console.typesafe.ai/keys creates, renames and deletes several keys, through Next.js server actions of a logged-in session, not through a documented API.
• werkdock before 0.13.2 wrote the environment of each build to $WERKDOCK_HOME/instances/<id>/config.json, mode 0644 (werkdock PR#17); the procedure removes the directories of finished builds after the last secret.

Open Questions

None.

Follow-up PRs

• The human changes the source tokens of the statistics service by hand, since this PR does not know the file that holds them. A tool for it can follow once the human names that file.
• Each of the four secrets is created and revoked by hand. For WERKREVISOR_STATS_TOKEN and OPENROUTER_API_KEY, a tool can do it by API (Werkbaum #ops.rotate2.auto).