Files
werkrevisor/docs/prs/2026-10-05-PR#18-werkator-review-and-automerge.md
claude-code 0c61408302
werkator build succeeded after 00:43
werkrevisor 0.16.1: Werkator review, review only after WIP, merge by the bot, 100000-token limit (#18)
See docs/prs/2026-10-05-PR#18-werkator-review-and-automerge.md.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #18
Co-authored-by: claude-code <claude-code@javagil.de>
2026-10-06 07:36:29 +02:00

147 lines
7.3 KiB
Markdown

# PR#18: werkrevisor reviews Werkator, only a finished PR is reviewed, and a reviewed PR merges itself
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.
## The Problem
werkrevisor reviews werkprobe and werkrevisor, but not Werkator, the CI that runs it.
A green PR still waits for a human to click "Merge".
And a PR opened without "WIP: " is reviewed at once: with only its PR document, it would get GREEN and the bot's approval.
And a PR above 80000 input tokens gets no model review, e.g. werkrevisor PR#17 with 81468 tokens.
## Non-Goals
- No repository-wide automerge: Gitea schedules it per PR, so the bot schedules it after each review.
- No review on a label change: Werkator starts a review only for a new head commit or the end of WIP.
- No `.werkrevisor.toml` in Werkator yet: the host config holds all rules, so the review works before a Werkator PR.
## The Scenarios
### Feature: only a finished PR is reviewed, and a reviewed PR merges itself
#### Background
- The host config sets `[gate] require_wip_transition = true` and `automerge = true`.
- The branch protection of `main` requires the statuses `werkator` and `ai-review/gate` and one approval.
#### Scenario#18.01: a PR opened without "WIP: " gets no review and no approval
So that an unfinished PR can never merge after its first green review.
- **Given** a PR was opened with the title "Fix the gate", without "WIP: "
- **When** Werkator starts the review
- **Then** the verdict is ERROR with the reason `not_marked_ready`
- **and** werkrevisor calls no model, gives no approval and schedules no merge
##### Verified by
- [test_flow.ReadinessAndAutomergeTest.test_pr_never_marked_ready_gets_no_review_and_no_merge](../../tests/test_flow.py)
- [test_readiness.ReadinessTest.test_marked_ready_needs_a_recorded_change_from_wip](../../tests/test_readiness.py)
#### Scenario#18.02: the bot schedules the merge after the review
So that a finished PR merges without a click once the gate allows it.
- **Given** the title of a PR changed from "WIP: Fix the gate" to "Fix the gate"
- **When** the review ends with `<verdict>`
- **Then** `<result>`
| verdict | result |
|---|---|
| GREEN | the bot approves and schedules the merge, so Gitea merges at once |
| YELLOW or ORANGE | the bot schedules the merge, so the human approval merges the PR |
| RED or ERROR | no merge is scheduled |
The bot schedules a merge only if the protection of the base branch requires an approval and status checks;
otherwise Gitea would merge a YELLOW or ORANGE PR without a human look.
A PR set back to "WIP: " after it was finished is not reviewed until "WIP: " is removed again.
##### Verified by
- [test_flow.ReadinessAndAutomergeTest.test_green_approves_and_schedules_the_merge](../../tests/test_flow.py)
- [test_flow.ReadinessAndAutomergeTest.test_yellow_schedules_the_merge_before_the_human_approval](../../tests/test_flow.py)
- [test_flow.ReadinessAndAutomergeTest.test_red_and_error_never_schedule](../../tests/test_flow.py)
- [test_flow.ReadinessAndAutomergeTest.test_no_merge_without_an_approval_rule_on_the_base_branch](../../tests/test_flow.py)
- [test_readiness.ReadinessTest.test_only_the_last_wip_change_counts](../../tests/test_readiness.py)
- manual check on Werkator PR#127, see [Measurement](#measurement)
#### Scenario#18.03: automerge is only allowed together with the WIP transition
So that no config can merge a PR that was never marked as finished.
- **Given** a host config with `automerge = true`, but without `require_wip_transition = true`
- **When** werkrevisor starts
- **Then** werkrevisor stops with a config error
##### Verified by
- [test_flow.ReadinessAndAutomergeTest.test_automerge_needs_the_wip_transition](../../tests/test_flow.py)
### Feature: a repository admin sets the branch protection with one command
#### Scenario#18.04: `tools/branch-protection` saves the old rule and keeps its status checks
So that switching on the review gate in a repository loses no existing check and can be undone.
- **Given** the branch `main` has a protection rule that requires the status `werkator` and the status `lint`
- **When** a repository admin runs `tools/branch-protection` for `main`
with the statuses `werkator` and `ai-review/gate` and the approvers `ai-reviewer` and `mi`
- **Then** a file holds the previous rule
- **and** the rule requires the statuses `werkator`, `lint` and `ai-review/gate` and one approval of `ai-reviewer` or `mi`
##### Verified by
- [test_protection.BranchProtectionTest.test_existing_rule_is_saved_first_and_its_contexts_stay](../../tests/test_protection.py)
- [test_protection.BranchProtectionTest.test_missing_rule_is_created](../../tests/test_protection.py)
- [test_protection.BranchProtectionTest.test_dry_run_and_unchanged_rule_write_nothing](../../tests/test_protection.py)
### Feature: werkrevisor reviews Werkator
#### Scenario#18.05: Werkator PRs are reviewed with Werkator's own rules
So that a change to the deployment or the sandbox of Werkator always gets a human look.
- **Given** Werkator has no rules of its own for werkrevisor yet
- **When** a Werkator PR changes a deployment, sandbox or build file
- **Then** werkrevisor reviews it with the host rules for Werkator
- **and** the verdict is at least YELLOW
##### Verified by
- [test_config_budget_redact.ConfigTest.test_werkator_host_config_is_complete](../../tests/test_config_budget_redact.py)
## The Solution
- `config/werkator.toml` is a complete host config for `mi/werkator`.
Its dangerous paths cover the deployment, the sandbox, the trust gate and the build of Werkator.
Werkator writes open questions in the format of hsadmin-ng since its PR#127, so the same rule applies.
Only dated PR documents count; Werkator's template `TEAMPLATE.md` escapes the built-in exclusion of `TEMPLATE*.md`.
- `tools/branch-protection` sets the branch protection of the approval gate with the token of a repository admin;
it saves the previous rule as a JSON file and keeps its status contexts.
- `[gate] require_wip_transition` reads the Gitea timeline: only a recorded change from a "WIP: " title to one without
starts the review; otherwise the verdict is ERROR, without model call and approval.
- `[gate] automerge` lets the bot schedule the merge with `merge_when_checks_succeed` after GREEN, YELLOW and ORANGE;
it needs `yellow_gate = "approval"` and `require_wip_transition`.
- No tool removes the "WIP: " prefix: the Gitea UI or one `PATCH` of the PR title does it.
- The ops doc for git.javagil.de and the guide for dev.hostsharing.net describe the PR workflow;
werkprobe, werkrevisor and Werkator switch both settings on, hsadmin-ng keeps them off for now.
- The host configs of werkprobe, werkrevisor and Werkator raise `max_usd_per_review` from 0.40 to 0.48 USD.
The derived input limit grows from 80000 to 100000 tokens, well below the 150000 of `max_context_tokens`;
a review with one call costs at most about 0.24 USD.
- werkrevisor 0.16.1.
## Measurement
Pending: the manual check on werkprobe, whether Gitea 1.27 merges after a later human approval too.
## Prerequisite PRs
- PR#17: Jev judges the risk of every changed Liquibase changeset.
## Follow-up PRs
- Werkator gets its own review rules in `.werkrevisor.toml`, once the first reviews show which rules fit.