docs: the guide for dev.hostsharing.net names the state of werkrevisor 0.20.6 #38
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mi/werkrevisor#38
Reference in New Issue
Block a user
See the PR doc.
🤖 Generated with Claude Code
🟠 werkrevisor ORANGE for
8f7efcc, superseded by the result for9ad750a🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
docs_only.Reasons
Findings
[medium] Guide contradicts itself on gate setting and approvals at
docs/anleitung-dev.hostsharing.net.md:257, ORANGE, as a line comment[medium] PR doc contradicts guide on trial results at
docs/prs/2026-10-06-PR#38-dev-hostsharing-guide-current.md:24, hint, as a line comment[low] Unverifiable and unclear statements at
docs/anleitung-dev.hostsharing.net.md:235, hint, as a line commentClassifier
4 of 4 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 12163, calls 1, tokens in 13276 / out 828 (reasoning included in out), duration 24.0 s, cost 0.0348 USD (reported), classifier `jev-1.13.0` calls 4, tokens in 2852, cost 0.0001 USD, finding judge `jev-1.13.0` calls 2, tokens in 3007, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
c83d016).Reviewed head `8f7efcc66efbe05e9be215dd189b1c0cb55e3110` against base `c83d01690888bb1f8da62cc8bc4b3710308135bf`, werkrevisor 0.20.6.
werkrevisor: ORANGE / FINDINGS for
8f7efcc66efbe05e9be215dd189b1c0cb55e3110.The result comment of werkrevisor holds the details.
@@ -211,0 +232,4 @@Ein Dry run prüft den letzten Head eines PRs, also den Stand nach allen menschlichen Reviews.**Nur Regeln, 24 PRs:**#283 bricht mit „base commit unknown“ ab, obwohl Gitea seine Merge-Basis kennt; das ist ein Fehler in werkrevisor.[low] Unverifiable and unclear statements (werkrevisor)
hint
The note "Diese Anleitung ist noch nicht ausgeführt; nur die Erprobung in Schritt 6 lief schon" is fine. But "ob sie echt sind, prüft das Team" and "das ist ein Fehler in werkrevisor" (#283) state a known bug with no tracking reference, so it is unclear whether it is fixed or open. Also the table row for
block_on_official_review_requeststies the setting to GREEN review withdrawal, which only works from 0.20.2; the version is not named there.Fixed in
9ad750a: the #283 bug is marked as open, occurs only for old merged PRs and is listed in the PR doc for a follow-up; the withdrawn review request names werkrevisor 0.20.2; the unchecked ORANGE findings are worded as not yet checked.@@ -211,0 +254,4 @@| #315, #304, #286 | ORANGE | 0,06, 0,19, 0,32 || #302, #307, #306 | ORANGE, dazu harte Regeln | 0,30, 0,27, 0,25 |Von den 8 PRs ohne sperrende Regel hätte der Bot 4 freigegeben.[medium] Guide contradicts itself on gate setting and approvals (werkrevisor)
ORANGE; Jev: real 0.65, severity 2.5 of 3
The Gate item says
config/hsadmin-ng.tomlhasyellow_gate = "block"with nohuman_reviewers, so werkrevisor never approves. The trial result states "hätte der Bot 4 freigegeben" (the bot would have approved 4), which contradicts this. The PR doc also states "No change ofconfig/hsadmin-ng.toml", so the config stays as described. The reader cannot tell which behavior applies.Fixed in
9ad750a: the trial result says that 4 PRs would have become GREEN, which withyellow_gate = "block"ofconfig/hsadmin-ng.tomlis only a green status; the bot approves only with"approval".@@ -0,0 +21,4 @@## The Solution- The guide names werkrevisor 0.20.6 and what changed for the rollout, each with the version that brought it.- Step 6, the trial, holds the result of the rules-only dry runs over the last 25 merged PRs of hs.hsadmin.ng.[medium] PR doc contradicts guide on trial results (werkrevisor)
hint; Jev: real 0.74, severity 1.9 of 3
The PR document says step 6 holds "the result of the rules-only dry runs". The guide section also contains model runs (11 PRs, 2.08 USD, verdicts and costs). The solution description omits this and understates the change. The Problem section says the old state was "werkrevisor 0.8.0 to 0.15.0", but the guide mentions nothing of this range and the old text only named 0.5.0, 0.3.0 and 0.8.0.
Fixed in
9ad750a: the Solution names both trials, rules only over 24 PRs and with Sonnet 5.5 over 11 PRs for 2.08 USD, and the Problem names the state before werkrevisor 0.16.0 instead of a version range.Answer to the review of
8f7efcc(ORANGE):yellow_gate = "block"): fixed in9ad750a, resolved.9ad750a, resolved.9ad750a, resolved.9ad750a.🟡 werkrevisor YELLOW for
9ad750a, superseded by the result for58d2865🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
docs_only.Reasons
Findings
[medium] Contradiction in gate description at
docs/anleitung-dev.hostsharing.net.md:137-139, hint, as a line comment[medium] Doc claims unverified or inconsistent facts at
docs/anleitung-dev.hostsharing.net.md:275-276, hint, as a line comment[low] Unclear sentence and model name mismatch at
docs/prs/2026-10-06-PR#38-dev-hostsharing-guide-current.md:24-28, hint, as a line commentClassifier
3 of 4 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 12453, calls 1, tokens in 13577 / out 799 (reasoning included in out), duration 11.0 s, cost 0.0351 USD (reported), classifier `jev-1.13.0` calls 4, tokens in 2926, cost 0.0001 USD, finding judge `jev-1.13.0` calls 2, tokens in 4127, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
c83d016).Reviewed head `9ad750a5787f01f1ce4b62718391d6750f9ce076` against base `c83d01690888bb1f8da62cc8bc4b3710308135bf`, werkrevisor 0.20.6.
werkrevisor: YELLOW / HUMAN_REVIEW for
9ad750a5787f01f1ce4b62718391d6750f9ce076.The result comment of werkrevisor holds the details.
@@ -141,3 +137,1 @@300.000 Zeichen sind etwa 86.000 der 150.000 Tokens und lassen Platz für den Diff eines PRs mit 1000 Zeilen.- **Engine:** `[engine] mode = "llm"` bleibt der Standard.Mit `combined` bewertet Jev zuerst die Abschnitte und Gherkin-Szenarien der PR-Dokumente unter `doc/PR/`;- **Gate:** `config/hsadmin-ng.toml` hat `yellow_gate = "block"` und keine `human_reviewers`:[medium] Contradiction in gate description (werkrevisor)
hint; Jev: real 0.68, severity 2.1 of 3
The gate bullet says the config has
yellow_gate = "block"and nohuman_reviewers, so werkrevisor "gibt aber nie frei". The Step 6 result later says "mityellow_gate = "block"... heißt das nur ein grüner Status". The doc also states that with automerge the bot merges GREEN immediately. It is unclear what happens on GREEN underblock. The reader cannot tell whether GREEN gives an approval or only a status. The text should say explicitly what GREEN does underblock.Fixed in
58d2865: the Gate item says that GREEN underblockis only a greenai-review/gatestatus, without approval and without automerge.@@ -216,3 +273,3 @@1. Ein Repository-Administrator führt `tools/branch-protection` aus werkrevisor aus, erst mit `--dry-run`:`GITEA_TOKEN=… tools/branch-protection https://dev.hostsharing.net/hostsharing/hs.hsadmin.ng master``--contexts GitTally,ai-review/gate --approvers ai-reviewer,<Reviewer> --block-admin-override`.`--contexts <Status der CI>,ai-review/gate --approvers ai-reviewer,<Reviewer> --block-admin-override`.[medium] Doc claims unverified or inconsistent facts (werkrevisor)
hint; Jev: real 0.60, severity 2.4 of 3
The header says "Diese Anleitung ist noch nicht ausgeführt; nur die Erprobung in Schritt 6 lief schon". The PR doc says the Werkator/GitTally question is still open, yet the table now says "mindestens 1.36.0" for Werkator while the branch-protection step names the status
werkator. The oldGitTallycontext was removed, so the guide now asserts the status name without verification, which may be wrong if vm4006 runs GitTally. The--contextshint should point to the open check.Fixed in
58d2865: the branch-protection step names both status names,werkatorandGitTally, and points to the open check in Voraussetzungen.@@ -0,0 +21,4 @@## The Solution- The guide names werkrevisor 0.20.6 and what changed for the rollout, each with the version that brought it.- Step 6, the trial, holds the result of the dry runs over the last 25 merged PRs of hs.hsadmin.ng:[low] Unclear sentence and model name mismatch (werkrevisor)
hint
The PR doc names "Sonnet 5.5" as the model, but the guide never names the model used in the dry run. The reader cannot check the claim against the guide. The sentence about the CI on vm4006 also mixes "notes of 2026-08" without a source link, so it is hard to verify.
Fixed in
58d2865: the guide names Claude Sonnet 5.5 for the model runs; the PR doc dates the GitTally note to the move of the CI on 2026-08-10.Answer to the review of
9ad750a(YELLOW):58d2865.block, CI status name, model of the trial): fixed in58d2865, resolved.58d2865.🟠 werkrevisor ORANGE for
58d2865, superseded by the result for052d13e🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
docs_only.Reasons
Findings
[medium] Contradiction in the Werkator check at
docs/anleitung-dev.hostsharing.net.md:55, ORANGE, as a line comment[medium] Broken command in step 7 at
docs/anleitung-dev.hostsharing.net.md:275-276, hint, as a line comment[low] Unclear statement about the step 6 trial at
docs/anleitung-dev.hostsharing.net.md:5, hint, as a line commentClassifier
4 of 4 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 12435, calls 1, tokens in 13570 / out 727 (reasoning included in out), duration 10.1 s, cost 0.0344 USD (reported), classifier `jev-1.13.0` calls 4, tokens in 2940, cost 0.0001 USD, finding judge `jev-1.13.0` calls 2, tokens in 3455, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
c83d016).Reviewed head `58d286544aee73d8e297aa62281cb0482ede67d0` against base `c83d01690888bb1f8da62cc8bc4b3710308135bf`, werkrevisor 0.20.6.
werkrevisor: ORANGE / FINDINGS for
58d286544aee73d8e297aa62281cb0482ede67d0.The result comment of werkrevisor holds the details.
@@ -4,1 +3,3 @@Diese Anleitung ist noch nicht ausgeführt.Stand: 2026-10-06, werkrevisor 0.20.6,nach dem Versuch auf git.javagil.de (siehe [betrieb-git.javagil.de.md](betrieb-git.javagil.de.md)).Diese Anleitung ist noch nicht ausgeführt; nur die Erprobung in Schritt 6 lief schon, siehe dort.[low] Unclear statement about the step 6 trial (werkrevisor)
hint
The header says "nur die Erprobung in Schritt 6 lief schon".
Step 6 reports dry runs with the model on only 11 of 25 PRs, and the rules-only run is incomplete because #283 aborts.
The text does not say that this is a partial trial, and "GREEN möglich" in the table is only partly backed by the model results.
Fixed in
052d13e: the header says that of the trial in step 6 the dry runs have run.@@ -52,3 +53,3 @@| ----------------------- | ------------------------------------------ | ------------------------------------------------------------------ || Gitea | 1.27.3 | erledigt: `GET https://dev.hostsharing.net/api/v1/version` || Werkator auf vm4006 | mindestens 1.36.0 | Fußzeile der Werkator-Oberfläche || Werkator auf vm4006 | mindestens 1.36.0 | Fußzeile der Oberfläche; prüfen, ob dort Werkator oder noch die GitTally-Implementierung läuft |[medium] Contradiction in the Werkator check (werkrevisor)
ORANGE; Jev: real 0.80, severity 2.5 of 3
The guide says the CI status is
werkatororGitTally, depending on what runs on vm4006.The whole guide, including the Werkator-specific
POST /api/repos/<repo>/builds/restartcall and the@ai-reviewbranch suffix, assumes Werkator.The guide does not say what to do if the GitTally implementation runs.
This makes the "prüfen" item a blocker without a stated outcome.
Fixed in
052d13e: the guide requires Werkator on vm4006 for the build definition, the restart and the status names; if the GitTally implementation still runs there, the switch to Werkator comes first.@@ -215,7 +273,9 @@ Erst nach Schritt 6 und erst, wenn ein paar echte PRs ohne Pflicht gelaufen sind1. Ein Repository-Administrator führt `tools/branch-protection` aus werkrevisor aus, erst mit `--dry-run`:`GITEA_TOKEN=… tools/branch-protection https://dev.hostsharing.net/hostsharing/hs.hsadmin.ng master`[medium] Broken command in step 7 (werkrevisor)
hint; Jev: real 0.81, severity 2.4 of 3
The
tools/branch-protectioncommand is split across two code spans.The first span ends with the URL and
master, and the second span starts with--contexts.A reader who copies the first span runs the tool without
--contexts,--approversand--block-admin-override, and the guide does not show--dry-run.The old text had the same split, but the PR now adds more text around it.
Fixed in
052d13e: step 7 shows the command as one code block with--dry-run, and nameswerkatoras status of the CI.Answer to the review of
58d2865(ORANGE): all three points fixed in052d13eand resolved: Werkator is a requirement, the branch-protection command is one dry run, and the header calls the trial dry runs.🟢 werkrevisor: GREEN / PASS
Gate: approved by werkrevisor; merge waits for 2 open hint(s): reply to each line comment and resolve it, then restart the review or push a commit.
Change kind:
docs_only.Reasons
Findings
[medium] Contradiction in review-request config at
docs/anleitung-dev.hostsharing.net.md:139-145, hint, as a line comment[low] Unverified or unclear claim in results at
docs/anleitung-dev.hostsharing.net.md:242-263, hint, as a line commentClassifier
4 of 4 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 12611, calls 1, tokens in 13787 / out 593 (reasoning included in out), duration 10.2 s, cost 0.0335 USD (reported), classifier `jev-1.13.0` calls 4, tokens in 2940, cost 0.0001 USD, finding judge `jev-1.13.0` calls 1, tokens in 2392, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
c83d016).Reviewed head `052d13e9e63504c3384d76e22029a02225ccd7f8` against base `c83d01690888bb1f8da62cc8bc4b3710308135bf`, werkrevisor 0.20.6.
werkrevisor: GREEN / PASS for
052d13e9e63504c3384d76e22029a02225ccd7f8.The result comment of werkrevisor holds the details.
@@ -141,3 +139,1 @@300.000 Zeichen sind etwa 86.000 der 150.000 Tokens und lassen Platz für den Diff eines PRs mit 1000 Zeilen.- **Engine:** `[engine] mode = "llm"` bleibt der Standard.Mit `combined` bewertet Jev zuerst die Abschnitte und Gherkin-Szenarien der PR-Dokumente unter `doc/PR/`;- **Gate:** `config/hsadmin-ng.toml` hat `yellow_gate = "block"` und keine `human_reviewers`:[medium] Contradiction in review-request config (werkrevisor)
hint; Jev: real 0.68, severity 2.1 of 3
The guide says RED never requests changes and a GREEN review withdraws its review request (0.20.2), but the diagram edge was changed to list "Review-Anfrage" without saying when it is sent. The Branch Protection table also sets
block_on_official_review_requeststotrue. Withyellow_gate = "block"and nohuman_reviewers(as stated forconfig/hsadmin-ng.toml), the guide does not say who gets review requests. A reader cannot tell whether the setting matters for the current config.Not changed: review requests exist only with
yellow_gate = "approval"andhuman_reviewers, which the Gate item names as the setting for the activation; with the currentblockthere are none. The branch-protection step comes after the activation, soblock_on_official_review_requestsbelongs to that state.@@ -211,0 +239,4 @@#283 bricht mit „base commit unknown“ ab, obwohl Gitea seine Merge-Basis kennt;das ist ein offener Fehler in werkrevisor, der nur bei alten gemergten PRs auftritt.| Regel | PRs | Folge |[low] Unverified or unclear claim in results (werkrevisor)
hint
The result section says "Ab 0.8.0" rules and numbers for hard paths are given, but the counts add up unclearly: 12 + 5 + 3 + 8 = 28 PRs for 24 analysed PRs, since rules overlap. The table does not say the categories overlap, so the table reads as inconsistent. Also, "4 GREEN" of 8 is stated, while the model table lists #311, #299, #292, #290 as GREEN, which matches, but the reasoning for the other 4 is not given.
Not changed: the categories of the rules table overlap, since one PR can change a hard path and be too large at once; the table counts PRs per rule, not a partition. The other 4 of the 8 PRs are listed with their verdict in the model table (#300 YELLOW, #315, #304, #286 ORANGE).
Answer to the review of
052d13e(GREEN, merge held for 2 hints): both answered without a change and resolved. Restarting the review to release the merge.werkrevisor, review restarted for
052d13e9e63504c3384d76e22029a02225ccd7f8: approved by werkrevisor; merge scheduled for when the checks succeed; review request of mi withdrawn.