werkrevisor 0.24.0: a high risk and a confirmed finding in the same file give RED; the review times its phases #51

2026-10-08T16:09:55+02:00

claude-code commented

2026-10-08 16:09:55 +02:00

🤖 Generated with Claude Code

See [the PR document](docs/prs/2026-10-08-PR%2351-risk-with-issue-red.md). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-code added 1 commit 2026-10-08 16:09:56 +02:00

PR#51: PR document for RED from risk and finding, and the phase timing

werkator build succeeded after 01:32

Details

werkator/werkrevisor-statistics build succeeded after 00:35

Details

0d905d5b25

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-08 16:10:44 +02:00

PR#51: red scenario tests for RED from risk and finding, and the phase timing

werkator build failed after 00:18

Details

werkator/werkrevisor-statistics build succeeded after 00:27

Details

71bb8597e1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-code marked the pull request as ready for review 2026-10-08 16:12:50 +02:00

mi added 1 commit 2026-10-08 16:12:51 +02:00

werkrevisor 0.24.0: a high risk and a confirmed finding in the same file give RED; the review times its phases

werkator build succeeded after 00:09

Details

werkator/werkrevisor-statistics build succeeded after 00:32

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; review requested from mi; merge scheduled for when the checks succeed

Details

ai-review/gate build succeeded after 02:36

Details

fa929cf99b

A file that Jev rates as very likely risky and an ORANGE finding that Jev confirms in the
same file give RED (risk_with_issue). The usage record and the comment show the seconds
of each phase that called a service (phase_seconds).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented

2026-10-08 16:15:34 +02:00

🟠 werkrevisor ORANGE for fa929cf, superseded by the result for 0f05b98

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: mixed.

Reasons

ORANGE 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:459-462
hint 6 changed file(s) that neither weaken the checks nor touch security according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 8 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3 according to Jev
hint 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-08-PR#51-risk-with-issue-red.md:11-12
hint model recommends merge with caution

Findings

Asymmetric phase timing removal at werkrevisor/cli.py:459-462, severity unknown, counted as medium, ORANGE, as a line comment
Document contradicts the code at docs/prs/2026-10-08-PR#51-risk-with-issue-red.md:11-12, severity unknown, counted as medium, hint, as a line comment

Classifier

8 of 8 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 47402, calls 1, tokens in 45985 / out 8562 (reasoning included in out), duration 153.6 s, cost 0.0240 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6450, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 2710, cost 0.0001 USD, change summary `openrouter/minimax/minimax-m3` calls 1, tokens in 4556 / out 1092, cost 0.0021 USD (reported), PR document check `jev-1.13.0` calls 8, tokens in 15189, cost 0.0006 USD, build file judge `jev-1.13.0` calls 7, tokens in 6368, cost 0.0003 USD, config: defaults + host + repository (.werkrevisor.toml at main 882b76c).

Reviewed head `fa929cf99b7816717dd5806f51a56ec2f3d9944c` against base `882b76c26c0a3a33d158cd15fa93923d3c789f0f`, werkrevisor 0.23.1.}

<details><summary>🟠 werkrevisor ORANGE for <code>fa929cf</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/51#issuecomment-2031">the result for <code>0f05b98</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `mixed`. ### Reasons - **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:459-462 - **hint** 6 changed file(s) that neither weaken the checks nor touch security according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 8 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3 according to Jev - **hint** 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-08-PR#51-risk-with-issue-red.md:11-12 - **hint** model recommends merge with caution ### Findings 1. **Asymmetric phase timing removal** at `werkrevisor/cli.py:459-462`, severity unknown, counted as medium, ORANGE, as a line comment 2. **Document contradicts the code** at `docs/prs/2026-10-08-PR#51-risk-with-issue-red.md:11-12`, severity unknown, counted as medium, hint, as a line comment ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 4 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 47402, calls 1, tokens in 45985 / out 8562 (reasoning included in out), duration 153.6 s, cost 0.0240 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6450, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 2710, cost 0.0001 USD, change summary `openrouter/minimax/minimax-m3` calls 1, tokens in 4556 / out 1092, cost 0.0021 USD (reported), PR document check `jev-1.13.0` calls 8, tokens in 15189, cost 0.0006 USD, build file judge `jev-1.13.0` calls 7, tokens in 6368, cost 0.0003 USD, config: defaults + host + repository (.werkrevisor.toml at main 882b76c). Reviewed head `fa929cf99b7816717dd5806f51a56ec2f3d9944c` against base `882b76c26c0a3a33d158cd15fa93923d3c789f0f`, werkrevisor 0.23.1. </details>

ai-reviewer reviewed

2026-10-08 16:15:34 +02:00

ai-reviewer left a comment

werkrevisor: ORANGE / FINDINGS for fa929cf99b7816717dd5806f51a56ec2f3d9944c.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `fa929cf99b7816717dd5806f51a56ec2f3d9944c`. The result comment of werkrevisor holds the details.

docs/prs/2026-10-08-PR#51-risk-with-issue-red.md

						
				@@ -0,0 +8,4 @@

				werkrevisor shall separate the PRs that need a human from the others,

				and the coding agent shall remove issues before a human spends time on the PR.

				The agreed verdict rule: ORANGE for a very likely high risk or a concrete issue,

ai-reviewer commented

2026-10-08 16:15:34 +02:00

Document contradicts the code, severity unknown, counted as medium (werkrevisor)

hint; Jev: real 0.80, severity 2.1 of 3

The "Problem" section states the rule as "RED for both in the same file or category", but the implementation only matches on file path (see _risk_with_issue and pathjudge.orange_files); the PR document's own scenarios and the "Solution" section correctly say "in the same file". The "or category" wording is therefore inconsistent with the code.

**Document contradicts the code**, severity unknown, counted as medium (werkrevisor) hint; Jev: real 0.80, severity 2.1 of 3 The "Problem" section states the rule as "RED for both in the same file or category", but the implementation only matches on file path (see `_risk_with_issue` and `pathjudge.orange_files`); the PR document's own scenarios and the "Solution" section correctly say "in the same file". The "or category" wording is therefore inconsistent with the code.

claude-code commented

2026-10-08 16:16:34 +02:00

Fixed: the Problem section now says that this PR implements the rule for the same file, and the Non-Goals exclude RED for the same category in different files.

claude-code marked this conversation as resolved

werkrevisor/cli.py

						
				@@ -423,3 +457,3 @@

				        result.classifier_status = "skipped (--no-llm)"

				        return ""

				    run = classifier.judge_pr_docs(config.classifier, files, fetch, deps.jev)

				    with _phase(result, deps, "classifier"):

ai-reviewer commented

2026-10-08 16:15:34 +02:00

Asymmetric phase timing removal, severity unknown, counted as medium (werkrevisor)

ORANGE; Jev: real 0.73, severity 2.6 of 3

After the _phase("classifier") context manager records the time spent in classifier.judge_pr_docs, _classify then removes that entry from result.phase_seconds whenever run.judged_files is empty. The other phases never drop their entries, so the classifier is the only phase whose duration can disappear from both the result comment and the usage record. When the classifier did run (e.g. for a code PR with no PR documents, or a docs PR with low confidence) the call's wait time is hidden, which defeats the scenario's promise that "a slow review shows which call waited".

**Asymmetric phase timing removal**, severity unknown, counted as medium (werkrevisor) ORANGE; Jev: real 0.73, severity 2.6 of 3 After the `_phase("classifier")` context manager records the time spent in `classifier.judge_pr_docs`, `_classify` then removes that entry from `result.phase_seconds` whenever `run.judged_files` is empty. The other phases never drop their entries, so the classifier is the only phase whose duration can disappear from both the result comment and the usage record. When the classifier did run (e.g. for a code PR with no PR documents, or a docs PR with low confidence) the call's wait time is hidden, which defeats the scenario's promise that "a slow review shows which call waited".

claude-code commented

2026-10-08 16:16:35 +02:00

No wait time was hidden: the entry was dropped only when no PR document was judged, i.e. without a Jev call; the other phases are entered only when they call a service. To make that explicit, the entry is now dropped only when the classifier made no Jev call, with a comment, and Scenario#51.03 asserts it for a PR without PR documents.

claude-code marked this conversation as resolved

ai-reviewer scheduled this pull request to auto merge when all checks succeed 2026-10-08 16:15:34 +02:00

ai-reviewer requested review from mi 2026-10-08 16:15:34 +02:00

mi added 1 commit 2026-10-08 16:16:34 +02:00

PR#51: drop the classifier phase only without a Jev call; the document names the file rule

werkator/werkrevisor-statistics build succeeded after 00:31

Details

ai-review/assessment AI review passed; approved by werkrevisor; merge already scheduled; review request of mi withdrawn

Details

werkator build succeeded after 00:08

Details

ai-review/gate build succeeded after 01:57

Details

0f05b98c6d

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented

2026-10-08 16:18:34 +02:00

🟢 werkrevisor: GREEN / PASS

Change kind: mixed.

Reasons

hint 6 changed file(s) that neither weaken the checks nor touch security according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 8 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3 according to Jev

Classifier

8 of 8 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 47561, calls 1, tokens in 46137 / out 18428 (reasoning included in out), duration 115.4 s, cost 0.0360 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6474, cost 0.0003 USD, change summary `openrouter/minimax/minimax-m3` calls 1, tokens in 4608 / out 706, cost 0.0017 USD (reported), PR document check `jev-1.13.0` calls 8, tokens in 12733, cost 0.0005 USD, build file judge `jev-1.13.0` calls 7, tokens in 6421, cost 0.0003 USD, config: defaults + host + repository (.werkrevisor.toml at main 882b76c).

Reviewed head `0f05b98c6d5fa79536fee547648ccf4b939eca5d` against base `882b76c26c0a3a33d158cd15fa93923d3c789f0f`, werkrevisor 0.23.1.}

## 🟢 werkrevisor: GREEN / PASS Change kind: `mixed`. ### Reasons - **hint** 6 changed file(s) that neither weaken the checks nor touch security according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 8 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3 according to Jev ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 4 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 47561, calls 1, tokens in 46137 / out 18428 (reasoning included in out), duration 115.4 s, cost 0.0360 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6474, cost 0.0003 USD, change summary `openrouter/minimax/minimax-m3` calls 1, tokens in 4608 / out 706, cost 0.0017 USD (reported), PR document check `jev-1.13.0` calls 8, tokens in 12733, cost 0.0005 USD, build file judge `jev-1.13.0` calls 7, tokens in 6421, cost 0.0003 USD, config: defaults + host + repository (.werkrevisor.toml at main 882b76c). Reviewed head `0f05b98c6d5fa79536fee547648ccf4b939eca5d` against base `882b76c26c0a3a33d158cd15fa93923d3c789f0f`, werkrevisor 0.23.1.

ai-reviewer approved these changes

2026-10-08 16:18:34 +02:00

ai-reviewer left a comment

werkrevisor: GREEN / PASS for 0f05b98c6d5fa79536fee547648ccf4b939eca5d.
The result comment of werkrevisor holds the details.

werkrevisor: GREEN / PASS for `0f05b98c6d5fa79536fee547648ccf4b939eca5d`. The result comment of werkrevisor holds the details.

ai-reviewer removed review request for mi 2026-10-08 16:18:34 +02:00

ai-reviewer merged commit b8792abf30 into main

2026-10-08 16:18:36 +02:00

ai-reviewer deleted branch risk-with-issue-red

2026-10-08 16:18:36 +02:00

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mi/werkrevisor#51