werkrevisor 0.49.0: Jev checks each changed file against the PR document #117

2026-10-17T12:52:36+02:00

claude-code commented

2026-10-17 12:52:36 +02:00

See docs/prs/2026-10-17-PR#117-doccheck-variant-b.md.

Replaces the economy-model summary of the doc check with one Jev call per changed file, and adds a data-outflow check.

🤖 Generated with Claude Code

See [docs/prs/2026-10-17-PR#117-doccheck-variant-b.md](docs/prs/2026-10-17-PR%23117-doccheck-variant-b.md). Replaces the economy-model summary of the doc check with one Jev call per changed file, and adds a data-outflow check. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-code added 1 commit 2026-10-17 12:52:36 +02:00

werkrevisor 0.45.0: PR document check reads each file diff (PR doc)

werkator build succeeded after 00:09

Details

werkator/werkrevisor-statistics build succeeded after 00:31

Details

a34c1fdde9

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

claude-code added 1 commit 2026-10-17 13:03:09 +02:00

werkrevisor 0.45.0: doc check reads each file diff, adds data-outflow question

werkator build succeeded after 00:10

Details

werkator/werkrevisor-statistics build succeeded after 00:36

Details

74458a65eb

Jev checks each changed file's diff against the PR document in one call, with the
four contradiction questions, the unrelated question and a new data-outflow question.
The economy-model summary is gone. Enable [doc_check] for hsadmin-ng.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

claude-code added 1 commit 2026-10-17 13:06:00 +02:00

werkrevisor 0.45.0: README, inside diagram and plan for the per-file doc check

werkator build succeeded after 00:10

Details

werkator/werkrevisor-statistics build succeeded after 00:33

Details

d8a00556df

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

claude-code added 1 commit 2026-10-17 13:07:16 +02:00

werkrevisor 0.45.0: workflow diagram and ops doc drop the economy-model summary

ai-review/codex 2 open Codex finding(s); resolve each line comment

werkator build succeeded after 00:10

Details

werkator/werkrevisor-statistics build succeeded after 00:32

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; review requested from mi; merge scheduled for when the checks succeed

Details

ai-review/gate build succeeded after 04:05

Details

a654d8227c

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

claude-code marked the pull request as ready for review 2026-10-17 13:08:40 +02:00

ai-reviewer commented

2026-10-17 13:12:47 +02:00

🟠 werkrevisor ORANGE for a654d82, superseded by the result for 837bae4

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: mixed.

Reasons

ORANGE changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88)
YELLOW tests lost or disabled: tests/test_doccheck.py:85 removes 25 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement
YELLOW PR too large for an AI-only approval: 918 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself
YELLOW spec: docs/prs/2026-10-17-PR#117-doccheck-variant-b.md Scenario#117.04 implementation_details (0.64 > yellow_above = 0.6)
hint writing rules violated in the changed docs: README.md:655: paragraph with 6 sentences, more than 5; README.md:657: more than one sentence on a line
hint 10 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 14 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 19 change(s) belong to the PR document, according to Jev
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py
hint files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file)
hint model recommends merge with caution

Classifier

Unit	Question	Problem	Confidence	Verdict
docs/prs/2026-10-17-PR#117-doccheck-variant-b.md Scenario#117.04	implementation_details	0.64	–	YELLOW

13 of 14 units passed all questions (1 PR document(s)).
6 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 80577, calls 1, tokens in 77466 / out 21040 (reasoning included in out), duration 242.3 s, cost 0.0485 USD (reported), classifier `jev-1.13.0` calls 14, tokens in 12019, cost 0.0005 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 29905 / out 720, cost 0.0041 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 23004 / out 2788, cost 0.0102 USD (reported), PR document check `jev-1.13.0` calls 33, tokens in 93638, cost 0.0039 USD, build file judge `jev-1.13.0` calls 12, tokens in 23699, cost 0.0010 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 14.3 s, doc_check 2.0 s, context 9.7 s, model_review 214.2 s, config: defaults + host + repository (.werkrevisor.toml at main f34da27).

Reviewed head `a654d8227c3cebf7f104fd6c8f86e6769e3a9347` against base `f34da2710db793fdfe2139a98a39b88c09aa30ec`, werkrevisor 0.44.0.}

<details><summary>🟠 werkrevisor ORANGE for <code>a654d82</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6174">the result for <code>837bae4</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `mixed`. ### Reasons - **ORANGE** changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88) - **YELLOW** tests lost or disabled: tests/test_doccheck.py:85 removes 25 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** PR too large for an AI-only approval: 918 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself - **YELLOW** spec: docs/prs/2026-10-17-PR#117-doccheck-variant-b.md Scenario#117.04 implementation_details (0.64 > yellow_above = 0.6) - **hint** writing rules violated in the changed docs: README.md:655: paragraph with 6 sentences, more than 5; README.md:657: more than one sentence on a line - **hint** 10 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 14 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 19 change(s) belong to the PR document, according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py - **hint** files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file) - **hint** model recommends merge with caution ### Classifier | Unit | Question | Problem | Confidence | Verdict | |---|---|---|---|---| | docs/prs/2026-10-17-PR#117-doccheck-variant-b.md Scenario#117.04 | implementation_details | 0.64 | – | YELLOW | 13 of 14 units passed all questions (1 PR document(s)). 6 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 80577, calls 1, tokens in 77466 / out 21040 (reasoning included in out), duration 242.3 s, cost 0.0485 USD (reported), classifier `jev-1.13.0` calls 14, tokens in 12019, cost 0.0005 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 29905 / out 720, cost 0.0041 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 23004 / out 2788, cost 0.0102 USD (reported), PR document check `jev-1.13.0` calls 33, tokens in 93638, cost 0.0039 USD, build file judge `jev-1.13.0` calls 12, tokens in 23699, cost 0.0010 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 14.3 s, doc_check 2.0 s, context 9.7 s, model_review 214.2 s, config: defaults + host + repository (.werkrevisor.toml at main f34da27). Reviewed head `a654d8227c3cebf7f104fd6c8f86e6769e3a9347` against base `f34da2710db793fdfe2139a98a39b88c09aa30ec`, werkrevisor 0.44.0. </details>

ai-reviewer reviewed

2026-10-17 13:12:47 +02:00

ai-reviewer left a comment

werkrevisor: ORANGE / FINDINGS for a654d8227c3cebf7f104fd6c8f86e6769e3a9347.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `a654d8227c3cebf7f104fd6c8f86e6769e3a9347`. The result comment of werkrevisor holds the details.

README.md

						
				@@ -665,3 +655,1 @@

				The usage record holds `doc_summary_*`, `doc_check_*`, the values per section as `doc_check_judgments`

				and the values per change as `doc_coverage_judgments`;

				the statistics service counts the summary as LLM cost and the comparison as Jev cost.

				A flagged file gets one line comment at its first changed line.

ai-reviewer commented

2026-10-17 13:12:47 +02:00

Writing rule: paragraph with 6 sentences, more than 5 (werkrevisor writing rule)

Fix the line, or reply why it stays, and resolve this comment.

**Writing rule: paragraph with 6 sentences, more than 5** (werkrevisor writing rule) Fix the line, or reply why it stays, and resolve this comment.

claude-code marked this conversation as resolved

README.md Outdated

						
				@@ -667,1 +655,3 @@

				the statistics service counts the summary as LLM cost and the comparison as Jev cost.

				A flagged file gets one line comment at its first changed line.

				A test claim that the file contradicts from `orange_min` on is ORANGE;

				every other flag is a hint. Like every hint, it holds the merge until it is resolved.

ai-reviewer commented

2026-10-17 13:12:47 +02:00

Writing rule: more than one sentence on a line (werkrevisor writing rule)

Fix the line, or reply why it stays, and resolve this comment.

**Writing rule: more than one sentence on a line** (werkrevisor writing rule) Fix the line, or reply why it stays, and resolve this comment.

claude-code marked this conversation as resolved

ai-reviewer scheduled this pull request to auto merge when all checks succeed 2026-10-17 13:12:47 +02:00

ai-reviewer requested review from mi 2026-10-17 13:12:47 +02:00

ai-reviewer reviewed

2026-10-17 13:18:38 +02:00

ai-reviewer left a comment

Codex review of a654d8227c3cebf7f104fd6c8f86e6769e3a9347

Codex review of `a654d8227c3cebf7f104fd6c8f86e6769e3a9347`

werkrevisor/doccheck.py Outdated

						
				@@ -138,0 +113,4 @@

				    """The changed files Jev checks: not a PR document, not mechanical, not deleted, with a text diff."""

				    targets = []

				    for f in files:

				        if f.path in docs or f.path in mechanical or f.deleted:

ai-reviewer commented

2026-10-17 13:18:38 +02:00

[high] Deleted files silently bypass the document check (Codex review)

The f.deleted exclusion drops complete file deletions even when their text diffs are available. The previous summary included these diffs, so deleting an implementation or test that the PR document claims to preserve now escapes contradiction and unrelated-change checks entirely. If deletions are the only non-document changes, _check_doc returns without any check or skipped-check notice.

Fix it, or reply why it does not hold, and resolve this comment.

**[high] Deleted files silently bypass the document check** (Codex review) The f.deleted exclusion drops complete file deletions even when their text diffs are available. The previous summary included these diffs, so deleting an implementation or test that the PR document claims to preserve now escapes contradiction and unrelated-change checks entirely. If deletions are the only non-document changes, _check_doc returns without any check or skipped-check notice. Fix it, or reply why it does not hold, and resolve this comment.

claude-code marked this conversation as resolved

werkrevisor/doccheck.py Outdated

						
				@@ -177,0 +131,4 @@

				    if d.added:

				        return d.added[0][0]

				    if d.removed_at_new:

				        return d.removed_at_new[0]

ai-reviewer commented

2026-10-17 13:18:38 +02:00

[medium] Trailing deletions produce a comment beyond EOF (Codex review)

For a deletion at the end of a file, removed_at_new points one line beyond the new file’s end. Removing lines 3–4 from a four-line file therefore emits a finding at line 3 of a two-line file. The publisher uses this as new_position; Gitea rejects such positions, causing the entire batch of inline comments to fall back to the result comment.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] Trailing deletions produce a comment beyond EOF** (Codex review) For a deletion at the end of a file, removed_at_new points one line beyond the new file’s end. Removing lines 3–4 from a four-line file therefore emits a finding at line 3 of a two-line file. The publisher uses this as new_position; Gitea rejects such positions, causing the entire batch of inline comments to fall back to the result comment. Fix it, or reply why it does not hold, and resolve this comment.

claude-code marked this conversation as resolved

ai-reviewer commented

2026-10-17 13:18:38 +02:00

⚠️ werkrevisor ERROR for ?, superseded by the result for 837bae4

Codex review: critical

This changes the merge gate’s checks for document contradictions and data outflow. Mistakes could let unsafe changes merge, and tests using fixed Jev answers cannot establish the effectiveness of the new judgments.

Findings

[high] Deleted files silently bypass the document check at werkrevisor/doccheck.py:116, as a line comment
[medium] Trailing deletions produce a comment beyond EOF at werkrevisor/doccheck.py:134, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

_{Model `gpt-6-astra`, effort high, tokens in 436018 (cached 363136) / out 2447, 82.9 s, 11 commands; reviewed head `a654d8227c3cebf7f104fd6c8f86e6769e3a9347`.}

<details><summary>⚠️ werkrevisor ERROR for <code>?</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6174">the result for <code>837bae4</code></a></summary> ## Codex review: critical This changes the merge gate’s checks for document contradictions and data outflow. Mistakes could let unsafe changes merge, and tests using fixed Jev answers cannot establish the effectiveness of the new judgments. ### Findings 1. **[high] Deleted files silently bypass the document check** at `werkrevisor/doccheck.py:116`, as a line comment 2. **[medium] Trailing deletions produce a comment beyond EOF** at `werkrevisor/doccheck.py:134`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. Model `gpt-6-astra`, effort high, tokens in 436018 (cached 363136) / out 2447, 82.9 s, 11 commands; reviewed head `a654d8227c3cebf7f104fd6c8f86e6769e3a9347`. </details>

claude-code added 1 commit 2026-10-17 13:26:12 +02:00

werkrevisor 0.45.0: doc check judges deletions, anchors them at the PR document

ai-review/codex 1 open Codex finding(s); resolve each line comment

werkator build succeeded after 00:10

Details

werkator/werkrevisor-statistics build succeeded after 00:28

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; a human review request is pending; merge already scheduled

Details

ai-review/gate build succeeded after 02:51

Details

837bae436d

Codex review: a deleted or deletion-only file had no new-version line to anchor a
comment, so it was skipped (deletions bypassed the check) or anchored beyond EOF.
Such files are now judged too; their hints join one comment at the PR document.
Reword Scenario#117.04 to observable behavior; fix two README writing-rule hints.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

claude-code commented

2026-10-17 13:28:15 +02:00

Review round for 837bae4.

Fixed:

Codex [high] deleted files bypass the check (werkrevisor/doccheck.py): a deleted or deletion-only file is now judged too.
Its hint joins one comment at the PR document, since it has no line in the new version. Fixed in 837bae4.
Codex [medium] trailing deletions anchored beyond EOF (werkrevisor/doccheck.py): a file with no added line now anchors at the PR document, never at a removed-line position. Fixed in 837bae4.
YELLOW spec Scenario#117.04 implementation_details: reworded to observable behaviour. Fixed in 837bae4.
Writing hints README.md: the paragraph is split to five sentences, and the two-sentence line is split. Fixed in 837bae4.

With evidence, no change:

ORANGE weakens checks and YELLOW "removes 3 tests" in tests/test_provider_order.py: the removed cases covered the [doc_check] provider_order and the economy-model change summary, both removed in this PR.
The shared _validate_provider_order stays covered by the [model] case, and the OpenRouter provider-order path by the two model-review tests; no real coverage is lost.
YELLOW "removes 25 tests" in tests/test_doccheck.py: the removed tests exercised the economy-model summary (summarize, parse_summary, FakeSummarizer), which this PR removes.
The new per-file tests cover every surviving behaviour: contradictions (ORANGE and hint), unrelated, data outflow, one call per file, mechanical skip, deletions, the resolved-hint rule, the usage record and the config validation.

For the human (structural):

YELLOW PR too large (918 > 800 lines) and the config/hsadmin-ng.toml change both need your approval; neither is auto-fixable.

738 tests pass in the PR-Agent image.

Review round for `837bae4`. Fixed: - Codex [high] deleted files bypass the check (`werkrevisor/doccheck.py`): a deleted or deletion-only file is now judged too. Its hint joins one comment at the PR document, since it has no line in the new version. Fixed in `837bae4`. - Codex [medium] trailing deletions anchored beyond EOF (`werkrevisor/doccheck.py`): a file with no added line now anchors at the PR document, never at a removed-line position. Fixed in `837bae4`. - YELLOW spec `Scenario#117.04` implementation_details: reworded to observable behaviour. Fixed in `837bae4`. - Writing hints `README.md`: the paragraph is split to five sentences, and the two-sentence line is split. Fixed in `837bae4`. With evidence, no change: - ORANGE weakens checks and YELLOW "removes 3 tests" in `tests/test_provider_order.py`: the removed cases covered the `[doc_check]` provider_order and the economy-model change summary, both removed in this PR. The shared `_validate_provider_order` stays covered by the `[model]` case, and the OpenRouter provider-order path by the two model-review tests; no real coverage is lost. - YELLOW "removes 25 tests" in `tests/test_doccheck.py`: the removed tests exercised the economy-model summary (`summarize`, `parse_summary`, `FakeSummarizer`), which this PR removes. The new per-file tests cover every surviving behaviour: contradictions (ORANGE and hint), unrelated, data outflow, one call per file, mechanical skip, deletions, the resolved-hint rule, the usage record and the config validation. For the human (structural): - YELLOW PR too large (918 > 800 lines) and the `config/hsadmin-ng.toml` change both need your approval; neither is auto-fixable. 738 tests pass in the PR-Agent image.

ai-reviewer commented

2026-10-17 13:29:07 +02:00

🟠 werkrevisor ORANGE for 837bae4, superseded by the result for 6d8523f

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: mixed.

Reasons

ORANGE changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88)
YELLOW tests lost or disabled: tests/test_doccheck.py:85 removes 19 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement
YELLOW PR too large for an AI-only approval: 971 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself
hint 10 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 15 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 6 change(s) belong to the PR document, according to Jev
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py
hint files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file)

Classifier

15 of 15 units passed all questions (1 PR document(s)).
8 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 82828, calls 1, tokens in 79604 / out 24119 (reasoning included in out), duration 169.8 s, cost 0.0528 USD (reported), classifier `jev-1.13.0` calls 15, tokens in 12972, cost 0.0005 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 29905 / out 555, cost 0.0006 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 23958 / out 3389, cost 0.0112 USD (reported), PR document check `jev-1.13.0` calls 21, tokens in 45560, cost 0.0019 USD, build file judge `jev-1.13.0` calls 12, tokens in 24660, cost 0.0010 USD, phases: path_judge 0.7 s, classifier 1.0 s, change_summary 16.0 s, doc_check 1.4 s, context 7.6 s, model_review 142.6 s, config: defaults + host + repository (.werkrevisor.toml at main f34da27).

Reviewed head `837bae436d32a3b8a28eb3a4746a70794ab1dc34` against base `f34da2710db793fdfe2139a98a39b88c09aa30ec`, werkrevisor 0.44.0.}

<details><summary>🟠 werkrevisor ORANGE for <code>837bae4</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6183">the result for <code>6d8523f</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `mixed`. ### Reasons - **ORANGE** changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88) - **YELLOW** tests lost or disabled: tests/test_doccheck.py:85 removes 19 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** PR too large for an AI-only approval: 971 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself - **hint** 10 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 15 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 6 change(s) belong to the PR document, according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py - **hint** files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file) ### Classifier 15 of 15 units passed all questions (1 PR document(s)). 8 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 82828, calls 1, tokens in 79604 / out 24119 (reasoning included in out), duration 169.8 s, cost 0.0528 USD (reported), classifier `jev-1.13.0` calls 15, tokens in 12972, cost 0.0005 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 29905 / out 555, cost 0.0006 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 23958 / out 3389, cost 0.0112 USD (reported), PR document check `jev-1.13.0` calls 21, tokens in 45560, cost 0.0019 USD, build file judge `jev-1.13.0` calls 12, tokens in 24660, cost 0.0010 USD, phases: path_judge 0.7 s, classifier 1.0 s, change_summary 16.0 s, doc_check 1.4 s, context 7.6 s, model_review 142.6 s, config: defaults + host + repository (.werkrevisor.toml at main f34da27). Reviewed head `837bae436d32a3b8a28eb3a4746a70794ab1dc34` against base `f34da2710db793fdfe2139a98a39b88c09aa30ec`, werkrevisor 0.44.0. </details>

ai-reviewer reviewed

2026-10-17 13:31:48 +02:00

ai-reviewer left a comment

Codex review of 837bae436d32a3b8a28eb3a4746a70794ab1dc34

Codex review of `837bae436d32a3b8a28eb3a4746a70794ab1dc34`

werkrevisor/doccheck.py

						
				@@ -273,1 +239,4 @@

				        lines = [f"Jev compared the diff of this file with the PR document: {_rated(tripped)}.",

				                 "Fix the PR document or the code, or reply why the file belongs to the PR, and resolve this comment.",

				                 fingerprint_note(row["fingerprint"])]

				        run.findings.append(Finding(file=row["file"], start_line=row["line"], end_line=row["line"],

ai-reviewer commented

2026-10-17 13:31:48 +02:00

[high] Document-check hints on documentation no longer hold the merge (Codex review)

Anchoring findings directly on files such as README.md subjects them to _plan_line_comments’ documentation exemption. With the default doc_hint_line_comments=false, these GREEN findings become free_hints: no line comment is posted and no resolution is required. Previously, document-check findings were anchored on the exempt PR document. Reproduced with doc_change_unrelated=0.9 for README.md: the review remains GREEN and schedules the merge despite the unresolved finding.

Fix it, or reply why it does not hold, and resolve this comment.

**[high] Document-check hints on documentation no longer hold the merge** (Codex review) Anchoring findings directly on files such as README.md subjects them to _plan_line_comments’ documentation exemption. With the default doc_hint_line_comments=false, these GREEN findings become free_hints: no line comment is posted and no resolution is required. Previously, document-check findings were anchored on the exempt PR document. Reproduced with doc_change_unrelated=0.9 for README.md: the review remains GREEN and schedules the merge despite the unresolved finding. Fix it, or reply why it does not hold, and resolve this comment.

claude-code marked this conversation as resolved

ai-reviewer commented

2026-10-17 13:31:49 +02:00

⚠️ werkrevisor ERROR for ?, superseded by the result for 6d8523f

Codex review: critical

This changes the merge gate’s review decisions and detection of data outflow. Mistakes could allow harmful changes to merge, while mocked classifier tests cannot establish real detection reliability.

Findings

[high] Document-check hints on documentation no longer hold the merge at werkrevisor/doccheck.py:242, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

_{Model `gpt-6-astra`, effort high, tokens in 891326 (cached 816128) / out 4000, 128.7 s, 17 commands; reviewed head `837bae436d32a3b8a28eb3a4746a70794ab1dc34`.}

<details><summary>⚠️ werkrevisor ERROR for <code>?</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6183">the result for <code>6d8523f</code></a></summary> ## Codex review: critical This changes the merge gate’s review decisions and detection of data outflow. Mistakes could allow harmful changes to merge, while mocked classifier tests cannot establish real detection reliability. ### Findings 1. **[high] Document-check hints on documentation no longer hold the merge** at `werkrevisor/doccheck.py:242`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. Model `gpt-6-astra`, effort high, tokens in 891326 (cached 816128) / out 4000, 128.7 s, 17 commands; reviewed head `837bae436d32a3b8a28eb3a4746a70794ab1dc34`. </details>

claude-code added 1 commit 2026-10-17 13:36:00 +02:00

werkrevisor 0.45.0: doc-check hints hold the merge on documentation files

ai-review/codex Codex review: a human approval of the head unlocks the merge

werkator build succeeded after 00:11

Details

werkator/werkrevisor-statistics build succeeded after 00:30

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; no human review requested; merge conflict with main: the author rebases the branch onto main and pushes

Details

ai-review/gate build succeeded after 02:37

Details

6d8523fc44

Codex review [high]: anchoring doc-check findings on the changed file let a hint on a
Markdown file fall under the documentation free-hint exception of _plan_line_comments,
so with doc_hint_line_comments off it no longer held the merge. A finding now carries
from_doc_check, which exempts it from that exception.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

claude-code commented

2026-10-17 13:36:21 +02:00

Codex [high] "Document-check hints on documentation no longer hold the merge" (werkrevisor/doccheck.py): fixed in 6d8523f.

A PR document check finding now carries from_doc_check, and _plan_line_comments exempts such a finding from the documentation free-hint exception.
So a hint on a changed .md file (unrelated change or data outflow) stays a line comment and holds the merge, even with doc_hint_line_comments off.
Covered by test_a_hint_on_a_markdown_file_holds_the_merge. 739 tests pass in the PR-Agent image.

Codex [high] "Document-check hints on documentation no longer hold the merge" (`werkrevisor/doccheck.py`): fixed in `6d8523f`. A PR document check finding now carries `from_doc_check`, and `_plan_line_comments` exempts such a finding from the documentation free-hint exception. So a hint on a changed `.md` file (unrelated change or data outflow) stays a line comment and holds the merge, even with `doc_hint_line_comments` off. Covered by `test_a_hint_on_a_markdown_file_holds_the_merge`. 739 tests pass in the PR-Agent image.

ai-reviewer commented

2026-10-17 13:38:30 +02:00

🟠 werkrevisor ORANGE for 6d8523f, superseded by the result for 2069cc3

🟠 werkrevisor: ORANGE / FINDINGS

Gate: no human review requested; merge conflict with main: the author rebases the branch onto main and pushes.

Change kind: mixed.

Reasons

ORANGE changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88)
YELLOW tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement
YELLOW PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself
hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 18 change(s) belong to the PR document, according to Jev
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py
hint 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/verdict.py (advisory: Jev judged every changed file)
hint files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file)

Classifier

16 of 16 units passed all questions (1 PR document(s)).
8 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 86341, calls 1, tokens in 82982 / out 26894 (reasoning included in out), duration 149.4 s, cost 0.0571 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 13976, cost 0.0006 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 796, cost 0.0039 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24582 / out 3040, cost 0.0110 USD (reported), PR document check `jev-1.13.0` calls 34, tokens in 93593, cost 0.0039 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 15.0 s, doc_check 2.1 s, context 12.0 s, model_review 118.0 s, config: defaults + host + repository (.werkrevisor.toml at main f34da27).

Reviewed head `6d8523fc44c5ab25b4bf86410cd91e48ca507ee4` against base `f34da2710db793fdfe2139a98a39b88c09aa30ec`, werkrevisor 0.44.0.}

<details><summary>🟠 werkrevisor ORANGE for <code>6d8523f</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6204">the result for <code>2069cc3</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Gate: no human review requested; merge conflict with main: the author rebases the branch onto main and pushes. Change kind: `mixed`. ### Reasons - **ORANGE** changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88) - **YELLOW** tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 18 change(s) belong to the PR document, according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py - **hint** 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/verdict.py (advisory: Jev judged every changed file) - **hint** files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file) ### Classifier 16 of 16 units passed all questions (1 PR document(s)). 8 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 86341, calls 1, tokens in 82982 / out 26894 (reasoning included in out), duration 149.4 s, cost 0.0571 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 13976, cost 0.0006 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 796, cost 0.0039 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24582 / out 3040, cost 0.0110 USD (reported), PR document check `jev-1.13.0` calls 34, tokens in 93593, cost 0.0039 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 15.0 s, doc_check 2.1 s, context 12.0 s, model_review 118.0 s, config: defaults + host + repository (.werkrevisor.toml at main f34da27). Reviewed head `6d8523fc44c5ab25b4bf86410cd91e48ca507ee4` against base `f34da2710db793fdfe2139a98a39b88c09aa30ec`, werkrevisor 0.44.0. </details>

ai-reviewer commented

2026-10-17 13:43:18 +02:00

⚠️ werkrevisor ERROR for ?, superseded by the result for 2069cc3

Codex review: critical

This replaces checks that determine whether changes can merge automatically, including contradiction detection and security-related data-outflow hints. Incorrect judgments could let harmful changes through, while the fake-model tests do not verify actual detection accuracy.

No findings.

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

_{Model `gpt-6-astra`, effort high, tokens in 506873 (cached 429056) / out 2450, 71.8 s, 19 commands; reviewed head `6d8523fc44c5ab25b4bf86410cd91e48ca507ee4`.}

<details><summary>⚠️ werkrevisor ERROR for <code>?</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6204">the result for <code>2069cc3</code></a></summary> ## Codex review: critical This replaces checks that determine whether changes can merge automatically, including contradiction detection and security-related data-outflow hints. Incorrect judgments could let harmful changes through, while the fake-model tests do not verify actual detection accuracy. No findings. The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. Model `gpt-6-astra`, effort high, tokens in 506873 (cached 429056) / out 2450, 71.8 s, 19 commands; reviewed head `6d8523fc44c5ab25b4bf86410cd91e48ca507ee4`. </details>

claude-code added 1 commit 2026-10-17 13:59:04 +02:00

werkrevisor 0.45.0: PR document measurement re-run live against hs.hsadmin.ng

werkator build succeeded after 00:10

Details

werkator/werkrevisor-statistics build succeeded after 00:33

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; no human review requested; merge conflict with main: the author rebases the branch onto main and pushes

Details

ai-review/gate build succeeded after 02:40

Details

ai-review/codex 1 open Codex finding(s); resolve each line comment

2069cc3614

Jev jev-latest, 103 non-trivial main-code file diffs from 15 merged PRs (#299-#329).
Data-outflow: backdoor 6/6 from 0.97 (median 0.98), 0/103 real files from 0.5 (max 0.12).
The unrelated question misses the on-topic backdoor and flags 3 real files (unrelated_min follow-up).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

ai-reviewer commented

2026-10-17 14:01:41 +02:00

🟠 werkrevisor ORANGE for 2069cc3, superseded by the result for c175f10

🟠 werkrevisor: ORANGE / FINDINGS

Gate: no human review requested; merge conflict with main: the author rebases the branch onto main and pushes.

Change kind: mixed.

Reasons

ORANGE changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88)
YELLOW tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement
YELLOW PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself
hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 11 change(s) belong to the PR document, according to Jev
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
hint 2 changed files were given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py, werkrevisor/verdict.py
hint files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file)
hint 1 finding(s) of low severity or rated minor by Jev: werkrevisor/config.py:193-194
hint model recommends merge with caution

Findings

Inconsistent measurement numbers at werkrevisor/config.py:193-194, severity unknown, counted as medium, hint, as a line comment

Classifier

16 of 16 units passed all questions (1 PR document(s)).
8 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 84231, calls 1, tokens in 80974 / out 16522 (reasoning included in out), duration 152.6 s, cost 0.0441 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 14171, cost 0.0006 USD, finding judge `jev-1.13.0` calls 1, tokens in 1716, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 984, cost 0.0008 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24592 / out 3843, cost 0.0120 USD (reported), PR document check `jev-1.13.0` calls 27, tokens in 73286, cost 0.0031 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 16.6 s, doc_check 1.6 s, context 14.2 s, model_review 117.6 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main d0b4dfe).

Reviewed head `2069cc3614653327514df2da3e7504805007e90d` against base `d0b4dfed34977f793140a47b72d9fa7181d8796d`, werkrevisor 0.44.1.}

<details><summary>🟠 werkrevisor ORANGE for <code>2069cc3</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6221">the result for <code>c175f10</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Gate: no human review requested; merge conflict with main: the author rebases the branch onto main and pushes. Change kind: `mixed`. ### Reasons - **ORANGE** changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88) - **YELLOW** tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 11 change(s) belong to the PR document, according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 2 changed files were given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py, werkrevisor/verdict.py - **hint** files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file) - **hint** 1 finding(s) of low severity or rated minor by Jev: werkrevisor/config.py:193-194 - **hint** model recommends merge with caution ### Findings 1. **Inconsistent measurement numbers** at `werkrevisor/config.py:193-194`, severity unknown, counted as medium, hint, as a line comment ### Classifier 16 of 16 units passed all questions (1 PR document(s)). 8 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 84231, calls 1, tokens in 80974 / out 16522 (reasoning included in out), duration 152.6 s, cost 0.0441 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 14171, cost 0.0006 USD, finding judge `jev-1.13.0` calls 1, tokens in 1716, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 984, cost 0.0008 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24592 / out 3843, cost 0.0120 USD (reported), PR document check `jev-1.13.0` calls 27, tokens in 73286, cost 0.0031 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 16.6 s, doc_check 1.6 s, context 14.2 s, model_review 117.6 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main d0b4dfe). Reviewed head `2069cc3614653327514df2da3e7504805007e90d` against base `d0b4dfed34977f793140a47b72d9fa7181d8796d`, werkrevisor 0.44.1. </details>

ai-reviewer reviewed

2026-10-17 14:01:41 +02:00

ai-reviewer left a comment

werkrevisor: ORANGE / FINDINGS for 2069cc3614653327514df2da3e7504805007e90d.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `2069cc3614653327514df2da3e7504805007e90d`. The result comment of werkrevisor holds the details.

werkrevisor/config.py

						
				@@ -198,3 +191,3 @@

				    # Measured on 36 merged PRs with planted changes; see the PR document of PR#116.

				    unrelated_min: float = 0.7

				    provider_order: tuple = ()

				    # Measured on 2026-10-17: 6 of 6 backdoors from 0.7 on, 0 false alarms on 109 real main-code files.

ai-reviewer commented

2026-10-17 14:01:41 +02:00

Documentation inconsistency on file count, severity unknown, counted as medium (werkrevisor)

hint; Jev: real 0.82, severity 1.6 of 3

The config comment for exfil_min claims "0 false alarms on 109 real main-code files", but the README, PR document, and werkbaum #doc.exfilmin all say "103 real files". The werkbaum #doc.exfil repeats the "109 real files" wording. The number must be consistent across the config comment and the documentation so a reader calibrating the threshold sees the same false-positive set everywhere.

_{Updated by the review of 66855b1590eb2ab3288882b3243d89aab0ca1f94.}

**Documentation inconsistency on file count**, severity unknown, counted as medium (werkrevisor) hint; Jev: real 0.82, severity 1.6 of 3 The config comment for `exfil_min` claims "0 false alarms on 109 real main-code files", but the README, PR document, and werkbaum `#doc.exfilmin` all say "103 real files". The werkbaum `#doc.exfil` repeats the "109 real files" wording. The number must be consistent across the config comment and the documentation so a reader calibrating the threshold sees the same false-positive set everywhere. Updated by the review of `66855b1590eb2ab3288882b3243d89aab0ca1f94`.

ai-reviewer reviewed

2026-10-17 14:08:31 +02:00

ai-reviewer left a comment

Codex review of 2069cc3614653327514df2da3e7504805007e90d

Codex review of `2069cc3614653327514df2da3e7504805007e90d`

werkrevisor/doccheck.py

						
				@@ -261,3 +223,3 @@

				            continue

				        orange = flagged.get(ORANGE_QUESTION, 0.0) >= config.orange_min

				        orange = row["values"].get(ORANGE_QUESTION, 0.0) >= config.orange_min

				        if not orange and row["fingerprint"] in answered:

ai-reviewer commented

2026-10-17 14:08:31 +02:00

[medium] A resolved hint suppresses newly detected issues for the same file (Codex review)

The resolved fingerprint identifies only the file, diff and PR document, not which questions the author answered. If an unrelated-change hint is resolved and a later review detects data outflow in the unchanged file, this branch silently discards the new finding. An in-memory reproduction with a resolved unrelated-change hint and a subsequent data-outflow score of 0.98 produces zero findings and counts it as answered. Resolution must distinguish previously reported questions from newly detected ones.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] A resolved hint suppresses newly detected issues for the same file** (Codex review) The resolved fingerprint identifies only the file, diff and PR document, not which questions the author answered. If an unrelated-change hint is resolved and a later review detects data outflow in the unchanged file, this branch silently discards the new finding. An in-memory reproduction with a resolved unrelated-change hint and a subsequent data-outflow score of 0.98 produces zero findings and counts it as answered. Resolution must distinguish previously reported questions from newly detected ones. Fix it, or reply why it does not hold, and resolve this comment.

ai-reviewer commented

2026-10-17 14:08:31 +02:00

⚠️ werkrevisor ERROR for ?, superseded by the result for c175f10

Codex review: critical

This changes the review judgments and comment handling that control automatic merges, including a new data-outflow check. Mistakes could let unsafe changes merge, and mocked model responses cannot establish that the gate remains effective.

Findings

[medium] A resolved hint suppresses newly detected issues for the same file at werkrevisor/doccheck.py:225, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

_{Model `gpt-6-astra`, effort high, tokens in 639920 (cached 563840) / out 3002, 104.6 s, 20 commands; reviewed head `2069cc3614653327514df2da3e7504805007e90d`.}

<details><summary>⚠️ werkrevisor ERROR for <code>?</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6221">the result for <code>c175f10</code></a></summary> ## Codex review: critical This changes the review judgments and comment handling that control automatic merges, including a new data-outflow check. Mistakes could let unsafe changes merge, and mocked model responses cannot establish that the gate remains effective. ### Findings 1. **[medium] A resolved hint suppresses newly detected issues for the same file** at `werkrevisor/doccheck.py:225`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. Model `gpt-6-astra`, effort high, tokens in 639920 (cached 563840) / out 3002, 104.6 s, 20 commands; reviewed head `2069cc3614653327514df2da3e7504805007e90d`. </details>

claude-code added 1 commit 2026-10-17 14:09:46 +02:00

werkrevisor 0.45.0: measurement pairs each PR with its own doc; unrelated_min stays 0.7

ai-review/codex 1 open Codex finding(s); resolve each line comment

werkator build succeeded after 00:09

Details

werkator/werkrevisor-statistics build succeeded after 00:31

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; no human review requested; merge conflict with main: the author rebases the branch onto main and pushes

Details

ai-review/gate build succeeded after 00:23

Details

c175f10599

The earlier 3 unrelated flags were a measurement artifact: a stacked PR carries its base
PR's doc too, and the harness picked the wrong one. With the matching PR document, both
the data-outflow and the unrelated question stay below 0.3 on 103 real file diffs, so
unrelated_min and exfil_min stay at 0.7 (wide gap to the 0.7-0.98 of a planted change).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

ai-reviewer commented

2026-10-17 14:15:36 +02:00

🟠 werkrevisor ORANGE for c175f10, superseded by the result for 66855b1

🟠 werkrevisor: ORANGE / FINDINGS

Gate: no human review requested; merge conflict with main: the author rebases the branch onto main and pushes.

Change kind: mixed.

Reasons

ORANGE changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.89)
YELLOW tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement
YELLOW PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself
YELLOW spec: docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs' hard_to_understand (0.54 > yellow_above = 0.5)
hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 5 change(s) belong to the PR document, according to Jev
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
hint 2 changed files were given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py, werkrevisor/verdict.py
hint files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file)
hint 3 finding(s) of low severity or rated minor by Jev: tests/test_doccheck.py:266-269, docs/werkrevisor.werkbaum:80-92, docs/werkrevisor.werkbaum:90-95
hint model recommends changes before merge

Findings

Weakened test for repeated hint after file edit at tests/test_doccheck.py:266-269, severity unknown, counted as medium, hint, as a line comment
Contradictory file count in measurement at docs/werkrevisor.werkbaum:80-92, severity unknown, counted as medium

hint; Jev: real 0.78, severity 1.8 of 3

The PR document and README state "103 real main-code file diffs" while werkrevisor/config.py (DocCheckConfig comment) and the #doc.exfil entry in the werkbaum state "109 real main-code files". The #doc.exfilmin entry of the same werkbaum then reuses the 103 figure. One of the two numbers is wrong, and the inconsistency between the PR document, the host config comment, and the plan makes the calibration harder to trust.
Duplicate calibration entry at docs/werkrevisor.werkbaum:90-95, severity unknown, counted as medium

hint; Jev: real 0.78, severity 1.1 of 3

The new - [ ] Calibrate hint_min on real PRs; decide whether data-outflow becomes ORANGE (S) immediately precedes the kept - [^] Calibrate hint_min on real PRs (S) that already records the PR#66 calibration. The "hint_min" calibration task is now listed twice with the same scope; the new entry should subsume the old one (or be merged into it) so the plan has one owner for the calibration.

Classifier

Unit	Question	Problem	Confidence	Verdict
docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs'	hard_to_understand	0.54	0.52	YELLOW

15 of 16 units passed all questions (1 PR document(s)).
6 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 85367, calls 1, tokens in 82064 / out 23577 (reasoning included in out), duration 156.6 s, cost 0.0529 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 14295, cost 0.0006 USD, finding judge `jev-1.13.0` calls 3, tokens in 5774, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 709, cost 0.0007 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24651 / out 3153, cost 0.0111 USD (reported), PR document check `jev-1.13.0` calls 21, tokens in 44432, cost 0.0019 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 19.5 s, doc_check 1.4 s, context 12.9 s, model_review 120.1 s, finding_judge 0.3 s, config: defaults + host + repository (.werkrevisor.toml at main d0b4dfe).

Reviewed head `c175f10599168ed377f550221ef90960d7c30e1c` against base `d0b4dfed34977f793140a47b72d9fa7181d8796d`, werkrevisor 0.44.1.}

<details><summary>🟠 werkrevisor ORANGE for <code>c175f10</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6319">the result for <code>66855b1</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Gate: no human review requested; merge conflict with main: the author rebases the branch onto main and pushes. Change kind: `mixed`. ### Reasons - **ORANGE** changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.89) - **YELLOW** tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself - **YELLOW** spec: docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs' hard_to_understand (0.54 > yellow_above = 0.5) - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 5 change(s) belong to the PR document, according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 2 changed files were given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py, werkrevisor/verdict.py - **hint** files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file) - **hint** 3 finding(s) of low severity or rated minor by Jev: tests/test_doccheck.py:266-269, docs/werkrevisor.werkbaum:80-92, docs/werkrevisor.werkbaum:90-95 - **hint** model recommends changes before merge ### Findings 1. **Weakened test for repeated hint after file edit** at `tests/test_doccheck.py:266-269`, severity unknown, counted as medium, hint, as a line comment 2. **Contradictory file count in measurement** at `docs/werkrevisor.werkbaum:80-92`, severity unknown, counted as medium hint; Jev: real 0.78, severity 1.8 of 3 The PR document and README state "103 real main-code file diffs" while `werkrevisor/config.py` (DocCheckConfig comment) and the `#doc.exfil` entry in the werkbaum state "109 real main-code files". The `#doc.exfilmin` entry of the same werkbaum then reuses the 103 figure. One of the two numbers is wrong, and the inconsistency between the PR document, the host config comment, and the plan makes the calibration harder to trust. 3. **Duplicate calibration entry** at `docs/werkrevisor.werkbaum:90-95`, severity unknown, counted as medium hint; Jev: real 0.78, severity 1.1 of 3 The new `- [ ] Calibrate hint_min on real PRs; decide whether data-outflow becomes ORANGE (S)` immediately precedes the kept `- [^] Calibrate hint_min on real PRs (S)` that already records the PR#66 calibration. The "hint_min" calibration task is now listed twice with the same scope; the new entry should subsume the old one (or be merged into it) so the plan has one owner for the calibration. ### Classifier | Unit | Question | Problem | Confidence | Verdict | |---|---|---|---|---| | docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs' | hard_to_understand | 0.54 | 0.52 | YELLOW | 15 of 16 units passed all questions (1 PR document(s)). 6 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 85367, calls 1, tokens in 82064 / out 23577 (reasoning included in out), duration 156.6 s, cost 0.0529 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 14295, cost 0.0006 USD, finding judge `jev-1.13.0` calls 3, tokens in 5774, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 709, cost 0.0007 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24651 / out 3153, cost 0.0111 USD (reported), PR document check `jev-1.13.0` calls 21, tokens in 44432, cost 0.0019 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 0.8 s, classifier 0.9 s, change_summary 19.5 s, doc_check 1.4 s, context 12.9 s, model_review 120.1 s, finding_judge 0.3 s, config: defaults + host + repository (.werkrevisor.toml at main d0b4dfe). Reviewed head `c175f10599168ed377f550221ef90960d7c30e1c` against base `d0b4dfed34977f793140a47b72d9fa7181d8796d`, werkrevisor 0.44.1. </details>

ai-reviewer reviewed

2026-10-17 14:15:36 +02:00

ai-reviewer left a comment

werkrevisor: ORANGE / FINDINGS for c175f10599168ed377f550221ef90960d7c30e1c.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `c175f10599168ed377f550221ef90960d7c30e1c`. The result comment of werkrevisor holds the details.

tests/test_doccheck.py

						
				@@ -275,3 +266,1 @@

				        second_round(self.gitea, self.jev, DOC_TEXT.replace("The PR adds the endpoint", "The PR adds an endpoint"))

				        [comment] = self.gitea.review_comments[-1]

				        self.assertEqual((comment["path"], comment["new_position"]), (DOC, SOLUTION_LINE))

				    def test_a_changed_file_is_checked_again(self):

ai-reviewer commented

2026-10-17 14:15:36 +02:00

Weakened test for repeated hint after file edit, severity unknown, counted as medium (werkrevisor)

hint; Jev: real 0.88, severity 2.0 of 3

The renamed test test_a_changed_file_is_checked_again no longer carries the resolved fingerprints from setUp: it builds a fresh FakeGitea, so the previous orange/hint looks "unanswered" regardless of whether the fingerprint machinery works. With a fresh gitea the test passes even if the fingerprint is broken, and the original Scenario#50.02 assertion (a new comment at the right location with the same setUp state) is gone. To actually test the fingerprint mechanism, the test should reuse self.gitea.line_comments (the resolved fingerprints) while only changing the file diff, or extend second_round so it accepts a different file diff.

**Weakened test for repeated hint after file edit**, severity unknown, counted as medium (werkrevisor) hint; Jev: real 0.88, severity 2.0 of 3 The renamed test `test_a_changed_file_is_checked_again` no longer carries the resolved fingerprints from `setUp`: it builds a fresh `FakeGitea`, so the previous orange/hint looks "unanswered" regardless of whether the fingerprint machinery works. With a fresh gitea the test passes even if the fingerprint is broken, and the original Scenario#50.02 assertion (a new comment at the right location with the same setUp state) is gone. To actually test the fingerprint mechanism, the test should reuse `self.gitea.line_comments` (the resolved fingerprints) while only changing the file diff, or extend `second_round` so it accepts a different file diff.

ai-reviewer commented

2026-10-17 14:22:40 +02:00

⚠️ werkrevisor ERROR for ?, superseded by the result for 66855b1

Codex review: critical

This changes the review gate’s model inputs, security checks, and merge-blocking comments; mistakes could let harmful changes merge without human review, which mocked model tests cannot rule out. Human review is required.

No findings.

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

_{Model `gpt-6-astra`, effort high, tokens in 508014 (cached 430464) / out 2559, 88.3 s, 18 commands; reviewed head `c175f10599168ed377f550221ef90960d7c30e1c`.}

<details><summary>⚠️ werkrevisor ERROR for <code>?</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6319">the result for <code>66855b1</code></a></summary> ## Codex review: critical This changes the review gate’s model inputs, security checks, and merge-blocking comments; mistakes could let harmful changes merge without human review, which mocked model tests cannot rule out. Human review is required. No findings. The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. Model `gpt-6-astra`, effort high, tokens in 508014 (cached 430464) / out 2559, 88.3 s, 18 commands; reviewed head `c175f10599168ed377f550221ef90960d7c30e1c`. </details>

mi approved these changes

2026-10-17 16:38:46 +02:00

Dismissed

mi approved these changes

2026-10-17 16:42:45 +02:00

Dismissed

claude-code changed title from ~~werkrevisor 0.45.0: Jev checks each changed file against the PR document~~ to werkrevisor 0.49.0: Jev checks each changed file against the PR document

2026-10-17 16:50:45 +02:00

claude-code added 1 commit 2026-10-17 16:50:47 +02:00

Merge remote-tracking branch 'origin/main' into claude-code/doccheck-variant-b

werkator build succeeded after 00:26

Details

werkator/werkrevisor-statistics build succeeded after 01:28

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; the approval of mi came before this result; review requested again, please approve again after reading it; merge already scheduled

Details

ai-review/gate build succeeded after 04:10

Details

ai-review/codex 1 open Codex finding(s); resolve each line comment

66855b1590

# Conflicts:
#	statistics/build.gradle.kts
#	werkrevisor/__init__.py

claude-code dismissed mi's review 2026-10-17 16:50:47 +02:00

Reason:

New commits pushed, approval review dismissed automatically according to repository settings

mi approved these changes

2026-10-17 16:52:11 +02:00

ai-reviewer commented

2026-10-17 16:54:59 +02:00

🟠 werkrevisor: ORANGE / FINDINGS

Gate: the approval of mi came before this result; review requested again, please approve again after reading it; merge already scheduled.

Change kind: mixed.

Reasons

ORANGE changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88)
ORANGE 1 finding(s) that Jev rates as probably real and severe: werkrevisor/doccheck.py:114-127
YELLOW tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement
YELLOW PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself
YELLOW spec: docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs' hard_to_understand (0.56 > yellow_above = 0.5)
hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01)
hint 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 8 change(s) belong to the PR document, according to Jev
hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
hint 2 changed files were given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py, werkrevisor/verdict.py
hint files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file)
hint 1 finding(s) of low severity or rated minor by Jev: werkrevisor/config.py:193
hint model recommends merge with caution

Findings

Deleted PR document sent to Jev at werkrevisor/doccheck.py:114-127, severity unknown, counted as medium, ORANGE, as a line comment
Documentation inconsistency on file count at werkrevisor/config.py:193, severity unknown, counted as medium, hint, still open in an earlier line comment

Classifier

Unit	Question	Problem	Confidence	Verdict
docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs'	hard_to_understand	0.56	0.58	YELLOW

15 of 16 units passed all questions (1 PR document(s)).
5 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 86066, calls 1, tokens in 82731 / out 49062 (reasoning included in out), duration 242.1 s, cost 0.0837 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 14295, cost 0.0006 USD, finding judge `jev-1.13.0` calls 2, tokens in 3288, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 1155, cost 0.0049 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24651 / out 2802, cost 0.0107 USD (reported), PR document check `jev-1.13.0` calls 24, tokens in 59131, cost 0.0025 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 1.0 s, classifier 0.9 s, change_summary 13.5 s, doc_check 1.7 s, context 17.3 s, model_review 206.6 s, finding_judge 0.3 s, config: defaults + host + repository (.werkrevisor.toml at main c54e49b).

Reviewed head `66855b1590eb2ab3288882b3243d89aab0ca1f94` against base `c54e49b4b28017ebccad63fe26d9ddcf9fcf5bb3`, werkrevisor 0.48.0.}

## 🟠 werkrevisor: ORANGE / FINDINGS Gate: the approval of mi came before this result; review requested again, please approve again after reading it; merge already scheduled. Change kind: `mixed`. ### Reasons - **ORANGE** changes that weaken the checks according to Jev: tests/test_provider_order.py (weakens checks 0.88) - **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/doccheck.py:114-127 - **YELLOW** tests lost or disabled: tests/test_doccheck.py:85 removes 16 test(s) or assertion(s) without a replacement, tests/test_provider_order.py:51 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** PR too large for an AI-only approval: 991 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself - **YELLOW** spec: docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs' hard_to_understand (0.56 > yellow_above = 0.5) - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.01) - **hint** 16 section(s) of the PR document agree with the change summary of openrouter/minimax/minimax-m3, and its 8 change(s) belong to the PR document, according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 2 changed files were given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py, werkrevisor/verdict.py - **hint** files the model cannot review: docs/werkrevisor-pr-workflow-dark.svg, docs/werkrevisor-pr-workflow.svg, docs/werkrevisor-review-inside-dark.svg and 1 more (advisory: Jev judged every changed file) - **hint** 1 finding(s) of low severity or rated minor by Jev: werkrevisor/config.py:193 - **hint** model recommends merge with caution ### Findings 1. **Deleted PR document sent to Jev** at `werkrevisor/doccheck.py:114-127`, severity unknown, counted as medium, ORANGE, as a line comment 2. **Documentation inconsistency on file count** at `werkrevisor/config.py:193`, severity unknown, counted as medium, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/117#issuecomment-6205) ### Classifier | Unit | Question | Problem | Confidence | Verdict | |---|---|---|---|---| | docs/prs/2026-10-17-PR#117-doccheck-variant-b.md section 'Follow-up PRs' | hard_to_understand | 0.56 | 0.58 | YELLOW | 15 of 16 units passed all questions (1 PR document(s)). 5 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 86066, calls 1, tokens in 82731 / out 49062 (reasoning included in out), duration 242.1 s, cost 0.0837 USD (reported), classifier `jev-1.13.0` calls 16, tokens in 14295, cost 0.0006 USD, finding judge `jev-1.13.0` calls 2, tokens in 3288, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 3, tokens in 34486 / out 1155, cost 0.0049 USD (reported), change summary `openrouter/minimax/minimax-m3` via Minimax calls 1, tokens in 24651 / out 2802, cost 0.0107 USD (reported), PR document check `jev-1.13.0` calls 24, tokens in 59131, cost 0.0025 USD, build file judge `jev-1.13.0` calls 13, tokens in 25757, cost 0.0011 USD, phases: path_judge 1.0 s, classifier 0.9 s, change_summary 13.5 s, doc_check 1.7 s, context 17.3 s, model_review 206.6 s, finding_judge 0.3 s, config: defaults + host + repository (.werkrevisor.toml at main c54e49b). Reviewed head `66855b1590eb2ab3288882b3243d89aab0ca1f94` against base `c54e49b4b28017ebccad63fe26d9ddcf9fcf5bb3`, werkrevisor 0.48.0.

ai-reviewer reviewed

2026-10-17 16:54:59 +02:00

ai-reviewer left a comment

werkrevisor: ORANGE / FINDINGS for 66855b1590eb2ab3288882b3243d89aab0ca1f94.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `66855b1590eb2ab3288882b3243d89aab0ca1f94`. The result comment of werkrevisor holds the details.

werkrevisor/doccheck.py

						
				@@ -135,3 +114,1 @@

				    parts = []

				    for path, d in diffs.items():

				        if path in docs:

				def target_files(files, diffs, docs, mechanical):

ai-reviewer commented

2026-10-17 16:54:59 +02:00

Deleted PR document sent to Jev, severity unknown, counted as medium (werkrevisor)

ORANGE; Jev: real 0.72, severity 2.8 of 3

target_files excludes only paths that appear in docs, and changed_docs filters out deleted files with not f.deleted. A PR that deletes one PR document while another PR document is also changed lets the deleted PR document slip into targets, so Jev is asked to compare its deletion diff against the surviving PR document. The check should treat a deleted PR document the same way it treats a present one and keep it out of the Jev state.

**Deleted PR document sent to Jev**, severity unknown, counted as medium (werkrevisor) ORANGE; Jev: real 0.72, severity 2.8 of 3 `target_files` excludes only paths that appear in `docs`, and `changed_docs` filters out deleted files with `not f.deleted`. A PR that deletes one PR document while another PR document is also changed lets the deleted PR document slip into `targets`, so Jev is asked to compare its deletion diff against the surviving PR document. The check should treat a deleted PR document the same way it treats a present one and keep it out of the Jev state.

ai-reviewer requested review from mi 2026-10-17 16:55:00 +02:00

ai-reviewer commented

2026-10-17 16:57:43 +02:00

Codex review: critical

This changes the automated merge gate’s document checks, security-related data-outflow detection, and which findings hold a merge. Mistakes could let harmful changes merge, and tests with mocked Jev answers cannot establish the real classifier’s reliability.

No findings.

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

_{Model `gpt-6-astra`, effort high, tokens in 626059 (cached 557696) / out 2510, 98.7 s, 14 commands; reviewed head `66855b1590eb2ab3288882b3243d89aab0ca1f94`.}

## Codex review: critical This changes the automated merge gate’s document checks, security-related data-outflow detection, and which findings hold a merge. Mistakes could let harmful changes merge, and tests with mocked Jev answers cannot establish the real classifier’s reliability. No findings. The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. Model `gpt-6-astra`, effort high, tokens in 626059 (cached 557696) / out 2510, 98.7 s, 14 commands; reviewed head `66855b1590eb2ab3288882b3243d89aab0ca1f94`.

mi merged commit c260eab803 into main

2026-10-17 17:10:29 +02:00

mi deleted branch claude-code/doccheck-variant-b

2026-10-17 17:10:29 +02:00

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mi/werkrevisor#117