werkrevisor 0.20.0: a cheap model selects the excerpt of a file too large for the review model #31

2026-10-06T14:16:26+02:00

claude-code commented

2026-10-06 14:16:26 +02:00

See docs/prs/2026-10-06-PR#31-excerpt-selection.md.

🤖 Generated with Claude Code

See [docs/prs/2026-10-06-PR#31-excerpt-selection.md](docs/prs/2026-10-06-PR%2331-excerpt-selection.md). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-code added 2 commits 2026-10-06 14:16:27 +02:00

fix: PR#30 hints: no stale wording about change requests, simpler review event, no duplicate test 48ed8eca5e

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs: PR#31 doc, a cheap model selects the excerpt

werkator build succeeded after 00:31

Details

werkator/server build succeeded after 00:27

Details

04a319c566

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-06 14:21:54 +02:00

excerpt selection by a cheap model: [excerpt] select = "model", validated line ranges, fixed window as fallback

werkator build succeeded after 00:17

Details

werkator/server build succeeded after 00:27

Details

53994f2efb

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-06 14:22:19 +02:00

stats service counts the excerpt selection as LLM cost

werkator build succeeded after 00:10

Details

werkator/server build succeeded after 00:29

Details

9efc42f550

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-06 14:52:33 +02:00

werkrevisor 0.20.0: a cheap model selects the excerpt of a file too large for the review model

werkator build succeeded after 00:08

Details

ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; review requested from mi; merge scheduled for when the checks succeed

Details

ai-review/gate build succeeded after 00:15

Details

werkator/server build succeeded after 00:32

Details

bdf286bf0f

[excerpt] select = "model" lets GPT-6 Luna choose line ranges, checked by werkrevisor, with the fixed window as fallback;
measured on 4 historical heads in private bench repositories, switched on for the example host configs of the mi repositories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-code marked the pull request as ready for review 2026-10-06 14:52:33 +02:00

ai-reviewer commented

2026-10-06 14:52:55 +02:00

🟠 werkrevisor ORANGE for bdf286b, superseded by the result for 8f71589

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: mixed.

Reasons

ORANGE 2 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:261-267, werkrevisor/excerpt.py:141-143
YELLOW tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement
YELLOW risky paths changed: werkrevisor/config.py (werkrevisor/config.py)
YELLOW spec: docs/prs/2026-10-06-PR#31-excerpt-selection.md Scenario#31.01 given_when_then_misused (0.66 > yellow_above = 0.6)
YELLOW 2 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/report.py
hint writing rules violated in the changed docs: README.md:453: more than one sentence on a line; README.md:466: line longer than 128 characters (134)
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.03, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:60-66
hint model recommends merge with caution

Findings

[medium] Secrets in the diff do not stop the selection at werkrevisor/cli.py:261-267, ORANGE, as a line comment
[medium] Unhandled non-numeric usage values abort selection at werkrevisor/excerpt.py:141-143, ORANGE, as a line comment
[high] Weakened assertion in RED test at tests/test_line_comments.py:60-66, hint, as a line comment

Classifier

Unit	Question	Problem	Confidence	Verdict
docs/prs/2026-10-06-PR#31-excerpt-selection.md Scenario#31.01	given_when_then_misused	0.66	–	YELLOW

8 of 9 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 99492, calls 1, tokens in 98581 / out 675 (reasoning included in out), duration 13.4 s, cost 0.2039 USD (reported), classifier `jev-1.13.0` calls 9, tokens in 7513, cost 0.0003 USD, finding judge `jev-1.13.0` calls 3, tokens in 4308, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6).

Reviewed head `bdf286bf0f333cc684cdde82be8d418ba0e247e7` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3.}

<details><summary>🟠 werkrevisor ORANGE for <code>bdf286b</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/31#issuecomment-1340">the result for <code>8f71589</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `mixed`. ### Reasons - **ORANGE** 2 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:261-267, werkrevisor/excerpt.py:141-143 - **YELLOW** tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** risky paths changed: werkrevisor/config.py (werkrevisor/config.py) - **YELLOW** spec: docs/prs/2026-10-06-PR#31-excerpt-selection.md Scenario#31.01 given_when_then_misused (0.66 > yellow_above = 0.6) - **YELLOW** 2 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/report.py - **hint** writing rules violated in the changed docs: README.md:453: more than one sentence on a line; README.md:466: line longer than 128 characters (134) - **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.03, risk 0.00) - **hint** 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:60-66 - **hint** model recommends merge with caution ### Findings 1. **[medium] Secrets in the diff do not stop the selection** at `werkrevisor/cli.py:261-267`, ORANGE, as a line comment 2. **[medium] Unhandled non-numeric usage values abort selection** at `werkrevisor/excerpt.py:141-143`, ORANGE, as a line comment 3. **[high] Weakened assertion in RED test** at `tests/test_line_comments.py:60-66`, hint, as a line comment ### Classifier | Unit | Question | Problem | Confidence | Verdict | |---|---|---|---|---| | docs/prs/2026-10-06-PR#31-excerpt-selection.md Scenario#31.01 | given_when_then_misused | 0.66 | – | YELLOW | 8 of 9 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 99492, calls 1, tokens in 98581 / out 675 (reasoning included in out), duration 13.4 s, cost 0.2039 USD (reported), classifier `jev-1.13.0` calls 9, tokens in 7513, cost 0.0003 USD, finding judge `jev-1.13.0` calls 3, tokens in 4308, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6). Reviewed head `bdf286bf0f333cc684cdde82be8d418ba0e247e7` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3. </details>

ai-reviewer reviewed

2026-10-06 14:52:55 +02:00

ai-reviewer left a comment

werkrevisor: ORANGE / FINDINGS for bdf286bf0f333cc684cdde82be8d418ba0e247e7.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `bdf286bf0f333cc684cdde82be8d418ba0e247e7`. The result comment of werkrevisor holds the details.

tests/test_line_comments.py

						
				@@ -63,4 +59,2 @@

				        self.assertIs(outcome.result.verdict, Verdict.RED)

				        self.assertEqual([r[0] for r in gitea.created_reviews], ["COMMENT"])

				    def test_red_gives_a_comment_review_and_a_failed_gate(self):

ai-reviewer commented

2026-10-06 14:52:55 +02:00

[high] Weakened assertion in RED test (werkrevisor)

hint; Jev: real 0.46, severity 1.9 of 3

The assertion assertNotIn("REQUEST_CHANGES", ...) was removed from test_red_gives_a_comment_review_and_a_failed_gate. The remaining assertEqual(..., ["COMMENT"]) implies it, so strictness is mostly kept. Still, the removal is a formal weakening of an existing test, as the instruction requires reporting.

**[high] Weakened assertion in RED test** (werkrevisor) hint; Jev: real 0.46, severity 1.9 of 3 The assertion `assertNotIn("REQUEST_CHANGES", ...)` was removed from `test_red_gives_a_comment_review_and_a_failed_gate`. The remaining `assertEqual(..., ["COMMENT"])` implies it, so strictness is mostly kept. Still, the removal is a formal weakening of an existing test, as the instruction requires reporting.

claude-code commented

2026-10-06 15:10:56 +02:00

Not changed: this is the PR#30 follow-up. The RED case moved from test_review_kind_follows_the_verdict into test_red_gives_a_comment_review_and_a_failed_gate, which asserts the review list ["COMMENT"] and exit code 20; that list excludes REQUEST_CHANGES, so the removed assertNotIn added nothing. Left open for the human reviewer.

🤖 Generated with Claude Code

Not changed: this is the PR#30 follow-up. The RED case moved from `test_review_kind_follows_the_verdict` into `test_red_gives_a_comment_review_and_a_failed_gate`, which asserts the review list `["COMMENT"]` and exit code 20; that list excludes `REQUEST_CHANGES`, so the removed `assertNotIn` added nothing. Left open for the human reviewer. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

werkrevisor/cli.py Outdated

						
				@@ -248,6 +258,15 @@ def _judge_db_schema(config, deps, use_llm, result, rules, diffs, fetch, merge_b

				    result.reasons.extend(dbschema.reasons(run, config.db_schema))

				def _excerpt_selector(config, deps, use_llm, result):

ai-reviewer commented

2026-10-06 14:52:55 +02:00

[medium] Secrets in the diff do not stop the selection (werkrevisor)

ORANGE; Jev: real 0.85, severity 3.0 of 3

The README and the PR document say a diff with a secret gets no selection. _excerpt_selector does not check for secrets reasons in rules.reasons, unlike _classify, _judge_db_schema and _judge_paths. The diff and the full file text, which may hold a secret, are sent to the cheap model at OpenRouter.

**[medium] Secrets in the diff do not stop the selection** (werkrevisor) ORANGE; Jev: real 0.85, severity 3.0 of 3 The README and the PR document say a diff with a secret gets no selection. `_excerpt_selector` does not check for `secrets` reasons in `rules.reasons`, unlike `_classify`, `_judge_db_schema` and `_judge_paths`. The diff and the full file text, which may hold a secret, are sent to the cheap model at OpenRouter.

claude-code commented

2026-10-06 15:10:56 +02:00

Not a defect in the current flow, but guarded explicitly now: rules.py always sets skip_llm for a secret, so cli.py collects no full files and calls no selection. Since the last commit _excerpt_selector also checks the secrets reason itself; test_a_secret_in_the_diff_gets_no_selection covers it.

🤖 Generated with Claude Code

Not a defect in the current flow, but guarded explicitly now: `rules.py` always sets `skip_llm` for a secret, so `cli.py` collects no full files and calls no selection. Since the last commit `_excerpt_selector` also checks the `secrets` reason itself; `test_a_secret_in_the_diff_gets_no_selection` covers it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-code marked this conversation as resolved

werkrevisor/excerpt.py Outdated

						
				@@ -0,0 +138,4 @@

				    return select

				def _account(run, usage, config):

ai-reviewer commented

2026-10-06 14:52:55 +02:00

[medium] Unhandled non-numeric usage values abort selection (werkrevisor)

ORANGE; Jev: real 0.84, severity 2.9 of 3

_account calls int(usage.get("prompt_tokens") or 0). If OpenRouter reports a non-numeric value, this raises ValueError, which make_selector does not catch (it catches only SelectionError). The exception propagates out of collect and the whole review ends as ERROR instead of falling back to the fixed window.

**[medium] Unhandled non-numeric usage values abort selection** (werkrevisor) ORANGE; Jev: real 0.84, severity 2.9 of 3 `_account` calls `int(usage.get("prompt_tokens") or 0)`. If OpenRouter reports a non-numeric value, this raises `ValueError`, which `make_selector` does not catch (it catches only `SelectionError`). The exception propagates out of `collect` and the whole review ends as ERROR instead of falling back to the fixed window.

claude-code commented

2026-10-06 15:10:55 +02:00

Fixed in 8f71589: a malformed usage field (no integer, a boolean, a list instead of an object) now makes the selection cost an estimate instead of ending the review. Test test_malformed_usage_values_make_the_cost_an_estimate, red without the fix.

🤖 Generated with Claude Code

Fixed in 8f71589: a malformed usage field (no integer, a boolean, a list instead of an object) now makes the selection cost an estimate instead of ending the review. Test `test_malformed_usage_values_make_the_cost_an_estimate`, red without the fix. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-code marked this conversation as resolved

ai-reviewer scheduled this pull request to auto merge when all checks succeed 2026-10-06 14:52:55 +02:00

ai-reviewer requested review from mi 2026-10-06 14:52:55 +02:00

mi added 1 commit 2026-10-06 14:54:06 +02:00

review: malformed usage values make the selection cost an estimate; a secret in the diff gets no selection, tested

ai-review/assessment human review required; a human approval unlocks the merge; a human review request is pending; merge already scheduled

Details

ai-review/gate build succeeded after 00:10

Details

werkator build succeeded after 00:12

Details

werkator/server build succeeded after 00:28

Details

8f71589e5b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented

2026-10-06 14:54:22 +02:00

🟡 werkrevisor YELLOW for 8f71589, superseded by the result for 0abde98

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

YELLOW tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement
YELLOW risky paths changed: werkrevisor/config.py (werkrevisor/config.py)
YELLOW 2 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/report.py
YELLOW no AI review: the PR needs 100563 input tokens (64879 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 100000 derived from [budget] max_usd_per_review = 0.48
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)

Classifier

9 of 9 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 100563, duration 6.5 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 9, tokens in 7589, cost 0.0003 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6).

Reviewed head `8f71589e5bb8ae9227148b60cb87eabfb254a588` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3.}

<details><summary>🟡 werkrevisor YELLOW for <code>8f71589</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/31#issuecomment-1342">the result for <code>0abde98</code></a></summary> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** risky paths changed: werkrevisor/config.py (werkrevisor/config.py) - **YELLOW** 2 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/report.py - **YELLOW** no AI review: the PR needs 100563 input tokens (64879 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 100000 derived from [budget] max_usd_per_review = 0.48 - **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00) ### Classifier 9 of 9 units passed all questions (1 PR document(s)). 2 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 100563, duration 6.5 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 9, tokens in 7589, cost 0.0003 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6). Reviewed head `8f71589e5bb8ae9227148b60cb87eabfb254a588` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3. </details>

mi added 1 commit 2026-10-06 14:57:07 +02:00

review: a too large prompt first loses the full files, then the excerpts, before the review is given up

ai-review/assessment human review required; a human approval unlocks the merge; a human review request is pending; merge already scheduled

Details

ai-review/gate build succeeded after 00:08

Details

werkator build succeeded after 00:09

Details

werkator/server build succeeded after 00:27

Details

0abde9877e

PR#29 raised the full-file total to 160000 characters but forgot the token count factor;
the review of this PR then found no room for the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented

2026-10-06 14:57:21 +02:00

🟡 werkrevisor YELLOW for 0abde98, superseded by the result for 37142cd

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

YELLOW tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement
YELLOW risky paths changed: werkrevisor/config.py (werkrevisor/config.py)
YELLOW 4 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/excerpt.py, werkrevisor/fullfiles.py and 1 more
YELLOW no AI review: the PR needs 107138 input tokens (69121 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 100000 derived from [budget] max_usd_per_review = 0.48
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)

Classifier

10 of 10 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 107138, duration 5.7 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 10, tokens in 8669, cost 0.0004 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6).

Reviewed head `0abde9877ebc37a86eb26f26a3c08244134328a8` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3.}

<details><summary>🟡 werkrevisor YELLOW for <code>0abde98</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/31#issuecomment-1345">the result for <code>37142cd</code></a></summary> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** risky paths changed: werkrevisor/config.py (werkrevisor/config.py) - **YELLOW** 4 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/excerpt.py, werkrevisor/fullfiles.py and 1 more - **YELLOW** no AI review: the PR needs 107138 input tokens (69121 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 100000 derived from [budget] max_usd_per_review = 0.48 - **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00) ### Classifier 10 of 10 units passed all questions (1 PR document(s)). 4 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 107138, duration 5.7 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 10, tokens in 8669, cost 0.0004 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6). Reviewed head `0abde9877ebc37a86eb26f26a3c08244134328a8` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3. </details>

claude-code commented

2026-10-06 14:59:47 +02:00

Assessment of the werkrevisor findings (as of the last commit):

The CI reviews of the last two commits ran with the installed 0.19.3, which gives up a prompt above the input limit: "no AI review: the PR needs 107138 input tokens". That is the defect of PR#29 that this PR fixes. A local review with the code of this PR (dry run, reported as local review) reduced the context to excerpts and got a full AI review; its findings are handled below.

Fixed:

excerpt.py (ORANGE), malformed usage values of OpenRouter ended the run with a ValueError: they now make the cost an estimate. Test added, red without the fix.
cli.py, a too large prompt gave up the review: it now gets excerpts instead of full files, then the diff only, with the hint context_reduced (Scenario#31.04). Test added, red without the fix.
cli.py (local review), a selection dropped by the reduction was reported as used: the usage record marks it excerpt_discarded; its cost stays, since it was paid.
cli.py:293-299 (ORANGE and local review), secrets: rules.py always sets skip_llm for a secret, so no full files and no selection are collected; _excerpt_selector now also checks the secrets reason explicitly, and a test covers it.
Scenario#31.01 (YELLOW, Given/When/Then misused): rewritten with concrete lines.
README writing hints: wrapped.

Not changed:

YELLOW tests/test_line_comments.py:63, 3 assertions removed: these are the PR#30 follow-up; the RED case moved into test_red_gives_a_comment_review_and_a_failed_gate, which asserts the COMMENT review and exit code 20.
YELLOW werkrevisor/config.py: risky path by design (new [excerpt] section).
YELLOW files only as diff in the CI review: an effect of 0.19.3; with this PR they go as excerpt.

Deployment after the merge: rootfs 0.20.0, and [excerpt] select = "model" in the Werkator machine configs of werkprobe, werkrevisor and werkator; hsadmin-ng keeps the fixed window.

🤖 Generated with Claude Code

Assessment of the werkrevisor findings (as of the last commit): The CI reviews of the last two commits ran with the installed 0.19.3, which gives up a prompt above the input limit: "no AI review: the PR needs 107138 input tokens". That is the defect of PR#29 that this PR fixes. A local review with the code of this PR (dry run, reported as local review) reduced the context to excerpts and got a full AI review; its findings are handled below. Fixed: - `excerpt.py` (ORANGE), malformed usage values of OpenRouter ended the run with a ValueError: they now make the cost an estimate. Test added, red without the fix. - `cli.py`, a too large prompt gave up the review: it now gets excerpts instead of full files, then the diff only, with the hint `context_reduced` (Scenario#31.04). Test added, red without the fix. - `cli.py` (local review), a selection dropped by the reduction was reported as used: the usage record marks it `excerpt_discarded`; its cost stays, since it was paid. - `cli.py:293-299` (ORANGE and local review), secrets: `rules.py` always sets `skip_llm` for a secret, so no full files and no selection are collected; `_excerpt_selector` now also checks the `secrets` reason explicitly, and a test covers it. - Scenario#31.01 (YELLOW, Given/When/Then misused): rewritten with concrete lines. - README writing hints: wrapped. Not changed: - YELLOW `tests/test_line_comments.py:63`, 3 assertions removed: these are the PR#30 follow-up; the RED case moved into `test_red_gives_a_comment_review_and_a_failed_gate`, which asserts the COMMENT review and exit code 20. - YELLOW `werkrevisor/config.py`: risky path by design (new `[excerpt]` section). - YELLOW files only as diff in the CI review: an effect of 0.19.3; with this PR they go as excerpt. Deployment after the merge: rootfs 0.20.0, and `[excerpt] select = "model"` in the Werkator machine configs of werkprobe, werkrevisor and werkator; hsadmin-ng keeps the fixed window. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

mi added 1 commit 2026-10-06 14:59:48 +02:00

review: a selection dropped by the context reduction is marked as discarded; explicit secret guard for the selection

ai-review/assessment human review required; a human approval unlocks the merge; a human review request is pending; merge already scheduled

Details

ai-review/gate build succeeded after 00:08

Details

werkator build succeeded after 00:09

Details

werkator/server build succeeded after 00:27

Details

37142cdaca

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented

2026-10-06 15:00:02 +02:00

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

YELLOW tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement
YELLOW risky paths changed: werkrevisor/config.py (werkrevisor/config.py)
YELLOW 3 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/fullfiles.py, werkrevisor/report.py
YELLOW no AI review: the PR needs 105191 input tokens (67865 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 100000 derived from [budget] max_usd_per_review = 0.48
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 changed file was given to the AI as excerpt with 40 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/cli.py

Classifier

10 of 10 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 105191, duration 6.0 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 10, tokens in 8688, cost 0.0004 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6).

Reviewed head `37142cdaca18f5c9cdb1b7b9f7bbc18eb9de1e78` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3.}

## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** tests lost or disabled: tests/test_line_comments.py:63 removes 3 test(s) or assertion(s) without a replacement - **YELLOW** risky paths changed: werkrevisor/config.py (werkrevisor/config.py) - **YELLOW** 3 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 160000, and no excerpt fits the limits either): werkrevisor/config.py, werkrevisor/fullfiles.py, werkrevisor/report.py - **YELLOW** no AI review: the PR needs 105191 input tokens (67865 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 100000 derived from [budget] max_usd_per_review = 0.48 - **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00) - **hint** 1 changed file was given to the AI as excerpt with 40 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/cli.py ### Classifier 10 of 10 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 105191, duration 6.0 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 10, tokens in 8688, cost 0.0004 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base 70cfca6). Reviewed head `37142cdaca18f5c9cdb1b7b9f7bbc18eb9de1e78` against base `70cfca60b4c3f3cf676c2f6440ae428607b100c6`, werkrevisor 0.19.3.

claude-code commented

2026-10-06 15:10:56 +02:00

Answer to the werkrevisor result for 37142cd:

YELLOW "no AI review: the PR needs 105191 input tokens": the CI still runs werkrevisor 0.19.3, which gives up a prompt above the input limit; this PR fixes that (Scenario#31.04). A local review with the code of this PR reduced the context to excerpts and reviewed the PR; its findings are answered in my previous comment.
YELLOW 3 files only as diff (config.py, fullfiles.py, report.py): the same effect of 0.19.3; with this PR they go as excerpt.
YELLOW tests/test_line_comments.py:63, 3 assertions removed: the PR#30 follow-up, answered in the line comment.
YELLOW werkrevisor/config.py: risky path by design, the new [excerpt] section.
Hint server/build.gradle.kts: version bump, judged harmless by Jev.

No open finding of this round needs a code change.

🤖 Generated with Claude Code

Answer to the werkrevisor result for `37142cd`: - YELLOW "no AI review: the PR needs 105191 input tokens": the CI still runs werkrevisor 0.19.3, which gives up a prompt above the input limit; this PR fixes that (Scenario#31.04). A local review with the code of this PR reduced the context to excerpts and reviewed the PR; its findings are answered in my previous comment. - YELLOW 3 files only as diff (`config.py`, `fullfiles.py`, `report.py`): the same effect of 0.19.3; with this PR they go as excerpt. - YELLOW `tests/test_line_comments.py:63`, 3 assertions removed: the PR#30 follow-up, answered in the line comment. - YELLOW `werkrevisor/config.py`: risky path by design, the new `[excerpt]` section. - Hint `server/build.gradle.kts`: version bump, judged harmless by Jev. No open finding of this round needs a code change. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

mi approved these changes

2026-10-06 15:11:40 +02:00

ai-reviewer merged commit 6989c4b7dd into main

2026-10-06 15:11:41 +02:00

ai-reviewer deleted branch claude-code/excerpt-selection

2026-10-06 15:11:41 +02:00

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mi/werkrevisor#31