werkrevisor 0.15.0: the colors say how certain a finding is, and Jev judges the model findings #16

Merged

mi merged 6 commits from claude-code/color-semantics into main 2026-10-05 21:51:50 +02:00

Conversation 3 Commits 6 Files Changed 25

+926 -161

claude-code commented 2026-10-05 21:16:34 +02:00

Collaborator

Copy Link

Copy Source

See docs/prs/2026-10-05-PR#16-colors-by-certainty.md.

Replaces PR#15: dangerous paths give YELLOW, not ORANGE.

🤖 Generated with Claude Code

See docs/prs/2026-10-05-PR#16-colors-by-certainty.md. Replaces PR#15: dangerous paths give YELLOW, not ORANGE. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-code added 1 commit 2026-10-05 21:16:34 +02:00

PR#16 doc: the colors say how certain a finding is, and Jev judges the model findings ... b9a1830470

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-05 21:23:51 +02:00

werkrevisor 0.15.0: the colors say how certain a finding is, and Jev judges the model findings ... 2d68869c25

RED is a certain finding, ORANGE a probably real one, YELLOW a dangerous change without a finding.
Jev rates each medium or high model finding; low findings and the model's recommendation are hints.
Size limits give YELLOW, only lost or disabled tests block, and mixed is AI-approvable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-05 21:31:42 +02:00

PR#16 doc: dry run of the rules on 25 hsadmin-ng PRs, open decisions on writing rules and full-file context ... f333be3f1e

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-05 21:38:22 +02:00

werkrevisor 0.15.0: a violated writing rule is a hint; PR#16 doc records the dry run and the answered questions ... 80d20df6f4

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-code marked the pull request as ready for review 2026-10-05 21:38:27 +02:00

ai-reviewer commented 2026-10-05 21:38:35 +02:00

Collaborator

Copy Link

Copy Source

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

• YELLOW test changes are not purely additive: tests/fakes.py:231 changes an existing test line, tests/test_change_kinds.py:10 changes an existing test line, tests/test_change_kinds.py:47 changes an existing test line and 59 more
• YELLOW change kind 'mixed' (main code, tests, other files, docs); AI-approvable are only: docs only, tests only added, docs and tests, refactoring with unchanged tests, feature with purely additive tests, harmless files only, mechanical changes only
• YELLOW risky paths changed: .werkrevisor.toml (built in), config/hsadmin-ng.toml (config/), config/werkprobe.toml (config/) and 1 more
• YELLOW the changed .werkrevisor.toml uses settings unknown to the installed werkrevisor 0.14.0: [categories] auto_approvable 'mixed'; deploy a version that knows them right after the merge, or every review will end with ERROR
• YELLOW spec: docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03 implementation_details (0.63 > yellow_above = 0.6)
• YELLOW 9 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 120000): tests/test_flow.py, tests/test_verdict.py, tests/test_writing.py and 6 more
• YELLOW no AI review: the PR needs 114594 input tokens (73931 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 80000 derived from [budget] max_usd_per_review = 0.40

Classifier

Unit	Question	Problem	Confidence	Verdict
docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03	implementation_details	0.63	–	YELLOW

18 of 19 units passed all questions (1 PR document(s)).
8 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 114594, duration 6.4 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 19, tokens in 16837, cost 0.0007 USD, config: defaults + host + repository (.werkrevisor.toml at base e8b256e).
Reviewed head `80d20df6f47b8dc5504bb9240740fb1e9898647a` against base `e8b256ef92ad88780065ce295ababfe9e188e826`, werkrevisor 0.14.0.}

<!-- werkrevisor:80d20df6f47b8dc5504bb9240740fb1e9898647a --> <!-- werkrevisor-verdict:YELLOW --> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** test changes are not purely additive: tests/fakes.py:231 changes an existing test line, tests/test_change_kinds.py:10 changes an existing test line, tests/test_change_kinds.py:47 changes an existing test line and 59 more - **YELLOW** change kind 'mixed' (main code, tests, other files, docs); AI-approvable are only: docs only, tests only added, docs and tests, refactoring with unchanged tests, feature with purely additive tests, harmless files only, mechanical changes only - **YELLOW** risky paths changed: .werkrevisor.toml (built in), config/hsadmin-ng.toml (config/**), config/werkprobe.toml (config/**) and 1 more - **YELLOW** the changed .werkrevisor.toml uses settings unknown to the installed werkrevisor 0.14.0: [categories] auto_approvable 'mixed'; deploy a version that knows them right after the merge, or every review will end with ERROR - **YELLOW** spec: docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03 implementation_details (0.63 > yellow_above = 0.6) - **YELLOW** 9 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 120000): tests/test_flow.py, tests/test_verdict.py, tests/test_writing.py and 6 more - **YELLOW** no AI review: the PR needs 114594 input tokens (73931 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 80000 derived from [budget] max_usd_per_review = 0.40 ### Classifier | Unit | Question | Problem | Confidence | Verdict | |---|---|---|---|---| | docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03 | implementation_details | 0.63 | – | YELLOW | 18 of 19 units passed all questions (1 PR document(s)). 8 answers ignored because of a confidence below [classifier] min_confidence. _{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 114594, duration 6.4 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 19, tokens in 16837, cost 0.0007 USD, config: defaults + host + repository (.werkrevisor.toml at base e8b256e).<br> Reviewed head `80d20df6f47b8dc5504bb9240740fb1e9898647a` against base `e8b256ef92ad88780065ce295ababfe9e188e826`, werkrevisor 0.14.0.}

ai-reviewer requested review from mi 2026-10-05 21:38:35 +02:00

mi added 1 commit 2026-10-05 21:42:47 +02:00

werkrevisor 0.15.0: an open TODO.<topic> item in the PR document is deferred work and only a hint ... efe6b5a6bc

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented 2026-10-05 21:42:55 +02:00

Collaborator

Copy Link

Copy Source

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

• YELLOW test changes are not purely additive: tests/fakes.py:231 changes an existing test line, tests/test_change_kinds.py:10 changes an existing test line, tests/test_change_kinds.py:47 changes an existing test line and 62 more
• YELLOW change kind 'mixed' (main code, tests, other files, docs); AI-approvable are only: docs only, tests only added, docs and tests, refactoring with unchanged tests, feature with purely additive tests, harmless files only, mechanical changes only
• YELLOW risky paths changed: .werkrevisor.toml (built in), config/hsadmin-ng.toml (config/), config/werkprobe.toml (config/) and 1 more
• YELLOW the changed .werkrevisor.toml uses settings unknown to the installed werkrevisor 0.14.0: [categories] auto_approvable 'mixed'; deploy a version that knows them right after the merge, or every review will end with ERROR
• YELLOW spec: docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03 implementation_details (0.61 > yellow_above = 0.6)
• YELLOW 11 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 120000): tests/test_flow.py, tests/test_verdict.py, tests/test_writing.py and 8 more
• YELLOW no AI review: the PR needs 121226 input tokens (78210 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 80000 derived from [budget] max_usd_per_review = 0.40

Classifier

Unit	Question	Problem	Confidence	Verdict
docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03	implementation_details	0.61	–	YELLOW

19 of 20 units passed all questions (1 PR document(s)).
9 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 121226, duration 6.8 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 20, tokens in 17914, cost 0.0008 USD, config: defaults + host + repository (.werkrevisor.toml at base e8b256e).
Reviewed head `efe6b5a6bc5ccc4de69d4a2d18a97f97cadadf45` against base `e8b256ef92ad88780065ce295ababfe9e188e826`, werkrevisor 0.14.0.}

<!-- werkrevisor:efe6b5a6bc5ccc4de69d4a2d18a97f97cadadf45 --> <!-- werkrevisor-verdict:YELLOW --> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** test changes are not purely additive: tests/fakes.py:231 changes an existing test line, tests/test_change_kinds.py:10 changes an existing test line, tests/test_change_kinds.py:47 changes an existing test line and 62 more - **YELLOW** change kind 'mixed' (main code, tests, other files, docs); AI-approvable are only: docs only, tests only added, docs and tests, refactoring with unchanged tests, feature with purely additive tests, harmless files only, mechanical changes only - **YELLOW** risky paths changed: .werkrevisor.toml (built in), config/hsadmin-ng.toml (config/**), config/werkprobe.toml (config/**) and 1 more - **YELLOW** the changed .werkrevisor.toml uses settings unknown to the installed werkrevisor 0.14.0: [categories] auto_approvable 'mixed'; deploy a version that knows them right after the merge, or every review will end with ERROR - **YELLOW** spec: docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03 implementation_details (0.61 > yellow_above = 0.6) - **YELLOW** 11 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 120000): tests/test_flow.py, tests/test_verdict.py, tests/test_writing.py and 8 more - **YELLOW** no AI review: the PR needs 121226 input tokens (78210 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 80000 derived from [budget] max_usd_per_review = 0.40 ### Classifier | Unit | Question | Problem | Confidence | Verdict | |---|---|---|---|---| | docs/prs/2026-10-05-PR#16-colors-by-certainty.md Scenario#16.03 | implementation_details | 0.61 | – | YELLOW | 19 of 20 units passed all questions (1 PR document(s)). 9 answers ignored because of a confidence below [classifier] min_confidence. _{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 121226, duration 6.8 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 20, tokens in 17914, cost 0.0008 USD, config: defaults + host + repository (.werkrevisor.toml at base e8b256e).<br> Reviewed head `efe6b5a6bc5ccc4de69d4a2d18a97f97cadadf45` against base `e8b256ef92ad88780065ce295ababfe9e188e826`, werkrevisor 0.14.0.}

mi added 1 commit 2026-10-05 21:48:20 +02:00

review: a new test method no longer makes up for a lost assertion, a failed Jev answer counts one call, PR#16 scenarios in order ... d2c42cb810

Fixes the two medium findings and the low finding of the model dry run on PR#16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented 2026-10-05 21:48:33 +02:00

Collaborator

Copy Link

Copy Source

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

• YELLOW test changes are not purely additive: tests/fakes.py:231 changes an existing test line, tests/test_change_kinds.py:10 changes an existing test line, tests/test_change_kinds.py:47 changes an existing test line and 62 more
• YELLOW change kind 'mixed' (main code, tests, other files, docs); AI-approvable are only: docs only, tests only added, docs and tests, refactoring with unchanged tests, feature with purely additive tests, harmless files only, mechanical changes only
• YELLOW risky paths changed: .werkrevisor.toml (built in), config/hsadmin-ng.toml (config/), config/werkprobe.toml (config/) and 1 more
• YELLOW the changed .werkrevisor.toml uses settings unknown to the installed werkrevisor 0.14.0: [categories] auto_approvable 'mixed'; deploy a version that knows them right after the merge, or every review will end with ERROR
• YELLOW 11 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 120000): tests/test_flow.py, tests/test_rules.py, tests/test_writing.py and 8 more
• YELLOW no AI review: the PR needs 120598 input tokens (77805 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 80000 derived from [budget] max_usd_per_review = 0.40

Classifier

20 of 20 units passed all questions (1 PR document(s)).
8 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 120598, duration 7.0 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 20, tokens in 17910, cost 0.0008 USD, config: defaults + host + repository (.werkrevisor.toml at base e8b256e).
Reviewed head `d2c42cb810196cb9c6430c718f34a9c57b15a40e` against base `e8b256ef92ad88780065ce295ababfe9e188e826`, werkrevisor 0.14.0.}

<!-- werkrevisor:d2c42cb810196cb9c6430c718f34a9c57b15a40e --> <!-- werkrevisor-verdict:YELLOW --> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** test changes are not purely additive: tests/fakes.py:231 changes an existing test line, tests/test_change_kinds.py:10 changes an existing test line, tests/test_change_kinds.py:47 changes an existing test line and 62 more - **YELLOW** change kind 'mixed' (main code, tests, other files, docs); AI-approvable are only: docs only, tests only added, docs and tests, refactoring with unchanged tests, feature with purely additive tests, harmless files only, mechanical changes only - **YELLOW** risky paths changed: .werkrevisor.toml (built in), config/hsadmin-ng.toml (config/**), config/werkprobe.toml (config/**) and 1 more - **YELLOW** the changed .werkrevisor.toml uses settings unknown to the installed werkrevisor 0.14.0: [categories] auto_approvable 'mixed'; deploy a version that knows them right after the merge, or every review will end with ERROR - **YELLOW** 11 changed files were given to the AI only as diff, not in full, so it could not check them against the rest of their file (all full files together would exceed [review] max_full_files_chars = 120000): tests/test_flow.py, tests/test_rules.py, tests/test_writing.py and 8 more - **YELLOW** no AI review: the PR needs 120598 input tokens (77805 counted with o200k_base × [model] token_count_factor = 1.55), more than the limit of 80000 derived from [budget] max_usd_per_review = 0.40 ### Classifier 20 of 20 units passed all questions (1 PR document(s)). 8 answers ignored because of a confidence below [classifier] min_confidence. _{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (too large), tokens counted before the call 120598, duration 7.0 s, no model call, cost 0 USD, classifier `jev-1.13.0` calls 20, tokens in 17910, cost 0.0008 USD, config: defaults + host + repository (.werkrevisor.toml at base e8b256e).<br> Reviewed head `d2c42cb810196cb9c6430c718f34a9c57b15a40e` against base `e8b256ef92ad88780065ce295ababfe9e188e826`, werkrevisor 0.14.0.}

mi merged commit c50ad05877 into main 2026-10-05 21:51:50 +02:00

mi deleted branch claude-code/color-semantics 2026-10-05 21:51:50 +02:00

Sign in to join this conversation.

Reviewers

No Reviewers

mi

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

ai-reviewer claude-code mi

No Assignees

3 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mi/werkrevisor#16