werkrevisor 0.20.4: Gradle build files reach the model #35
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mi/werkrevisor#35
Reference in New Issue
Block a user
See the PR doc.
🤖 Generated with Claude Code
🟡 werkrevisor YELLOW for
a59d494, superseded by the result forfdc6231🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Settings read before they are applied at
werkrevisor/review.py:71, YELLOW, as a line comment[low] Scenario text is unclear at
docs/prs/2026-10-06-PR#35-gradle-files-reach-the-model.md:30, hint, as a line commentClassifier
4 of 5 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 25447, calls 1, tokens in 25713 / out 615 (reasoning included in out), duration 35.4 s, cost 0.0576 USD (reported), classifier `jev-1.13.0` calls 5, tokens in 3679, cost 0.0002 USD, finding judge `jev-1.13.0` calls 1, tokens in 1501, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
d7f7f2f).Reviewed head `a59d494d0a1a1eaf9d0f14dd9d61ab613add4fb6` against base `d7f7f2f7fb65ac53389a8cb22994ec33ec9b6cc6`, werkrevisor 0.20.3.
werkrevisor: YELLOW / HUMAN_REVIEW for
a59d494d0a1a1eaf9d0f14dd9d61ab613add4fb6.The result comment of werkrevisor holds the details.
@@ -0,0 +27,4 @@#### Scenario#35.01: a changed Gradle file is in the prompt of the modelSo that the model reviews the build code it depends on.[low] Scenario text is unclear (werkrevisor)
hint
The
So thatline of Scenario#35.01 says the model reviews "the build code it depends on". The model does not depend on build code, so the sentence is unclear. Say that the model reviews the changed build code.The classifier also rated this scenario as implementation details; the Given/Then steps are fine, but the rationale line is the vague part.
Fixed in
fdc6231: the So-that names what the model reviews (dependencies, repositories, plugins); title and Then no longer speak of the prompt.@@ -68,6 +68,7 @@ def pr_agent_settings(config, pr, gitea_token, openrouter_key, extra_instruction"pr_description.enable_large_pr_handling": False,"pr_description.max_ai_calls": 1,"model_routing.enable": False,"bad_extensions.default": reviewer_bad_extensions(),[medium] Settings read before they are applied (werkrevisor)
YELLOW; Jev: real 0.52, severity 2.6 of 3
pr_agent_settingscallsreviewer_bad_extensions(), which readsget_settings().bad_extensions.default, to build the forced values.apply_settingsthen writes the filtered list back into the same key.If
pr_agent_settingsorapply_settingsruns twice in one process, the second run filters an already filtered list, which is harmless. Butis_reviewable_by_pr_agentalso reads the live settings. If it runs beforeapply_settings, or in a process where the settings were never forced, it reads whatever the settings hold at that point.A repository or global settings file could change that list. The "forced settings" invariant in the
pr_agent_settingsdocstring is therefore weakened for this key, because the value derives from mutable state instead of a constant.Fixed in
fdc6231:reviewer_bad_extensionsreads the list once fromlanguage_extensions.tomlshipped with PR-Agent, no longer from the live settings;test_reviewability_does_not_follow_changed_settingsfailed before and passes now.Answer to the review of
a59d494(YELLOW):werkrevisor/review.py:71(settings read before they are applied): fixed infdc6231, thread resolved.implementation_detailsand hint on its wording: fixed infdc6231, thread resolved.statistics/build.gradle.kts: only the version bump.🟡 werkrevisor YELLOW for
fdc6231, superseded by the result for9d0a32c🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Doc contradicts code on Kotlin Gradle files at
docs/prs/2026-10-06-PR#35-gradle-files-reach-the-model.md:17, hint, as a line comment[low] Vague Then clause in scenario at
docs/prs/2026-10-06-PR#35-gradle-files-reach-the-model.md:34, hint, as a line comment[low] Missing blank line before comment and wrong order of definitions at
werkrevisor/review.py:353-364, hint, as a line commentClassifier
4 of 5 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 26360, calls 1, tokens in 26714 / out 663 (reasoning included in out), duration 10.9 s, cost 0.0601 USD (reported), classifier `jev-1.13.0` calls 5, tokens in 3652, cost 0.0002 USD, finding judge `jev-1.13.0` calls 1, tokens in 1224, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
d7f7f2f).Reviewed head `fdc62318df3b3884464db2dcffb88f70c906f23b` against base `d7f7f2f7fb65ac53389a8cb22994ec33ec9b6cc6`, werkrevisor 0.20.3.
werkrevisor: YELLOW / HUMAN_REVIEW for
fdc62318df3b3884464db2dcffb88f70c906f23b.The result comment of werkrevisor holds the details.
@@ -0,0 +14,4 @@Gradle files are build code: they hold dependencies, repositories and plugins,which a review must see.`build.gradle.kts` is not affected, since its extension is `kts`.[medium] Doc contradicts code on Kotlin Gradle files (werkrevisor)
hint; Jev: real 0.60, severity 2.1 of 3
The doc says
build.gradle.ktsis not affected because its extension iskts. The new integration test listsbuild.gradle.ktsas reviewable, but that is not evidence that the unchanged code handles it, and the PR changesstatistics/build.gradle.ktsitself. The Problem section and test list do not state how the claim is verified, so the doc is unclear about whether the test confirms it.Clarified in
9d0a32c:build.gradle.ktswas never in the list, since its extension iskts;test_gradle_files_are_reviewable_but_binary_files_are_notcovers it as reviewable.@@ -0,0 +31,4 @@- **Given** a PR that changes `settings.gradle` and `src/a.py`- **When** werkrevisor reviews the PR- **Then** the model reviews the changes of both files[low] Vague Then clause in scenario (werkrevisor)
hint
The Then step "the model reviews the changes of both files" is vague: the test only checks that the files appear in the prompt, not that the model reviews them. Say "both files appear in the model prompt". The classifier also rated this scenario as vague.
Changed in
9d0a32c: the Then names both files. It stays at what the model gets, not at what a test sees in the prompt, since the previous round rated the prompt wording as implementation detail.@@ -347,6 +350,20 @@ def run_model_review(config, pr, gitea_token, openrouter_key, extra_instructions)# PR-Agent lists these among binary formats, but they hold build code that a review must see.[low] Missing blank line before comment and wrong order of definitions (werkrevisor)
hint
pr_agent_settingscallsreviewer_bad_extensions(), which is defined at the end of the module, afterrun_model_review. This works at runtime but separates related helpers and makes the file harder to follow. Move the definition abovepr_agent_settings.Fixed in
9d0a32c:REVIEWED_EXTENSIONSandreviewer_bad_extensionsnow precedepr_agent_settings.Answer to the review of
fdc6231(YELLOW):vague_thenand its hint: the Then names both files in9d0a32c, thread resolved.build.gradle.kts: clarified in9d0a32c, thread resolved.werkrevisor/review.py: fixed in9d0a32c, thread resolved.statistics/build.gradle.kts: only the version bump.🟢 werkrevisor: GREEN / PASS
Gate: approved by werkrevisor; merge waits for 2 open hint(s): reply to each line comment and resolve it, then restart the review or push a commit.
Change kind:
mixed.Reasons
Findings
[medium] Doc contradicts code on build.gradle.kts at
docs/prs/2026-10-06-PR#35-gradle-files-reach-the-model.md:17, hint, as a line comment[medium] Failure mode of reading the shipped TOML at
werkrevisor/review.py:35-42, hint, as a line commentClassifier
5 of 5 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 26304, calls 1, tokens in 26680 / out 619 (reasoning included in out), duration 10.8 s, cost 0.0595 USD (reported), classifier `jev-1.13.0` calls 5, tokens in 3679, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 2764, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at based7f7f2f).Reviewed head `9d0a32c2c0a09e7ff0145bdf11292f3895d083a4` against base `d7f7f2f7fb65ac53389a8cb22994ec33ec9b6cc6`, werkrevisor 0.20.3.
werkrevisor: GREEN / PASS for
9d0a32c2c0a09e7ff0145bdf11292f3895d083a4.The result comment of werkrevisor holds the details.
@@ -0,0 +14,4 @@Gradle files are build code: they hold dependencies, repositories and plugins,which a review must see.`build.gradle.kts` was not affected, since its extension is `kts`; the tests below cover it too.[medium] Doc contradicts code on build.gradle.kts (werkrevisor)
hint; Jev: real 0.74, severity 2.0 of 3
The doc says
build.gradle.kts"was not affected, since its extension iskts" and that the tests cover it. The test list only includes abuild.gradle.ktscase inis_reviewable_by_pr_agent. The doc mentions no test that sends it through the prompt. Also,ktsis not verified to be absent from PR-Agent'sbad_extensions; the claim rests on an unverified assumption. Suggest checking thatktsis absent and stating this precisely.Checked in the pinned PR-Agent 0.47.0 image:
ktsis neither inbad_extensions.defaultnor inextra.is_reviewable_by_pr_agent("build.gradle.kts")returning True in the test is exactly that check, since reviewability only depends on that list. Not changed.@@ -29,0 +32,4 @@REVIEWED_EXTENSIONS = frozenset({"gradle"})@functools.cache[medium] Failure mode of reading the shipped TOML (werkrevisor)
hint; Jev: real 0.72, severity 2.0 of 3
reviewer_bad_extensionsreads a private file path inside PR-Agent (settings/language_extensions.toml) and thebad_extensions.defaultkey. If the layout differs, it raisesFileNotFoundErrororKeyErrorat settings build time. This happens before any model call and fails the whole run, and no test covers this error path. The path is fixed to 0.47.0 and the PR-Agent image is pinned, so the realistic impact is limited. The testtest_every_forced_setting_exists_in_pr_agentdoes exercise the happy path.Not changed, on purpose: the image is pinned by digest, and the test suite reads the file in every run (
test_gradle_files_are_reviewable_but_binary_files_are_not), so a changed layout fails the build of the upgrade PR. At runtime, a missing file fails the review before any model call, which is the safe direction.Answer to the review of
9d0a32c(GREEN, merge held for 2 hints):build.gradle.kts: verified thatktsis no bad extension of PR-Agent 0.47.0; not changed, resolved.Restarting the review to release the merge.
werkrevisor, review restarted for
9d0a32c2c0a09e7ff0145bdf11292f3895d083a4: approved by werkrevisor; merge scheduled for when the checks succeed; review request of mi withdrawn.