A misreported build status waits for the human (v1.42.3) #159
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mi/werkator#159
Reference in New Issue
Block a user
See the PR document under docs/prs/.
🤖 Generated with Claude Code
🟡 werkrevisor YELLOW for
947793f, superseded by the result for60e18ea🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
Category name inconsistency at
docs/prs/2026-10-14-PR#159-gate-status-misreported.md:12-19, severity unknown, counted as medium, hint, as a line commentAmbiguous phrasing in Measurement at
docs/prs/2026-10-14-PR#159-gate-status-misreported.md:47, severity unknown, counted as medium, hint, as a line commentClassifier
5 of 6 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 20716, calls 1, tokens in 19903 / out 16175 (reasoning included in out), duration 273.8 s, cost 0.0252 USD (reported), classifier `jev-1.13.0` calls 6, tokens in 4404, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 4398, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 31422 / out 199, cost 0.0040 USD (reported), build file judge `jev-1.13.0` calls 2, tokens in 2104, cost 0.0001 USD, phases: path_judge 0.3 s, classifier 0.5 s, context 4.3 s, model_review 267.9 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main2 answers ignored because of a confidence below [classifier] min_confidence.
15b1251).Reviewed head `947793f27f351560c68dae97f8da8a228dd30dad` against base `15b125126912d7c2124c222b8ecd8a839c12aa5d`, werkrevisor 0.41.2.
werkrevisor: YELLOW / HUMAN_REVIEW for
947793f27f351560c68dae97f8da8a228dd30dad.The result comment of werkrevisor holds the details.
@@ -0,0 +9,4 @@- [werkrevisor PR#106](https://git.javagil.de/mi/werkrevisor/pulls/106): werkrevisor did not see a weakened gate, Codex did.- [werkrevisor PR#107](https://git.javagil.de/mi/werkrevisor/pulls/107): the category `gate_weakened` of werkrevisor.## ProblemCategory name inconsistency, severity unknown, counted as medium (werkrevisor)
hint; Jev: real 0.88, severity 1.7 of 3
The Problem section cites werkrevisor categories
weakens_checksandsecuritywith the specific value "0.07 or less for a timed-out build reported assuccess", but the Measurement table uses different category names (gate_weakened,status_misreported,pr_decides_checks) and never listsweakens_checksorsecurity. The 0.07 figure does not appear anywhere in the document. A reader cannot verify the 0.07 claim from the document alone, and must infer (from the related links to werkrevisor PR#106 and PR#107) thatgate_weakenedis the post-PR#107 successor ofweakens_checksand that the problem statement refers to pre-PR#107 behavior. The document should explain this evolution and either include the historical categories in the measurement table or drop the specific numbers and link to PR#106/PR#107.Fixed in
60e18ea: the table now showsweakens_checksandsecurity, the questions werkrevisor asks today, with the 0.07 of the timed-out build;gate_weakenedis a category of werkrevisor PR#107, not a successor of them.@@ -0,0 +44,4 @@| a pull-request target build takes the settings of the PR | 0.15 | 0.14 | 0.39 || highest of the 22 merged PRs | 0.17 | 0.25 | 0.25 |`status_misreported` separates the two misreported statuses from the merged PRs; 0.6 lies between both.Ambiguous phrasing in Measurement, severity unknown, counted as medium (werkrevisor)
hint; Jev: real 0.74, severity 1.2 of 3
The sentence "0.6 lies between both" does not specify what "both" refers to. Reading it as "between the two misreported status values 0.75 and 0.85" is numerically false (0.6 is below both). The intended reading appears to be "between the highest merged-PR value (0.25) and the lowest misreported-status value (0.75)", but that interpretation is not stated.
Fixed in
60e18ea: the document now says that 0.6 lies between 0.25, the highest merged PR, and 0.75, the lower misreported status.⚠️ werkrevisor ERROR for
?, superseded by the result for60e18eaCodex review: critical
This changes the review gate that determines whether future changes require human approval; a configuration mistake could compromise that protection without Werkator’s tests detecting it. Repository policy explicitly requires human review for changes to .werkrevisor.toml.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 147226 (cached 112768) / out 948, 35.2 s, 10 commands; reviewed head `947793f27f351560c68dae97f8da8a228dd30dad`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.🟡 werkrevisor YELLOW for
60e18ea, superseded by the result foreea70ce🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Classifier
5 of 6 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 20859, calls 1, tokens in 20036 / out 8787 (reasoning included in out), duration 159.1 s, cost 0.0165 USD (reported), classifier `jev-1.13.0` calls 6, tokens in 4527, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 31422 / out 279, cost 0.0005 USD (reported), build file judge `jev-1.13.0` calls 2, tokens in 2104, cost 0.0001 USD, phases: path_judge 0.3 s, classifier 0.4 s, context 6.2 s, model_review 151.7 s, config: defaults + host + repository (.werkrevisor.toml at main1 answer ignored because of a confidence below [classifier] min_confidence.
15b1251).Reviewed head `60e18ea2cf773e2e0d1df1b029b2f7fb577378f2` against base `15b125126912d7c2124c222b8ecd8a839c12aa5d`, werkrevisor 0.41.2.
⚠️ werkrevisor ERROR for
?, superseded by the result foreea70ceCodex review: critical
This changes the review gate’s risk classification and mandatory human-review paths in .werkrevisor.toml. A configuration mistake could affect automatic merging without being detected by Werkator’s tests; the repository workflow also explicitly requires human approval for changes to this file.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 146325 (cached 112000) / out 915, 41.3 s, 10 commands; reviewed head `60e18ea2cf773e2e0d1df1b029b2f7fb577378f2`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Classifier
6 of 6 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 19414, calls 1, tokens in 18661 / out 19503 (reasoning included in out), duration 416.8 s, cost 0.0290 USD (reported), classifier `jev-1.13.0` calls 6, tokens in 4615, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 31422 / out 189, cost 0.0004 USD (reported), build file judge `jev-1.13.0` calls 2, tokens in 2104, cost 0.0001 USD, phases: path_judge 0.3 s, classifier 0.5 s, context 5.0 s, model_review 409.3 s, config: defaults + host + repository (.werkrevisor.toml at main1 answer ignored because of a confidence below [classifier] min_confidence.
15b1251).Reviewed head `eea70ce89712747f4f6d8d12443459343408744c` against base `15b125126912d7c2124c222b8ecd8a839c12aa5d`, werkrevisor 0.41.2.
Codex review: critical
This changes the review gates in .werkrevisor.toml, where a mistake could allow unsafe changes to merge without being detected by application tests. The repository workflow explicitly requires human approval for changes to this file.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 148856 (cached 114176) / out 1009, 37.0 s, 9 commands; reviewed head `eea70ce89712747f4f6d8d12443459343408744c`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.