werkrevisor 0.75.0: the cost of the PR Lens diagrams in the usage record, the result comment and the statistics #140

Merged
ai-reviewer merged 14 commits from claude-code/diagram-cost into main 2026-10-25 11:20:35 +01:00
Collaborator

See the PR document under docs/prs/.

🤖 Generated with Claude Code

See the PR document under docs/prs/. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
claude-code changed title from WIP: werkrevisor 0.66.0: the cost of the PR Lens diagrams in the usage record, the result comment and the statistics to WIP: werkrevisor 0.67.0: the cost of the PR Lens diagrams in the usage record, the result comment and the statistics 2026-10-23 18:25:38 +02:00
claude-code marked the pull request as ready for review 2026-10-23 18:30:54 +02:00
Collaborator
🟡 werkrevisor YELLOW for f0b4e4a, superseded by the result for 15dd358

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test
  • hint writing rules violated in the changed docs: docs/prs/2026-10-23-PR#140-diagram-cost.md:94: paragraph with 6 sentences, more than 5
  • hint 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02)
  • hint 12 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint the model review failed: TimeoutError: review exceeded 720 seconds (advisory: Jev judged every changed file)

Classifier

9 of 9 units passed all questions (1 PR document(s)).
5 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 56964, duration 733.0 s, cost unknown, classifier `jev-1.14.0` calls 9, tokens in 7181, cost 0.0003 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25495 / out 311, cost 0.0033 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 35192, cost 0.0015 USD, build file judge `jev-1.14.0` calls 10, tokens in 10982, cost 0.0005 USD, phases: path_judge 0.6 s, classifier 0.5 s, doc_check 0.6 s, context 8.1 s, model_review 722.7 s, config: defaults + host + repository (.werkrevisor.toml at main bad4782).
Reviewed head `f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3` against base `bad4782b99657b8a662db8b9333722c04619962e`, werkrevisor 0.65.2.

Diagrams

Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.

pr-lens-v-containers.svg

pr-lens-v-flow.svg

<!-- werkrevisor:f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3 --> <!-- werkrevisor-verdict:YELLOW --> <!-- werkrevisor-blocks:63409cde741958f25faba06dad424b3c,0e4219813569a2d1c065c10ba1484a61,d21e9a7256a79225097563dbbbe8a1c0,279ceb5c617659daa18bd3a576d54d9b,bde4d00d50992e84778f1ef8ae10d041,1d81fca8c42dbc68a8d130de0a42144b,0876df68568537d89da1bce76aa5a6d8,dd7390dea91edd0b4fb6707f447bb596,d5faadd9a5721f2e14276d12358bdb94,307098a735b318115a5d947f02ed8df9,911fff4991e0402eb539885689b26d85,8a49ae3c2cc1553554b7e938e7d2d221,9c5fe7e4c38bdc08691e292081d93ce7,0dee2ccc6cb7f9df3b332adcba209d6c,97ae6d34524bf5ac1bcaae7cfe1abb0d,683904167c90880f9394e05f2239f937,706f24f14f3caae33a87765bc63bba3e,ce44fe9624a95971d7e1fb8a5c819c03,dfc5e3c3eb88b186fff7c7a8b8130f02,d65111f4d74aae37689965efe54df69c,f9e2e3fb1f55296ff5420943cde9126a,847171037bae7a881992c05215d913c6,5ffb2d9443d0febb9533622afa206898,88e964d4ce7a2ef100b2784bd3e03776,c953214c00ea0365b24d859338dea927,0d4b1f06bb63ae5eaa5ecdc9eee188db,0de1f38acc3b2dac8c4b35bbe70ec881,f1585989851248a2cdd3839f190db11a,9c3053e2775f4d0593a78067cded19d7,38711d39767625743151649b51835f88,8568dc89fa52197e9b0d00aa56585fa2,a6d371d72925492d60f8e4495736daf9,d8b5adf93f08f525491e9bfbe337be2f,2268c92ee72701d419ac138890534913,891cb918b9ed302be63a10634b2fbe8f,d0961487cc001692775811eb5a3fb2bf,7edee309ed0d536adf129697841d130d,f25b2d9d5b10a5bc84cac33c7fd4d44b,3a46395c910fbba407ca48bbd1c79c9b,d2ffa4118b6f1812521f1a38c0798580,8445171d6fd82c2c4d76f1d633bb7199,4cc22115da27696787cecd65bfb65952,8e822399fc5dcf5eb33b233ccbbe8215,50f471144dae8364c94ec7a04c245f2c --> <!-- werkrevisor:diagrams --> <!-- werkrevisor-folded --> <details><summary>🟡 werkrevisor YELLOW for <code>f0b4e4a</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/140#issuecomment-7473">the result for <code>15dd358</code></a></summary> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test - **hint** writing rules violated in the changed docs: docs/prs/2026-10-23-PR#140-diagram-cost.md:94: paragraph with 6 sentences, more than 5 - **hint** 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02) - **hint** 12 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** the model review failed: TimeoutError: review exceeded 720 seconds (advisory: Jev judged every changed file) ### Classifier 9 of 9 units passed all questions (1 PR document(s)). 5 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 56964, duration 733.0 s, cost unknown, classifier `jev-1.14.0` calls 9, tokens in 7181, cost 0.0003 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25495 / out 311, cost 0.0033 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 35192, cost 0.0015 USD, build file judge `jev-1.14.0` calls 10, tokens in 10982, cost 0.0005 USD, phases: path_judge 0.6 s, classifier 0.5 s, doc_check 0.6 s, context 8.1 s, model_review 722.7 s, config: defaults + host + repository (.werkrevisor.toml at main bad4782).<br> Reviewed head `f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3` against base `bad4782b99657b8a662db8b9333722c04619962e`, werkrevisor 0.65.2. </sub> ### Diagrams Drawn by a model from the diff with [PR Lens](https://github.com/coldteadotai/pr-lens): a view of what the PR touches, not a statement of what it does. ![pr-lens-v-containers.svg](https://git.javagil.de/attachments/f7823d32-2383-4cdb-92e3-b3522210a180) ![pr-lens-v-flow.svg](https://git.javagil.de/attachments/e767d03b-17e6-48a6-b089-cbe1539959af) </details>
ai-reviewer left a comment
Collaborator

werkrevisor: YELLOW / HUMAN_REVIEW for f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3.
The result comment of werkrevisor holds the details.

werkrevisor: YELLOW / HUMAN_REVIEW for `f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3`. The result comment of werkrevisor holds the details. <!-- werkrevisor-review:f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3 -->
@@ -0,0 +91,4 @@
### Test of the Release of a Held Merge
This paragraph breaks the writing rule of at most 5 sentences on purpose.
Collaborator

Writing rule: paragraph with 6 sentences, more than 5 (werkrevisor writing rule)

Fix the line, or reply why it stays, and resolve this comment.

**Writing rule: paragraph with 6 sentences, more than 5** (werkrevisor writing rule) Fix the line, or reply why it stays, and resolve this comment.
Author
Collaborator

Fixed in 15dd358: the deliberate paragraph is removed; the test of the release moves to a simpler PR.

Fixed in `15dd358`: the deliberate paragraph is removed; the test of the release moves to a simpler PR.
claude-code marked this conversation as resolved
ai-reviewer scheduled this pull request to auto merge when all checks succeed 2026-10-23 18:43:14 +02:00
ai-reviewer requested review from mi 2026-10-23 18:43:14 +02:00
Author
Collaborator

Answer to the review of f0b4e4a:

  • YELLOW "tests lost or disabled: tests/test_pr_lens_script.py:62": the new test class is skipped only without Node.js. The PR-Agent image of the documented local test command has no Node.js; the rootfs of werkrevisor, where Werkator runs the tests, has /usr/bin/node, the same Node.js that runs PR Lens in production. No existing test is disabled. This is left to the human.
  • Hint "paragraph with 6 sentences" in the PR document: on purpose, see its section "Test of the Release of a Held Merge"; it stays until the decision on the YELLOW reason.
Answer to the review of `f0b4e4a`: - YELLOW "tests lost or disabled: tests/test_pr_lens_script.py:62": the new test class is skipped only without Node.js. The PR-Agent image of the documented local test command has no Node.js; the rootfs of werkrevisor, where Werkator runs the tests, has `/usr/bin/node`, the same Node.js that runs PR Lens in production. No existing test is disabled. This is left to the human. - Hint "paragraph with 6 sentences" in the PR document: on purpose, see its section "Test of the Release of a Held Merge"; it stays until the decision on the YELLOW reason.
ai-reviewer left a comment
Collaborator

Codex review of f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3

Codex review of `f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3` <!-- werkrevisor-codex -->
@@ -203,1 +204,3 @@
boundedSum("cost_usd, excerpt_cost_usd and doc_summary_cost_usd", listOf(it, excerptCost, docSummaryCost))
boundedSum(
"cost_usd, excerpt_cost_usd, doc_summary_cost_usd and diagram_cost_usd",
listOf(it, excerptCost, docSummaryCost, diagramCost),
Collaborator

[medium] Unknown diagram costs are presented as a complete total (Codex review)

When diagram_calls is positive but diagram_cost_usd is null, boundedSum silently drops the unknown cost and preserves the review's cost_status. For a reported review cost, statistics therefore show the partial total as known, with unknownCostReviews unchanged and no estimated-cost indicator. Distinguish an unpriced drawing from a run that drew nothing.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] Unknown diagram costs are presented as a complete total** (Codex review) When diagram_calls is positive but diagram_cost_usd is null, boundedSum silently drops the unknown cost and preserves the review's cost_status. For a reported review cost, statistics therefore show the partial total as known, with unknownCostReviews unchanged and no estimated-cost indicator. Distinguish an unpriced drawing from a run that drew nothing. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in 00a7b82: a record with diagram_calls > 0 and no diagram_cost_usd makes the cost of the review unknown, so it counts in unknownCostReviews. Test: Scenario#140.03 in UsageApiIntegrationTest.

Fixed in `00a7b82`: a record with `diagram_calls > 0` and no `diagram_cost_usd` makes the cost of the review unknown, so it counts in `unknownCostReviews`. Test: Scenario#140.03 in `UsageApiIntegrationTest`.
claude-code marked this conversation as resolved
@@ -68,5 +94,6 @@ try {
? error.details.split("\n").map((l) => l.trim()).filter(Boolean).slice(0, 2).join(" ") : "";
process.stderr.write(`pr-lens: ${error?.code ?? error?.name ?? "Error"}: ${error?.message ?? error}`
+ `${detail ? ` (${detail})` : ""}\n`);
process.stdout.write(JSON.stringify({ usage: usageOut() }));
Collaborator

[medium] A timeout loses usage from completed calls (Codex review)

Usage is emitted only when drawing finishes or throws. If the first model call completes but the repair call exceeds timeout_seconds, subprocess.run kills Node without executing this catch block. prlens.draw then raises DiagramError without usage, so the record reports zero diagram calls and loses the first call's known cost.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] A timeout loses usage from completed calls** (Codex review) Usage is emitted only when drawing finishes or throws. If the first model call completes but the repair call exceeds timeout_seconds, subprocess.run kills Node without executing this catch block. prlens.draw then raises DiagramError without usage, so the record reports zero diagram calls and loses the first call's known cost. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in 00a7b82: pr_lens.mjs writes the usage to stderr after each call; on a timeout prlens.draw reads the last line from the killed process. Test: test_a_killed_pr_lens_names_the_usage_of_its_finished_calls.

Fixed in `00a7b82`: `pr_lens.mjs` writes the usage to stderr after each call; on a timeout `prlens.draw` reads the last line from the killed process. Test: `test_a_killed_pr_lens_names_the_usage_of_its_finished_calls`.
claude-code marked this conversation as resolved
Collaborator

Codex review: not critical

The changes add diagram usage accounting and reporting without changing authorization, credentials, merge decisions, or existing data. The identified risks concern incomplete cost reporting.

Findings

  1. [medium] A timeout loses usage from completed calls at werkrevisor/pr_lens.mjs:97, as a line comment
  2. [medium] Unknown diagram costs are presented as a complete total at statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:206, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

Model `gpt-6-astra`, effort high, tokens in 459108 (cached 411648) / out 2508, 100.2 s, 14 commands; reviewed head `f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3`.
<!-- werkrevisor-codex:f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3 --> <!-- werkrevisor-codex-critical:false --> <!-- werkrevisor-codex-unattached:0 --> <!-- werkrevisor-codex --> ## Codex review: not critical The changes add diagram usage accounting and reporting without changing authorization, credentials, merge decisions, or existing data. The identified risks concern incomplete cost reporting. ### Findings 1. **[medium] A timeout loses usage from completed calls** at `werkrevisor/pr_lens.mjs:97`, as a line comment 2. **[medium] Unknown diagram costs are presented as a complete total** at `statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:206`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. <sub> Model `gpt-6-astra`, effort high, tokens in 459108 (cached 411648) / out 2508, 100.2 s, 14 commands; reviewed head `f0b4e4aa186b8cf5db81a890c520f2f6e4f1fdc3`. </sub>
Collaborator
🟠 werkrevisor ORANGE for 15dd358, superseded by the result for 00a7b82

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: mixed.

Reasons

  • ORANGE 2 finding(s) that Jev rates as probably real and severe: werkrevisor/pr_lens.mjs:22-30, werkrevisor/cli.py:1323-1328
  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test
  • hint 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02)
  • hint 12 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint 1 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:1349-1362
  • hint model recommends changes before merge

Findings

  1. [medium] Failed Call at werkrevisor/pr_lens.mjs:22-30, ORANGE, as a line comment

  2. [medium] Callback Contract at werkrevisor/cli.py:1323-1328, ORANGE, as a line comment

  3. [medium] Missing Cost at werkrevisor/cli.py:1349-1362, hint, as a line comment

Classifier

8 of 8 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 57780, calls 1, tokens in 55744 / out 34265 (reasoning included in out), duration 404.1 s, cost 0.0578 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6459, cost 0.0003 USD, finding judge `jev-1.14.0` calls 3, tokens in 5047, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25495 / out 380, cost 0.0034 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 33992, cost 0.0014 USD, build file judge `jev-1.14.0` calls 10, tokens in 10982, cost 0.0005 USD, phases: path_judge 0.6 s, classifier 0.4 s, doc_check 0.7 s, context 7.7 s, model_review 393.5 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main bad4782).
Reviewed head `15dd3582cc28789f6a20e96f0611848837dd5636` against base `bad4782b99657b8a662db8b9333722c04619962e`, werkrevisor 0.65.2.

Diagrams

Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.

pr-lens-arch-containers.svg

pr-lens-flow-diagram-cost.svg

<!-- werkrevisor:15dd3582cc28789f6a20e96f0611848837dd5636 --> <!-- werkrevisor-verdict:ORANGE --> <!-- werkrevisor-blocks:63409cde741958f25faba06dad424b3c,134e9e7c4e20da097e94506deaddc9fe,d21e9a7256a79225097563dbbbe8a1c0,279ceb5c617659daa18bd3a576d54d9b,bde4d00d50992e84778f1ef8ae10d041,1d81fca8c42dbc68a8d130de0a42144b,0876df68568537d89da1bce76aa5a6d8,dd7390dea91edd0b4fb6707f447bb596,d5faadd9a5721f2e14276d12358bdb94,307098a735b318115a5d947f02ed8df9,911fff4991e0402eb539885689b26d85,8a49ae3c2cc1553554b7e938e7d2d221,9c5fe7e4c38bdc08691e292081d93ce7,0dee2ccc6cb7f9df3b332adcba209d6c,97ae6d34524bf5ac1bcaae7cfe1abb0d,683904167c90880f9394e05f2239f937,706f24f14f3caae33a87765bc63bba3e,ce44fe9624a95971d7e1fb8a5c819c03,dfc5e3c3eb88b186fff7c7a8b8130f02,d65111f4d74aae37689965efe54df69c,f9e2e3fb1f55296ff5420943cde9126a,847171037bae7a881992c05215d913c6,5ffb2d9443d0febb9533622afa206898,88e964d4ce7a2ef100b2784bd3e03776,c953214c00ea0365b24d859338dea927,0d4b1f06bb63ae5eaa5ecdc9eee188db,0de1f38acc3b2dac8c4b35bbe70ec881,f1585989851248a2cdd3839f190db11a,9c3053e2775f4d0593a78067cded19d7,38711d39767625743151649b51835f88,8568dc89fa52197e9b0d00aa56585fa2,a6d371d72925492d60f8e4495736daf9,d8b5adf93f08f525491e9bfbe337be2f,2268c92ee72701d419ac138890534913,891cb918b9ed302be63a10634b2fbe8f,d0961487cc001692775811eb5a3fb2bf,7edee309ed0d536adf129697841d130d,f25b2d9d5b10a5bc84cac33c7fd4d44b,3a46395c910fbba407ca48bbd1c79c9b,d2ffa4118b6f1812521f1a38c0798580,8445171d6fd82c2c4d76f1d633bb7199,4cc22115da27696787cecd65bfb65952,8e822399fc5dcf5eb33b233ccbbe8215,50f471144dae8364c94ec7a04c245f2c --> <!-- werkrevisor:diagrams --> <!-- werkrevisor-folded --> <details><summary>🟠 werkrevisor ORANGE for <code>15dd358</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/140#issuecomment-7501">the result for <code>00a7b82</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `mixed`. ### Reasons - **ORANGE** 2 finding(s) that Jev rates as probably real and severe: werkrevisor/pr_lens.mjs:22-30, werkrevisor/cli.py:1323-1328 - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test - **hint** 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02) - **hint** 12 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:1349-1362 - **hint** model recommends changes before merge ### Findings 1. **[medium] Failed Call** at `werkrevisor/pr_lens.mjs:22-30`, ORANGE, as a line comment 2. **[medium] Callback Contract** at `werkrevisor/cli.py:1323-1328`, ORANGE, as a line comment 3. **[medium] Missing Cost** at `werkrevisor/cli.py:1349-1362`, hint, as a line comment ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 57780, calls 1, tokens in 55744 / out 34265 (reasoning included in out), duration 404.1 s, cost 0.0578 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6459, cost 0.0003 USD, finding judge `jev-1.14.0` calls 3, tokens in 5047, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25495 / out 380, cost 0.0034 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 33992, cost 0.0014 USD, build file judge `jev-1.14.0` calls 10, tokens in 10982, cost 0.0005 USD, phases: path_judge 0.6 s, classifier 0.4 s, doc_check 0.7 s, context 7.7 s, model_review 393.5 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main bad4782).<br> Reviewed head `15dd3582cc28789f6a20e96f0611848837dd5636` against base `bad4782b99657b8a662db8b9333722c04619962e`, werkrevisor 0.65.2. </sub> ### Diagrams Drawn by a model from the diff with [PR Lens](https://github.com/coldteadotai/pr-lens): a view of what the PR touches, not a statement of what it does. ![pr-lens-arch-containers.svg](https://git.javagil.de/attachments/51edae18-fce4-44c2-8864-be88713d0b07) ![pr-lens-flow-diagram-cost.svg](https://git.javagil.de/attachments/3c1a24f2-9253-4153-b757-eb7e594a2545) </details>
ai-reviewer left a comment
Collaborator

werkrevisor: ORANGE / FINDINGS for 15dd3582cc28789f6a20e96f0611848837dd5636.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `15dd3582cc28789f6a20e96f0611848837dd5636`. The result comment of werkrevisor holds the details. <!-- werkrevisor-review:15dd3582cc28789f6a20e96f0611848837dd5636 -->
@@ -1321,0 +1320,4 @@
def _draw(config, pr, deps, diff, result):
"""The usage goes into result.diagrams, also of a drawing that failed after a model call."""
try:
drawing = deps.draw_diagrams(config, pr, diff)
Collaborator

[medium] Callback Contract (werkrevisor)

ORANGE; Jev: real 0.79, severity 2.9 of 3

Deps.draw_diagrams is still documented as returning a list of (name, SVG) tuples, but _draw now unconditionally reads .usage and .files from a Drawing. A callback following the documented contract raises AttributeError, so its diagrams and usage are skipped. Update the contract and callers, or support the former return shape.

**[medium] Callback Contract** (werkrevisor) ORANGE; Jev: real 0.79, severity 2.9 of 3 `Deps.draw_diagrams` is still documented as returning a list of `(name, SVG)` tuples, but `_draw` now unconditionally reads `.usage` and `.files` from a `Drawing`. A callback following the documented contract raises `AttributeError`, so its diagrams and usage are skipped. Update the contract and callers, or support the former return shape.
Author
Collaborator

Fixed in 00a7b82: the comment of Deps.draw_diagrams names the new contract, prlens.Drawing and prlens.DiagramError with its usage; all callers return a Drawing.

Fixed in `00a7b82`: the comment of `Deps.draw_diagrams` names the new contract, `prlens.Drawing` and `prlens.DiagramError` with its usage; all callers return a `Drawing`.
claude-code marked this conversation as resolved
@@ -1336,3 +1347,3 @@
if not diff:
return
drawn = deps.draw_diagrams(config, pr, diff)
drawn = _draw(config, pr, deps, diff, result)
Collaborator

[medium] Missing Cost (werkrevisor)

hint; Jev: real 0.80, severity 2.4 of 3

The new cost line is added only when at least one asset was attached. If PR Lens returns no SVGs, or if every create_comment_asset call fails, _draw has already recorded a paid model call but the result comment receives no Diagrams section and never names that cost. Add the usage section when usage exists even if no asset could be attached.

**[medium] Missing Cost** (werkrevisor) hint; Jev: real 0.80, severity 2.4 of 3 The new cost line is added only when at least one asset was attached. If PR Lens returns no SVGs, or if every `create_comment_asset` call fails, `_draw` has already recorded a paid model call but the result comment receives no Diagrams section and never names that cost. Add the usage section when usage exists even if no asset could be attached.
Author
Collaborator

Fixed in 00a7b82: a drawing that cost something but attached no diagram gets the section "Diagrams" with "No diagram was attached." and the cost line. Test: test_a_drawing_without_a_diagram_names_its_cost_in_the_comment.

Fixed in `00a7b82`: a drawing that cost something but attached no diagram gets the section "Diagrams" with "No diagram was attached." and the cost line. Test: `test_a_drawing_without_a_diagram_names_its_cost_in_the_comment`.
claude-code marked this conversation as resolved
@@ -13,0 +19,4 @@
// OpenRouter adds the cost of the call to its usage only on request.
request = { ...init, body: JSON.stringify({ ...JSON.parse(init.body), usage: { include: true } }) };
}
const response = await originalFetch(url, request);
Collaborator

[medium] Failed Call (werkrevisor)

ORANGE; Jev: real 0.90, severity 2.9 of 3

If originalFetch rejects after OpenRouter accepted the request, such as when the connection drops while reading the response, the wrapper never reaches its accounting block. The outer handler then emits calls: 0 and a known $0 cost even though the request may have been billed. Count the attempted call and mark its cost unknown when fetch rejects.

**[medium] Failed Call** (werkrevisor) ORANGE; Jev: real 0.90, severity 2.9 of 3 If `originalFetch` rejects after OpenRouter accepted the request, such as when the connection drops while reading the response, the wrapper never reaches its accounting block. The outer handler then emits `calls: 0` and a known `$0` cost even though the request may have been billed. Count the attempted call and mark its cost unknown when `fetch` rejects.
Author
Collaborator

Fixed in 00a7b82: the wrapper counts the call before fetch, and a rejected fetch makes the cost unknown. Test: test_a_call_without_an_answer_counts_with_an_unknown_cost.

Fixed in `00a7b82`: the wrapper counts the call before `fetch`, and a rejected `fetch` makes the cost unknown. Test: `test_a_call_without_an_answer_counts_with_an_unknown_cost`.
claude-code marked this conversation as resolved
ai-reviewer left a comment
Collaborator

Codex review of 15dd3582cc28789f6a20e96f0611848837dd5636

Codex review of `15dd3582cc28789f6a20e96f0611848837dd5636` <!-- werkrevisor-codex -->
@@ -203,1 +204,3 @@
boundedSum("cost_usd, excerpt_cost_usd and doc_summary_cost_usd", listOf(it, excerptCost, docSummaryCost))
boundedSum(
"cost_usd, excerpt_cost_usd, doc_summary_cost_usd and diagram_cost_usd",
listOf(it, excerptCost, docSummaryCost, diagramCost),
Collaborator

[medium] Unknown diagram costs become apparently complete totals (Codex review)

The script deliberately returns diagram_cost_usd=null when a response lacks cost information. boundedSum drops that null, while costStatus remains the review's original status. Thus a reported $0.10 review with paid diagram calls of unknown cost appears as a fully known $0.10 total, without increasing unknownCostReviews or marking the total estimated. Distinguish missing diagram costs after calls from diagrams that were never requested.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] Unknown diagram costs become apparently complete totals** (Codex review) The script deliberately returns diagram_cost_usd=null when a response lacks cost information. boundedSum drops that null, while costStatus remains the review's original status. Thus a reported $0.10 review with paid diagram calls of unknown cost appears as a fully known $0.10 total, without increasing unknownCostReviews or marking the total estimated. Distinguish missing diagram costs after calls from diagrams that were never requested. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in 00a7b82, the same finding as on f0b4e4a (this review was of 15dd358): diagram_calls > 0 without diagram_cost_usd makes the review cost unknown.

Fixed in `00a7b82`, the same finding as on `f0b4e4a` (this review was of `15dd358`): `diagram_calls > 0` without `diagram_cost_usd` makes the review cost unknown.
claude-code marked this conversation as resolved
@@ -60,6 +85,7 @@ try {
flows: document.flows.length,
attempts,
truncated: truncatedAt !== undefined,
usage: usageOut(),
Collaborator

[medium] Completed-call usage is lost when drawing times out (Codex review)

Usage is emitted only when drawing finishes or throws. If the first paid response completes but a repair call exceeds timeout_seconds, subprocess.run kills Node before either output path runs. The Python timeout handler then supplies no usage, so the record reports zero diagram calls and no cost despite the completed paid call. Preserve usage incrementally and recover it on timeout.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] Completed-call usage is lost when drawing times out** (Codex review) Usage is emitted only when drawing finishes or throws. If the first paid response completes but a repair call exceeds timeout_seconds, subprocess.run kills Node before either output path runs. The Python timeout handler then supplies no usage, so the record reports zero diagram calls and no cost despite the completed paid call. Preserve usage incrementally and recover it on timeout. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in 00a7b82, the same finding as on f0b4e4a (this review was of 15dd358): the usage goes to stderr after each call and survives a timeout.

Fixed in `00a7b82`, the same finding as on `f0b4e4a` (this review was of `15dd358`): the usage goes to stderr after each call and survives a timeout.
claude-code marked this conversation as resolved
Collaborator

Codex review: not critical

The change adds diagram usage accounting and display without changing authorization, merge decisions, CI gates, or existing data. Its failure modes affect reported costs rather than security or merge safety.

Findings

  1. [medium] Completed-call usage is lost when drawing times out at werkrevisor/pr_lens.mjs:88, as a line comment
  2. [medium] Unknown diagram costs become apparently complete totals at statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:206, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

Model `gpt-6-astra`, effort high, tokens in 398128 (cached 351360) / out 2045, 84.7 s, 8 commands; reviewed head `15dd3582cc28789f6a20e96f0611848837dd5636`.
<!-- werkrevisor-codex:15dd3582cc28789f6a20e96f0611848837dd5636 --> <!-- werkrevisor-codex-critical:false --> <!-- werkrevisor-codex-unattached:0 --> <!-- werkrevisor-codex --> ## Codex review: not critical The change adds diagram usage accounting and display without changing authorization, merge decisions, CI gates, or existing data. Its failure modes affect reported costs rather than security or merge safety. ### Findings 1. **[medium] Completed-call usage is lost when drawing times out** at `werkrevisor/pr_lens.mjs:88`, as a line comment 2. **[medium] Unknown diagram costs become apparently complete totals** at `statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:206`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. <sub> Model `gpt-6-astra`, effort high, tokens in 398128 (cached 351360) / out 2045, 84.7 s, 8 commands; reviewed head `15dd3582cc28789f6a20e96f0611848837dd5636`. </sub>
Collaborator
⚠️ werkrevisor ERROR for 00a7b82, superseded by the result for 4cac73e

⚠️ werkrevisor: ERROR / review not completed, human action needed

Change kind: mixed.

Reasons

  • ERROR the model answer reached the output token limit [model] max_output_tokens = 64000
  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test
  • hint 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02)
  • hint 12 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint 1 finding(s) of low severity or rated minor by Jev: werkrevisor/pr_lens.mjs
  • hint model recommends changes before merge
  • hint model rates the risk as high (advisory: Jev judged every changed file)

Findings

  1. Issue at werkrevisor/pr_lens.mjs, severity unknown, counted as medium

    hint; Jev: real 0.15, severity 0.5 of 3

Classifier

8 of 8 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 60003, calls 1, tokens in 57881 / out 64000 (reasoning included in out), duration 676.1 s, cost 0.0941 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6576, cost 0.0003 USD, finding judge `jev-1.14.0` calls 1, tokens in 576, cost 0.0000 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25869 / out 380, cost 0.0034 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 36682, cost 0.0015 USD, build file judge `jev-1.14.0` calls 10, tokens in 12224, cost 0.0005 USD, phases: path_judge 0.5 s, classifier 0.4 s, doc_check 0.5 s, context 6.5 s, model_review 667.3 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main bad4782).
Reviewed head `00a7b8264a859317fcf75eb1a0e83443f0e5b48a` against base `bad4782b99657b8a662db8b9333722c04619962e`, werkrevisor 0.65.2.
<!-- werkrevisor-error:00a7b8264a859317fcf75eb1a0e83443f0e5b48a --> <!-- werkrevisor-folded --> <details><summary>⚠️ werkrevisor ERROR for <code>00a7b82</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/140#issuecomment-7962">the result for <code>4cac73e</code></a></summary> ## ⚠️ werkrevisor: ERROR / review not completed, human action needed Change kind: `mixed`. ### Reasons - **ERROR** the model answer reached the output token limit [model] max_output_tokens = 64000 - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test - **hint** 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02) - **hint** 12 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 finding(s) of low severity or rated minor by Jev: werkrevisor/pr_lens.mjs - **hint** model recommends changes before merge - **hint** model rates the risk as high (advisory: Jev judged every changed file) ### Findings 1. **Issue** at `werkrevisor/pr_lens.mjs`, severity unknown, counted as medium hint; Jev: real 0.15, severity 0.5 of 3 ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 60003, calls 1, tokens in 57881 / out 64000 (reasoning included in out), duration 676.1 s, cost 0.0941 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6576, cost 0.0003 USD, finding judge `jev-1.14.0` calls 1, tokens in 576, cost 0.0000 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25869 / out 380, cost 0.0034 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 36682, cost 0.0015 USD, build file judge `jev-1.14.0` calls 10, tokens in 12224, cost 0.0005 USD, phases: path_judge 0.5 s, classifier 0.4 s, doc_check 0.5 s, context 6.5 s, model_review 667.3 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main bad4782).<br> Reviewed head `00a7b8264a859317fcf75eb1a0e83443f0e5b48a` against base `bad4782b99657b8a662db8b9333722c04619962e`, werkrevisor 0.65.2. </sub> </details>
claude-code changed title from werkrevisor 0.67.0: the cost of the PR Lens diagrams in the usage record, the result comment and the statistics to werkrevisor 0.73.0: the cost of the PR Lens diagrams in the usage record, the result comment and the statistics 2026-10-24 23:08:38 +02:00
claude-code force-pushed claude-code/diagram-cost from 00a7b8264a to 4cac73e395 2026-10-24 23:08:40 +02:00 Compare
Collaborator
🟠 werkrevisor ORANGE for 4cac73e, superseded by the result for 293848b

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: mixed.

Reasons

  • ORANGE 2 finding(s) that Jev rates as probably real and severe: werkrevisor/pr_lens.mjs:21-38, werkrevisor/prlens.py:127-132
  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test
  • hint 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02)
  • hint 12 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt with 40 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint excerpt selection by openrouter/openai/gpt-6-luna failed, the fixed window was used: werkrevisor/cli.py (a range lies outside the file)
  • hint 1 finding(s) possibly severe, but Jev doubts that they are real: statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:189-210 (advisory: Jev judged every changed file)
  • hint model recommends changes before merge
  • hint model rates the risk as high (advisory: Jev judged every changed file)

Findings

  1. [high] In-Flight Cost at werkrevisor/pr_lens.mjs:21-38, ORANGE, as a line comment

  2. [medium] Usage Dropped at werkrevisor/prlens.py:127-132, ORANGE, as a line comment

  3. [medium] Model Attribution at statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:189-210, YELLOW, as a line comment

Classifier

8 of 8 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 63609, calls 1, tokens in 61375 / out 40466 (reasoning included in out), duration 356.3 s, cost 0.0669 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6572, cost 0.0003 USD, finding judge `jev-1.14.0` calls 3, tokens in 5428, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25937 / out 454, cost 0.0035 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 36659, cost 0.0015 USD, build file judge `jev-1.14.0` calls 10, tokens in 12224, cost 0.0005 USD, phases: path_judge 0.6 s, classifier 0.4 s, doc_check 0.6 s, context 6.3 s, model_review 347.8 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main f856ea9).
Reviewed head `4cac73e39564e77f86d82e6ced43f1e45e9d161c` against base `f856ea9ff0b5d90f3e3fc59ad12002a2d5be8da0`, werkrevisor 0.71.0.

Diagrams

Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.

pr-lens-containers.svg

pr-lens-cost-flow.svg

<!-- werkrevisor:4cac73e39564e77f86d82e6ced43f1e45e9d161c --> <!-- werkrevisor-verdict:ORANGE --> <!-- werkrevisor-blocks:4e36668f56b3adede9dd8803092e1494,b4cc2152ca0e2daeeb5fd61822f32b07,d21e9a7256a79225097563dbbbe8a1c0,279ceb5c617659daa18bd3a576d54d9b,bde4d00d50992e84778f1ef8ae10d041,1d81fca8c42dbc68a8d130de0a42144b,caae5a8f8c592c4a4f05db166dec71b8,9a7b294e234d1752504ba48a0d271e89,d5faadd9a5721f2e14276d12358bdb94,1c06d69232f6c0d0939602aa623eb89c,9d45acbe7decdc04bc8e5c4f92739e21,25f8e7a0617e7f9768b11140fde5d2ef,9c5fe7e4c38bdc08691e292081d93ce7,0dee2ccc6cb7f9df3b332adcba209d6c,97ae6d34524bf5ac1bcaae7cfe1abb0d,683904167c90880f9394e05f2239f937,304120908522508bbd1392258c305d18,ce44fe9624a95971d7e1fb8a5c819c03,f9aada625221b8ce392789822bfbe146,c1ece810470b1c571dcb410785b8e622,b072325581e333dab96566adb6da6087,f9e2e3fb1f55296ff5420943cde9126a,847171037bae7a881992c05215d913c6,5ffb2d9443d0febb9533622afa206898,88e964d4ce7a2ef100b2784bd3e03776,b8f4341b68df8e27798c434282972232,67ff34b71195f89fdeba6730bc396d17,0d4b1f06bb63ae5eaa5ecdc9eee188db,0de1f38acc3b2dac8c4b35bbe70ec881,f1585989851248a2cdd3839f190db11a,36587aa143b8b220c02c7f3b4d696bd3,a6d371d72925492d60f8e4495736daf9,d8b5adf93f08f525491e9bfbe337be2f,e509a6d8194bebb81bbac964a333777e,891cb918b9ed302be63a10634b2fbe8f,13e08a9484adff429d655ab5546fe018,0a3de30906fbe728a6b9a8cda6108144,53c50ddc271dbcbbed91b5ee9b813871,536339a64687886483750b5cfc329a52,3a46395c910fbba407ca48bbd1c79c9b,d2ffa4118b6f1812521f1a38c0798580,cf76790835076baee94331ab15009e44,4b225314d4bf1d606bdd9218dd9fa145,8e822399fc5dcf5eb33b233ccbbe8215,50f471144dae8364c94ec7a04c245f2c --> <!-- werkrevisor:diagrams --> <!-- werkrevisor-folded --> <details><summary>🟠 werkrevisor ORANGE for <code>4cac73e</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/140#issuecomment-7989">the result for <code>293848b</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `mixed`. ### Reasons - **ORANGE** 2 finding(s) that Jev rates as probably real and severe: werkrevisor/pr_lens.mjs:21-38, werkrevisor/prlens.py:127-132 - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:62 disables a test - **hint** 9 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02) - **hint** 12 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt with 40 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** excerpt selection by openrouter/openai/gpt-6-luna failed, the fixed window was used: werkrevisor/cli.py (a range lies outside the file) - **hint** 1 finding(s) possibly severe, but Jev doubts that they are real: statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:189-210 (advisory: Jev judged every changed file) - **hint** model recommends changes before merge - **hint** model rates the risk as high (advisory: Jev judged every changed file) ### Findings 1. **[high] In-Flight Cost** at `werkrevisor/pr_lens.mjs:21-38`, ORANGE, as a line comment 2. **[medium] Usage Dropped** at `werkrevisor/prlens.py:127-132`, ORANGE, as a line comment 3. **[medium] Model Attribution** at `statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:189-210`, YELLOW, as a line comment ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 2 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 63609, calls 1, tokens in 61375 / out 40466 (reasoning included in out), duration 356.3 s, cost 0.0669 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6572, cost 0.0003 USD, finding judge `jev-1.14.0` calls 3, tokens in 5428, cost 0.0002 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 25937 / out 454, cost 0.0035 USD (reported), PR document check `jev-1.14.0` calls 12, tokens in 36659, cost 0.0015 USD, build file judge `jev-1.14.0` calls 10, tokens in 12224, cost 0.0005 USD, phases: path_judge 0.6 s, classifier 0.4 s, doc_check 0.6 s, context 6.3 s, model_review 347.8 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main f856ea9).<br> Reviewed head `4cac73e39564e77f86d82e6ced43f1e45e9d161c` against base `f856ea9ff0b5d90f3e3fc59ad12002a2d5be8da0`, werkrevisor 0.71.0. </sub> ### Diagrams Drawn by a model from the diff with [PR Lens](https://github.com/coldteadotai/pr-lens): a view of what the PR touches, not a statement of what it does. ![pr-lens-containers.svg](https://git.javagil.de/attachments/3b2c6ee6-a467-4319-8a4b-3723585d4c05) ![pr-lens-cost-flow.svg](https://git.javagil.de/attachments/a22f63b7-8eef-4660-9f16-32f0267f7b0b) </details>
ai-reviewer left a comment
Collaborator

werkrevisor: ORANGE / FINDINGS for 4cac73e39564e77f86d82e6ced43f1e45e9d161c.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `4cac73e39564e77f86d82e6ced43f1e45e9d161c`. The result comment of werkrevisor holds the details. <!-- werkrevisor-review:4cac73e39564e77f86d82e6ced43f1e45e9d161c -->
@@ -186,6 +186,9 @@ class UsageApiController(
// Read also without cost_usd, so that an invalid value is refused although an unknown review cost ignores it.
val excerptCost = body.number("excerpt_cost_usd")
val docSummaryCost = body.number("doc_summary_cost_usd")
val diagramCost = body.number("diagram_cost_usd")
Collaborator

[medium] Model Attribution (werkrevisor)

YELLOW; Jev: real 0.56, severity 2.9 of 3

When [diagrams] model differs from the review model, this parser ignores the emitted diagram_model and stores the added diagram cost in a UsageRecord whose only model is the review model. The existing per-model statistics therefore charge the diagram spend to the review model and show no separate diagram-model total. Store and attribute the diagram model separately.

**[medium] Model Attribution** (werkrevisor) YELLOW; Jev: real 0.56, severity 2.9 of 3 When `[diagrams] model` differs from the review model, this parser ignores the emitted `diagram_model` and stores the added diagram cost in a `UsageRecord` whose only model is the review `model`. The existing per-model statistics therefore charge the diagram spend to the review model and show no separate diagram-model total. Store and attribute the diagram model separately.
Author
Collaborator

Fixed in dfbbec4: the statistics store diagram_model and diagram_cost_usd in their own columns and charge the drawing to the diagram model in byModel; the record names the model as openrouter/<slug>, like the review model. Test: Scenario#140.03 in UsageApiIntegrationTest.

Fixed in `dfbbec4`: the statistics store `diagram_model` and `diagram_cost_usd` in their own columns and charge the drawing to the diagram model in `byModel`; the record names the model as `openrouter/<slug>`, like the review model. Test: Scenario#140.03 in `UsageApiIntegrationTest`.
claude-code marked this conversation as resolved
@@ -13,0 +18,4 @@
// OpenRouter adds the cost of the call to its usage only on request.
const request = typeof init?.body === "string"
? { ...init, body: JSON.stringify({ ...JSON.parse(init.body), usage: { include: true } }) } : init;
usage.calls += 1;
Collaborator

[high] In-Flight Cost (werkrevisor)

ORANGE; Jev: real 0.79, severity 3.0 of 3

When the drawing timeout kills Node while a chat-completion request is awaiting its response, that request has no usage line because reportUsage() runs only after fetch settles. The record can therefore omit the call entirely or retain only a known partial cost, while OpenRouter may still have billed the in-flight request. Emit an in-flight usage record with an unknown cost, or otherwise mark timed-out usage as unknown. This also breaks the documented guarantee that a killed drawing reports the calls it made.

**[high] In-Flight Cost** (werkrevisor) ORANGE; Jev: real 0.79, severity 3.0 of 3 When the drawing timeout kills Node while a chat-completion request is awaiting its response, that request has no usage line because `reportUsage()` runs only after `fetch` settles. The record can therefore omit the call entirely or retain only a known partial cost, while OpenRouter may still have billed the in-flight request. Emit an in-flight usage record with an unknown cost, or otherwise mark timed-out usage as unknown. This also breaks the documented guarantee that a killed drawing reports the calls it made.
Author
Collaborator

Fixed in dfbbec4: pr_lens.mjs writes a usage line with an unknown cost before each call, so a call killed in flight counts with an unknown cost. Test: test_a_call_in_flight_counts_with_an_unknown_cost.

Fixed in `dfbbec4`: `pr_lens.mjs` writes a usage line with an unknown cost before each call, so a call killed in flight counts with an unknown cost. Test: `test_a_call_in_flight_counts_with_an_unknown_cost`.
claude-code marked this conversation as resolved
@@ -80,3 +124,4 @@
raise DiagramError("PR Lens answered without diagrams", _usage(request["model"], completed.stderr)) from None
drawn, names = [], set()
for svg in svgs[:config.max_diagrams]:
name = f"{ASSET_PREFIX}{re.sub(r'[^a-z0-9-]+', '-', str(svg.get('view', '')).lower()).strip('-') or 'view'}"
Collaborator

[medium] Usage Dropped (werkrevisor)

ORANGE; Jev: real 0.88, severity 2.9 of 3

Usage is parsed only when draw() successfully constructs a Drawing. If the renderer returns an SVG entry without a string svg field, svg["svg"].encode(...) raises a raw KeyError or AttributeError instead of DiagramError. Because _draw() handles only DiagramError, the already billed model call is omitted from result.diagrams and the usage record. Parse the usage immediately after the subprocess and wrap all post-call processing failures in a DiagramError carrying it.

**[medium] Usage Dropped** (werkrevisor) ORANGE; Jev: real 0.88, severity 2.9 of 3 Usage is parsed only when `draw()` successfully constructs a `Drawing`. If the renderer returns an SVG entry without a string `svg` field, `svg["svg"].encode(...)` raises a raw `KeyError` or `AttributeError` instead of `DiagramError`. Because `_draw()` handles only `DiagramError`, the already billed model call is omitted from `result.diagrams` and the usage record. Parse the usage immediately after the subprocess and wrap all post-call processing failures in a `DiagramError` carrying it.
Author
Collaborator

Fixed in dfbbec4: draw reads the usage right after the subprocess, and every failure of the post-processing raises a DiagramError with it. Test: test_a_malformed_diagram_keeps_the_usage.

Fixed in `dfbbec4`: `draw` reads the usage right after the subprocess, and every failure of the post-processing raises a `DiagramError` with it. Test: `test_a_malformed_diagram_keeps_the_usage`.
claude-code marked this conversation as resolved
ai-reviewer left a comment
Collaborator

Codex review of 4cac73e39564e77f86d82e6ced43f1e45e9d161c

Codex review of `4cac73e39564e77f86d82e6ced43f1e45e9d161c` <!-- werkrevisor-codex -->
@@ -201,3 +204,2 @@
costUsd =
body.number("cost_usd")?.let {
boundedSum("cost_usd, excerpt_cost_usd and doc_summary_cost_usd", listOf(it, excerptCost, docSummaryCost))
body.number("cost_usd")?.takeUnless { diagramUnpriced }?.let {
Collaborator

[medium] Unknown diagram cost is treated as known zero when the review model was skipped (Codex review)

When diagramUnpriced is true, costUsd becomes null but costStatus remains unchanged. A review skipped because its PR document has open questions can still draw diagrams and sends cost_status="none". UsageRecord.costKnown explicitly treats "none" as known even with null costUsd, so these unpriced diagram calls count as zero and do not increase unknownCostReviews. Set the aggregate cost status to unknown when diagramUnpriced is true.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] Unknown diagram cost is treated as known zero when the review model was skipped** (Codex review) When diagramUnpriced is true, costUsd becomes null but costStatus remains unchanged. A review skipped because its PR document has open questions can still draw diagrams and sends cost_status="none". UsageRecord.costKnown explicitly treats "none" as known even with null costUsd, so these unpriced diagram calls count as zero and do not increase unknownCostReviews. Set the aggregate cost status to unknown when diagramUnpriced is true. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in 293848b: an unpriced drawing sets the status of the record to unknown, also when the review itself had none. Test: Scenario#140.03 in UsageApiIntegrationTest.

Fixed in `293848b`: an unpriced drawing sets the status of the record to `unknown`, also when the review itself had `none`. Test: Scenario#140.03 in `UsageApiIntegrationTest`.
claude-code marked this conversation as resolved
@@ -13,0 +18,4 @@
// OpenRouter adds the cost of the call to its usage only on request.
const request = typeof init?.body === "string"
? { ...init, body: JSON.stringify({ ...JSON.parse(init.body), usage: { include: true } }) } : init;
usage.calls += 1;
Collaborator

[medium] Timeouts omit the in-flight model call (Codex review)

The call counter is incremented before fetching, but usage is emitted only after the response or an exception. When Python kills the process during a pending call, neither happens. A timeout during the first call therefore records no diagram usage; during a repair call it reports only the earlier calls with an apparently complete cost. Emit pending-call usage with unknown cost before awaiting the response.

Fix it, or reply why it does not hold, and resolve this comment.

**[medium] Timeouts omit the in-flight model call** (Codex review) The call counter is incremented before fetching, but usage is emitted only after the response or an exception. When Python kills the process during a pending call, neither happens. A timeout during the first call therefore records no diagram usage; during a repair call it reports only the earlier calls with an apparently complete cost. Emit pending-call usage with unknown cost before awaiting the response. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in dfbbec4 (rebased as 293848b), the same as werkrevisor's "In-Flight Cost": a usage line with an unknown cost precedes each call.

Fixed in `dfbbec4` (rebased as `293848b`), the same as werkrevisor's "In-Flight Cost": a usage line with an unknown cost precedes each call.
claude-code marked this conversation as resolved
Collaborator

Codex review: not critical

The change adds diagram usage accounting and presentation without changing authorization, credentials, merge gates, or existing stored data. Its defects affect reported costs and are testable accounting errors.

Security effect: none; fails closed: no; a test of the closed case: no.

Findings

  1. [medium] Timeouts omit the in-flight model call at werkrevisor/pr_lens.mjs:21, as a line comment
  2. [medium] Unknown diagram cost is treated as known zero when the review model was skipped at statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:205, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

Model `gpt-6-astra`, effort high, tokens in 533742 (cached 464896) / out 2948, 64.7 s, 13 commands; reviewed head `4cac73e39564e77f86d82e6ced43f1e45e9d161c`.
<!-- werkrevisor-codex:4cac73e39564e77f86d82e6ced43f1e45e9d161c --> <!-- werkrevisor-codex-critical:false --> <!-- werkrevisor-codex-unattached:0 --> <!-- werkrevisor-codex --> ## Codex review: not critical The change adds diagram usage accounting and presentation without changing authorization, credentials, merge gates, or existing stored data. Its defects affect reported costs and are testable accounting errors. Security effect: none; fails closed: no; a test of the closed case: no. ### Findings 1. **[medium] Timeouts omit the in-flight model call** at `werkrevisor/pr_lens.mjs:21`, as a line comment 2. **[medium] Unknown diagram cost is treated as known zero when the review model was skipped** at `statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:205`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. <sub> Model `gpt-6-astra`, effort high, tokens in 533742 (cached 464896) / out 2948, 64.7 s, 13 commands; reviewed head `4cac73e39564e77f86d82e6ced43f1e45e9d161c`. </sub>
claude-code force-pushed claude-code/diagram-cost from dfbbec48b1 to 293848b995 2026-10-24 23:24:48 +02:00 Compare
Collaborator
🟡 werkrevisor YELLOW for 293848b, superseded by the result for ce17701

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test
  • hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02)
  • hint 14 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-23-PR#140-diagram-cost.md:76

Findings

  1. [low] Inaccurate Section Description at docs/prs/2026-10-23-PR#140-diagram-cost.md:76

    hint

    The PR document says that a drawing which attached no diagram "gets the section with this line only",
    but prlens.section also inserts the sentence "No diagram was attached." before the cost line, so the
    comment contains two lines. The new test in tests/test_prlens.py asserts exactly that two-line form.
    Please correct the description so that it matches the implemented comment.

Classifier

8 of 8 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 70202, calls 1, tokens in 67843 / out 39632 (reasoning included in out), duration 318.6 s, cost 0.0679 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6663, cost 0.0003 USD, finding judge `none` calls 0, tokens in 0, cost 0.0000 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 26984 / out 490, cost 0.0036 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 44675, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 15276, cost 0.0006 USD, phases: path_judge 0.6 s, classifier 0.3 s, doc_check 0.8 s, context 8.6 s, model_review 307.6 s, finding_judge 0.0 s, config: defaults + host + repository (.werkrevisor.toml at main 568ae2b).
Reviewed head `293848b995853457ae9d29594e05930fc6127b5b` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.

Diagrams

Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.

pr-lens-view-containers.svg

pr-lens-view-cost-flow.svg

<!-- werkrevisor:293848b995853457ae9d29594e05930fc6127b5b --> <!-- werkrevisor-verdict:YELLOW --> <!-- werkrevisor-blocks:23c8393df3cd2e42333e944054898332,3bab67b65c6a1012577c19e1d2d3a0de,d21e9a7256a79225097563dbbbe8a1c0,279ceb5c617659daa18bd3a576d54d9b,bde4d00d50992e84778f1ef8ae10d041,1d81fca8c42dbc68a8d130de0a42144b,580a19b6ff4d32c9426dd7d0dfd19729,60393956e8f584722626331c9ae9676c,d1806eb9f3dde5308e3a18e59252d25d,2cb55a77d858874fa77ca8721429a3b5,9a7b294e234d1752504ba48a0d271e89,502b799bb6e02922b19708f5d0cd86a2,8f4fbac0747049bf6f2cf0ad2c1daa85,b9d42ea28e9688f3a38b63b1956e7c58,2bf190bb49bb6881ebfb79787ba10c72,4f379d72be0a68e0bca5612023ebfb06,dcf9a0459aff8360687e2e3fed1c16d2,10948ffe15e063a39e1d399e6a648b52,9436e50cdc1738a2ce15e89588c9f06a,e61bf47cd5b0b14b9ab407bd099d455f,2bd656785ea2f89343120df3a785495a,e102569b5cc212a4c24667cb87c882f1,9c5fe7e4c38bdc08691e292081d93ce7,0dee2ccc6cb7f9df3b332adcba209d6c,97ae6d34524bf5ac1bcaae7cfe1abb0d,683904167c90880f9394e05f2239f937,2a052dccee75e378b39b4af8bfea94ba,ce44fe9624a95971d7e1fb8a5c819c03,58102414ac20acf40fa93d6ab385ab67,3684f01b5460861efd4d8e9254c8f7fe,b072325581e333dab96566adb6da6087,f9e2e3fb1f55296ff5420943cde9126a,847171037bae7a881992c05215d913c6,5ffb2d9443d0febb9533622afa206898,88e964d4ce7a2ef100b2784bd3e03776,b8f4341b68df8e27798c434282972232,67ff34b71195f89fdeba6730bc396d17,0d4b1f06bb63ae5eaa5ecdc9eee188db,0de1f38acc3b2dac8c4b35bbe70ec881,f1585989851248a2cdd3839f190db11a,879f39390c8504abe82a2b82f2334fbe,a6d371d72925492d60f8e4495736daf9,d8b5adf93f08f525491e9bfbe337be2f,e509a6d8194bebb81bbac964a333777e,6a1f5da4342608150cd22ff1dcd9e1e8,4f785e13ae89a19844e200ff7034dcde,db4387c2bd58b0e4d5ece61e9b055834,e69daf80e5ed7479eb36d589a06e6fc4,3a46395c910fbba407ca48bbd1c79c9b,d2ffa4118b6f1812521f1a38c0798580,cf76790835076baee94331ab15009e44,4b225314d4bf1d606bdd9218dd9fa145,8e822399fc5dcf5eb33b233ccbbe8215,7907f8d04ae5d6344fc672657cba211d --> <!-- werkrevisor:diagrams --> <!-- werkrevisor-folded --> <details><summary>🟡 werkrevisor YELLOW for <code>293848b</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/140#issuecomment-8010">the result for <code>ce17701</code></a></summary> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02) - **hint** 14 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-23-PR#140-diagram-cost.md:76 ### Findings 1. **[low] Inaccurate Section Description** at `docs/prs/2026-10-23-PR#140-diagram-cost.md:76` hint The PR document says that a drawing which attached no diagram "gets the section with this line only", but `prlens.section` also inserts the sentence "No diagram was attached." before the cost line, so the comment contains two lines. The new test in `tests/test_prlens.py` asserts exactly that two-line form. Please correct the description so that it matches the implemented comment. ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 70202, calls 1, tokens in 67843 / out 39632 (reasoning included in out), duration 318.6 s, cost 0.0679 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6663, cost 0.0003 USD, finding judge `none` calls 0, tokens in 0, cost 0.0000 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 26984 / out 490, cost 0.0036 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 44675, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 15276, cost 0.0006 USD, phases: path_judge 0.6 s, classifier 0.3 s, doc_check 0.8 s, context 8.6 s, model_review 307.6 s, finding_judge 0.0 s, config: defaults + host + repository (.werkrevisor.toml at main 568ae2b).<br> Reviewed head `293848b995853457ae9d29594e05930fc6127b5b` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0. </sub> ### Diagrams Drawn by a model from the diff with [PR Lens](https://github.com/coldteadotai/pr-lens): a view of what the PR touches, not a statement of what it does. ![pr-lens-view-containers.svg](https://git.javagil.de/attachments/d6301d2e-60a5-4c77-a82b-8cdfe090cce7) ![pr-lens-view-cost-flow.svg](https://git.javagil.de/attachments/a251b788-950e-4bbd-8cf0-ef77353044c9) </details>
Author
Collaborator

Answer to the review of 293848b, fixed in ce17701:

  • YELLOW "tests lost or disabled: tests/test_pr_lens_script.py:66": unchanged, see my earlier answer; the class is skipped only without Node.js, which the PR-Agent image lacks and the rootfs of the CI has. Left to the human.
  • Hint "Inaccurate Section Description": holds; the PR document now names the sentence "No diagram was attached." and the cost line.
Answer to the review of `293848b`, fixed in `ce17701`: - YELLOW "tests lost or disabled: tests/test_pr_lens_script.py:66": unchanged, see my earlier answer; the class is skipped only without Node.js, which the PR-Agent image lacks and the rootfs of the CI has. Left to the human. - Hint "Inaccurate Section Description": holds; the PR document now names the sentence "No diagram was attached." and the cost line.
ai-reviewer left a comment
Collaborator

Codex review of 293848b995853457ae9d29594e05930fc6127b5b

Codex review of `293848b995853457ae9d29594e05930fc6127b5b` <!-- werkrevisor-codex -->
@@ -242,0 +243,4 @@
// A drawing counts for the model that drew it, which need not be the review model.
val own = Sums.of(recordsByModel[model].orEmpty(), billedByModel[model].orEmpty()) { 0.0 }
val drawn = records.filter { it.costKnown && it.diagramModel == model }.sumOf { it.diagramCostUsd ?: 0.0 }
Group(model, own.copy(llmCostUsd = own.llmCostUsd + drawn))
Collaborator

[low] Per-model local costs retain the diagram cost under the wrong model (Codex review)

The model split adjusts llmCostUsd but leaves localCostUsd including diagrams charged to another model. For example, a published review costing $0.10 and a local review costing $0.02 on model A, with $0.21 of local diagrams on model B, produce costPerPullRequestUsd = -$0.11 for A instead of $0.10. Attribute localCostUsd to the diagram model alongside llmCostUsd.

Fix it, or reply why it does not hold, and resolve this comment.

**[low] Per-model local costs retain the diagram cost under the wrong model** (Codex review) The model split adjusts llmCostUsd but leaves localCostUsd including diagrams charged to another model. For example, a published review costing $0.10 and a local review costing $0.02 on model A, with $0.21 of local diagrams on model B, produce costPerPullRequestUsd = -$0.11 for A instead of $0.10. Attribute localCostUsd to the diagram model alongside llmCostUsd. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in 29885bf: localCostUsd by model moves a local drawing to the diagram model, like llmCostUsd; the review model keeps a positive cost per PR. Test: "a local drawing counts for the diagram model also in the local cost"; it fails without the fix.

Fixed in `29885bf`: `localCostUsd` by model moves a local drawing to the diagram model, like `llmCostUsd`; the review model keeps a positive cost per PR. Test: "a local drawing counts for the diagram model also in the local cost"; it fails without the fix.
claude-code marked this conversation as resolved
Collaborator

Codex review: not critical

The changes concern usage accounting, comment output, and additive statistics columns. They do not change authorization, credentials, merge gates, or destructively modify existing data.

Security effect: none; fails closed: no; a test of the closed case: no.

Findings

  1. [low] Per-model local costs retain the diagram cost under the wrong model at statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/Statistics.kt:246, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

Model `gpt-6-astra`, effort high, tokens in 389727 (cached 318720) / out 2583, 74.0 s, 13 commands; reviewed head `293848b995853457ae9d29594e05930fc6127b5b`.
<!-- werkrevisor-codex:293848b995853457ae9d29594e05930fc6127b5b --> <!-- werkrevisor-codex-critical:false --> <!-- werkrevisor-codex-unattached:0 --> <!-- werkrevisor-codex --> ## Codex review: not critical The changes concern usage accounting, comment output, and additive statistics columns. They do not change authorization, credentials, merge gates, or destructively modify existing data. Security effect: none; fails closed: no; a test of the closed case: no. ### Findings 1. **[low] Per-model local costs retain the diagram cost under the wrong model** at `statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/Statistics.kt:246`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. <sub> Model `gpt-6-astra`, effort high, tokens in 389727 (cached 318720) / out 2583, 74.0 s, 13 commands; reviewed head `293848b995853457ae9d29594e05930fc6127b5b`. </sub>
Collaborator
🟠 werkrevisor ORANGE for ce17701, superseded by the result for 29885bf

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: mixed.

Reasons

  • ORANGE 1 finding(s) that Jev rates as probably real and severe: statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:205-207
  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test
  • hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02)
  • hint 14 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py
  • hint model recommends changes before merge

Findings

  1. [medium] Unbounded Total at statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:205-207, ORANGE, as a line comment

Classifier

8 of 8 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 65944, calls 1, tokens in 63738 / out 52183 (reasoning included in out), duration 420.9 s, cost 0.0817 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6672, cost 0.0003 USD, finding judge `jev-1.14.0` calls 1, tokens in 1586, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 34390 / out 574, cost 0.0015 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 44801, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 15276, cost 0.0006 USD, phases: path_judge 0.5 s, classifier 0.4 s, doc_check 0.6 s, context 9.1 s, model_review 409.3 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main 568ae2b).
Reviewed head `ce177018c8110d7a10c587c49ca05bb3f8a60187` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.

Diagrams

Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.

pr-lens-containers.svg

pr-lens-cost-flow.svg

<!-- werkrevisor:ce177018c8110d7a10c587c49ca05bb3f8a60187 --> <!-- werkrevisor-verdict:ORANGE --> <!-- werkrevisor-blocks:23c8393df3cd2e42333e944054898332,ce5f5686a0d6431ea55ff2b73de9ea40,d21e9a7256a79225097563dbbbe8a1c0,279ceb5c617659daa18bd3a576d54d9b,bde4d00d50992e84778f1ef8ae10d041,1d81fca8c42dbc68a8d130de0a42144b,580a19b6ff4d32c9426dd7d0dfd19729,60393956e8f584722626331c9ae9676c,d1806eb9f3dde5308e3a18e59252d25d,2cb55a77d858874fa77ca8721429a3b5,9a7b294e234d1752504ba48a0d271e89,502b799bb6e02922b19708f5d0cd86a2,8f4fbac0747049bf6f2cf0ad2c1daa85,b9d42ea28e9688f3a38b63b1956e7c58,2bf190bb49bb6881ebfb79787ba10c72,4f379d72be0a68e0bca5612023ebfb06,dcf9a0459aff8360687e2e3fed1c16d2,10948ffe15e063a39e1d399e6a648b52,9436e50cdc1738a2ce15e89588c9f06a,e61bf47cd5b0b14b9ab407bd099d455f,2bd656785ea2f89343120df3a785495a,e102569b5cc212a4c24667cb87c882f1,9c5fe7e4c38bdc08691e292081d93ce7,0dee2ccc6cb7f9df3b332adcba209d6c,97ae6d34524bf5ac1bcaae7cfe1abb0d,683904167c90880f9394e05f2239f937,2a052dccee75e378b39b4af8bfea94ba,ce44fe9624a95971d7e1fb8a5c819c03,58102414ac20acf40fa93d6ab385ab67,3684f01b5460861efd4d8e9254c8f7fe,b072325581e333dab96566adb6da6087,f9e2e3fb1f55296ff5420943cde9126a,847171037bae7a881992c05215d913c6,5ffb2d9443d0febb9533622afa206898,88e964d4ce7a2ef100b2784bd3e03776,b8f4341b68df8e27798c434282972232,67ff34b71195f89fdeba6730bc396d17,0d4b1f06bb63ae5eaa5ecdc9eee188db,0de1f38acc3b2dac8c4b35bbe70ec881,f1585989851248a2cdd3839f190db11a,879f39390c8504abe82a2b82f2334fbe,a6d371d72925492d60f8e4495736daf9,d8b5adf93f08f525491e9bfbe337be2f,e509a6d8194bebb81bbac964a333777e,6a1f5da4342608150cd22ff1dcd9e1e8,4f785e13ae89a19844e200ff7034dcde,db4387c2bd58b0e4d5ece61e9b055834,e69daf80e5ed7479eb36d589a06e6fc4,3a46395c910fbba407ca48bbd1c79c9b,d2ffa4118b6f1812521f1a38c0798580,cf76790835076baee94331ab15009e44,4b225314d4bf1d606bdd9218dd9fa145,8e822399fc5dcf5eb33b233ccbbe8215,7907f8d04ae5d6344fc672657cba211d --> <!-- werkrevisor:diagrams --> <!-- werkrevisor-folded --> <details><summary>🟠 werkrevisor ORANGE for <code>ce17701</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/140#issuecomment-8034">the result for <code>29885bf</code></a></summary> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `mixed`. ### Reasons - **ORANGE** 1 finding(s) that Jev rates as probably real and severe: statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:205-207 - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02) - **hint** 14 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py - **hint** model recommends changes before merge ### Findings 1. **[medium] Unbounded Total** at `statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/UsageApiController.kt:205-207`, ORANGE, as a line comment ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 65944, calls 1, tokens in 63738 / out 52183 (reasoning included in out), duration 420.9 s, cost 0.0817 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6672, cost 0.0003 USD, finding judge `jev-1.14.0` calls 1, tokens in 1586, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 34390 / out 574, cost 0.0015 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 44801, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 15276, cost 0.0006 USD, phases: path_judge 0.5 s, classifier 0.4 s, doc_check 0.6 s, context 9.1 s, model_review 409.3 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main 568ae2b).<br> Reviewed head `ce177018c8110d7a10c587c49ca05bb3f8a60187` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0. </sub> ### Diagrams Drawn by a model from the diff with [PR Lens](https://github.com/coldteadotai/pr-lens): a view of what the PR touches, not a statement of what it does. ![pr-lens-containers.svg](https://git.javagil.de/attachments/89519fd7-cf60-44d0-9d89-5ad07684d4d3) ![pr-lens-cost-flow.svg](https://git.javagil.de/attachments/6efd4e47-f8d3-4def-8fa3-28247a60627b) </details>
ai-reviewer left a comment
Collaborator

werkrevisor: ORANGE / FINDINGS for ce177018c8110d7a10c587c49ca05bb3f8a60187.
The result comment of werkrevisor holds the details.

werkrevisor: ORANGE / FINDINGS for `ce177018c8110d7a10c587c49ca05bb3f8a60187`. The result comment of werkrevisor holds the details. <!-- werkrevisor-review:ce177018c8110d7a10c587c49ca05bb3f8a60187 -->
@@ -200,3 +203,3 @@
// an unknown review cost stays unknown.
costUsd =
body.number("cost_usd")?.let {
body.number("cost_usd")?.takeUnless { diagramUnpriced }?.let {
Collaborator

[medium] Unbounded Total (werkrevisor)

ORANGE; Jev: real 0.74, severity 3.0 of 3

diagram_cost_usd is checked only against the per-value maximum and is omitted from the boundedSum enforcing the service's total LLM-cost limit. For example, a record with cost_usd=90, excerpt_cost_usd=5, doc_summary_cost_usd=5, and diagram_cost_usd=50 is accepted even though the statistics report a $150 LLM cost, violating the stated $100 total-cost invariant. Include the known diagram cost in the same bounded sum so over-limit records are rejected consistently.

**[medium] Unbounded Total** (werkrevisor) ORANGE; Jev: real 0.74, severity 3.0 of 3 `diagram_cost_usd` is checked only against the per-value maximum and is omitted from the `boundedSum` enforcing the service's total LLM-cost limit. For example, a record with `cost_usd=90`, `excerpt_cost_usd=5`, `doc_summary_cost_usd=5`, and `diagram_cost_usd=50` is accepted even though the statistics report a $150 LLM cost, violating the stated $100 total-cost invariant. Include the known diagram cost in the same bounded sum so over-limit records are rejected consistently.
Author
Collaborator

Fixed in 29885bf: the bound of the whole LLM cost of a review includes diagram_cost_usd; a record above it is refused. Test: Scenario#140.03 in UsageApiIntegrationTest.

Fixed in `29885bf`: the bound of the whole LLM cost of a review includes `diagram_cost_usd`; a record above it is refused. Test: Scenario#140.03 in `UsageApiIntegrationTest`.
claude-code marked this conversation as resolved
ai-reviewer left a comment
Collaborator

Codex review of ce177018c8110d7a10c587c49ca05bb3f8a60187

Codex review of `ce177018c8110d7a10c587c49ca05bb3f8a60187` <!-- werkrevisor-codex -->
@@ -242,0 +243,4 @@
// A drawing counts for the model that drew it, which need not be the review model.
val own = Sums.of(recordsByModel[model].orEmpty(), billedByModel[model].orEmpty()) { 0.0 }
val drawn = records.filter { it.costKnown && it.diagramModel == model }.sumOf { it.diagramCostUsd ?: 0.0 }
Group(model, own.copy(llmCostUsd = own.llmCostUsd + drawn))
Collaborator

[low] Move local diagram costs with their model allocation (Codex review)

The model grouping reallocates diagram spending in llmCostUsd, but localCostUsd still includes the entire local run under its review model. With one $0.10 CI review and one $0.10 local review whose diagrams cost $0.30 on another model, the review model reports costPerPullRequestUsd as -$0.20 instead of $0.10. Allocate the local diagram cost to the drawing model as well.

Fix it, or reply why it does not hold, and resolve this comment.

**[low] Move local diagram costs with their model allocation** (Codex review) The model grouping reallocates diagram spending in llmCostUsd, but localCostUsd still includes the entire local run under its review model. With one $0.10 CI review and one $0.10 local review whose diagrams cost $0.30 on another model, the review model reports costPerPullRequestUsd as -$0.20 instead of $0.10. Allocate the local diagram cost to the drawing model as well. Fix it, or reply why it does not hold, and resolve this comment. <!-- werkrevisor-codex -->
Author
Collaborator

Fixed in 29885bf: localCostUsd by model moves a local drawing to the diagram model, like llmCostUsd; the review model keeps a positive cost per PR. Test: "a local drawing counts for the diagram model also in the local cost"; it fails without the fix.

Fixed in `29885bf`: `localCostUsd` by model moves a local drawing to the diagram model, like `llmCostUsd`; the review model keeps a positive cost per PR. Test: "a local drawing counts for the diagram model also in the local cost"; it fails without the fix.
claude-code marked this conversation as resolved
Collaborator

Codex review: not critical

The change adds diagram-cost accounting and reporting with additive database columns. It does not change authentication, authorization, merge gates, or irreversibly modify existing records.

Security effect: none; fails closed: no; a test of the closed case: no.

Findings

  1. [low] Move local diagram costs with their model allocation at statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/Statistics.kt:246, as a line comment

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

Model `gpt-6-astra`, effort high, tokens in 400431 (cached 336896) / out 1639, 51.8 s, 9 commands; reviewed head `ce177018c8110d7a10c587c49ca05bb3f8a60187`.
<!-- werkrevisor-codex:ce177018c8110d7a10c587c49ca05bb3f8a60187 --> <!-- werkrevisor-codex-critical:false --> <!-- werkrevisor-codex-unattached:0 --> <!-- werkrevisor-codex --> ## Codex review: not critical The change adds diagram-cost accounting and reporting with additive database columns. It does not change authentication, authorization, merge gates, or irreversibly modify existing records. Security effect: none; fails closed: no; a test of the closed case: no. ### Findings 1. **[low] Move local diagram costs with their model allocation** at `statistics/src/main/kotlin/de/javagil/werkrevisor/statistics/Statistics.kt:246`, as a line comment The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. <sub> Model `gpt-6-astra`, effort high, tokens in 400431 (cached 336896) / out 1639, 51.8 s, 9 commands; reviewed head `ce177018c8110d7a10c587c49ca05bb3f8a60187`. </sub>
mi approved these changes 2026-10-24 23:52:56 +02:00
Dismissed
Collaborator
🟡 werkrevisor YELLOW for 29885bf, superseded by the result for 5db1bdd

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Gate: the approval of mi came before this result; review requested again, please approve again after reading it; merge already scheduled.

Change kind: mixed.

Reasons

  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test
  • hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02)
  • hint 14 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint 1 finding(s) of low severity or rated minor by Jev: statistics/src/test/kotlin/de/javagil/werkrevisor/statistics/UsageApiIntegrationTest.kt:330
  • hint model recommends changes before merge

Findings

  1. [medium] Failing Test at statistics/src/test/kotlin/de/javagil/werkrevisor/statistics/UsageApiIntegrationTest.kt:330, hint, as a line comment

Classifier

8 of 8 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 71607, calls 1, tokens in 69239 / out 60898 (reasoning included in out), duration 477.6 s, cost 0.0938 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6699, cost 0.0003 USD, finding judge `jev-1.14.0` calls 1, tokens in 1975, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 26984 / out 376, cost 0.0005 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 45960, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 16057, cost 0.0007 USD, phases: path_judge 0.5 s, classifier 0.3 s, doc_check 0.6 s, context 11.7 s, model_review 463.5 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main 568ae2b).
Reviewed head `29885bf8e6b99c870b1aee4f2a6795f91461a9a6` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.

Diagrams

Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.

pr-lens-view-containers.svg

pr-lens-view-flow.svg

<!-- werkrevisor:29885bf8e6b99c870b1aee4f2a6795f91461a9a6 --> <!-- werkrevisor-verdict:YELLOW --> <!-- werkrevisor-blocks:23c8393df3cd2e42333e944054898332,59169b5c861973ea76a1fec1b4f64c0c,d21e9a7256a79225097563dbbbe8a1c0,279ceb5c617659daa18bd3a576d54d9b,bde4d00d50992e84778f1ef8ae10d041,1d81fca8c42dbc68a8d130de0a42144b,580a19b6ff4d32c9426dd7d0dfd19729,60393956e8f584722626331c9ae9676c,d1806eb9f3dde5308e3a18e59252d25d,3a37ef3b899ac8f267040afbbed14b33,ce3e07d4252b5a91cc4897247b813b73,7585a24557fcbcb36edc4013f179f3da,502b799bb6e02922b19708f5d0cd86a2,8f4fbac0747049bf6f2cf0ad2c1daa85,b9d42ea28e9688f3a38b63b1956e7c58,2bf190bb49bb6881ebfb79787ba10c72,4e327c2ee94154ea154f014a05f02cfd,4f379d72be0a68e0bca5612023ebfb06,dcf9a0459aff8360687e2e3fed1c16d2,10948ffe15e063a39e1d399e6a648b52,9436e50cdc1738a2ce15e89588c9f06a,e61bf47cd5b0b14b9ab407bd099d455f,61c25e5903b730afc0a09632bef57ea4,e102569b5cc212a4c24667cb87c882f1,9c5fe7e4c38bdc08691e292081d93ce7,0dee2ccc6cb7f9df3b332adcba209d6c,97ae6d34524bf5ac1bcaae7cfe1abb0d,683904167c90880f9394e05f2239f937,2a052dccee75e378b39b4af8bfea94ba,ce44fe9624a95971d7e1fb8a5c819c03,58102414ac20acf40fa93d6ab385ab67,3684f01b5460861efd4d8e9254c8f7fe,b072325581e333dab96566adb6da6087,f9e2e3fb1f55296ff5420943cde9126a,847171037bae7a881992c05215d913c6,5ffb2d9443d0febb9533622afa206898,88e964d4ce7a2ef100b2784bd3e03776,b8f4341b68df8e27798c434282972232,67ff34b71195f89fdeba6730bc396d17,0d4b1f06bb63ae5eaa5ecdc9eee188db,0de1f38acc3b2dac8c4b35bbe70ec881,f1585989851248a2cdd3839f190db11a,879f39390c8504abe82a2b82f2334fbe,a6d371d72925492d60f8e4495736daf9,d8b5adf93f08f525491e9bfbe337be2f,e509a6d8194bebb81bbac964a333777e,6a1f5da4342608150cd22ff1dcd9e1e8,4f785e13ae89a19844e200ff7034dcde,db4387c2bd58b0e4d5ece61e9b055834,e69daf80e5ed7479eb36d589a06e6fc4,3a46395c910fbba407ca48bbd1c79c9b,d2ffa4118b6f1812521f1a38c0798580,cf76790835076baee94331ab15009e44,4b225314d4bf1d606bdd9218dd9fa145,8e822399fc5dcf5eb33b233ccbbe8215,7907f8d04ae5d6344fc672657cba211d --> <!-- werkrevisor:diagrams --> <!-- werkrevisor-folded --> <details><summary>🟡 werkrevisor YELLOW for <code>29885bf</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/140#issuecomment-8172">the result for <code>5db1bdd</code></a></summary> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Gate: the approval of mi came before this result; review requested again, please approve again after reading it; merge already scheduled. Change kind: `mixed`. ### Reasons - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.02, risk 0.00, security 0.02) - **hint** 14 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 finding(s) of low severity or rated minor by Jev: statistics/src/test/kotlin/de/javagil/werkrevisor/statistics/UsageApiIntegrationTest.kt:330 - **hint** model recommends changes before merge ### Findings 1. **[medium] Failing Test** at `statistics/src/test/kotlin/de/javagil/werkrevisor/statistics/UsageApiIntegrationTest.kt:330`, hint, as a line comment ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 71607, calls 1, tokens in 69239 / out 60898 (reasoning included in out), duration 477.6 s, cost 0.0938 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6699, cost 0.0003 USD, finding judge `jev-1.14.0` calls 1, tokens in 1975, cost 0.0001 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 1, tokens in 26984 / out 376, cost 0.0005 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 45960, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 16057, cost 0.0007 USD, phases: path_judge 0.5 s, classifier 0.3 s, doc_check 0.6 s, context 11.7 s, model_review 463.5 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main 568ae2b).<br> Reviewed head `29885bf8e6b99c870b1aee4f2a6795f91461a9a6` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0. </sub> ### Diagrams Drawn by a model from the diff with [PR Lens](https://github.com/coldteadotai/pr-lens): a view of what the PR touches, not a statement of what it does. ![pr-lens-view-containers.svg](https://git.javagil.de/attachments/dd4ff8b4-891e-4c3f-a1ca-8139f3c61438) ![pr-lens-view-flow.svg](https://git.javagil.de/attachments/2f7eccb2-3b53-4e84-8835-576dbac22ccd) </details>
ai-reviewer left a comment
Collaborator

werkrevisor: YELLOW / HUMAN_REVIEW for 29885bf8e6b99c870b1aee4f2a6795f91461a9a6.
The result comment of werkrevisor holds the details.

werkrevisor: YELLOW / HUMAN_REVIEW for `29885bf8e6b99c870b1aee4f2a6795f91461a9a6`. The result comment of werkrevisor holds the details. <!-- werkrevisor-review:29885bf8e6b99c870b1aee4f2a6795f91461a9a6 -->
@@ -287,0 +327,4 @@
),
).andExpect(status().isCreated)
val byModel = stats("reader")["details"]["byModel"].associate { it["key"].asString() to it["sums"] }
byModel["openrouter/anthropic/claude-sonnet-5.5"]!!["costPerPullRequestUsd"].asDouble() shouldBe (0.1005 plusOrMinus 1e-9)
Collaborator

[medium] Failing Test (werkrevisor)

hint; Jev: real 0.74, severity 2.1 of 3

This assertion cannot pass with the current byModel implementation. Only the Sonnet review record contributes a pull-request denominator, while the Opus group contains only diagramModel entries; its pullRequests count is therefore zero and costPerPullRequestUsd is 0.0, not 0.1005. The new integration test fails on every run. Check the Sonnet or global metric here, or define the intended denominator for diagram-only model groups before merging.

**[medium] Failing Test** (werkrevisor) hint; Jev: real 0.74, severity 2.1 of 3 This assertion cannot pass with the current `byModel` implementation. Only the Sonnet review record contributes a pull-request denominator, while the Opus group contains only `diagramModel` entries; its `pullRequests` count is therefore zero and `costPerPullRequestUsd` is `0.0`, not `0.1005`. The new integration test fails on every run. Check the Sonnet or global metric here, or define the intended denominator for diagram-only model groups before merging.
Author
Collaborator

Does not hold: the assertion reads the group openrouter/anthropic/claude-sonnet-5.5, the review model, not the Opus group. Its one published review counts as the pull request; (0.1 + 0.02 + 2 × 0.0005 − (0.02 + 0.0005)) / 1 = 0.1005. The test passes: the build werkator/werkrevisor-statistics of 29885bf is green, and locally all 32 tests of UsageApiIntegrationTest pass. The Opus group is checked only for its localCostUsd.

Does not hold: the assertion reads the group `openrouter/anthropic/claude-sonnet-5.5`, the review model, not the Opus group. Its one published review counts as the pull request; (0.1 + 0.02 + 2 × 0.0005 − (0.02 + 0.0005)) / 1 = 0.1005. The test passes: the build `werkator/werkrevisor-statistics` of `29885bf` is green, and locally all 32 tests of `UsageApiIntegrationTest` pass. The Opus group is checked only for its `localCostUsd`.
claude-code marked this conversation as resolved
ai-reviewer requested review from mi 2026-10-24 23:58:06 +02:00
Collaborator

Codex review: not critical

The change adds cost accounting, presentation, and additive database columns. It does not change authorization, credential scopes, merge gates, or destructively modify existing records.

Security effect: hardens; fails closed: no; a test of the closed case: yes.

No findings.

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

Model `gpt-6-astra`, effort high, tokens in 291471 (cached 245632) / out 2253, 71.1 s, 11 commands; reviewed head `29885bf8e6b99c870b1aee4f2a6795f91461a9a6`.
<!-- werkrevisor-codex:29885bf8e6b99c870b1aee4f2a6795f91461a9a6 --> <!-- werkrevisor-codex-critical:false --> <!-- werkrevisor-codex-unattached:0 --> <!-- werkrevisor-codex --> ## Codex review: not critical The change adds cost accounting, presentation, and additive database columns. It does not change authorization, credential scopes, merge gates, or destructively modify existing records. Security effect: hardens; fails closed: no; a test of the closed case: yes. No findings. The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. <sub> Model `gpt-6-astra`, effort high, tokens in 291471 (cached 245632) / out 2253, 71.1 s, 11 commands; reviewed head `29885bf8e6b99c870b1aee4f2a6795f91461a9a6`. </sub>
claude-code changed title from werkrevisor 0.73.0: the cost of the PR Lens diagrams in the usage record, the result comment and the statistics to werkrevisor 0.75.0: the cost of the PR Lens diagrams in the usage record, the result comment and the statistics 2026-10-25 10:37:21 +01:00
claude-code added 14 commits 2026-10-25 10:37:22 +01:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pr_lens.mjs reads the usage of each OpenRouter answer, also of a failed drawing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test of the release of a held merge moves to a simpler PR, as the human decided.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review findings of round 1 (werkrevisor and Codex).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review findings of round 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex finding of round 4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review hint of round 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review findings of round 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR document: version 0.75.0 after the rebase onto 0.74.1
werkator build succeeded after 00:54
werkator/werkrevisor-statistics build succeeded after 00:39
ai-review/assessment human review required; a human approval unlocks the merge; a human review request is pending; merge already scheduled
ai-review/gate build succeeded after 07:04
ai-review/codex done ✅: Codex review passed
5db1bddb9f
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude-code force-pushed claude-code/diagram-cost from 29885bf8e6 to 5db1bddb9f 2026-10-25 10:37:22 +01:00 Compare
claude-code dismissed mi's review 2026-10-25 10:37:22 +01:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

Collaborator

🟡 werkrevisor: YELLOW / HUMAN_REVIEW

Change kind: mixed.

Reasons

  • YELLOW tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test
  • hint 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev
  • hint 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02)
  • hint 14 changed file(s) agree with the PR document according to Jev
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py
  • hint 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py
  • hint model recommends merge with caution

Classifier

8 of 8 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.

Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 66834, calls 1, tokens in 64602 / out 43510 (reasoning included in out), duration 345.8 s, cost 0.0716 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6699, cost 0.0003 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 34390 / out 669, cost 0.0046 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 45960, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 16057, cost 0.0007 USD, phases: path_judge 0.7 s, classifier 0.4 s, doc_check 0.9 s, context 10.6 s, model_review 332.0 s, config: defaults + host + repository (.werkrevisor.toml at main 4f9319c).
Reviewed head `5db1bddb9f6922c3fd5734f0a249a4e9b8c9397c` against base `4f9319cefe72e806fa2a843515be7dfd97bd792f`, werkrevisor 0.74.1.

Diagrams

Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.

pr-lens-containers.svg

pr-lens-statistics-components.svg

pr-lens-cost-flow.svg

<!-- werkrevisor:5db1bddb9f6922c3fd5734f0a249a4e9b8c9397c --> <!-- werkrevisor-verdict:YELLOW --> <!-- werkrevisor-blocks:23c8393df3cd2e42333e944054898332,737f477708bfa2a21726383d8de07d1f,d21e9a7256a79225097563dbbbe8a1c0,279ceb5c617659daa18bd3a576d54d9b,bde4d00d50992e84778f1ef8ae10d041,1d81fca8c42dbc68a8d130de0a42144b,c83936de5780882ecfb95be43f4a5c4d,60393956e8f584722626331c9ae9676c,d1806eb9f3dde5308e3a18e59252d25d,3a37ef3b899ac8f267040afbbed14b33,ce3e07d4252b5a91cc4897247b813b73,7585a24557fcbcb36edc4013f179f3da,502b799bb6e02922b19708f5d0cd86a2,8f4fbac0747049bf6f2cf0ad2c1daa85,b9d42ea28e9688f3a38b63b1956e7c58,2bf190bb49bb6881ebfb79787ba10c72,4e327c2ee94154ea154f014a05f02cfd,4f379d72be0a68e0bca5612023ebfb06,dcf9a0459aff8360687e2e3fed1c16d2,10948ffe15e063a39e1d399e6a648b52,9436e50cdc1738a2ce15e89588c9f06a,e61bf47cd5b0b14b9ab407bd099d455f,61c25e5903b730afc0a09632bef57ea4,e102569b5cc212a4c24667cb87c882f1,9c5fe7e4c38bdc08691e292081d93ce7,0dee2ccc6cb7f9df3b332adcba209d6c,97ae6d34524bf5ac1bcaae7cfe1abb0d,683904167c90880f9394e05f2239f937,2a052dccee75e378b39b4af8bfea94ba,ce44fe9624a95971d7e1fb8a5c819c03,58102414ac20acf40fa93d6ab385ab67,ef3ee6d2aa6b462da0561296994f25b5,b072325581e333dab96566adb6da6087,f9e2e3fb1f55296ff5420943cde9126a,847171037bae7a881992c05215d913c6,5ffb2d9443d0febb9533622afa206898,88e964d4ce7a2ef100b2784bd3e03776,b8f4341b68df8e27798c434282972232,67ff34b71195f89fdeba6730bc396d17,0d4b1f06bb63ae5eaa5ecdc9eee188db,0de1f38acc3b2dac8c4b35bbe70ec881,f1585989851248a2cdd3839f190db11a,879f39390c8504abe82a2b82f2334fbe,a6d371d72925492d60f8e4495736daf9,d8b5adf93f08f525491e9bfbe337be2f,e509a6d8194bebb81bbac964a333777e,6a1f5da4342608150cd22ff1dcd9e1e8,4f785e13ae89a19844e200ff7034dcde,db4387c2bd58b0e4d5ece61e9b055834,e69daf80e5ed7479eb36d589a06e6fc4,3a46395c910fbba407ca48bbd1c79c9b,d2ffa4118b6f1812521f1a38c0798580,cf76790835076baee94331ab15009e44,4b225314d4bf1d606bdd9218dd9fa145,8e822399fc5dcf5eb33b233ccbbe8215,7907f8d04ae5d6344fc672657cba211d --> ## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW Change kind: `mixed`. ### Reasons - **YELLOW** tests lost or disabled: tests/test_pr_lens_script.py:66 disables a test - **hint** 11 changed file(s) that neither weaken the checks, touch security nor show a risk category of the repository config according to Jev - **hint** 1 build change(s) of low risk according to Jev: statistics/build.gradle.kts (weakens checks 0.03, risk 0.00, security 0.02) - **hint** 14 changed file(s) agree with the PR document according to Jev - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (each larger than [review] max_full_file_chars = 80000): werkrevisor/cli.py - **hint** 1 changed file was given to the AI as excerpt of the selected lines and 5 lines around each changed line, not in full (all full files together would exceed [review] max_full_files_chars = 160000): werkrevisor/report.py - **hint** model recommends merge with caution ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 2 answers ignored because of a confidence below [classifier] min_confidence. <sub> Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 66834, calls 1, tokens in 64602 / out 43510 (reasoning included in out), duration 345.8 s, cost 0.0716 USD (reported), classifier `jev-1.14.0` calls 8, tokens in 6699, cost 0.0003 USD, excerpt selection `openrouter/openai/gpt-6-luna` calls 2, tokens in 34390 / out 669, cost 0.0046 USD (reported), PR document check `jev-1.14.0` calls 14, tokens in 45960, cost 0.0019 USD, build file judge `jev-1.14.0` calls 12, tokens in 16057, cost 0.0007 USD, phases: path_judge 0.7 s, classifier 0.4 s, doc_check 0.9 s, context 10.6 s, model_review 332.0 s, config: defaults + host + repository (.werkrevisor.toml at main 4f9319c).<br> Reviewed head `5db1bddb9f6922c3fd5734f0a249a4e9b8c9397c` against base `4f9319cefe72e806fa2a843515be7dfd97bd792f`, werkrevisor 0.74.1. </sub> <!-- werkrevisor:diagrams --> ### Diagrams Drawn by a model from the diff with [PR Lens](https://github.com/coldteadotai/pr-lens): a view of what the PR touches, not a statement of what it does. ![pr-lens-containers.svg](https://git.javagil.de/attachments/1f01adb9-c231-4919-885d-d6027459ae34) ![pr-lens-statistics-components.svg](https://git.javagil.de/attachments/15b2c549-f2c0-49ac-8bb8-81c1da896178) ![pr-lens-cost-flow.svg](https://git.javagil.de/attachments/a53a8310-ecae-4693-8812-690ffa4a3947)
Collaborator

Codex review: not critical

The change adds diagram-cost accounting, presentation and database columns without modifying existing records, credentials, authorization or merge gates. Its likely failure modes are incorrect statistics or missing cost information.

Security effect: hardens; fails closed: no; a test of the closed case: yes.

No findings.

The merge waits for the status ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.

Model `gpt-6-astra`, effort high, tokens in 325091 (cached 258048) / out 2131, 102.1 s, 13 commands; reviewed head `5db1bddb9f6922c3fd5734f0a249a4e9b8c9397c`.
<!-- werkrevisor-codex:5db1bddb9f6922c3fd5734f0a249a4e9b8c9397c --> <!-- werkrevisor-codex-critical:false --> <!-- werkrevisor-codex-unattached:0 --> <!-- werkrevisor-codex --> ## Codex review: not critical The change adds diagram-cost accounting, presentation and database columns without modifying existing records, credentials, authorization or merge gates. Its likely failure modes are incorrect statistics or missing cost information. Security effect: hardens; fails closed: no; a test of the closed case: yes. No findings. The merge waits for the status `ai-review/codex`: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here. <sub> Model `gpt-6-astra`, effort high, tokens in 325091 (cached 258048) / out 2131, 102.1 s, 13 commands; reviewed head `5db1bddb9f6922c3fd5734f0a249a4e9b8c9397c`. </sub>
mi approved these changes 2026-10-25 11:20:34 +01:00
ai-reviewer merged commit 9131a08453 into main 2026-10-25 11:20:35 +01:00
ai-reviewer deleted branch claude-code/diagram-cost 2026-10-25 11:20:35 +01:00
Sign in to join this conversation.
No Reviewers
No labels
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mi/werkrevisor#140