werkrevisor 0.17.0: findings as line comments, earlier results folded #20

Merged

ai-reviewer merged 6 commits from claude-code/findings-as-line-comments into main 2026-10-06 08:18:52 +02:00

Conversation 5 Commits 6 Files Changed 10

+543 -50

claude-code commented 2026-10-06 08:04:58 +02:00

Collaborator

Copy Link

Copy Source

See docs/prs/2026-10-06-PR#20-findings-as-line-comments.md.

🤖 Generated with Claude Code

See docs/prs/2026-10-06-PR#20-findings-as-line-comments.md. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-code added 1 commit 2026-10-06 08:04:59 +02:00

PR#20 doc: findings become line comments, and earlier results fold away ... ade49c52a0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-code marked the pull request as ready for review 2026-10-06 08:09:03 +02:00

mi added 1 commit 2026-10-06 08:09:05 +02:00

werkrevisor 0.17.0: findings as line comments, earlier results folded ... fedf11470d

The bot review carries each finding with a line in a changed file as a line comment that a human can resolve.
A new result comment folds the bot's results for earlier commits into one line with a details block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mi added 1 commit 2026-10-06 08:09:14 +02:00

README: split a long paragraph of the approval mode ... c471a50e69

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented 2026-10-06 08:09:42 +02:00

Collaborator

Copy Link

Copy Source

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: feature_with_tests.

Reasons

• ORANGE 3 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:365-370, werkrevisor/cli.py:539-551, werkrevisor/cli.py:482-485
• YELLOW risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
• hint model recommends merge with caution

Findings

1. [high] Failed line-comment review is not retried for COMMENT events at werkrevisor/cli.py:365-370

   ORANGE; Jev: real 0.78, severity 3.0 of 3

   When Gitea refuses the line comments with HTTP 400 or 422 and the event is COMMENT, the code swallows the error and posts no review at all. That is acceptable. For APPROVED or REQUEST_CHANGES, the retry without comments runs, but result.attached stays empty, which is correct. However, the fallback create_review call is not wrapped. A failure there propagates and turns the run into ERROR after the first review attempt may already have partly succeeded. A 422 may also come from a self-approval or an already-submitted review, not from the line comments. Then the retry hides the real cause, and the 422 path silently drops the review for COMMENT. The condition e.status in (400, 422) is too broad to identify a line-position error.

2. [medium] Folding earlier results is skipped when the result comment already exists at werkrevisor/cli.py:539-551

   ORANGE; Jev: real 0.86, severity 2.8 of 3

   _fold_earlier runs only after a new comment is created in _post_comment. If the new comment is created but the fold edits fail (swallowed by except Exception: pass), no later run folds them, because the head's result is then found as existing and returns early. Failures are invisible, so earlier results stay open with no log. Also, pr.head_sha in body skips any comment that merely mentions the head SHA, not only its own result.

3. [medium] Review submitted before result comment, so its failure leaves partial state at werkrevisor/cli.py:482-485

   ORANGE; Jev: real 0.80, severity 3.0 of 3

   In the approval gate, _publish_review now runs before _post_comment. If the comment post fails after an APPROVED review was submitted, the run ends as ERROR, but the PR already carries the bot approval. The earlier order posted the comment first. A rerun then sees no existing comment, finds the existing approval, and does not undo it. With automerge, this can unlock a merge despite the ERROR status.

Classifier

7 of 7 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74297, calls 1, tokens in 73659 / out 864 (reasoning included in out), duration 14.5 s, cost 0.1560 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5802, cost 0.0002 USD, finding judge `jev-1.13.0` calls 3, tokens in 4896, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).
Reviewed head `c471a50e6936e6e3be06b99bca5ea88566214074` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

<!-- werkrevisor:c471a50e6936e6e3be06b99bca5ea88566214074 --> <!-- werkrevisor-verdict:ORANGE --> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `feature_with_tests`. ### Reasons - **ORANGE** 3 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:365-370, werkrevisor/cli.py:539-551, werkrevisor/cli.py:482-485 - **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py) - **hint** model recommends merge with caution ### Findings 1. **[high] Failed line-comment review is not retried for COMMENT events** at `werkrevisor/cli.py:365-370` ORANGE; Jev: real 0.78, severity 3.0 of 3 When Gitea refuses the line comments with HTTP 400 or 422 and the event is `COMMENT`, the code swallows the error and posts no review at all. That is acceptable. For `APPROVED` or `REQUEST_CHANGES`, the retry without comments runs, but `result.attached` stays empty, which is correct. However, the fallback `create_review` call is not wrapped. A failure there propagates and turns the run into ERROR after the first review attempt may already have partly succeeded. A 422 may also come from a self-approval or an already-submitted review, not from the line comments. Then the retry hides the real cause, and the 422 path silently drops the review for `COMMENT`. The condition `e.status in (400, 422)` is too broad to identify a line-position error. 2. **[medium] Folding earlier results is skipped when the result comment already exists** at `werkrevisor/cli.py:539-551` ORANGE; Jev: real 0.86, severity 2.8 of 3 `_fold_earlier` runs only after a new comment is created in `_post_comment`. If the new comment is created but the fold edits fail (swallowed by `except Exception: pass`), no later run folds them, because the head's result is then found as existing and returns early. Failures are invisible, so earlier results stay open with no log. Also, `pr.head_sha in body` skips any comment that merely mentions the head SHA, not only its own result. 3. **[medium] Review submitted before result comment, so its failure leaves partial state** at `werkrevisor/cli.py:482-485` ORANGE; Jev: real 0.80, severity 3.0 of 3 In the approval gate, `_publish_review` now runs before `_post_comment`. If the comment post fails after an `APPROVED` review was submitted, the run ends as ERROR, but the PR already carries the bot approval. The earlier order posted the comment first. A rerun then sees no existing comment, finds the existing approval, and does not undo it. With automerge, this can unlock a merge despite the ERROR status. ### Classifier 7 of 7 units passed all questions (1 PR document(s)). 2 answers ignored because of a confidence below [classifier] min_confidence. _{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74297, calls 1, tokens in 73659 / out 864 (reasoning included in out), duration 14.5 s, cost 0.1560 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5802, cost 0.0002 USD, finding judge `jev-1.13.0` calls 3, tokens in 4896, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).<br> Reviewed head `c471a50e6936e6e3be06b99bca5ea88566214074` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

ai-reviewer scheduled this pull request to auto merge when all checks succeed 2026-10-06 08:09:43 +02:00

ai-reviewer requested review from mi 2026-10-06 08:09:43 +02:00

mi added 1 commit 2026-10-06 08:14:13 +02:00

review: result comment before the bot review, narrower retry, fold on every publish; open line comments are reused ... 30e82b202e

Fixes the three ORANGE findings of the werkrevisor review on PR#20 and adds the reuse of unresolved line comments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented 2026-10-06 08:14:26 +02:00

Collaborator

Copy Link

Copy Source

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: feature_with_tests.

Reasons

• ORANGE 2 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:551-554, werkrevisor/cli.py:548-551
• YELLOW risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
• YELLOW 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (each larger than [review] max_full_file_chars = 40000): werkrevisor/cli.py
• hint 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:53-65
• hint model recommends merge with caution

Findings

1. [medium] Failed edit after refused line comments is not caught at werkrevisor/cli.py:551-554

   ORANGE; Jev: real 0.86, severity 2.9 of 3

   In _publish, gitea.edit_issue_comment(created["id"], body) runs after _publish_review returned True and is not in a try block. If the edit fails, the exception ends the run and the finding text is lost. The review already exists with no line comments, and the result comment still says "as a line comment" for findings that have none. A rerun does not repair it, because the result comment for this head already exists and fresh is false.

2. [medium] Rerun after a failed review gives no line comments at werkrevisor/cli.py:548-551

   ORANGE; Jev: real 0.86, severity 3.0 of 3

   The line comments are planned only when existing is None. If the result comment is posted but _publish_review then fails (e.g. HTTP 500), the next run finds the comment, so fresh is false. The review is then not submitted at all, and an ORANGE or GREEN PR gets no approval or line comments. The comment still says "as a line comment" for findings that were never attached. The old code retried the review on rerun. No test covers this path.

3. [low] Scenario table contradicts the code at docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:53-65

   hint

   Scenario#20.03 says YELLOW or ORANGE gives COMMENT and RED gives REQUEST_CHANGES. In _review_event, GREEN and RED only follow the verdict when yellow_gate == "approval"; otherwise the kind is COMMENT. The test covers this (gate="block"), but the document does not state it.

Classifier

8 of 8 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 64401, calls 1, tokens in 63686 / out 736 (reasoning included in out), duration 13.0 s, cost 0.1347 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6874, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 3239, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).
Reviewed head `30e82b202e13337f7a02c0cd5980603b46ae58a4` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

<!-- werkrevisor:30e82b202e13337f7a02c0cd5980603b46ae58a4 --> <!-- werkrevisor-verdict:ORANGE --> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `feature_with_tests`. ### Reasons - **ORANGE** 2 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:551-554, werkrevisor/cli.py:548-551 - **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py) - **YELLOW** 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (each larger than [review] max_full_file_chars = 40000): werkrevisor/cli.py - **hint** 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:53-65 - **hint** model recommends merge with caution ### Findings 1. **[medium] Failed edit after refused line comments is not caught** at `werkrevisor/cli.py:551-554` ORANGE; Jev: real 0.86, severity 2.9 of 3 In `_publish`, `gitea.edit_issue_comment(created["id"], body)` runs after `_publish_review` returned True and is not in a try block. If the edit fails, the exception ends the run and the finding text is lost. The review already exists with no line comments, and the result comment still says "as a line comment" for findings that have none. A rerun does not repair it, because the result comment for this head already exists and `fresh` is false. 2. **[medium] Rerun after a failed review gives no line comments** at `werkrevisor/cli.py:548-551` ORANGE; Jev: real 0.86, severity 3.0 of 3 The line comments are planned only when `existing is None`. If the result comment is posted but `_publish_review` then fails (e.g. HTTP 500), the next run finds the comment, so `fresh` is false. The review is then not submitted at all, and an ORANGE or GREEN PR gets no approval or line comments. The comment still says "as a line comment" for findings that were never attached. The old code retried the review on rerun. No test covers this path. 3. **[low] Scenario table contradicts the code** at `docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:53-65` hint Scenario#20.03 says YELLOW or ORANGE gives COMMENT and RED gives REQUEST_CHANGES. In `_review_event`, GREEN and RED only follow the verdict when `yellow_gate == "approval"`; otherwise the kind is COMMENT. The test covers this (`gate="block"`), but the document does not state it. ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 2 answers ignored because of a confidence below [classifier] min_confidence. _{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 64401, calls 1, tokens in 63686 / out 736 (reasoning included in out), duration 13.0 s, cost 0.1347 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6874, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 3239, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).<br> Reviewed head `30e82b202e13337f7a02c0cd5980603b46ae58a4` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

mi added 1 commit 2026-10-06 08:15:34 +02:00

review: a failed review moves the findings into the result comment, and a rerun submits the missing review ... c0b7619164

Fixes the two ORANGE findings and the hint of the werkrevisor review on PR#20.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented 2026-10-06 08:15:57 +02:00

Collaborator

Copy Link

Copy Source

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: feature_with_tests.

Reasons

• ORANGE 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:545-559
• YELLOW risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
• YELLOW 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (each larger than [review] max_full_file_chars = 40000): werkrevisor/cli.py
• hint 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:391-400, docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:111-113
• hint model recommends merge with caution

Findings

1. [high] Rerun after failed review loses line comments at werkrevisor/cli.py:545-559

   ORANGE; Jev: real 0.76, severity 2.9 of 3

   Line comments are planned only when existing is None (fresh). If create_review fails with a non-4xx error, _carry_findings moves the findings into the result comment, and the rerun submits the missing review with comments=[]. This is acceptable as a fallback, but a rerun after a refusal-free failure of an APPROVED or REQUEST_CHANGES review no longer posts line comments. The announced "as a line comment" text is replaced only if result.attached is non-empty and created has an id. On a rerun, created is the existing comment, which may already be edited. This path is only partly covered by tests.

2. [medium] Refusal fallback can leave stale announcements at werkrevisor/cli.py:391-400

   hint; Jev: real 0.78, severity 2.5 of 3

   In _publish_review, when Gitea refuses the line comments for an APPROVED or REQUEST_CHANGES event, the review is re-created without comments. If that second create_review raises, the exception goes to _publish, which calls _carry_findings. The line_comments_refused reason is not yet appended, so the comment will not explain the cause. No test covers this branch (a second refusal for a non-COMMENT event).

3. [low] PR document contradicts code on second refusal at docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:111-113

   hint

   The Solution section says a second refusal "ends the run with ERROR". In the code, a non-4xx failure or a second create_review exception propagates and is handled by the caller. For COMMENT events there is no second call at all. The statement is only correct for APPROVED and REQUEST_CHANGES, and the document does not say so.

Classifier

8 of 8 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 66007, calls 1, tokens in 65302 / out 809 (reasoning included in out), duration 13.0 s, cost 0.1387 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6932, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 3522, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).
Reviewed head `c0b761916428a25093fc3164d02d6364b0892ae1` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

<!-- werkrevisor:c0b761916428a25093fc3164d02d6364b0892ae1 --> <!-- werkrevisor-verdict:ORANGE --> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `feature_with_tests`. ### Reasons - **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:545-559 - **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py) - **YELLOW** 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (each larger than [review] max_full_file_chars = 40000): werkrevisor/cli.py - **hint** 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:391-400, docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:111-113 - **hint** model recommends merge with caution ### Findings 1. **[high] Rerun after failed review loses line comments** at `werkrevisor/cli.py:545-559` ORANGE; Jev: real 0.76, severity 2.9 of 3 Line comments are planned only when `existing is None` (`fresh`). If `create_review` fails with a non-4xx error, `_carry_findings` moves the findings into the result comment, and the rerun submits the missing review with `comments=[]`. This is acceptable as a fallback, but a rerun after a refusal-free failure of an `APPROVED` or `REQUEST_CHANGES` review no longer posts line comments. The announced "as a line comment" text is replaced only if `result.attached` is non-empty and `created` has an `id`. On a rerun, `created` is the existing comment, which may already be edited. This path is only partly covered by tests. 2. **[medium] Refusal fallback can leave stale announcements** at `werkrevisor/cli.py:391-400` hint; Jev: real 0.78, severity 2.5 of 3 In `_publish_review`, when Gitea refuses the line comments for an `APPROVED` or `REQUEST_CHANGES` event, the review is re-created without comments. If that second `create_review` raises, the exception goes to `_publish`, which calls `_carry_findings`. The `line_comments_refused` reason is not yet appended, so the comment will not explain the cause. No test covers this branch (a second refusal for a non-COMMENT event). 3. **[low] PR document contradicts code on second refusal** at `docs/prs/2026-10-06-PR#20-findings-as-line-comments.md:111-113` hint The Solution section says a second refusal "ends the run with ERROR". In the code, a non-4xx failure or a second `create_review` exception propagates and is handled by the caller. For `COMMENT` events there is no second call at all. The statement is only correct for `APPROVED` and `REQUEST_CHANGES`, and the document does not say so. ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 2 answers ignored because of a confidence below [classifier] min_confidence. _{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 66007, calls 1, tokens in 65302 / out 809 (reasoning included in out), duration 13.0 s, cost 0.1387 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6932, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 3522, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).<br> Reviewed head `c0b761916428a25093fc3164d02d6364b0892ae1` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

mi added 1 commit 2026-10-06 08:16:36 +02:00

review: the refusal reason is recorded before the second call, and PR#20 doc states the fallback of a rerun ... 2e737a5f74

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-reviewer commented 2026-10-06 08:16:55 +02:00

Collaborator

Copy Link

Copy Source

🟠 werkrevisor: ORANGE / FINDINGS

Change kind: feature_with_tests.

Reasons

• ORANGE 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:548-559
• YELLOW risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
• YELLOW 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (each larger than [review] max_full_file_chars = 40000): werkrevisor/cli.py
• YELLOW 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:593-607
• hint 1 finding(s) of low severity or rated minor by Jev: tests/fakes.py:165-183
• hint model recommends merge with caution

Findings

1. [medium] Rerun after failed review loses findings and the YELLOW review path is untested at werkrevisor/cli.py:548-559

   ORANGE; Jev: real 0.81, severity 3.0 of 3

   In _publish, comments is planned only when existing is None. If the first run posted the result comment with "as a line comment" and _publish_review then failed, _carry_findings edits the comment to carry the findings, but only if the edit succeeds. If that edit also fails (logged only to stderr), the comment still says "as a line comment" while no line comments exist, and a rerun reuses existing and never plans line comments. The findings are then lost from the PR. The logged fallback mitigates this only in CI logs.

2. [medium] Fold may fold the result for the current head's rerun of another comment or foreign PR text at werkrevisor/cli.py:593-607

   YELLOW; Jev: real 0.58, severity 2.7 of 3

   _fold_earlier folds every bot comment that has a werkrevisor marker line without the current head SHA. An ERROR comment (werkrevisor-error:<sha>) of the current head is kept, but if created is {} (an error comment already existed), new_url is empty and nothing folds. If a result for the current head is posted by a concurrent job after the first one, the earlier-head result is folded, which is fine. The edge case to check is a rerun on an older head after a newer one exists: it would fold the newer head's result as "superseded" by the older result, because is_result_for only compares against the running head.

3. [medium] Test weakened: old assertions removed or replaced at tests/fakes.py:165-183

   hint; Jev: real 0.50, severity 2.0 of 3

   test_orange_requests_human_review_without_bot_review previously asserted created_reviews == []; it now asserts a COMMENT review, which matches the new behavior. However, the old assertion that the review body links the comment (#issuecomment-1) was replaced by a fixed string check, and test_comment_shows_the_jev_values/test_hint_is_shown_as_hint no longer check the old comment layout. These are intentional changes, but nothing tests the yellow_gate = "block"/pass_with_review_request branch where a COMMENT review is published for YELLOW with findings. Only ORANGE with the approval gate and one block case are covered.

Classifier

8 of 8 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.

_{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 66182, calls 1, tokens in 65454 / out 947 (reasoning included in out), duration 14.6 s, cost 0.1404 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6989, cost 0.0003 USD, finding judge `jev-1.13.0` calls 3, tokens in 5217, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).
Reviewed head `2e737a5f740168906882e3355dacaee56bd6ac07` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

<!-- werkrevisor:2e737a5f740168906882e3355dacaee56bd6ac07 --> <!-- werkrevisor-verdict:ORANGE --> ## 🟠 werkrevisor: ORANGE / FINDINGS Change kind: `feature_with_tests`. ### Reasons - **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:548-559 - **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py) - **YELLOW** 1 changed file was given to the AI only as diff, not in full, so it could not check it against the rest of its file (each larger than [review] max_full_file_chars = 40000): werkrevisor/cli.py - **YELLOW** 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:593-607 - **hint** 1 finding(s) of low severity or rated minor by Jev: tests/fakes.py:165-183 - **hint** model recommends merge with caution ### Findings 1. **[medium] Rerun after failed review loses findings and the YELLOW review path is untested** at `werkrevisor/cli.py:548-559` ORANGE; Jev: real 0.81, severity 3.0 of 3 In `_publish`, `comments` is planned only when `existing is None`. If the first run posted the result comment with "as a line comment" and `_publish_review` then failed, `_carry_findings` edits the comment to carry the findings, but only if the edit succeeds. If that edit also fails (logged only to stderr), the comment still says "as a line comment" while no line comments exist, and a rerun reuses `existing` and never plans line comments. The findings are then lost from the PR. The logged fallback mitigates this only in CI logs. 2. **[medium] Fold may fold the result for the current head's rerun of another comment or foreign PR text** at `werkrevisor/cli.py:593-607` YELLOW; Jev: real 0.58, severity 2.7 of 3 `_fold_earlier` folds every bot comment that has a `werkrevisor` marker line without the current head SHA. An ERROR comment (`werkrevisor-error:<sha>`) of the current head is kept, but if `created` is `{}` (an error comment already existed), `new_url` is empty and nothing folds. If a result for the current head is posted by a concurrent job after the first one, the earlier-head result is folded, which is fine. The edge case to check is a rerun on an older head after a newer one exists: it would fold the newer head's result as "superseded" by the older result, because `is_result_for` only compares against the running head. 3. **[medium] Test weakened: old assertions removed or replaced** at `tests/fakes.py:165-183` hint; Jev: real 0.50, severity 2.0 of 3 `test_orange_requests_human_review_without_bot_review` previously asserted `created_reviews == []`; it now asserts a COMMENT review, which matches the new behavior. However, the old assertion that the review body links the comment (`#issuecomment-1`) was replaced by a fixed string check, and `test_comment_shows_the_jev_values`/`test_hint_is_shown_as_hint` no longer check the old comment layout. These are intentional changes, but nothing tests the `yellow_gate = "block"`/`pass_with_review_request` branch where a COMMENT review is published for YELLOW with findings. Only ORANGE with the `approval` gate and one `block` case are covered. ### Classifier 8 of 8 units passed all questions (1 PR document(s)). 3 answers ignored because of a confidence below [classifier] min_confidence. _{Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 66182, calls 1, tokens in 65454 / out 947 (reasoning included in out), duration 14.6 s, cost 0.1404 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6989, cost 0.0003 USD, finding judge `jev-1.13.0` calls 3, tokens in 5217, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base 4c21614).<br> Reviewed head `2e737a5f740168906882e3355dacaee56bd6ac07` against base `4c216143257ecd75423ad01ee8e78e314a740a7a`, werkrevisor 0.16.1.}

mi approved these changes 2026-10-06 08:18:50 +02:00

mi left a comment

Owner

Copy Link

Copy Source

Die Edge-Cases ignorieren wir mal.

Die Edge-Cases ignorieren wir mal.

ai-reviewer merged commit 7aee874ec7 into main 2026-10-06 08:18:52 +02:00

ai-reviewer deleted branch claude-code/findings-as-line-comments 2026-10-06 08:18:52 +02:00

Sign in to join this conversation.

Reviewers

No Reviewers

mi

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

ai-reviewer claude-code mi

No Assignees

3 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mi/werkrevisor#20