YELLOW 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:207-215
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:460-473
hint model recommends merge with caution
Findings
[medium] Update note regex only matches a lowercase hex head SHA at werkrevisor/cli.py:207-215, YELLOW, as a line comment
[medium] Edited line comment may be re-edited every round after redaction or <!-- escaping at werkrevisor/cli.py:460-473, hint, as a line comment
Classifier
7 of 7 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 58753, calls 1, tokens in 57327 / out 527 (reasoning included in out), duration 16.1 s, cost 0.1199 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5573, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 3358, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `a0bffc1408b48421502a1b88110e3c342b77736e` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:a0bffc1408b48421502a1b88110e3c342b77736e -->
<!-- werkrevisor-verdict:YELLOW -->
<!-- werkrevisor-folded -->
<details><summary>🟡 werkrevisor YELLOW for <code>a0bffc1</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1409">the result for <code>704fd0b</code></a></summary>
## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind: `mixed`.
### Reasons
- **YELLOW** 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:207-215
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:460-473
- **hint** model recommends merge with caution
### Findings
1. **[medium] Update note regex only matches a lowercase hex head SHA** at `werkrevisor/cli.py:207-215`, YELLOW, as a line comment
2. **[medium] Edited line comment may be re-edited every round after redaction or `<!--` escaping** at `werkrevisor/cli.py:460-473`, hint, as a line comment
### Classifier
7 of 7 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 58753, calls 1, tokens in 57327 / out 527 (reasoning included in out), duration 16.1 s, cost 0.1199 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5573, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 3358, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `a0bffc1408b48421502a1b88110e3c342b77736e` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: YELLOW / HUMAN_REVIEW for a0bffc1408b48421502a1b88110e3c342b77736e.
The result comment of werkrevisor holds the details.
werkrevisor: YELLOW / HUMAN_REVIEW for `a0bffc1408b48421502a1b88110e3c342b77736e`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:a0bffc1408b48421502a1b88110e3c342b77736e -->
[medium] Update note regex only matches a lowercase hex head SHA (werkrevisor)
YELLOW; Jev: real 0.18, severity 2.6 of 3
_UPDATE_NOTE ends with $, which matches before a trailing newline but not before other trailing whitespace. If Gitea returns the body with trailing whitespace or CRLF line endings, the note is not stripped. The comparison then fails and the comment is edited again each round, and the note is stacked on a body that already holds one. This is uncertain because it depends on Gitea's body normalization.
**[medium] Update note regex only matches a lowercase hex head SHA** (werkrevisor)
YELLOW; Jev: real 0.18, severity 2.6 of 3
`_UPDATE_NOTE` ends with `$`, which matches before a trailing newline but not before other trailing whitespace. If Gitea returns the body with trailing whitespace or CRLF line endings, the note is not stripped. The comparison then fails and the comment is edited again each round, and the note is stacked on a body that already holds one. This is uncertain because it depends on Gitea's body normalization.
Fixed in 704fd0b: _UPDATE_NOTE accepts any blanks and CRLF around the note and upper-case hex; the comparison normalizes line endings. Stacking cannot happen anyway: an edit replaces the whole body by the new text with exactly one note.
Fixed in 704fd0b: `_UPDATE_NOTE` accepts any blanks and CRLF around the note and upper-case hex; the comparison normalizes line endings. Stacking cannot happen anyway: an edit replaces the whole body by the new text with exactly one note.
[medium] Edited line comment may be re-edited every round after redaction or <!-- escaping (werkrevisor)
hint; Jev: real 0.80, severity 2.0 of 3
The no-change check compares without_update_note(earlier["body"]) with redactor(text). Gitea may return a body that differs from what was posted, for example after normalization of line endings. In that case the comment is edited and marked as edited on every round, which the PR document says it avoids. The test fake stores the body verbatim, so this path is not covered. The effect is limited to extra edits, not data loss.
**[medium] Edited line comment may be re-edited every round after redaction or `<!--` escaping** (werkrevisor)
hint; Jev: real 0.80, severity 2.0 of 3
The no-change check compares `without_update_note(earlier["body"])` with `redactor(text)`. Gitea may return a body that differs from what was posted, for example after normalization of line endings. In that case the comment is edited and marked as edited on every round, which the PR document says it avoids. The test fake stores the body verbatim, so this path is not covered. The effect is limited to extra edits, not data loss.
Fixed in 704fd0b: the no-change check now ignores line endings and trailing blanks (report.same_line_comment_text), so a body that Gitea normalized is no change; the test test_a_reused_line_comment_with_the_same_text_stays_untouched covers a body with CRLF and trailing blanks (red without the fix). Redaction and the <!-- escaping are not affected: both are deterministic, the earlier comment was posted through the same redactor and line_comment.
Fixed in 704fd0b: the no-change check now ignores line endings and trailing blanks (`report.same_line_comment_text`), so a body that Gitea normalized is no change; the test `test_a_reused_line_comment_with_the_same_text_stays_untouched` covers a body with CRLF and trailing blanks (red without the fix). Redaction and the `<!--` escaping are not affected: both are deterministic, the earlier comment was posted through the same redactor and `line_comment`.
claude-code marked this conversation as resolved
ai-reviewer
scheduled this pull request to auto merge when all checks succeed 2026-10-06 15:49:32 +02:00
ai-reviewer
requested review from mi 2026-10-06 15:49:32 +02:00
YELLOW werkrevisor/cli.py:207-215 (CRLF/trailing whitespace defeats the note regex): fixed in 704fd0b, thread resolved.
hint werkrevisor/cli.py:460-473 (re-edit every round after normalization): fixed in 704fd0b, thread resolved.
hint build change server/build.gradle.kts: only the version bump to 0.20.2.
hint merge with caution: no further action.
Answer to the review of `a0bffc1` (YELLOW):
- YELLOW `werkrevisor/cli.py:207-215` (CRLF/trailing whitespace defeats the note regex): fixed in 704fd0b, thread resolved.
- hint `werkrevisor/cli.py:460-473` (re-edit every round after normalization): fixed in 704fd0b, thread resolved.
- hint build change `server/build.gradle.kts`: only the version bump to 0.20.2.
- hint merge with caution: no further action.
6 of 7 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 59279, calls 1, tokens in 57864 / out 758 (reasoning included in out), duration 13.5 s, cost 0.1233 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5593, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 3208, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `704fd0bbc0531f5d7274bf18c0a5796e58533107` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:704fd0bbc0531f5d7274bf18c0a5796e58533107 -->
<!-- werkrevisor-verdict:YELLOW -->
<!-- werkrevisor-folded -->
<details><summary>🟡 werkrevisor YELLOW for <code>704fd0b</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1417">the result for <code>c2043de</code></a></summary>
## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind: `mixed`.
### Reasons
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.01 given_when_then_misused (0.63 > yellow_above = 0.6)
- **YELLOW** 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:460-473
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:715-716, werkrevisor/report.py:207-218
- **hint** model recommends merge with caution
### Findings
1. **[medium] Edit via issue-comment endpoint for line comments** at `werkrevisor/cli.py:460-473`, YELLOW, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1406)
2. **[medium] Line comment is edited before the result comment is posted** at `werkrevisor/cli.py:715-716`, hint, as a line comment
3. **[low] Update note regex is not anchored to bot-written text** at `werkrevisor/report.py:207-218`, hint, as a line comment
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.01 | given_when_then_misused | 0.63 | – | YELLOW |
6 of 7 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 59279, calls 1, tokens in 57864 / out 758 (reasoning included in out), duration 13.5 s, cost 0.1233 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5593, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 3208, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `704fd0bbc0531f5d7274bf18c0a5796e58533107` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: YELLOW / HUMAN_REVIEW for 704fd0bbc0531f5d7274bf18c0a5796e58533107.
The result comment of werkrevisor holds the details.
werkrevisor: YELLOW / HUMAN_REVIEW for `704fd0bbc0531f5d7274bf18c0a5796e58533107`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:704fd0bbc0531f5d7274bf18c0a5796e58533107 -->
[medium] Line comment is edited before the result comment is posted (werkrevisor)
hint; Jev: real 0.71, severity 2.1 of 3
_plan_line_comments now writes to Gitea (edits earlier line comments) before _post_comment runs. If the later steps fail, for example the result comment post or a superseded head, the earlier line comment already holds text from a review of a head that never got a result. The old behaviour had no write in the planning phase, so the "result comment comes first" docstring of _publish no longer holds for these edits. The impact is limited to wording and the update note naming the head.
**[medium] Line comment is edited before the result comment is posted** (werkrevisor)
hint; Jev: real 0.71, severity 2.1 of 3
`_plan_line_comments` now writes to Gitea (edits earlier line comments) before `_post_comment` runs. If the later steps fail, for example the result comment post or a superseded head, the earlier line comment already holds text from a review of a head that never got a result. The old behaviour had no write in the planning phase, so the "result comment comes first" docstring of `_publish` no longer holds for these edits. The impact is limited to wording and the update note naming the head.
Not changed, on purpose. The edit only brings the wording of the open finding up to date and names the head that wrote it; if the result comment fails afterwards, the next run of that head or of a later head writes the result and edits the text again. The docstring of _publish is about the approval, which still never stands without its result comment. Moving the edits behind the result comment would lose the fallback: a failed edit puts the new text into the result comment, so the result comment must know the outcome of the edit first. Left open for the human reviewer.
Not changed, on purpose. The edit only brings the wording of the open finding up to date and names the head that wrote it; if the result comment fails afterwards, the next run of that head or of a later head writes the result and edits the text again. The docstring of `_publish` is about the approval, which still never stands without its result comment. Moving the edits behind the result comment would lose the fallback: a failed edit puts the new text into the result comment, so the result comment must know the outcome of the edit first. Left open for the human reviewer.
[low] Update note regex is not anchored to bot-written text (werkrevisor)
hint
same_line_comment_text strips a trailing <sub>Updated by ...</sub> from the earlier body. This is harmless for bot comments, but a model finding ending with that literal text would also be stripped before comparison, so a real change would not be detected. This needs deliberate model output and has low impact.
**[low] Update note regex is not anchored to bot-written text** (werkrevisor)
hint
`same_line_comment_text` strips a trailing `<sub>Updated by ...</sub>` from the earlier body. This is harmless for bot comments, but a model finding ending with that literal text would also be stripped before comparison, so a real change would not be detected. This needs deliberate model output and has low impact.
Not changed: only a model text that ends with exactly this note line, with a hex SHA, is affected, and the only consequence is one skipped edit of wording; the finding itself stays open and listed. Left open for the human reviewer.
Not changed: only a model text that ends with exactly this note line, with a hex SHA, is affected, and the only consequence is one skipped edit of wording; the finding itself stays open and listed. Left open for the human reviewer.
YELLOW spec Scenario#33.01 given_when_then_misused: the When step held state; fixed in c2043de, the different wording is now a Given.
YELLOW werkrevisor/cli.py:460-473 (edit via the issue-comment endpoint): PATCH /issues/comments/{id} edits review comments too on Gitea 1.27; it was checked on git.javagil.de before this PR.
This finding also showed a real, older bug: werkrevisor 0.20.1 reused my reply 1406 as the "earlier line comment", because Gitea lists a reply among the comments of the review that started the thread; the reply also counted as an open hint, and with this PR the bot would have tried to edit it.
Fixed in c2043de: only comments written by the bot count, with Scenario#33.04 and red/green tests.
hint werkrevisor/cli.py:715-716 (edit before the result comment): not changed, reasons in the thread.
hint werkrevisor/report.py:207-218 (note regex not tied to bot text): not changed, reasons in the thread.
hint build change server/build.gradle.kts: only the version bump.
The reviews run with the deployed 0.20.1, so this PR still shows the old behaviour: the reused comment kept its text.
Answer to the review of `704fd0b` (YELLOW):
- YELLOW spec Scenario#33.01 `given_when_then_misused`: the When step held state; fixed in c2043de, the different wording is now a Given.
- YELLOW `werkrevisor/cli.py:460-473` (edit via the issue-comment endpoint): `PATCH /issues/comments/{id}` edits review comments too on Gitea 1.27; it was checked on git.javagil.de before this PR.
This finding also showed a real, older bug: werkrevisor 0.20.1 reused my reply 1406 as the "earlier line comment", because Gitea lists a reply among the comments of the review that started the thread; the reply also counted as an open hint, and with this PR the bot would have tried to edit it.
Fixed in c2043de: only comments written by the bot count, with Scenario#33.04 and red/green tests.
- hint `werkrevisor/cli.py:715-716` (edit before the result comment): not changed, reasons in the thread.
- hint `werkrevisor/report.py:207-218` (note regex not tied to bot text): not changed, reasons in the thread.
- hint build change `server/build.gradle.kts`: only the version bump.
The reviews run with the deployed 0.20.1, so this PR still shows the old behaviour: the reused comment kept its text.
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:426-430, docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:69-82
hint model recommends merge with caution
Findings
[medium] Redacted text compared against the unredacted body, and the held-hint count includes reused comments at werkrevisor/cli.py:426-430, hint, as a line comment
[low] Scenario 33.04 names a test file that does not contain one of its tests at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:69-82, hint, as a line comment
7 of 8 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 65440, calls 1, tokens in 63836 / out 678 (reasoning included in out), duration 16.0 s, cost 0.1345 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6647, cost 0.0003 USD, finding judge `jev-1.13.0` calls 1, tokens in 1688, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `c2043deb92eb15ed8d59ff8607a1366b064a340b` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:c2043deb92eb15ed8d59ff8607a1366b064a340b -->
<!-- werkrevisor-verdict:YELLOW -->
<!-- werkrevisor-folded -->
<details><summary>🟡 werkrevisor YELLOW for <code>c2043de</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1425">the result for <code>f572e65</code></a></summary>
## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind: `mixed`.
### Reasons
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.01 given_when_then_misused (0.71 > yellow_above = 0.6)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:426-430, docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:69-82
- **hint** model recommends merge with caution
### Findings
1. **[medium] Redacted text compared against the unredacted body, and the held-hint count includes reused comments** at `werkrevisor/cli.py:426-430`, hint, as a line comment
2. **[low] Scenario 33.04 names a test file that does not contain one of its tests** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:69-82`, hint, as a line comment
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.01 | given_when_then_misused | 0.71 | – | YELLOW |
7 of 8 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 65440, calls 1, tokens in 63836 / out 678 (reasoning included in out), duration 16.0 s, cost 0.1345 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6647, cost 0.0003 USD, finding judge `jev-1.13.0` calls 1, tokens in 1688, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `c2043deb92eb15ed8d59ff8607a1366b064a340b` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: YELLOW / HUMAN_REVIEW for c2043deb92eb15ed8d59ff8607a1366b064a340b.
The result comment of werkrevisor holds the details.
werkrevisor: YELLOW / HUMAN_REVIEW for `c2043deb92eb15ed8d59ff8607a1366b064a340b`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:c2043deb92eb15ed8d59ff8607a1366b064a340b -->
[low] Scenario 33.04 names a test file that does not contain one of its tests (werkrevisor)
hint
Both entries under "Verified by" for Scenario#33.04 are listed, but the merge-hold test is in tests/test_hints_hold_merge.py, and the scenario's Given says the bot comment is "resolved" while only one test sets it up this way. The link text is consistent with the code. However, the scenario title claims a reply is "neither reused nor a hint", and the two tests each cover only one half. This is acceptable, but the Given/When/Then mixes both halves in a single scenario, which makes it hard to read.
**[low] Scenario 33.04 names a test file that does not contain one of its tests** (werkrevisor)
hint
Both entries under "Verified by" for Scenario#33.04 are listed, but the merge-hold test is in `tests/test_hints_hold_merge.py`, and the scenario's Given says the bot comment is "resolved" while only one test sets it up this way. The link text is consistent with the code. However, the scenario title claims a reply is "neither reused nor a hint", and the two tests each cover only one half. This is acceptable, but the Given/When/Then mixes both halves in a single scenario, which makes it hard to read.
[medium] Redacted text compared against the unredacted body, and the held-hint count includes reused comments (werkrevisor)
hint; Jev: real 0.51, severity 2.1 of 3
_update_line_comment redacts text before comparing it with earlier["body"]. Redacted text is also what gets posted, so a secret in the model text is stored as [REDACTED] and the comparison against the stored body stays consistent. This part is fine. A real problem is that a reply from the bot itself in the same thread (author equals bot_login) is still treated as a reusable line comment, because the filter checks only the author, not whether the comment starts a thread. A bot reply would then be edited and counted as a hint. This is an edge case, because the bot does not reply to threads in the code shown. The uncertainty is whether the bot ever writes replies.
**[medium] Redacted text compared against the unredacted body, and the held-hint count includes reused comments** (werkrevisor)
hint; Jev: real 0.51, severity 2.1 of 3
`_update_line_comment` redacts `text` before comparing it with `earlier["body"]`. Redacted text is also what gets posted, so a secret in the model text is stored as `[REDACTED]` and the comparison against the stored body stays consistent. This part is fine. A real problem is that a reply from the bot itself in the same thread (author equals `bot_login`) is still treated as a reusable line comment, because the filter checks only the author, not whether the comment starts a thread. A bot reply would then be edited and counted as a hint. This is an edge case, because the bot does not reply to threads in the code shown. The uncertainty is whether the bot ever writes replies.
Not changed: werkrevisor never replies in a thread; it posts line comments only as part of its own review. Should it ever reply, its reply would be bot text in an open thread, and reusing or counting it would be right. Left open for the human reviewer.
Not changed: werkrevisor never replies in a thread; it posts line comments only as part of its own review. Should it ever reply, its reply would be bot text in an open thread, and reusing or counting it would be right. Left open for the human reviewer.
YELLOW spec Scenario#33.01 given_when_then_misused: reworded in f572e65 with concrete texts: the Given holds the earlier comment, the When the new review, the Then the result.
hint werkrevisor/cli.py:426-430 (a bot reply would count): not changed, the bot never replies; reasons in the thread.
hint PR doc Scenario#33.04 mixes two halves: fixed in f572e65, split into #33.04 and #33.05, thread resolved.
hint build change server/build.gradle.kts: only the version bump.
Answer to the review of `c2043de` (YELLOW):
- YELLOW spec Scenario#33.01 `given_when_then_misused`: reworded in f572e65 with concrete texts: the Given holds the earlier comment, the When the new review, the Then the result.
- hint `werkrevisor/cli.py:426-430` (a bot reply would count): not changed, the bot never replies; reasons in the thread.
- hint PR doc Scenario#33.04 mixes two halves: fixed in f572e65, split into #33.04 and #33.05, thread resolved.
- hint build change `server/build.gradle.kts`: only the version bump.
Gate: approved by werkrevisor; merge waits for 10 open hint(s): reply to each line comment and resolve it, then restart the review or push a commit.
Change kind: mixed.
Reasons
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:453-454, werkrevisor/cli.py:462-475
hint model recommends merge with caution
Findings
[medium] Reused comment is edited while publishing can still fail at werkrevisor/cli.py:453-454, hint, as a line comment
[medium] Line comment body with a redacted text and the update note differ in comparison at werkrevisor/cli.py:462-475, hint, still open in an earlier line comment
Classifier
9 of 9 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 65479, calls 1, tokens in 63872 / out 637 (reasoning included in out), duration 12.9 s, cost 0.1341 USD (reported), classifier `jev-1.13.0` calls 9, tokens in 7526, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 3291, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `f572e656d1ae88e70a2b0d1a36a054371e42692e` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:f572e656d1ae88e70a2b0d1a36a054371e42692e -->
<!-- werkrevisor-verdict:GREEN -->
<!-- werkrevisor-folded -->
<details><summary>🟢 werkrevisor GREEN for <code>f572e65</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1435">the result for <code>dfd5f69</code></a></summary>
## 🟢 werkrevisor: GREEN / PASS
Gate: approved by werkrevisor; merge waits for 10 open hint(s): reply to each line comment and resolve it, then restart the review or push a commit.
Change kind: `mixed`.
### Reasons
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:453-454, werkrevisor/cli.py:462-475
- **hint** model recommends merge with caution
### Findings
1. **[medium] Reused comment is edited while publishing can still fail** at `werkrevisor/cli.py:453-454`, hint, as a line comment
2. **[medium] Line comment body with a redacted text and the update note differ in comparison** at `werkrevisor/cli.py:462-475`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1406)
### Classifier
9 of 9 units passed all questions (1 PR document(s)).
2 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 65479, calls 1, tokens in 63872 / out 637 (reasoning included in out), duration 12.9 s, cost 0.1341 USD (reported), classifier `jev-1.13.0` calls 9, tokens in 7526, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 3291, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `f572e656d1ae88e70a2b0d1a36a054371e42692e` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: GREEN / PASS for f572e656d1ae88e70a2b0d1a36a054371e42692e.
The result comment of werkrevisor holds the details.
werkrevisor: GREEN / PASS for `f572e656d1ae88e70a2b0d1a36a054371e42692e`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:f572e656d1ae88e70a2b0d1a36a054371e42692e -->
[medium] Reused comment is edited while publishing can still fail (werkrevisor)
hint; Jev: real 0.76, severity 2.4 of 3
_plan_line_comments edits the earlier line comment before the result comment is posted and before the review is submitted. If _post_comment raises afterwards, the run ends in ERROR but the earlier comment already carries the new text and the "Updated by" note. A rerun on the same head then finds an equal text, so it makes no edit and does not report the finding in the result comment. The side effect also breaks the docstring contract of _publish that the result comment comes first. The impact is limited: the edited line comment keeps the finding visible.
**[medium] Reused comment is edited while publishing can still fail** (werkrevisor)
hint; Jev: real 0.76, severity 2.4 of 3
`_plan_line_comments` edits the earlier line comment before the result comment is posted and before the review is submitted. If `_post_comment` raises afterwards, the run ends in ERROR but the earlier comment already carries the new text and the "Updated by" note. A rerun on the same head then finds an equal text, so it makes no edit and does not report the finding in the result comment. The side effect also breaks the docstring contract of `_publish` that the result comment comes first. The impact is limited: the edited line comment keeps the finding visible.
Not changed: the same point as the thread at werkrevisor/cli.py:715; the result comment must know the outcome of the edit to carry the text after a failed edit, and the next review writes the wording again. Left open for the human reviewer.
Not changed: the same point as the thread at `werkrevisor/cli.py:715`; the result comment must know the outcome of the edit to carry the text after a failed edit, and the next review writes the wording again. Left open for the human reviewer.
claude-code marked this conversation as resolved
ai-reviewer
canceled auto merging this pull request when all checks succeed 2026-10-06 16:09:00 +02:00
Answer to the review of f572e65 (GREEN, merge held for 10 open hints):
hint werkrevisor/cli.py:453-454 (edit while publishing can fail): not changed, as in the thread at cli.py:715.
hint werkrevisor/cli.py:462-475 (redacted text and update note in the comparison): its text is lost, since 0.20.1 reused my reply 1406 again, which is the bug this PR fixes. From the title: the comparison strips the note from the stored body and compares it with the redacted new text; the posted body is exactly the redacted text plus the note, so both sides match. Not changed.
6 of the 10 open hints are my replies, counted by the deployed 0.20.1; with 0.20.2 only the 4 open bot comments count.
Answer to the review of `f572e65` (GREEN, merge held for 10 open hints):
- hint `werkrevisor/cli.py:453-454` (edit while publishing can fail): not changed, as in the thread at `cli.py:715`.
- hint `werkrevisor/cli.py:462-475` (redacted text and update note in the comparison): its text is lost, since 0.20.1 reused my reply 1406 again, which is the bug this PR fixes. From the title: the comparison strips the note from the stored body and compares it with the redacted new text; the posted body is exactly the redacted text plus the note, so both sides match. Not changed.
- 6 of the 10 open hints are my replies, counted by the deployed 0.20.1; with 0.20.2 only the 4 open bot comments count.
ai-reviewer
scheduled this pull request to auto merge when all checks succeed 2026-10-06 16:21:41 +02:00
ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; a human review request is pending; merge already scheduled
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:557-561, docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:95-103
hint model recommends merge with caution
Findings
[medium] Withdrawal of review requests has no test with the automerge flag off or a matching case at werkrevisor/cli.py:530-533, ORANGE, as a line comment
[medium] Held-merge hint count ignores finding text carried in the result comment at werkrevisor/cli.py:557-561, hint, as a line comment
[low] Scenario#33.06 misuses Given/When/Then at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:95-103, hint, as a line comment
9 of 10 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 73240, calls 1, tokens in 71417 / out 749 (reasoning included in out), duration 14.0 s, cost 0.1503 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8574, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3425, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `dfd5f6972d230c671e518537f57f7575dab88251` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:dfd5f6972d230c671e518537f57f7575dab88251 -->
<!-- werkrevisor-verdict:ORANGE -->
<!-- werkrevisor-folded -->
<details><summary>🟠 werkrevisor ORANGE for <code>dfd5f69</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1445">the result for <code>08ecc13</code></a></summary>
## 🟠 werkrevisor: ORANGE / FINDINGS
Change kind: `mixed`.
### Reasons
- **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:530-533
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.06 given_when_then_misused (0.62 > yellow_above = 0.6)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 2 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:557-561, docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:95-103
- **hint** model recommends merge with caution
### Findings
1. **[medium] Withdrawal of review requests has no test with the automerge flag off or a matching case** at `werkrevisor/cli.py:530-533`, ORANGE, as a line comment
2. **[medium] Held-merge hint count ignores finding text carried in the result comment** at `werkrevisor/cli.py:557-561`, hint, as a line comment
3. **[low] Scenario#33.06 misuses Given/When/Then** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:95-103`, hint, as a line comment
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.06 | given_when_then_misused | 0.62 | – | YELLOW |
9 of 10 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 73240, calls 1, tokens in 71417 / out 749 (reasoning included in out), duration 14.0 s, cost 0.1503 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8574, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3425, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `dfd5f6972d230c671e518537f57f7575dab88251` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: ORANGE / FINDINGS for dfd5f6972d230c671e518537f57f7575dab88251.
The result comment of werkrevisor holds the details.
werkrevisor: ORANGE / FINDINGS for `dfd5f6972d230c671e518537f57f7575dab88251`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:dfd5f6972d230c671e518537f57f7575dab88251 -->
The scenario has two "Given" lines: an automerge setting and a historic YELLOW round. The "So that" line also states a mechanism rather than a goal. The classifier flagged this scenario. Rephrase it so each step has one clear role.
**[low] Scenario#33.06 misuses Given/When/Then** (werkrevisor)
hint
The scenario has two "Given" lines: an automerge setting and a historic YELLOW round. The "So that" line also states a mechanism rather than a goal. The classifier flagged this scenario. Rephrase it so each step has one clear role.
[medium] Withdrawal of review requests has no test with the automerge flag off or a matching case (werkrevisor)
ORANGE; Jev: real 0.87, severity 3.0 of 3
_withdraw_review_request withdraws requests when the merge is scheduled, but the withdrawal runs even when _schedule_merge returned "merge not scheduled" (for example when branch protection is missing or the merge call failed). The review request is then withdrawn without a scheduled merge, which drops the human's pending review and leaves the PR unlocked by neither path. Withdraw only when the scheduling note starts with "; merge scheduled" or "; merge already scheduled".
**[medium] Withdrawal of review requests has no test with the automerge flag off or a matching case** (werkrevisor)
ORANGE; Jev: real 0.87, severity 3.0 of 3
`_withdraw_review_request` withdraws requests when the merge is scheduled, but the withdrawal runs even when `_schedule_merge` returned "merge not scheduled" (for example when branch protection is missing or the merge call failed). The review request is then withdrawn without a scheduled merge, which drops the human's pending review and leaves the PR unlocked by neither path. Withdraw only when the scheduling note starts with "; merge scheduled" or "; merge already scheduled".
Fixed in 08ecc13: the request is withdrawn only after merge scheduled or merge already scheduled; test_a_merge_not_scheduled_keeps_the_review_request covers a missing branch protection (red without the fix).
Fixed in 08ecc13: the request is withdrawn only after `merge scheduled` or `merge already scheduled`; `test_a_merge_not_scheduled_keeps_the_review_request` covers a missing branch protection (red without the fix).
[medium] Held-merge hint count ignores finding text carried in the result comment (werkrevisor)
hint; Jev: real 0.75, severity 2.3 of 3
_open_hints excludes every index in result.reused from in_comment_only. A reused finding whose edit failed (reused_not_updated) now has its full text only in the result comment, yet it counts as neither in-comment nor attached. It is still counted through the open line comment, so the count is not lost, but the gate note says "reply to each line comment" while the new text sits elsewhere. No test covers the hold together with a failed edit.
**[medium] Held-merge hint count ignores finding text carried in the result comment** (werkrevisor)
hint; Jev: real 0.75, severity 2.3 of 3
`_open_hints` excludes every index in `result.reused` from `in_comment_only`. A reused finding whose edit failed (`reused_not_updated`) now has its full text only in the result comment, yet it counts as neither in-comment nor attached. It is still counted through the open line comment, so the count is not lost, but the gate note says "reply to each line comment" while the new text sits elsewhere. No test covers the hold together with a failed edit.
Not changed: the count stays right, as the finding says, since the open line comment counts. The author still replies to that line comment and resolves it; the result comment only carries the newer wording. Left open for the human reviewer.
Not changed: the count stays right, as the finding says, since the open line comment counts. The author still replies to that line comment and resolves it; the result comment only carries the newer wording. Left open for the human reviewer.
ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; a human review request is pending; merge already scheduled
ORANGE werkrevisor/cli.py:530-533 (withdrawal without a scheduled merge): fixed in 08ecc13, thread resolved.
YELLOW risky path werkrevisor/gitea.py: one new call, withdraw_review_request (DELETE /pulls/{n}/requested_reviewers); needs the human review.
YELLOW spec Scenario#33.06 given_when_then_misused: reworded in 08ecc13, thread resolved.
hint werkrevisor/cli.py:557-561 (hint count and text in the result comment): not changed, reasons in the thread.
hint build change server/build.gradle.kts: only the version bump.
Answer to the review of `dfd5f69` (ORANGE):
- ORANGE `werkrevisor/cli.py:530-533` (withdrawal without a scheduled merge): fixed in 08ecc13, thread resolved.
- YELLOW risky path `werkrevisor/gitea.py`: one new call, `withdraw_review_request` (`DELETE /pulls/{n}/requested_reviewers`); needs the human review.
- YELLOW spec Scenario#33.06 `given_when_then_misused`: reworded in 08ecc13, thread resolved.
- hint `werkrevisor/cli.py:557-561` (hint count and text in the result comment): not changed, reasons in the thread.
- hint build change `server/build.gradle.kts`: only the version bump.
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:118-125
hint model recommends merge with caution
Findings
[medium] Review request withdrawn although the merge may not follow at werkrevisor/cli.py:532-536, ORANGE, still open in an earlier line comment
[low] Document names a reproduction detail instead of the change at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:118-125, hint, as a line comment
Classifier
10 of 10 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74368, calls 1, tokens in 72692 / out 597 (reasoning included in out), duration 19.6 s, cost 0.1514 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8522, cost 0.0004 USD, finding judge `jev-1.13.0` calls 1, tokens in 1761, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `08ecc13cbbc31f5c85f41ab55f222749c62a453a` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:08ecc13cbbc31f5c85f41ab55f222749c62a453a -->
<!-- werkrevisor-verdict:ORANGE -->
<!-- werkrevisor-folded -->
<details><summary>🟠 werkrevisor ORANGE for <code>08ecc13</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1456">the result for <code>3d0fbbc</code></a></summary>
## 🟠 werkrevisor: ORANGE / FINDINGS
Change kind: `mixed`.
### Reasons
- **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:532-536
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:118-125
- **hint** model recommends merge with caution
### Findings
1. **[medium] Review request withdrawn although the merge may not follow** at `werkrevisor/cli.py:532-536`, ORANGE, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1440)
2. **[low] Document names a reproduction detail instead of the change** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:118-125`, hint, as a line comment
### Classifier
10 of 10 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74368, calls 1, tokens in 72692 / out 597 (reasoning included in out), duration 19.6 s, cost 0.1514 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8522, cost 0.0004 USD, finding judge `jev-1.13.0` calls 1, tokens in 1761, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `08ecc13cbbc31f5c85f41ab55f222749c62a453a` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: ORANGE / FINDINGS for 08ecc13cbbc31f5c85f41ab55f222749c62a453a.
The result comment of werkrevisor holds the details.
werkrevisor: ORANGE / FINDINGS for `08ecc13cbbc31f5c85f41ab55f222749c62a453a`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:08ecc13cbbc31f5c85f41ab55f222749c62a453a -->
[low] Document names a reproduction detail instead of the change (werkrevisor)
hint
The Solution bullets narrate the history of this PR ("0.20.1 reused a reply of claude-code in this PR", "the request of the YELLOW rounds blocked the merge scheduled by the GREEN review of f572e65"). This is incident narration that does not describe the change and goes stale. Scenario#33.03 also covers only two causes in a table but gives one When line, which makes the scenario harder to read.
**[low] Document names a reproduction detail instead of the change** (werkrevisor)
hint
The Solution bullets narrate the history of this PR ("0.20.1 reused a reply of claude-code in this PR", "the request of the YELLOW rounds blocked the merge scheduled by the GREEN review of f572e65"). This is incident narration that does not describe the change and goes stale. Scenario#33.03 also covers only two causes in a table but gives one `When` line, which makes the scenario harder to read.
Fixed in 308b44e: the two defects found in the reviews of this PR moved from the solution into the problem. Scenario#33.03 keeps its table: both causes lead to the same outcome, which the template allows as data-driven scenario.
Fixed in 308b44e: the two defects found in the reviews of this PR moved from the solution into the problem. Scenario#33.03 keeps its table: both causes lead to the same outcome, which the template allows as data-driven scenario.
ORANGE werkrevisor/cli.py:532-536 (request withdrawn although the merge may not follow): its text is lost again, since 0.20.1 reused my reply 1440, the bug this PR fixes. From the title: not changed. The request is withdrawn only once Gitea accepted the scheduled merge. If a check fails later, the PR stays unmerged; the author pushes a fix, and a review that is not GREEN requests the human review again. A GREEN head needs no human review, since the approval of the bot is the lock.
YELLOW risky path werkrevisor/gitea.py: one new call, withdraw_review_request; needs the human review.
hint PR doc narrates incidents in the solution: fixed in 308b44e, thread resolved.
hint build change server/build.gradle.kts: only the version bump.
Answer to the review of `08ecc13` (ORANGE):
- ORANGE `werkrevisor/cli.py:532-536` (request withdrawn although the merge may not follow): its text is lost again, since 0.20.1 reused my reply 1440, the bug this PR fixes. From the title: not changed. The request is withdrawn only once Gitea accepted the scheduled merge. If a check fails later, the PR stays unmerged; the author pushes a fix, and a review that is not GREEN requests the human review again. A GREEN head needs no human review, since the approval of the bot is the lock.
- YELLOW risky path `werkrevisor/gitea.py`: one new call, `withdraw_review_request`; needs the human review.
- hint PR doc narrates incidents in the solution: fixed in 308b44e, thread resolved.
- hint build change `server/build.gradle.kts`: only the version bump.
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:464-477
hint model recommends merge with caution
Findings
[medium] Withdrawal only uses requested_reviewers of a stale PR snapshot at werkrevisor/cli.py:579-590
ORANGE; Jev: real 0.67, severity 3.0 of 3
_withdraw_review_request reads requested_reviewers from pr_data, which _assert_current fetched before the publish step. For an approval YELLOW round the review request may also exist only as a REQUEST_REVIEW review, which _review_request_pending treats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request in requested_reviewers.
[medium] Reuse of a later finding after an update failure is untested at tests/test_line_comments.py:464-477
hint; Jev: real 0.54, severity 2.2 of 3
The updated set marks a comment as updated before the edit is tried. If the edit fails for the first finding, a second finding at the same comment gets reused_not_updated through the updated check, which is correct. A failed first edit is never retried by the second finding, so only one of two failed findings gets the update attempt. The tests cover each case separately but not the combination.
Classifier
10 of 10 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74388, calls 1, tokens in 72715 / out 554 (reasoning included in out), duration 12.2 s, cost 0.1510 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8528, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 2292, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `308b44ee1d022a01f3dbcdaf7ae5174012d2ee26` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:308b44ee1d022a01f3dbcdaf7ae5174012d2ee26 -->
<!-- werkrevisor-verdict:ORANGE -->
<!-- werkrevisor-folded -->
<details><summary>🟠 werkrevisor ORANGE for <code>308b44e</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1456">the result for <code>3d0fbbc</code></a></summary>
## 🟠 werkrevisor: ORANGE / FINDINGS
Change kind: `mixed`.
### Reasons
- **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:579-590
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:464-477
- **hint** model recommends merge with caution
### Findings
1. **[medium] Withdrawal only uses `requested_reviewers` of a stale PR snapshot** at `werkrevisor/cli.py:579-590`
ORANGE; Jev: real 0.67, severity 3.0 of 3
`_withdraw_review_request` reads `requested_reviewers` from `pr_data`, which `_assert_current` fetched before the publish step. For an `approval` YELLOW round the review request may also exist only as a `REQUEST_REVIEW` review, which `_review_request_pending` treats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request in `requested_reviewers`.
2. **[medium] Reuse of a later finding after an update failure is untested** at `tests/test_line_comments.py:464-477`
hint; Jev: real 0.54, severity 2.2 of 3
The `updated` set marks a comment as updated before the edit is tried. If the edit fails for the first finding, a second finding at the same comment gets `reused_not_updated` through the `updated` check, which is correct. A failed first edit is never retried by the second finding, so only one of two failed findings gets the update attempt. The tests cover each case separately but not the combination.
### Classifier
10 of 10 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74388, calls 1, tokens in 72715 / out 554 (reasoning included in out), duration 12.2 s, cost 0.1510 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8528, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 2292, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `308b44ee1d022a01f3dbcdaf7ae5174012d2ee26` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:464-477
hint model recommends merge with caution
Findings
[medium] Withdrawal only uses requested_reviewers of a stale PR snapshot at werkrevisor/cli.py:579-590
ORANGE; Jev: real 0.67, severity 3.0 of 3
_withdraw_review_request reads requested_reviewers from pr_data, which _assert_current fetched before the publish step. For an approval YELLOW round the review request may also exist only as a REQUEST_REVIEW review, which _review_request_pending treats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request in requested_reviewers.
[medium] Reuse of a later finding after an update failure is untested at tests/test_line_comments.py:464-477
hint; Jev: real 0.54, severity 2.2 of 3
The updated set marks a comment as updated before the edit is tried. If the edit fails for the first finding, a second finding at the same comment gets reused_not_updated through the updated check, which is correct. A failed first edit is never retried by the second finding, so only one of two failed findings gets the update attempt. The tests cover each case separately but not the combination.
Classifier
10 of 10 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74388, calls 1, tokens in 72715 / out 554 (reasoning included in out), duration 12.2 s, cost 0.1510 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8528, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 2292, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `308b44ee1d022a01f3dbcdaf7ae5174012d2ee26` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor-error:308b44ee1d022a01f3dbcdaf7ae5174012d2ee26 -->
<!-- werkrevisor-folded -->
<details><summary>⚠️ werkrevisor ERROR for <code>308b44e</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1456">the result for <code>3d0fbbc</code></a></summary>
## ⚠️ werkrevisor: ERROR / review not completed, human action needed
Change kind: `mixed`.
### Reasons
- **ERROR** GiteaError: POST /repos/mi/werkrevisor/pulls/33/reviews failed with HTTP 500
- **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:579-590
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:464-477
- **hint** model recommends merge with caution
### Findings
1. **[medium] Withdrawal only uses `requested_reviewers` of a stale PR snapshot** at `werkrevisor/cli.py:579-590`
ORANGE; Jev: real 0.67, severity 3.0 of 3
`_withdraw_review_request` reads `requested_reviewers` from `pr_data`, which `_assert_current` fetched before the publish step. For an `approval` YELLOW round the review request may also exist only as a `REQUEST_REVIEW` review, which `_review_request_pending` treats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request in `requested_reviewers`.
2. **[medium] Reuse of a later finding after an update failure is untested** at `tests/test_line_comments.py:464-477`
hint; Jev: real 0.54, severity 2.2 of 3
The `updated` set marks a comment as updated before the edit is tried. If the edit fails for the first finding, a second finding at the same comment gets `reused_not_updated` through the `updated` check, which is correct. A failed first edit is never retried by the second finding, so only one of two failed findings gets the update attempt. The tests cover each case separately but not the combination.
### Classifier
10 of 10 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74388, calls 1, tokens in 72715 / out 554 (reasoning included in out), duration 12.2 s, cost 0.1510 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8528, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 2292, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `308b44ee1d022a01f3dbcdaf7ae5174012d2ee26` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
ERROR POST /pulls/33/reviews HTTP 500: the finding at tests/test_line_comments.py:464-477 lies beyond the end of the file (224 lines), and Gitea answers such a line comment with 500, which 0.20.1 does not treat as refusal. Fixed in 3d0fbbc: 500 counts like 400 and 422, Scenario#33.07, red/green.
ORANGE werkrevisor/cli.py:579-590 (withdrawal reads only requested_reviewers): fixed in 3d0fbbc, a request listed only as REQUEST_REVIEW review is withdrawn too, the same sources as _review_request_pending. A request added during the run can only come from this run, which adds none at GREEN.
hint tests/test_line_comments.py:464-477 (a failed first edit is not retried by a second finding): not changed; one edit per comment and round is intended, and the second finding's text goes into the result comment.
YELLOW risky path werkrevisor/gitea.py: needs the human review.
The findings of this round have no line comments, since the review failed; this comment answers them.
Answer to the review of `308b44e` (ERROR):
- ERROR `POST /pulls/33/reviews` HTTP 500: the finding at `tests/test_line_comments.py:464-477` lies beyond the end of the file (224 lines), and Gitea answers such a line comment with 500, which 0.20.1 does not treat as refusal. Fixed in 3d0fbbc: 500 counts like 400 and 422, Scenario#33.07, red/green.
- ORANGE `werkrevisor/cli.py:579-590` (withdrawal reads only `requested_reviewers`): fixed in 3d0fbbc, a request listed only as `REQUEST_REVIEW` review is withdrawn too, the same sources as `_review_request_pending`. A request added during the run can only come from this run, which adds none at GREEN.
- hint `tests/test_line_comments.py:464-477` (a failed first edit is not retried by a second finding): not changed; one edit per comment and round is intended, and the second finding's text goes into the result comment.
- YELLOW risky path `werkrevisor/gitea.py`: needs the human review.
The findings of this round have no line comments, since the review failed; this comment answers them.
10 of 11 units passed all questions (1 PR document(s)).
5 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 78993, calls 1, tokens in 77260 / out 715 (reasoning included in out), duration 14.8 s, cost 0.1617 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9572, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 5174, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `3d0fbbc2aaef555958127831bc0974ea77a798a1` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:3d0fbbc2aaef555958127831bc0974ea77a798a1 -->
<!-- werkrevisor-verdict:YELLOW -->
<!-- werkrevisor-folded -->
<details><summary>🟡 werkrevisor YELLOW for <code>3d0fbbc</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1467">the result for <code>a356441</code></a></summary>
## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind: `mixed`.
### Reasons
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 given_when_then_misused (0.71 > yellow_above = 0.6)
- **YELLOW** 2 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:464-477, werkrevisor/cli.py:499-500
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:161-169
- **hint** model recommends changes before merge
### Findings
1. **[high] Line-comment edit may fail on real Gitea** at `werkrevisor/cli.py:464-477`, YELLOW, as a line comment
2. **[medium] HTTP 500 treated as a line-comment refusal** at `werkrevisor/cli.py:499-500`, YELLOW, as a line comment
3. **[medium] Reply filter has no test with an open reply to an open bot comment** at `tests/test_line_comments.py:161-169`, hint, as a line comment
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 | given_when_then_misused | 0.71 | – | YELLOW |
10 of 11 units passed all questions (1 PR document(s)).
5 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 78993, calls 1, tokens in 77260 / out 715 (reasoning included in out), duration 14.8 s, cost 0.1617 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9572, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 5174, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `3d0fbbc2aaef555958127831bc0974ea77a798a1` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: YELLOW / HUMAN_REVIEW for 3d0fbbc2aaef555958127831bc0974ea77a798a1.
The result comment of werkrevisor holds the details.
werkrevisor: YELLOW / HUMAN_REVIEW for `3d0fbbc2aaef555958127831bc0974ea77a798a1`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:3d0fbbc2aaef555958127831bc0974ea77a798a1 -->
[medium] Reply filter has no test with an open reply to an open bot comment (werkrevisor)
hint; Jev: real 0.71, severity 2.0 of 3
The new author filter in _open_bot_line_comments is tested only for a resolved bot comment with a human reply. There is no test that a human reply to an open bot comment is not counted as an extra hint or reuse target, which changes _open_hints counts.
**[medium] Reply filter has no test with an open reply to an open bot comment** (werkrevisor)
hint; Jev: real 0.71, severity 2.0 of 3
The new author filter in `_open_bot_line_comments` is tested only for a resolved bot comment with a human reply. There is no test that a human reply to an open bot comment is not counted as an extra hint or reuse target, which changes `_open_hints` counts.
[high] Line-comment edit may fail on real Gitea (werkrevisor)
YELLOW; Jev: real 0.45, severity 2.7 of 3
_update_line_comment edits a pull-review line comment via PATCH /issues/comments/{id}. The fake was changed to accept this for line comments ("Gitea edits issue comments and line comments under the same path"), which hides the question whether real Gitea honors it. The id comes from the review comments listing. If real Gitea rejects the id on that path, the code only falls back to the result comment, so the feature silently does nothing. This remains uncertain, since the real API was not checked here.
**[high] Line-comment edit may fail on real Gitea** (werkrevisor)
YELLOW; Jev: real 0.45, severity 2.7 of 3
`_update_line_comment` edits a pull-review line comment via `PATCH /issues/comments/{id}`. The fake was changed to accept this for line comments ("Gitea edits issue comments and line comments under the same path"), which hides the question whether real Gitea honors it. The id comes from the review comments listing. If real Gitea rejects the id on that path, the code only falls back to the result comment, so the feature silently does nothing. This remains uncertain, since the real API was not checked here.
Checked on git.javagil.de just now: PATCH /issues/comments/1406 with the unchanged body of line comment 1406 answered HTTP 200 with the same id, so Gitea 1.27 edits line comments under this path. Not changed.
Checked on git.javagil.de just now: `PATCH /issues/comments/1406` with the unchanged body of line comment 1406 answered HTTP 200 with the same id, so Gitea 1.27 edits line comments under this path. Not changed.
[medium] HTTP 500 treated as a line-comment refusal (werkrevisor)
YELLOW; Jev: real 0.59, severity 2.9 of 3
_publish_review now treats any HTTP 500 from create_review as a refusal of line comments and falls back to a review without them. A genuine transient server error then silently drops the line comments and produces a review that is carried in the result comment. For an APPROVED event the approval is retried once without comments. That is acceptable, but a transient 500 can no longer surface as ERROR. This is likely intended; the broader behavior change is worth confirming.
**[medium] HTTP 500 treated as a line-comment refusal** (werkrevisor)
YELLOW; Jev: real 0.59, severity 2.9 of 3
`_publish_review` now treats any HTTP 500 from `create_review` as a refusal of line comments and falls back to a review without them. A genuine transient server error then silently drops the line comments and produces a review that is carried in the result comment. For an APPROVED event the approval is retried once without comments. That is acceptable, but a transient 500 can no longer surface as ERROR. This is likely intended; the broader behavior change is worth confirming.
Not changed, intended: a transient 500 still gets a second POST without line comments; if that fails too, the run ends with ERROR. Otherwise the findings are in the result comment, so nothing is lost. A line beyond the file is a model error that would otherwise end every rerun of that head with ERROR. Left open for the human reviewer.
Not changed, intended: a transient 500 still gets a second POST without line comments; if that fails too, the run ends with ERROR. Otherwise the findings are in the result comment, so nothing is lost. A line beyond the file is a model error that would otherwise end every rerun of that head with ERROR. Left open for the human reviewer.
[medium] Withdrawal only uses requested_reviewers of a stale PR snapshot (werkrevisor)
ORANGE; Jev: real 0.67, severity 3.0 of 3
_withdraw_review_request reads requested_reviewers from pr_data, which _assert_current fetched before the publish step. For an approval YELLOW round the review request may also exist only as a REQUEST_REVIEW review, which _review_request_pending treats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request in requested_reviewers.
**[medium] Withdrawal only uses `requested_reviewers` of a stale PR snapshot** (werkrevisor)
ORANGE; Jev: real 0.67, severity 3.0 of 3
`_withdraw_review_request` reads `requested_reviewers` from `pr_data`, which `_assert_current` fetched before the publish step. For an `approval` YELLOW round the review request may also exist only as a `REQUEST_REVIEW` review, which `_review_request_pending` treats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request in `requested_reviewers`.
hint tests/test_line_comments.py:161-169 (open reply to an open hint): test added in a356441, resolved.
YELLOW risky path werkrevisor/gitea.py: needs the human review.
Answer to the review of `3d0fbbc` (YELLOW):
- YELLOW `werkrevisor/cli.py:464-477` (edit of line comments on real Gitea): verified with an unchanged edit of line comment 1406, HTTP 200; resolved.
- YELLOW `werkrevisor/cli.py:499-500` (HTTP 500 as refusal): intended, reasons in the thread, left open.
- YELLOW spec Scenario#33.07: reworded in a356441.
- hint `tests/test_line_comments.py:161-169` (open reply to an open hint): test added in a356441, resolved.
- YELLOW risky path `werkrevisor/gitea.py`: needs the human review.
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 3 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:455-456, docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, werkrevisor/cli.py:499-501
hint model recommends merge with caution
Findings
[medium] Line comment edited before the result comment is posted at werkrevisor/cli.py:455-456, hint, as a line comment
[low] Scenario#33.07 contradicts the code on the cause at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, hint, still open in an earlier line comment
[low] Any HTTP 500 is now treated as a refusal of line comments at werkrevisor/cli.py:499-501, hint, still open in an earlier line comment
10 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79357, calls 1, tokens in 77645 / out 786 (reasoning included in out), duration 13.5 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9588, cost 0.0004 USD, finding judge `jev-1.13.0` calls 1, tokens in 1493, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `a356441634415fc4556e0bc14416ee084559b6e2` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:a356441634415fc4556e0bc14416ee084559b6e2 -->
<!-- werkrevisor-verdict:YELLOW -->
<!-- werkrevisor-folded -->
<details><summary>🟡 werkrevisor YELLOW for <code>a356441</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1473">the result for <code>eeb7934</code></a></summary>
## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind: `mixed`.
### Reasons
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 given_when_then_misused (0.66 > yellow_above = 0.6)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 3 finding(s) of low severity or rated minor by Jev: werkrevisor/cli.py:455-456, docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, werkrevisor/cli.py:499-501
- **hint** model recommends merge with caution
### Findings
1. **[medium] Line comment edited before the result comment is posted** at `werkrevisor/cli.py:455-456`, hint, as a line comment
2. **[low] Scenario#33.07 contradicts the code on the cause** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1448)
3. **[low] Any HTTP 500 is now treated as a refusal of line comments** at `werkrevisor/cli.py:499-501`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1458)
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 | given_when_then_misused | 0.66 | – | YELLOW |
10 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79357, calls 1, tokens in 77645 / out 786 (reasoning included in out), duration 13.5 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9588, cost 0.0004 USD, finding judge `jev-1.13.0` calls 1, tokens in 1493, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `a356441634415fc4556e0bc14416ee084559b6e2` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: YELLOW / HUMAN_REVIEW for a356441634415fc4556e0bc14416ee084559b6e2.
The result comment of werkrevisor holds the details.
werkrevisor: YELLOW / HUMAN_REVIEW for `a356441634415fc4556e0bc14416ee084559b6e2`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:a356441634415fc4556e0bc14416ee084559b6e2 -->
[medium] Line comment edited before the result comment is posted (werkrevisor)
hint; Jev: real 0.71, severity 2.2 of 3
_plan_line_comments now edits earlier line comments while planning, before _post_comment and _publish_review run. If the result comment post or the review submission then fails, the earlier line comment already carries the new text and the "Updated by the review of <head>" note, but no result comment for that head exists. This contradicts the documented order "result comment first". The edit also runs on a path that was side-effect free before. Impact is limited because a rerun finds the text unchanged and does not edit again.
**[medium] Line comment edited before the result comment is posted** (werkrevisor)
hint; Jev: real 0.71, severity 2.2 of 3
`_plan_line_comments` now edits earlier line comments while planning, before `_post_comment` and `_publish_review` run. If the result comment post or the review submission then fails, the earlier line comment already carries the new text and the "Updated by the review of `<head>`" note, but no result comment for that head exists. This contradicts the documented order "result comment first". The edit also runs on a path that was side-effect free before. Impact is limited because a rerun finds the text unchanged and does not edit again.
Not changed: the same point as the threads at cli.py:715 and cli.py:453; the result comment must know whether the edit failed, so the edit comes first. Left open for the human reviewer.
Not changed: the same point as the threads at `cli.py:715` and `cli.py:453`; the result comment must know whether the edit failed, so the edit comes first. Left open for the human reviewer.
YELLOW spec Scenario#33.07 and hint "contradicts the code on the cause": reworded in eeb7934; the scenario starts from the finding beyond the file, and the Then names the outcome.
hint werkrevisor/cli.py:455-456 (edit before the result comment): repeated finding, not changed, see its thread.
hint werkrevisor/cli.py:499-501 (any HTTP 500 as refusal): repeated finding, answered in its thread 1458.
YELLOW risky path werkrevisor/gitea.py: needs the human review.
Answer to the review of `a356441` (YELLOW):
- YELLOW spec Scenario#33.07 and hint "contradicts the code on the cause": reworded in eeb7934; the scenario starts from the finding beyond the file, and the Then names the outcome.
- hint `werkrevisor/cli.py:455-456` (edit before the result comment): repeated finding, not changed, see its thread.
- hint `werkrevisor/cli.py:499-501` (any HTTP 500 as refusal): repeated finding, answered in its thread 1458.
- YELLOW risky path `werkrevisor/gitea.py`: needs the human review.
ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; a human review request is pending; merge already scheduled
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 2 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, werkrevisor/cli.py:455-456
hint model recommends merge with caution
Findings
[medium] Reuse also matches a bot comment that is a reply, and edit bypasses the resolver check at werkrevisor/cli.py:430-432, ORANGE, as a line comment
[medium] Scenario#33.07 contradicts the code and tests at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, hint, still open in an earlier line comment
[medium] Update of a reused line comment happens before the review is published at werkrevisor/cli.py:455-456, hint, still open in an earlier line comment
10 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79351, calls 1, tokens in 77634 / out 786 (reasoning included in out), duration 13.5 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9572, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4777, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `eeb79347d9f517e1f223a796e382124d42fbac55` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:eeb79347d9f517e1f223a796e382124d42fbac55 -->
<!-- werkrevisor-verdict:ORANGE -->
<!-- werkrevisor-folded -->
<details><summary>🟠 werkrevisor ORANGE for <code>eeb7934</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1479">the result for <code>0fcbb0f</code></a></summary>
## 🟠 werkrevisor: ORANGE / FINDINGS
Change kind: `mixed`.
### Reasons
- **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:430-432
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 given_when_then_misused (0.84 > yellow_above = 0.6)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 2 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, werkrevisor/cli.py:455-456
- **hint** model recommends merge with caution
### Findings
1. **[medium] Reuse also matches a bot comment that is a reply, and edit bypasses the resolver check** at `werkrevisor/cli.py:430-432`, ORANGE, as a line comment
2. **[medium] Scenario#33.07 contradicts the code and tests** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1448)
3. **[medium] Update of a reused line comment happens before the review is published** at `werkrevisor/cli.py:455-456`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1468)
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 | given_when_then_misused | 0.84 | – | YELLOW |
10 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79351, calls 1, tokens in 77634 / out 786 (reasoning included in out), duration 13.5 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9572, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4777, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `eeb79347d9f517e1f223a796e382124d42fbac55` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: ORANGE / FINDINGS for eeb79347d9f517e1f223a796e382124d42fbac55.
The result comment of werkrevisor holds the details.
werkrevisor: ORANGE / FINDINGS for `eeb79347d9f517e1f223a796e382124d42fbac55`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:eeb79347d9f517e1f223a796e382124d42fbac55 -->
[medium] Reuse also matches a bot comment that is a reply, and edit bypasses the resolver check (werkrevisor)
ORANGE; Jev: real 0.68, severity 2.6 of 3
_open_bot_line_comments now accepts every comment authored by the bot with an empty resolver. A bot reply inside a thread started by a human would be treated as a hint and edited. Such a comment is not necessarily a bot line comment of a finding. This is uncertain, since bots do not reply today, but the new edit makes the case destructive, and no test covers it.
**[medium] Reuse also matches a bot comment that is a reply, and edit bypasses the resolver check** (werkrevisor)
ORANGE; Jev: real 0.68, severity 2.6 of 3
`_open_bot_line_comments` now accepts every comment authored by the bot with an empty `resolver`. A bot reply inside a thread started by a human would be treated as a hint and edited. Such a comment is not necessarily a bot line comment of a finding. This is uncertain, since bots do not reply today, but the new edit makes the case destructive, and no test covers it.
Not changed. _open_bot_line_comments reads only the comments of reviews submitted by the bot, so a bot reply in a thread started by a human, which Gitea lists under the human's review, is never seen. A bot reply in its own thread does not exist: werkrevisor never calls the reply endpoint. Should it ever do so, that reply is bot text in an open thread of the bot. Left open for the human reviewer.
Not changed. `_open_bot_line_comments` reads only the comments of reviews submitted by the bot, so a bot reply in a thread started by a human, which Gitea lists under the human's review, is never seen. A bot reply in its own thread does not exist: werkrevisor never calls the reply endpoint. Should it ever do so, that reply is bot text in an open thread of the bot. Left open for the human reviewer.
ORANGE werkrevisor/cli.py:430-432 (a bot reply would be reused and edited): not changed, reasons in the thread; only bot reviews are read, and the bot writes no replies.
YELLOW spec Scenario#33.07 and hint "contradicts the code and tests": fixed in 0fcbb0f; the When is now the HTTP 500 of Gitea, as in the test, and the line beyond the file moved into the So-that as the known cause.
hint werkrevisor/cli.py:455-456 (edit before publishing): repeated, see thread 1468.
YELLOW risky path werkrevisor/gitea.py: needs the human review.
Answer to the review of `eeb7934` (ORANGE):
- ORANGE `werkrevisor/cli.py:430-432` (a bot reply would be reused and edited): not changed, reasons in the thread; only bot reviews are read, and the bot writes no replies.
- YELLOW spec Scenario#33.07 and hint "contradicts the code and tests": fixed in 0fcbb0f; the When is now the HTTP 500 of Gitea, as in the test, and the line beyond the file moved into the So-that as the known cause.
- hint `werkrevisor/cli.py:455-456` (edit before publishing): repeated, see thread 1468.
- YELLOW risky path `werkrevisor/gitea.py`: needs the human review.
ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; a human review request is pending; merge already scheduled
YELLOW 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:533-537
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-127
hint model recommends merge with caution
Findings
[medium] Edited reused comment is not kept in sync across runs when a 500 is refused at werkrevisor/cli.py:499-504, ORANGE, still open in an earlier line comment
[medium] Withdrawal skips reviewers whose login differs only in case from the request at werkrevisor/cli.py:533-537, YELLOW, as a line comment
[low] Scenario 33.07 contradicts the code for the COMMENT event at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-127, hint, still open in an earlier line comment
10 of 11 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79309, calls 1, tokens in 77601 / out 795 (reasoning included in out), duration 15.1 s, cost 0.1632 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9590, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3422, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d -->
<!-- werkrevisor-verdict:ORANGE -->
<!-- werkrevisor-folded -->
<details><summary>🟠 werkrevisor ORANGE for <code>0fcbb0f</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1485">the result for <code>c3ad49e</code></a></summary>
## 🟠 werkrevisor: ORANGE / FINDINGS
Change kind: `mixed`.
### Reasons
- **ORANGE** 1 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:499-504
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 given_when_then_misused (0.70 > yellow_above = 0.6)
- **YELLOW** 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:533-537
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-127
- **hint** model recommends merge with caution
### Findings
1. **[medium] Edited reused comment is not kept in sync across runs when a 500 is refused** at `werkrevisor/cli.py:499-504`, ORANGE, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1458)
2. **[medium] Withdrawal skips reviewers whose login differs only in case from the request** at `werkrevisor/cli.py:533-537`, YELLOW, as a line comment
3. **[low] Scenario 33.07 contradicts the code for the COMMENT event** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-127`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1448)
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.07 | given_when_then_misused | 0.70 | – | YELLOW |
10 of 11 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79309, calls 1, tokens in 77601 / out 795 (reasoning included in out), duration 15.1 s, cost 0.1632 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9590, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3422, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: ORANGE / FINDINGS for 0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d.
The result comment of werkrevisor holds the details.
werkrevisor: ORANGE / FINDINGS for `0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d -->
[medium] Withdrawal skips reviewers whose login differs only in case from the request (werkrevisor)
YELLOW; Jev: real 0.54, severity 2.7 of 3
_withdraw_review_request passes the logins as Gitea returned them, which is correct. However, the withdrawal is only attempted when the merge note equals exactly MERGE_SCHEDULED or MERGE_ALREADY_SCHEDULED; the comparison of rendered note strings is fragile, since _schedule_merge returns redactor-processed text for other branches. A future change of the note wording silently disables the withdrawal. Prefer a structured return value.
**[medium] Withdrawal skips reviewers whose login differs only in case from the request** (werkrevisor)
YELLOW; Jev: real 0.54, severity 2.7 of 3
`_withdraw_review_request` passes the logins as Gitea returned them, which is correct. However, the withdrawal is only attempted when the merge note equals exactly `MERGE_SCHEDULED` or `MERGE_ALREADY_SCHEDULED`; the comparison of rendered note strings is fragile, since `_schedule_merge` returns redactor-processed text for other branches. A future change of the note wording silently disables the withdrawal. Prefer a structured return value.
Not changed: the notes are the constants MERGE_SCHEDULED and MERGE_ALREADY_SCHEDULED, which _schedule_merge returns as they are; a change of the wording changes the constant and so the comparison. test_a_scheduled_merge_withdraws_the_review_request_of_an_earlier_round fails if the withdrawal is silently disabled. Left open for the human reviewer.
Not changed: the notes are the constants `MERGE_SCHEDULED` and `MERGE_ALREADY_SCHEDULED`, which `_schedule_merge` returns as they are; a change of the wording changes the constant and so the comparison. `test_a_scheduled_merge_withdraws_the_review_request_of_an_earlier_round` fails if the withdrawal is silently disabled. Left open for the human reviewer.
ORANGE werkrevisor/cli.py:499-504 (reused comment not in sync after a 500): its text is lost, since 0.20.1 reused my reply 1458. From the title: an edit of a reused comment happens before the review POST and is not affected by a later 500; the new findings of a refused review go into the result comment. Not changed.
YELLOW werkrevisor/cli.py:533-537 (note strings as condition): not changed, reasons in the thread.
YELLOW spec Scenario#33.07 and hint "contradicts the code for the COMMENT event": fixed in c3ad49e; the Then no longer claims a second review, which only an approval gets.
YELLOW risky path werkrevisor/gitea.py: needs the human review.
Answer to the review of `0fcbb0f` (ORANGE):
- ORANGE `werkrevisor/cli.py:499-504` (reused comment not in sync after a 500): its text is lost, since 0.20.1 reused my reply 1458. From the title: an edit of a reused comment happens before the review POST and is not affected by a later 500; the new findings of a refused review go into the result comment. Not changed.
- YELLOW `werkrevisor/cli.py:533-537` (note strings as condition): not changed, reasons in the thread.
- YELLOW spec Scenario#33.07 and hint "contradicts the code for the COMMENT event": fixed in c3ad49e; the Then no longer claims a second review, which only an approval gets.
- YELLOW risky path `werkrevisor/gitea.py`: needs the human review.
ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; a human review request is pending; merge already scheduled
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:37-39
hint model recommends merge with caution
Findings
[medium] HTTP 500 treated as a line-comment refusal hides real server errors at werkrevisor/cli.py:499-500, ORANGE, still open in an earlier line comment
[medium] Edit of reused line comment happens before the result comment and review are posted at werkrevisor/cli.py:455-456, ORANGE, still open in an earlier line comment
[medium] Missing test for failed withdrawal when line comments hit HTTP 500 on APPROVED event at tests/test_line_comments.py:37-39, hint, as a line comment
Classifier
11 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79298, calls 1, tokens in 77594 / out 795 (reasoning included in out), duration 13.7 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9579, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4723, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `c3ad49e96a6f68fff353d713b737a23b70ac5c88` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:c3ad49e96a6f68fff353d713b737a23b70ac5c88 -->
<!-- werkrevisor-verdict:ORANGE -->
<!-- werkrevisor-folded -->
<details><summary>🟠 werkrevisor ORANGE for <code>c3ad49e</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1491">the result for <code>aa33df4</code></a></summary>
## 🟠 werkrevisor: ORANGE / FINDINGS
Change kind: `mixed`.
### Reasons
- **ORANGE** 2 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:499-500, werkrevisor/cli.py:455-456
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: tests/test_line_comments.py:37-39
- **hint** model recommends merge with caution
### Findings
1. **[medium] HTTP 500 treated as a line-comment refusal hides real server errors** at `werkrevisor/cli.py:499-500`, ORANGE, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1458)
2. **[medium] Edit of reused line comment happens before the result comment and review are posted** at `werkrevisor/cli.py:455-456`, ORANGE, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1468)
3. **[medium] Missing test for failed withdrawal when line comments hit HTTP 500 on APPROVED event** at `tests/test_line_comments.py:37-39`, hint, as a line comment
### Classifier
11 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79298, calls 1, tokens in 77594 / out 795 (reasoning included in out), duration 13.7 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9579, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4723, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `c3ad49e96a6f68fff353d713b737a23b70ac5c88` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
werkrevisor: ORANGE / FINDINGS for c3ad49e96a6f68fff353d713b737a23b70ac5c88.
The result comment of werkrevisor holds the details.
werkrevisor: ORANGE / FINDINGS for `c3ad49e96a6f68fff353d713b737a23b70ac5c88`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:c3ad49e96a6f68fff353d713b737a23b70ac5c88 -->
[medium] Missing test for failed withdrawal when line comments hit HTTP 500 on APPROVED event (werkrevisor)
hint; Jev: real 0.78, severity 2.0 of 3
The new HTTP 500 handling in _publish_review retries an APPROVED review without line comments after the refusal. The test test_finding_without_a_usable_line_stays_in_the_result uses finding_run with the default gate="approval" and an ORANGE verdict, so only the COMMENT path is exercised. The retry-without-comments path for GREEN (event != "COMMENT") is not tested with status 500.
**[medium] Missing test for failed withdrawal when line comments hit HTTP 500 on APPROVED event** (werkrevisor)
hint; Jev: real 0.78, severity 2.0 of 3
The new HTTP 500 handling in `_publish_review` retries an APPROVED review without line comments after the refusal. The test `test_finding_without_a_usable_line_stays_in_the_result` uses `finding_run` with the default `gate="approval"` and an ORANGE verdict, so only the COMMENT path is exercised. The retry-without-comments path for GREEN (`event != "COMMENT"`) is not tested with status 500.
ORANGE werkrevisor/cli.py:499-500 (HTTP 500 as refusal) and werkrevisor/cli.py:455-456 (edit before publishing): repeated findings, not changed, reasons in the threads 1458 and 1468, left open for the human reviewer.
hint tests/test_line_comments.py:37-39 (no test of HTTP 500 on a GREEN review): test added in aa33df4, resolved.
YELLOW risky path werkrevisor/gitea.py: needs the human review.
The spec check passes now. From my side PR#33 is ready for the human review.
Answer to the review of `c3ad49e` (ORANGE):
- ORANGE `werkrevisor/cli.py:499-500` (HTTP 500 as refusal) and `werkrevisor/cli.py:455-456` (edit before publishing): repeated findings, not changed, reasons in the threads 1458 and 1468, left open for the human reviewer.
- hint `tests/test_line_comments.py:37-39` (no test of HTTP 500 on a GREEN review): test added in aa33df4, resolved.
- YELLOW risky path `werkrevisor/gitea.py`: needs the human review.
The spec check passes now. From my side PR#33 is ready for the human review.
YELLOW 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:535-536
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 2 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:85-97, werkrevisor/cli.py:455-456
hint model recommends merge with caution
Findings
[medium] Withdrawal skipped when the merge is held but earlier requests remain at werkrevisor/cli.py:535-536, YELLOW, still open in an earlier line comment
[medium] Scenario#33.05 contradicts the test and the code at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:85-97, hint, still open in an earlier line comment
[medium] Edit of a reused comment happens before the review is published at werkrevisor/cli.py:455-456, hint, still open in an earlier line comment
10 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79905, calls 1, tokens in 78208 / out 793 (reasoning included in out), duration 13.8 s, cost 0.1643 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9579, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4858, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `aa33df41af752b9071c4ac675f34c0b67042d028` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:aa33df41af752b9071c4ac675f34c0b67042d028 -->
<!-- werkrevisor-verdict:YELLOW -->
<!-- werkrevisor-folded -->
<details><summary>🟡 werkrevisor YELLOW for <code>aa33df4</code>, superseded by <a href="http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1495">the result for <code>d9b659a</code></a></summary>
## 🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind: `mixed`.
### Reasons
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **YELLOW** spec: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.05 given_when_then_misused (0.63 > yellow_above = 0.6)
- **YELLOW** 1 finding(s) possibly severe, but Jev doubts that they are real: werkrevisor/cli.py:535-536
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 2 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:85-97, werkrevisor/cli.py:455-456
- **hint** model recommends merge with caution
### Findings
1. **[medium] Withdrawal skipped when the merge is held but earlier requests remain** at `werkrevisor/cli.py:535-536`, YELLOW, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1480)
2. **[medium] Scenario#33.05 contradicts the test and the code** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:85-97`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1441)
3. **[medium] Edit of a reused comment happens before the review is published** at `werkrevisor/cli.py:455-456`, hint, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1468)
### Classifier
| Unit | Question | Problem | Confidence | Verdict |
|---|---|---|---|---|
| docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md Scenario#33.05 | given_when_then_misused | 0.63 | – | YELLOW |
10 of 11 units passed all questions (1 PR document(s)).
4 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79905, calls 1, tokens in 78208 / out 793 (reasoning included in out), duration 13.8 s, cost 0.1643 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9579, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4858, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `aa33df41af752b9071c4ac675f34c0b67042d028` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
</details>
YELLOW spec Scenario#33.05 and hint "contradicts the test and the code": fixed in d9b659a; the scenario now has a table for the resolved and the open hint, matching its two tests.
YELLOW werkrevisor/cli.py:535-536 (withdrawal skipped while the merge is held): intended, Scenario#33.06; a held merge waits for the author, and the human request stays until the merge is scheduled.
hint werkrevisor/cli.py:455-456 (edit before publishing): repeated, see thread 1468.
YELLOW risky path werkrevisor/gitea.py: needs the human review.
Answer to the review of `aa33df4` (YELLOW):
- YELLOW spec Scenario#33.05 and hint "contradicts the test and the code": fixed in d9b659a; the scenario now has a table for the resolved and the open hint, matching its two tests.
- YELLOW `werkrevisor/cli.py:535-536` (withdrawal skipped while the merge is held): intended, Scenario#33.06; a held merge waits for the author, and the human request stays until the merge is scheduled.
- hint `werkrevisor/cli.py:455-456` (edit before publishing): repeated, see thread 1468.
- YELLOW risky path `werkrevisor/gitea.py`: needs the human review.
ai-review/assessment AI findings; a developer approval after checking them unlocks the merge; approved by mi for this head commit; merge already scheduled
hint 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
hint 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:14-17
hint model recommends changes before merge
Findings
[medium] Line comment edited before the result comment is posted at werkrevisor/cli.py:455-456, ORANGE, still open in an earlier line comment
[medium] HTTP 500 treated as refusal of line comments at werkrevisor/cli.py:499-500, ORANGE, still open in an earlier line comment
[low] PR title and document scope are inconsistent at docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:14-17, hint, as a line comment
Classifier
11 of 11 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79818, calls 1, tokens in 78137 / out 786 (reasoning included in out), duration 14.2 s, cost 0.1641 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9618, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3041, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).
Reviewed head `d9b659a7e9e68faa92b2d62aa3d789cf93eec891` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
<!-- werkrevisor:d9b659a7e9e68faa92b2d62aa3d789cf93eec891 -->
<!-- werkrevisor-verdict:ORANGE -->
## 🟠 werkrevisor: ORANGE / FINDINGS
Change kind: `mixed`.
### Reasons
- **ORANGE** 2 finding(s) that Jev rates as probably real and severe: werkrevisor/cli.py:455-456, werkrevisor/cli.py:499-500
- **YELLOW** risky paths changed: werkrevisor/gitea.py (werkrevisor/gitea.py)
- **hint** 1 build change(s) of low risk according to Jev: server/build.gradle.kts (weakens checks 0.02, risk 0.00)
- **hint** 1 finding(s) of low severity or rated minor by Jev: docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:14-17
- **hint** model recommends changes before merge
### Findings
1. **[medium] Line comment edited before the result comment is posted** at `werkrevisor/cli.py:455-456`, ORANGE, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1468)
2. **[medium] HTTP 500 treated as refusal of line comments** at `werkrevisor/cli.py:499-500`, ORANGE, still open in [an earlier line comment](http://git.javagil.de/mi/werkrevisor/pulls/33#issuecomment-1458)
3. **[low] PR title and document scope are inconsistent** at `docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:14-17`, hint, as a line comment
### Classifier
11 of 11 units passed all questions (1 PR document(s)).
3 answers ignored because of a confidence below [classifier] min_confidence.
<sub>
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79818, calls 1, tokens in 78137 / out 786 (reasoning included in out), duration 14.2 s, cost 0.1641 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9618, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3041, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base dcdcf48).<br>
Reviewed head `d9b659a7e9e68faa92b2d62aa3d789cf93eec891` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
</sub>
werkrevisor: ORANGE / FINDINGS for d9b659a7e9e68faa92b2d62aa3d789cf93eec891.
The result comment of werkrevisor holds the details.
werkrevisor: ORANGE / FINDINGS for `d9b659a7e9e68faa92b2d62aa3d789cf93eec891`.
The result comment of werkrevisor holds the details.
<!-- werkrevisor-review:d9b659a7e9e68faa92b2d62aa3d789cf93eec891 -->
[low] PR title and document scope are inconsistent (werkrevisor)
hint
The PR title and the first Problem paragraph name only the reused line comment text. The document also covers reply filtering, withdrawal of review requests and HTTP 500 handling, and presents these as defects "of 0.20.1". The Problem section mixes the three extra defects into one long sentence chain with ; and and, which is hard to read. The unrelated changes make the PR harder to review and revert.
**[low] PR title and document scope are inconsistent** (werkrevisor)
hint
The PR title and the first Problem paragraph name only the reused line comment text. The document also covers reply filtering, withdrawal of review requests and HTTP 500 handling, and presents these as defects "of 0.20.1". The Problem section mixes the three extra defects into one long sentence chain with `;` and `and`, which is hard to read. The unrelated changes make the PR harder to review and revert.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
See the PR doc.
🤖 Generated with Claude Code
🟡 werkrevisor YELLOW for
a0bffc1, superseded by the result for704fd0b🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Update note regex only matches a lowercase hex head SHA at
werkrevisor/cli.py:207-215, YELLOW, as a line comment[medium] Edited line comment may be re-edited every round after redaction or
<!--escaping atwerkrevisor/cli.py:460-473, hint, as a line commentClassifier
7 of 7 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 58753, calls 1, tokens in 57327 / out 527 (reasoning included in out), duration 16.1 s, cost 0.1199 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5573, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 3358, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base2 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `a0bffc1408b48421502a1b88110e3c342b77736e` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: YELLOW / HUMAN_REVIEW for
a0bffc1408b48421502a1b88110e3c342b77736e.The result comment of werkrevisor holds the details.
@@ -203,5 +203,5 @@stop = ebreakfull = _reduced_context(reductions.pop(0), full, files, fetch, pr, config, deps, rules, diffs, result)prompt_section = full.section + hintsextra_chars = (len(instructions) + len(pr_data.get("title") or "") + len(pr_data.get("body") or "")[medium] Update note regex only matches a lowercase hex head SHA (werkrevisor)
YELLOW; Jev: real 0.18, severity 2.6 of 3
_UPDATE_NOTEends with$, which matches before a trailing newline but not before other trailing whitespace. If Gitea returns the body with trailing whitespace or CRLF line endings, the note is not stripped. The comparison then fails and the comment is edited again each round, and the note is stacked on a body that already holds one. This is uncertain because it depends on Gitea's body normalization.Fixed in
704fd0b:_UPDATE_NOTEaccepts any blanks and CRLF around the note and upper-case hex; the comparison normalizes line endings. Stacking cannot happen anyway: an edit replaces the whole body by the new text with exactly one note.@@ -453,6 +457,22 @@ def _plan_line_comments(gitea, pr, result, bot_login):return commentsdef _update_line_comment(gitea, pr, earlier, text, redactor, updated):[medium] Edited line comment may be re-edited every round after redaction or
<!--escaping (werkrevisor)hint; Jev: real 0.80, severity 2.0 of 3
The no-change check compares
without_update_note(earlier["body"])withredactor(text). Gitea may return a body that differs from what was posted, for example after normalization of line endings. In that case the comment is edited and marked as edited on every round, which the PR document says it avoids. The test fake stores the body verbatim, so this path is not covered. The effect is limited to extra edits, not data loss.Fixed in
704fd0b: the no-change check now ignores line endings and trailing blanks (report.same_line_comment_text), so a body that Gitea normalized is no change; the testtest_a_reused_line_comment_with_the_same_text_stays_untouchedcovers a body with CRLF and trailing blanks (red without the fix). Redaction and the<!--escaping are not affected: both are deterministic, the earlier comment was posted through the same redactor andline_comment.Answer to the review of
a0bffc1(YELLOW):werkrevisor/cli.py:207-215(CRLF/trailing whitespace defeats the note regex): fixed in704fd0b, thread resolved.werkrevisor/cli.py:460-473(re-edit every round after normalization): fixed in704fd0b, thread resolved.server/build.gradle.kts: only the version bump to 0.20.2.🟡 werkrevisor YELLOW for
704fd0b, superseded by the result forc2043de🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Edit via issue-comment endpoint for line comments at
werkrevisor/cli.py:460-473, YELLOW, still open in an earlier line comment[medium] Line comment is edited before the result comment is posted at
werkrevisor/cli.py:715-716, hint, as a line comment[low] Update note regex is not anchored to bot-written text at
werkrevisor/report.py:207-218, hint, as a line commentClassifier
6 of 7 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 59279, calls 1, tokens in 57864 / out 758 (reasoning included in out), duration 13.5 s, cost 0.1233 USD (reported), classifier `jev-1.13.0` calls 7, tokens in 5593, cost 0.0002 USD, finding judge `jev-1.13.0` calls 2, tokens in 3208, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base2 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `704fd0bbc0531f5d7274bf18c0a5796e58533107` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: YELLOW / HUMAN_REVIEW for
704fd0bbc0531f5d7274bf18c0a5796e58533107.The result comment of werkrevisor holds the details.
@@ -693,3 +713,3 @@"""The result comment comes first: if anything fails later, no bot approval stands without its comment."""fresh = existing is None and result.verdict is not Verdict.ERRORcomments = _plan_line_comments(gitea, pr, result, bot_login) if fresh else []comments = _plan_line_comments(gitea, pr, result, bot_login, redactor) if fresh else [][medium] Line comment is edited before the result comment is posted (werkrevisor)
hint; Jev: real 0.71, severity 2.1 of 3
_plan_line_commentsnow writes to Gitea (edits earlier line comments) before_post_commentruns. If the later steps fail, for example the result comment post or a superseded head, the earlier line comment already holds text from a review of a head that never got a result. The old behaviour had no write in the planning phase, so the "result comment comes first" docstring of_publishno longer holds for these edits. The impact is limited to wording and the update note naming the head.Not changed, on purpose. The edit only brings the wording of the open finding up to date and names the head that wrote it; if the result comment fails afterwards, the next run of that head or of a later head writes the result and edits the text again. The docstring of
_publishis about the approval, which still never stands without its result comment. Moving the edits behind the result comment would lose the fallback: a failed edit puts the new text into the result comment, so the result comment must know the outcome of the edit first. Left open for the human reviewer.@@ -195,6 +204,20 @@ def line_comment(f, v):return body.replace("<!--", "<!--")_UPDATE_NOTE = re.compile(r"\s*<sub>Updated by the review of `[0-9a-fA-F]+`\.</sub>\s*$")[low] Update note regex is not anchored to bot-written text (werkrevisor)
hint
same_line_comment_textstrips a trailing<sub>Updated by ...</sub>from the earlier body. This is harmless for bot comments, but a model finding ending with that literal text would also be stripped before comparison, so a real change would not be detected. This needs deliberate model output and has low impact.Not changed: only a model text that ends with exactly this note line, with a hex SHA, is affected, and the only consequence is one skipped edit of wording; the finding itself stays open and listed. Left open for the human reviewer.
Answer to the review of
704fd0b(YELLOW):given_when_then_misused: the When step held state; fixed inc2043de, the different wording is now a Given.werkrevisor/cli.py:460-473(edit via the issue-comment endpoint):PATCH /issues/comments/{id}edits review comments too on Gitea 1.27; it was checked on git.javagil.de before this PR.This finding also showed a real, older bug: werkrevisor 0.20.1 reused my reply 1406 as the "earlier line comment", because Gitea lists a reply among the comments of the review that started the thread; the reply also counted as an open hint, and with this PR the bot would have tried to edit it.
Fixed in
c2043de: only comments written by the bot count, with Scenario#33.04 and red/green tests.werkrevisor/cli.py:715-716(edit before the result comment): not changed, reasons in the thread.werkrevisor/report.py:207-218(note regex not tied to bot text): not changed, reasons in the thread.server/build.gradle.kts: only the version bump.The reviews run with the deployed 0.20.1, so this PR still shows the old behaviour: the reused comment kept its text.
🟡 werkrevisor YELLOW for
c2043de, superseded by the result forf572e65🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Redacted text compared against the unredacted body, and the held-hint count includes reused comments at
werkrevisor/cli.py:426-430, hint, as a line comment[low] Scenario 33.04 names a test file that does not contain one of its tests at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:69-82, hint, as a line commentClassifier
7 of 8 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 65440, calls 1, tokens in 63836 / out 678 (reasoning included in out), duration 16.0 s, cost 0.1345 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6647, cost 0.0003 USD, finding judge `jev-1.13.0` calls 1, tokens in 1688, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base2 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `c2043deb92eb15ed8d59ff8607a1366b064a340b` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: YELLOW / HUMAN_REVIEW for
c2043deb92eb15ed8d59ff8607a1366b064a340b.The result comment of werkrevisor holds the details.
@@ -0,0 +66,4 @@### Feature: only comments of the bot count as its line comments#### Scenario#33.04: a reply in the thread of a bot line comment is neither reused nor a hint[low] Scenario 33.04 names a test file that does not contain one of its tests (werkrevisor)
hint
Both entries under "Verified by" for Scenario#33.04 are listed, but the merge-hold test is in
tests/test_hints_hold_merge.py, and the scenario's Given says the bot comment is "resolved" while only one test sets it up this way. The link text is consistent with the code. However, the scenario title claims a reply is "neither reused nor a hint", and the two tests each cover only one half. This is acceptable, but the Given/When/Then mixes both halves in a single scenario, which makes it hard to read.Fixed in
f572e65: the reply case is split into Scenario#33.04 (not reused) and Scenario#33.05 (no hold), one test each.@@ -425,3 +425,2 @@for c in gitea.list_review_comments(pr.number, review["id"]) or ():if not c.get("resolver") and (c.get("position") or 0) > 0:found.append({"path": c.get("path"), "line": c["position"], "url": c.get("html_url") or ""})# A reply of a human is listed in the review that started the thread, but is no hint of the bot.[medium] Redacted text compared against the unredacted body, and the held-hint count includes reused comments (werkrevisor)
hint; Jev: real 0.51, severity 2.1 of 3
_update_line_commentredactstextbefore comparing it withearlier["body"]. Redacted text is also what gets posted, so a secret in the model text is stored as[REDACTED]and the comparison against the stored body stays consistent. This part is fine. A real problem is that a reply from the bot itself in the same thread (author equalsbot_login) is still treated as a reusable line comment, because the filter checks only the author, not whether the comment starts a thread. A bot reply would then be edited and counted as a hint. This is an edge case, because the bot does not reply to threads in the code shown. The uncertainty is whether the bot ever writes replies.Not changed: werkrevisor never replies in a thread; it posts line comments only as part of its own review. Should it ever reply, its reply would be bot text in an open thread, and reusing or counting it would be right. Left open for the human reviewer.
Answer to the review of
c2043de(YELLOW):given_when_then_misused: reworded inf572e65with concrete texts: the Given holds the earlier comment, the When the new review, the Then the result.werkrevisor/cli.py:426-430(a bot reply would count): not changed, the bot never replies; reasons in the thread.f572e65, split into #33.04 and #33.05, thread resolved.server/build.gradle.kts: only the version bump.🟢 werkrevisor GREEN for
f572e65, superseded by the result fordfd5f69🟢 werkrevisor: GREEN / PASS
Gate: approved by werkrevisor; merge waits for 10 open hint(s): reply to each line comment and resolve it, then restart the review or push a commit.
Change kind:
mixed.Reasons
Findings
[medium] Reused comment is edited while publishing can still fail at
werkrevisor/cli.py:453-454, hint, as a line comment[medium] Line comment body with a redacted text and the update note differ in comparison at
werkrevisor/cli.py:462-475, hint, still open in an earlier line commentClassifier
9 of 9 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 65479, calls 1, tokens in 63872 / out 637 (reasoning included in out), duration 12.9 s, cost 0.1341 USD (reported), classifier `jev-1.13.0` calls 9, tokens in 7526, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 3291, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base2 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `f572e656d1ae88e70a2b0d1a36a054371e42692e` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: GREEN / PASS for
f572e656d1ae88e70a2b0d1a36a054371e42692e.The result comment of werkrevisor holds the details.
@@ -446,6 +450,8 @@ def _plan_line_comments(gitea, pr, result, bot_login):and first - LINE_SLACK <= c["line"] <= last + LINE_SLACK), None)if earlier is not None:result.reused[i] = earlier["url"]if not _update_line_comment(gitea, pr, earlier, report.line_comment(f, v), redactor, updated):[medium] Reused comment is edited while publishing can still fail (werkrevisor)
hint; Jev: real 0.76, severity 2.4 of 3
_plan_line_commentsedits the earlier line comment before the result comment is posted and before the review is submitted. If_post_commentraises afterwards, the run ends in ERROR but the earlier comment already carries the new text and the "Updated by" note. A rerun on the same head then finds an equal text, so it makes no edit and does not report the finding in the result comment. The side effect also breaks the docstring contract of_publishthat the result comment comes first. The impact is limited: the edited line comment keeps the finding visible.Not changed: the same point as the thread at
werkrevisor/cli.py:715; the result comment must know the outcome of the edit to carry the text after a failed edit, and the next review writes the wording again. Left open for the human reviewer.Answer to the review of
f572e65(GREEN, merge held for 10 open hints):werkrevisor/cli.py:453-454(edit while publishing can fail): not changed, as in the thread atcli.py:715.werkrevisor/cli.py:462-475(redacted text and update note in the comparison): its text is lost, since 0.20.1 reused my reply 1406 again, which is the bug this PR fixes. From the title: the comparison strips the note from the stored body and compares it with the redacted new text; the posted body is exactly the redacted text plus the note, so both sides match. Not changed.werkrevisor, review restarted for
f572e656d1ae88e70a2b0d1a36a054371e42692e: approved by werkrevisor; merge scheduled for when the checks succeed.New commits pushed, approval review dismissed automatically according to repository settings
🟠 werkrevisor ORANGE for
dfd5f69, superseded by the result for08ecc13🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Withdrawal of review requests has no test with the automerge flag off or a matching case at
werkrevisor/cli.py:530-533, ORANGE, as a line comment[medium] Held-merge hint count ignores finding text carried in the result comment at
werkrevisor/cli.py:557-561, hint, as a line comment[low] Scenario#33.06 misuses Given/When/Then at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:95-103, hint, as a line commentClassifier
9 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 73240, calls 1, tokens in 71417 / out 749 (reasoning included in out), duration 14.0 s, cost 0.1503 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8574, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3425, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base4 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `dfd5f6972d230c671e518537f57f7575dab88251` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: ORANGE / FINDINGS for
dfd5f6972d230c671e518537f57f7575dab88251.The result comment of werkrevisor holds the details.
@@ -0,0 +92,4 @@### Feature: a GREEN review releases its review request#### Scenario#33.06: a scheduled merge withdraws the review request of an earlier round[low] Scenario#33.06 misuses Given/When/Then (werkrevisor)
hint
The scenario has two "Given" lines: an automerge setting and a historic YELLOW round. The "So that" line also states a mechanism rather than a goal. The classifier flagged this scenario. Rephrase it so each step has one clear role.
Fixed in
08ecc13: one Given (the open request), one When (the GREEN review schedules the merge), one Then; the So-that states the goal.@@ -506,3 +528,3 @@exit_code = 0 if cancelled else Verdict.YELLOW.exit_codereturn GateDecision(exit_code, "approved by werkrevisor" + note, held=True)return GateDecision(0, "approved by werkrevisor" + _schedule_merge(config, pr, gitea, redactor, base_ref))scheduled = _schedule_merge(config, pr, gitea, redactor, base_ref)[medium] Withdrawal of review requests has no test with the automerge flag off or a matching case (werkrevisor)
ORANGE; Jev: real 0.87, severity 3.0 of 3
_withdraw_review_requestwithdraws requests when the merge is scheduled, but the withdrawal runs even when_schedule_mergereturned "merge not scheduled" (for example when branch protection is missing or the merge call failed). The review request is then withdrawn without a scheduled merge, which drops the human's pending review and leaves the PR unlocked by neither path. Withdraw only when the scheduling note starts with "; merge scheduled" or "; merge already scheduled".Fixed in
08ecc13: the request is withdrawn only aftermerge scheduledormerge already scheduled;test_a_merge_not_scheduled_keeps_the_review_requestcovers a missing branch protection (red without the fix).@@ -553,5 +553,5 @@if not config.gate.automerge:return ""try:# Without these rules Gitea would merge a YELLOW or ORANGE PR as soon as the checks pass, unapproved.branch = gitea.get_branch(base_ref) or {}[medium] Held-merge hint count ignores finding text carried in the result comment (werkrevisor)
hint; Jev: real 0.75, severity 2.3 of 3
_open_hintsexcludes every index inresult.reusedfromin_comment_only. A reused finding whose edit failed (reused_not_updated) now has its full text only in the result comment, yet it counts as neither in-comment nor attached. It is still counted through the open line comment, so the count is not lost, but the gate note says "reply to each line comment" while the new text sits elsewhere. No test covers the hold together with a failed edit.Not changed: the count stays right, as the finding says, since the open line comment counts. The author still replies to that line comment and resolves it; the result comment only carries the newer wording. Left open for the human reviewer.
Answer to the review of
dfd5f69(ORANGE):werkrevisor/cli.py:530-533(withdrawal without a scheduled merge): fixed in08ecc13, thread resolved.werkrevisor/gitea.py: one new call,withdraw_review_request(DELETE /pulls/{n}/requested_reviewers); needs the human review.given_when_then_misused: reworded in08ecc13, thread resolved.werkrevisor/cli.py:557-561(hint count and text in the result comment): not changed, reasons in the thread.server/build.gradle.kts: only the version bump.🟠 werkrevisor ORANGE for
08ecc13, superseded by the result for3d0fbbc🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Review request withdrawn although the merge may not follow at
werkrevisor/cli.py:532-536, ORANGE, still open in an earlier line comment[low] Document names a reproduction detail instead of the change at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:118-125, hint, as a line commentClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74368, calls 1, tokens in 72692 / out 597 (reasoning included in out), duration 19.6 s, cost 0.1514 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8522, cost 0.0004 USD, finding judge `jev-1.13.0` calls 1, tokens in 1761, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base3 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `08ecc13cbbc31f5c85f41ab55f222749c62a453a` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: ORANGE / FINDINGS for
08ecc13cbbc31f5c85f41ab55f222749c62a453a.The result comment of werkrevisor holds the details.
@@ -0,0 +115,4 @@without a change of the text, ignoring that line, line endings and trailing blanks, it does not edit,since Gitea may return a body normalized otherwise than posted.- A failed edit, or a second finding at the same comment, puts the full text into the result comment.- Only comments written by the bot count as its line comments:[low] Document names a reproduction detail instead of the change (werkrevisor)
hint
The Solution bullets narrate the history of this PR ("0.20.1 reused a reply of claude-code in this PR", "the request of the YELLOW rounds blocked the merge scheduled by the GREEN review of f572e65"). This is incident narration that does not describe the change and goes stale. Scenario#33.03 also covers only two causes in a table but gives one
Whenline, which makes the scenario harder to read.Fixed in
308b44e: the two defects found in the reviews of this PR moved from the solution into the problem. Scenario#33.03 keeps its table: both causes lead to the same outcome, which the template allows as data-driven scenario.Answer to the review of
08ecc13(ORANGE):werkrevisor/cli.py:532-536(request withdrawn although the merge may not follow): its text is lost again, since 0.20.1 reused my reply 1440, the bug this PR fixes. From the title: not changed. The request is withdrawn only once Gitea accepted the scheduled merge. If a check fails later, the PR stays unmerged; the author pushes a fix, and a review that is not GREEN requests the human review again. A GREEN head needs no human review, since the approval of the bot is the lock.werkrevisor/gitea.py: one new call,withdraw_review_request; needs the human review.308b44e, thread resolved.server/build.gradle.kts: only the version bump.🟠 werkrevisor ORANGE for
308b44e, superseded by the result for3d0fbbc🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Withdrawal only uses
requested_reviewersof a stale PR snapshot atwerkrevisor/cli.py:579-590ORANGE; Jev: real 0.67, severity 3.0 of 3
_withdraw_review_requestreadsrequested_reviewersfrompr_data, which_assert_currentfetched before the publish step. For anapprovalYELLOW round the review request may also exist only as aREQUEST_REVIEWreview, which_review_request_pendingtreats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request inrequested_reviewers.[medium] Reuse of a later finding after an update failure is untested at
tests/test_line_comments.py:464-477hint; Jev: real 0.54, severity 2.2 of 3
The
updatedset marks a comment as updated before the edit is tried. If the edit fails for the first finding, a second finding at the same comment getsreused_not_updatedthrough theupdatedcheck, which is correct. A failed first edit is never retried by the second finding, so only one of two failed findings gets the update attempt. The tests cover each case separately but not the combination.Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74388, calls 1, tokens in 72715 / out 554 (reasoning included in out), duration 12.2 s, cost 0.1510 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8528, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 2292, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base3 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `308b44ee1d022a01f3dbcdaf7ae5174012d2ee26` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
⚠️ werkrevisor ERROR for
308b44e, superseded by the result for3d0fbbc⚠️ werkrevisor: ERROR / review not completed, human action needed
Change kind:
mixed.Reasons
Findings
[medium] Withdrawal only uses
requested_reviewersof a stale PR snapshot atwerkrevisor/cli.py:579-590ORANGE; Jev: real 0.67, severity 3.0 of 3
_withdraw_review_requestreadsrequested_reviewersfrompr_data, which_assert_currentfetched before the publish step. For anapprovalYELLOW round the review request may also exist only as aREQUEST_REVIEWreview, which_review_request_pendingtreats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request inrequested_reviewers.[medium] Reuse of a later finding after an update failure is untested at
tests/test_line_comments.py:464-477hint; Jev: real 0.54, severity 2.2 of 3
The
updatedset marks a comment as updated before the edit is tried. If the edit fails for the first finding, a second finding at the same comment getsreused_not_updatedthrough theupdatedcheck, which is correct. A failed first edit is never retried by the second finding, so only one of two failed findings gets the update attempt. The tests cover each case separately but not the combination.Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 74388, calls 1, tokens in 72715 / out 554 (reasoning included in out), duration 12.2 s, cost 0.1510 USD (reported), classifier `jev-1.13.0` calls 10, tokens in 8528, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 2292, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base3 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `308b44ee1d022a01f3dbcdaf7ae5174012d2ee26` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
Answer to the review of
308b44e(ERROR):POST /pulls/33/reviewsHTTP 500: the finding attests/test_line_comments.py:464-477lies beyond the end of the file (224 lines), and Gitea answers such a line comment with 500, which 0.20.1 does not treat as refusal. Fixed in3d0fbbc: 500 counts like 400 and 422, Scenario#33.07, red/green.werkrevisor/cli.py:579-590(withdrawal reads onlyrequested_reviewers): fixed in3d0fbbc, a request listed only asREQUEST_REVIEWreview is withdrawn too, the same sources as_review_request_pending. A request added during the run can only come from this run, which adds none at GREEN.tests/test_line_comments.py:464-477(a failed first edit is not retried by a second finding): not changed; one edit per comment and round is intended, and the second finding's text goes into the result comment.werkrevisor/gitea.py: needs the human review.The findings of this round have no line comments, since the review failed; this comment answers them.
🟡 werkrevisor YELLOW for
3d0fbbc, superseded by the result fora356441🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[high] Line-comment edit may fail on real Gitea at
werkrevisor/cli.py:464-477, YELLOW, as a line comment[medium] HTTP 500 treated as a line-comment refusal at
werkrevisor/cli.py:499-500, YELLOW, as a line comment[medium] Reply filter has no test with an open reply to an open bot comment at
tests/test_line_comments.py:161-169, hint, as a line commentClassifier
10 of 11 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 78993, calls 1, tokens in 77260 / out 715 (reasoning included in out), duration 14.8 s, cost 0.1617 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9572, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 5174, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base5 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `3d0fbbc2aaef555958127831bc0974ea77a798a1` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: YELLOW / HUMAN_REVIEW for
3d0fbbc2aaef555958127831bc0974ea77a798a1.The result comment of werkrevisor holds the details.
@@ -104,0 +158,4 @@self.assertEqual(result.count("still open in [an earlier line comment](https://x/77)"), 2)self.assertEqual(result.count("Breaks X."), 1)def test_a_reply_in_the_thread_of_a_bot_review_is_not_reused(self):[medium] Reply filter has no test with an open reply to an open bot comment (werkrevisor)
hint; Jev: real 0.71, severity 2.0 of 3
The new author filter in
_open_bot_line_commentsis tested only for a resolved bot comment with a human reply. There is no test that a human reply to an open bot comment is not counted as an extra hint or reuse target, which changes_open_hintscounts.Fixed in
a356441:test_an_open_reply_to_an_open_hint_is_no_second_hint, red without the author filter.@@ -453,6 +461,22 @@ def _plan_line_comments(gitea, pr, result, bot_login):return commentsdef _update_line_comment(gitea, pr, earlier, text, redactor, updated):[high] Line-comment edit may fail on real Gitea (werkrevisor)
YELLOW; Jev: real 0.45, severity 2.7 of 3
_update_line_commentedits a pull-review line comment viaPATCH /issues/comments/{id}. The fake was changed to accept this for line comments ("Gitea edits issue comments and line comments under the same path"), which hides the question whether real Gitea honors it. The id comes from the review comments listing. If real Gitea rejects the id on that path, the code only falls back to the result comment, so the feature silently does nothing. This remains uncertain, since the real API was not checked here.Checked on git.javagil.de just now:
PATCH /issues/comments/1406with the unchanged body of line comment 1406 answered HTTP 200 with the same id, so Gitea 1.27 edits line comments under this path. Not changed.@@ -473,3 +497,3 @@return Falseexcept GiteaError as e:if not comments or e.status not in (400, 422):# Gitea 1.27 answers a line beyond the end of the file with HTTP 500.[medium] HTTP 500 treated as a line-comment refusal (werkrevisor)
YELLOW; Jev: real 0.59, severity 2.9 of 3
_publish_reviewnow treats any HTTP 500 fromcreate_reviewas a refusal of line comments and falls back to a review without them. A genuine transient server error then silently drops the line comments and produces a review that is carried in the result comment. For an APPROVED event the approval is retried once without comments. That is acceptable, but a transient 500 can no longer surface as ERROR. This is likely intended; the broader behavior change is worth confirming.Not changed, intended: a transient 500 still gets a second POST without line comments; if that fails too, the run ends with ERROR. Otherwise the findings are in the result comment, so nothing is lost. A line beyond the file is a model error that would otherwise end every rerun of that head with ERROR. Left open for the human reviewer.
@@ -548,6 +576,20 @@ def _hold_merge(pr, gitea, redactor, count):return note, Truedef _withdraw_review_request(config, pr, gitea, redactor, pr_data):[medium] Withdrawal only uses
requested_reviewersof a stale PR snapshot (werkrevisor)ORANGE; Jev: real 0.67, severity 3.0 of 3
_withdraw_review_requestreadsrequested_reviewersfrompr_data, which_assert_currentfetched before the publish step. For anapprovalYELLOW round the review request may also exist only as aREQUEST_REVIEWreview, which_review_request_pendingtreats as a pending request. Such a request is not withdrawn and the merge stays blocked. The snapshot may also be stale if the request was changed during the run. Uncertain: whether Gitea reports every pending request inrequested_reviewers.Fixed in
3d0fbbc: requests listed only asREQUEST_REVIEWreview are withdrawn too.Answer to the review of
3d0fbbc(YELLOW):werkrevisor/cli.py:464-477(edit of line comments on real Gitea): verified with an unchanged edit of line comment 1406, HTTP 200; resolved.werkrevisor/cli.py:499-500(HTTP 500 as refusal): intended, reasons in the thread, left open.a356441.tests/test_line_comments.py:161-169(open reply to an open hint): test added ina356441, resolved.werkrevisor/gitea.py: needs the human review.🟡 werkrevisor YELLOW for
a356441, superseded by the result foreeb7934🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Line comment edited before the result comment is posted at
werkrevisor/cli.py:455-456, hint, as a line comment[low] Scenario#33.07 contradicts the code on the cause at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, hint, still open in an earlier line comment[low] Any HTTP 500 is now treated as a refusal of line comments at
werkrevisor/cli.py:499-501, hint, still open in an earlier line commentClassifier
10 of 11 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79357, calls 1, tokens in 77645 / out 786 (reasoning included in out), duration 13.5 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9588, cost 0.0004 USD, finding judge `jev-1.13.0` calls 1, tokens in 1493, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base4 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `a356441634415fc4556e0bc14416ee084559b6e2` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: YELLOW / HUMAN_REVIEW for
a356441634415fc4556e0bc14416ee084559b6e2.The result comment of werkrevisor holds the details.
@@ -446,6 +452,8 @@ def _plan_line_comments(gitea, pr, result, bot_login):and first - LINE_SLACK <= c["line"] <= last + LINE_SLACK), None)if earlier is not None:result.reused[i] = earlier["url"]if not _update_line_comment(gitea, pr, earlier, report.line_comment(f, v), redactor, updated):[medium] Line comment edited before the result comment is posted (werkrevisor)
hint; Jev: real 0.71, severity 2.2 of 3
_plan_line_commentsnow edits earlier line comments while planning, before_post_commentand_publish_reviewrun. If the result comment post or the review submission then fails, the earlier line comment already carries the new text and the "Updated by the review of<head>" note, but no result comment for that head exists. This contradicts the documented order "result comment first". The edit also runs on a path that was side-effect free before. Impact is limited because a rerun finds the text unchanged and does not edit again.Not changed: the same point as the threads at
cli.py:715andcli.py:453; the result comment must know whether the edit failed, so the edit comes first. Left open for the human reviewer.Answer to the review of
a356441(YELLOW):werkrevisor/cli.py:455-456(edit before the result comment): repeated finding, not changed, see its thread.werkrevisor/cli.py:499-501(any HTTP 500 as refusal): repeated finding, answered in its thread 1458.werkrevisor/gitea.py: needs the human review.🟠 werkrevisor ORANGE for
eeb7934, superseded by the result for0fcbb0f🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Reuse also matches a bot comment that is a reply, and edit bypasses the resolver check at
werkrevisor/cli.py:430-432, ORANGE, as a line comment[medium] Scenario#33.07 contradicts the code and tests at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-126, hint, still open in an earlier line comment[medium] Update of a reused line comment happens before the review is published at
werkrevisor/cli.py:455-456, hint, still open in an earlier line commentClassifier
10 of 11 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79351, calls 1, tokens in 77634 / out 786 (reasoning included in out), duration 13.5 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9572, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4777, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base4 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `eeb79347d9f517e1f223a796e382124d42fbac55` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: ORANGE / FINDINGS for
eeb79347d9f517e1f223a796e382124d42fbac55.The result comment of werkrevisor holds the details.
@@ -427,1 +428,3 @@found.append({"path": c.get("path"), "line": c["position"], "url": c.get("html_url") or ""})# A reply of a human is listed in the review that started the thread, but is no hint of the bot.author = (c.get("user") or {}).get("login") or ""if author == bot_login and not c.get("resolver") and (c.get("position") or 0) > 0:[medium] Reuse also matches a bot comment that is a reply, and edit bypasses the resolver check (werkrevisor)
ORANGE; Jev: real 0.68, severity 2.6 of 3
_open_bot_line_commentsnow accepts every comment authored by the bot with an emptyresolver. A bot reply inside a thread started by a human would be treated as a hint and edited. Such a comment is not necessarily a bot line comment of a finding. This is uncertain, since bots do not reply today, but the new edit makes the case destructive, and no test covers it.Not changed.
_open_bot_line_commentsreads only the comments of reviews submitted by the bot, so a bot reply in a thread started by a human, which Gitea lists under the human's review, is never seen. A bot reply in its own thread does not exist: werkrevisor never calls the reply endpoint. Should it ever do so, that reply is bot text in an open thread of the bot. Left open for the human reviewer.Answer to the review of
eeb7934(ORANGE):werkrevisor/cli.py:430-432(a bot reply would be reused and edited): not changed, reasons in the thread; only bot reviews are read, and the bot writes no replies.werkrevisor/cli.py:455-456(edit before publishing): repeated, see thread 1468.werkrevisor/gitea.py: needs the human review.🟠 werkrevisor ORANGE for
0fcbb0f, superseded by the result forc3ad49e🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Edited reused comment is not kept in sync across runs when a 500 is refused at
werkrevisor/cli.py:499-504, ORANGE, still open in an earlier line comment[medium] Withdrawal skips reviewers whose login differs only in case from the request at
werkrevisor/cli.py:533-537, YELLOW, as a line comment[low] Scenario 33.07 contradicts the code for the COMMENT event at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:119-127, hint, still open in an earlier line commentClassifier
10 of 11 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79309, calls 1, tokens in 77601 / out 795 (reasoning included in out), duration 15.1 s, cost 0.1632 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9590, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3422, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base3 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: ORANGE / FINDINGS for
0fcbb0f971e0e63780d1c8eeddf8ffd5ed7e631d.The result comment of werkrevisor holds the details.
@@ -506,3 +531,3 @@exit_code = 0 if cancelled else Verdict.YELLOW.exit_codereturn GateDecision(exit_code, "approved by werkrevisor" + note, held=True)return GateDecision(0, "approved by werkrevisor" + _schedule_merge(config, pr, gitea, redactor, base_ref))scheduled = _schedule_merge(config, pr, gitea, redactor, base_ref)[medium] Withdrawal skips reviewers whose login differs only in case from the request (werkrevisor)
YELLOW; Jev: real 0.54, severity 2.7 of 3
_withdraw_review_requestpasses the logins as Gitea returned them, which is correct. However, the withdrawal is only attempted when the merge note equals exactlyMERGE_SCHEDULEDorMERGE_ALREADY_SCHEDULED; the comparison of rendered note strings is fragile, since_schedule_mergereturns redactor-processed text for other branches. A future change of the note wording silently disables the withdrawal. Prefer a structured return value.Not changed: the notes are the constants
MERGE_SCHEDULEDandMERGE_ALREADY_SCHEDULED, which_schedule_mergereturns as they are; a change of the wording changes the constant and so the comparison.test_a_scheduled_merge_withdraws_the_review_request_of_an_earlier_roundfails if the withdrawal is silently disabled. Left open for the human reviewer.Answer to the review of
0fcbb0f(ORANGE):werkrevisor/cli.py:499-504(reused comment not in sync after a 500): its text is lost, since 0.20.1 reused my reply 1458. From the title: an edit of a reused comment happens before the review POST and is not affected by a later 500; the new findings of a refused review go into the result comment. Not changed.werkrevisor/cli.py:533-537(note strings as condition): not changed, reasons in the thread.werkrevisor/gitea.py: needs the human review.🟠 werkrevisor ORANGE for
c3ad49e, superseded by the result foraa33df4🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] HTTP 500 treated as a line-comment refusal hides real server errors at
werkrevisor/cli.py:499-500, ORANGE, still open in an earlier line comment[medium] Edit of reused line comment happens before the result comment and review are posted at
werkrevisor/cli.py:455-456, ORANGE, still open in an earlier line comment[medium] Missing test for failed withdrawal when line comments hit HTTP 500 on APPROVED event at
tests/test_line_comments.py:37-39, hint, as a line commentClassifier
11 of 11 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79298, calls 1, tokens in 77594 / out 795 (reasoning included in out), duration 13.7 s, cost 0.1631 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9579, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4723, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base4 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `c3ad49e96a6f68fff353d713b737a23b70ac5c88` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: ORANGE / FINDINGS for
c3ad49e96a6f68fff353d713b737a23b70ac5c88.The result comment of werkrevisor holds the details.
@@ -35,3 +35,3 @@def test_finding_without_a_usable_line_stays_in_the_result(self):for name, issues, start, refuse in (("no line", [APP], 0, False), ("other file", ["other.py"], 40, False),("refused by Gitea", [APP], 40, True)):("refused by Gitea", [APP], 40, True),[medium] Missing test for failed withdrawal when line comments hit HTTP 500 on APPROVED event (werkrevisor)
hint; Jev: real 0.78, severity 2.0 of 3
The new HTTP 500 handling in
_publish_reviewretries an APPROVED review without line comments after the refusal. The testtest_finding_without_a_usable_line_stays_in_the_resultusesfinding_runwith the defaultgate="approval"and an ORANGE verdict, so only the COMMENT path is exercised. The retry-without-comments path for GREEN (event != "COMMENT") is not tested with status 500.Fixed in
aa33df4:test_a_green_review_refused_with_http_500_approves_without_line_comments.Answer to the review of
c3ad49e(ORANGE):werkrevisor/cli.py:499-500(HTTP 500 as refusal) andwerkrevisor/cli.py:455-456(edit before publishing): repeated findings, not changed, reasons in the threads 1458 and 1468, left open for the human reviewer.tests/test_line_comments.py:37-39(no test of HTTP 500 on a GREEN review): test added inaa33df4, resolved.werkrevisor/gitea.py: needs the human review.The spec check passes now. From my side PR#33 is ready for the human review.
🟡 werkrevisor YELLOW for
aa33df4, superseded by the result ford9b659a🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Withdrawal skipped when the merge is held but earlier requests remain at
werkrevisor/cli.py:535-536, YELLOW, still open in an earlier line comment[medium] Scenario#33.05 contradicts the test and the code at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:85-97, hint, still open in an earlier line comment[medium] Edit of a reused comment happens before the review is published at
werkrevisor/cli.py:455-456, hint, still open in an earlier line commentClassifier
10 of 11 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79905, calls 1, tokens in 78208 / out 793 (reasoning included in out), duration 13.8 s, cost 0.1643 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9579, cost 0.0004 USD, finding judge `jev-1.13.0` calls 3, tokens in 4858, cost 0.0002 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base4 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `aa33df41af752b9071c4ac675f34c0b67042d028` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
Answer to the review of
aa33df4(YELLOW):werkrevisor/cli.py:535-536(withdrawal skipped while the merge is held): intended, Scenario#33.06; a held merge waits for the author, and the human request stays until the merge is scheduled.werkrevisor/cli.py:455-456(edit before publishing): repeated, see thread 1468.werkrevisor/gitea.py: needs the human review.🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Line comment edited before the result comment is posted at
werkrevisor/cli.py:455-456, ORANGE, still open in an earlier line comment[medium] HTTP 500 treated as refusal of line comments at
werkrevisor/cli.py:499-500, ORANGE, still open in an earlier line comment[low] PR title and document scope are inconsistent at
docs/prs/2026-10-06-PR#33-reused-line-comment-gets-new-text.md:14-17, hint, as a line commentClassifier
11 of 11 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 79818, calls 1, tokens in 78137 / out 786 (reasoning included in out), duration 14.2 s, cost 0.1641 USD (reported), classifier `jev-1.13.0` calls 11, tokens in 9618, cost 0.0004 USD, finding judge `jev-1.13.0` calls 2, tokens in 3041, cost 0.0001 USD, build file judge `jev-1.13.0` calls 1, tokens in 560, cost 0.0000 USD, config: defaults + host + repository (.werkrevisor.toml at base3 answers ignored because of a confidence below [classifier] min_confidence.
dcdcf48).Reviewed head `d9b659a7e9e68faa92b2d62aa3d789cf93eec891` against base `dcdcf48bed8f9fc8a7752b3fddce300c185e8358`, werkrevisor 0.20.1.
werkrevisor: ORANGE / FINDINGS for
d9b659a7e9e68faa92b2d62aa3d789cf93eec891.The result comment of werkrevisor holds the details.
@@ -0,0 +11,4 @@The model words a finding differently in each round and often sharpens it after a fix;the new wording is lost, and the author answers a text that the latest review no longer holds.The reviews of this PR showed two more defects of 0.20.1:[low] PR title and document scope are inconsistent (werkrevisor)
hint
The PR title and the first Problem paragraph name only the reused line comment text. The document also covers reply filtering, withdrawal of review requests and HTTP 500 handling, and presents these as defects "of 0.20.1". The Problem section mixes the three extra defects into one long sentence chain with
;andand, which is hard to read. The unrelated changes make the PR harder to review and revert.ai-reviewer referenced this pull request2026-10-07 10:36:22 +02:00