# PR#66: Hints on Documentation Stay in the Result Comment

> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

## Related Links

- [PR#65](https://git.javagil.de/mi/werkrevisor/pulls/65): a dry run writes the model prompt, used for the measurement below.

## The Problem

Many PRs need several review rounds, each for new small hints, e.g. werkprobe PR#12 with 5 rounds.
Each hint is a line comment, and on a GREEN PR every open line comment holds the merge.
So the coding agent answers or fixes each hint, and each fix starts a new round with new hints.

A frontier model, `anthropic/claude-opus-5.5`, rated 42 answered hints of werkrevisor PRs #32 and #48 to #64.
It saw the prompt from which the economy model wrote each hint, rebuilt with `tools/dry-run --prompt-out`
from a throwaway PR per reviewed head in `mi/werkrevisor-bench`.
It rated 14 hints as useful and 28 as noise.

| Hints (the third row is part of the first) | useful | noise |
|---|---|---|
| on a documentation file | 6 | 19 |
| on code | 8 | 9 |
| of these: PR document vs change summary (`[doc_check]`) | 1 | 4 |

The review round, Jev's values for "real" and the severity of the model did not separate useful hints from noise.
The three `[doc_check]` hints below 0.7 (0.62, 0.65, 0.69) were noise; the useful one had 0.77.

## Non-Goals

- No change for hints on code, for YELLOW, ORANGE and RED findings, and for the writing rules.
- No Jev question that rates whether a hint is worth a round: 42 ratings with a confidence of mostly 0.5 to 0.6
  are too few for it.

## The Scenarios

### Feature: a hint on documentation does not cost a review round

#### Scenario#66.01: a hint on a documentation file is no line comment

So that a small remark on wording does not hold the merge.

- **Given** a GREEN review with automerge
- **And** its only finding is a hint on a file of the category `docs`
- **When** werkrevisor publishes the review
- **Then** the hint stands in the result comment with its full text
- **And** it is no line comment
- **And** the merge is scheduled

##### Verified by

- [test_doc_hints.DocHintsTest.test_a_hint_on_documentation_is_no_line_comment](../../tests/test_doc_hints.py)

#### Scenario#66.02: a hint on code stays a line comment

So that the hints that most often name a real problem still get an answer.

- **Given** a GREEN review with automerge
- **And** its only finding is a hint on a file outside the category `docs`
- **When** werkrevisor publishes the review
- **Then** the hint is a line comment that holds the merge

##### Verified by

- [test_doc_hints.DocHintsTest.test_a_hint_on_code_stays_a_line_comment](../../tests/test_doc_hints.py)

#### Scenario#66.03: the old behaviour stays selectable

So that a repository that wants every hint answered can keep it.

- **Given** `[review] doc_hint_line_comments = true`
- **And** a GREEN review whose only finding is a hint on a file of the category `docs`
- **When** werkrevisor publishes the review
- **Then** the hint is a line comment that holds the merge

##### Verified by

- [test_doc_hints.DocHintsTest.test_doc_hint_line_comments_restores_the_line_comment](../../tests/test_doc_hints.py)

#### Scenario#66.04: a weak contradiction with the change summary is no hint

So that the PR document check reports only contradictions from 0.7 on.

- **Given** the default config
- **When** Jev rates a section of the PR document at 0.65 as a contradiction of the change summary
- **Then** the result has no hint for that section

##### Verified by

- [test_doccheck.DocCheckTest.test_a_contradiction_below_hint_min_is_no_hint](../../tests/test_doccheck.py)

## The Solution

- `[review] doc_hint_line_comments`, default `false`: a GREEN finding on a file of the category `docs`
  stays in the result comment and does not count as an open hint.
- `[doc_check] hint_min` from 0.6 to 0.7.
- A hint of the PR document check is a hint on documentation too, so it stays in the result comment.
  It comes back in each round while Jev rates its section as before;
  the resolved fingerprints of PR#50 apply only with `doc_hint_line_comments = true`.
- The README describes both; the Werkbaum records the measurement and the next ideas under `#model.rounds`.
- Version 0.31.0.