werkrevisor/docs/prs/2026-10-06-PR#31-excerpt-selection.md

# PR#31: a cheap model selects the excerpt of a file too large for the review model

> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

## The Problem

Since werkrevisor 0.19.2, a file beyond the full-file limits goes to the review model as excerpt:
each changed line with 40 lines before and after.
The lines that matter for a change are often elsewhere, though:
the function that the changed code calls, the callers of a changed function, a constant at the top of the file.
A fixed window around each change does not find them.

The cheap models of the model comparison of 2026-10-06 found too few defects to review on their own,
but reading a file and naming the relevant lines is a simpler task.

## Non-Goals

- No text of the cheap model reaches the review model: it returns line numbers only, which werkrevisor checks.
- No selection for files that go in full: they need none.
- No default switch-on before a measurement shows that the review model finds the known defects with the selected excerpt.

## The Scenarios

### Feature: a cheap model selects the excerpt

#### Scenario#31.01: the excerpt holds the changes and the lines that the cheap model selects

So that the review model sees what the change depends on, not only what surrounds it.

- **Given** `[excerpt] select = "model"`
  - **and** a PR changes line 150 of a file beyond the full-file limits
  - **and** the cheap model selects the lines 10 to 20 of that file
- **When** werkrevisor reviews the PR
- **Then** the review model gets an excerpt of the lines 10 to 20 and 145 to 155

##### Verified by

- [test_excerpt: test_the_excerpt_holds_the_changes_and_the_selected_lines](../../tests/test_excerpt.py)
- [test_excerpt: test_the_cheap_model_gets_the_diff_and_the_numbered_file](../../tests/test_excerpt.py)

#### Scenario#31.02: an unusable selection falls back to the fixed window

So that a failure of the cheap model never costs the review its context.

- **Given** `[excerpt] select = "model"`
- **When** the cheap model `<fails>`
- **Then** the excerpt holds each changed line with `[review] excerpt_context_lines` lines before and after,
  as without selection

| fails |
|---|
| answers with an error or not in time |
| answers with something that is no list of line ranges |
| selects more lines than `[excerpt] max_lines` |

##### Verified by

- [test_excerpt: test_an_unusable_selection_falls_back_to_the_fixed_window](../../tests/test_excerpt.py)
- [test_excerpt: test_failures_give_none_and_name_the_cause](../../tests/test_excerpt.py)
- [test_excerpt: test_anything_but_valid_line_numbers_is_refused_as_a_whole](../../tests/test_excerpt.py)

#### Scenario#31.03: the selection costs are part of the run

So that the dashboard shows what the selection costs.

- **Given** the cheap model selected the excerpt of one file
- **When** werkrevisor writes the usage record
- **Then** the record holds the model, the calls, the tokens and the costs of the selection
  - **and** the statistics service counts the costs as LLM costs

##### Verified by

- [test_excerpt: test_the_usage_record_holds_the_selection](../../tests/test_excerpt.py)
- [UsageApiIntegrationTest: the excerpt selection counts as LLM cost](../../server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt)

### Feature: a too large prompt keeps its review

#### Scenario#31.04: a prompt beyond the input limit first loses the full files, then the excerpts

So that a large PR still gets an AI review, as before werkrevisor 0.19.2 raised the full-file limits.

- **Given** the prompt with the full files exceeds the input limit of the review model
- **When** werkrevisor reviews the PR
- **Then** the review model gets `<context>`
  - **and** a hint says that the context was reduced

| fits the input limit | context |
|---|---|
| with excerpts | excerpts with the fixed window instead of the full files |
| only without excerpts | the diff only |

##### Verified by

- [test_fullfiles: test_a_too_large_prompt_first_loses_the_full_files_then_the_excerpts](../../tests/test_fullfiles.py)

## The Solution

- `[excerpt]`, a host-only section, sets `select = "context"` (default) or `"model"`, the model, its prices and limits.
- `werkrevisor/excerpt.py` asks the model through the OpenRouter chat API for JSON line ranges
  and refuses an answer as a whole unless it holds only valid line numbers within `max_lines`.
- The excerpt holds each changed line with `context_lines` lines and the selected ranges;
  a failed selection gets the fixed window of `[review] excerpt_context_lines` and the hint `excerpt_selection_failed`.
- No selection without a model call, for a file above 400000 characters, or for a diff with a secret;
  with a secret, werkrevisor collects no full files at all, as before.
- A malformed usage field of OpenRouter makes the selection cost an estimate instead of ending the review.
- The usage record and the comment name the model, calls, tokens and cost of the selection;
  the statistics service counts the cost as LLM cost.
- The README diagram shows the selection as optional step between the context and the review model.
- The host configs of werkrevisor, werkprobe and werkator switch the selection on at the deployment;
  hsadmin-ng keeps the fixed window, since the selection sends code to another provider.
- A prompt beyond the input limit gets excerpts with the fixed window instead of the full files, and then the diff only,
  with a hint `context_reduced`; the selection is not repeated, so the retry costs no more,
  and the usage record marks a selection that the reduction dropped as `excerpt_discarded`.
  PR#29 raised the total limit of the full files to 160000 characters but forgot `token_count_factor = 1.55` in its estimate,
  so the first review of this PR found no room for the model at all.
- werkrevisor 0.20.0.

## Measurement

On 2026-10-06, Claude Sonnet 5.5 reviewed 4 heads with 10 known real findings, copied into the private repositories
`mi/werkrevisor-bench` and `mi/hsadmin-ng-bench`, since a dry run always reviews the current head of a pull request.
`max_full_file_chars = 8000` made each changed source file an excerpt; `[review] max_findings` stayed 3.

| head (known findings) | full files | fixed window | GPT-6 Luna |
|---|---|---|---|
| werkrevisor PR#9 at eb2823f (2) | too large | 1 | 1 |
| werkrevisor PR#15 at b5bb590 (2) | 0 | 0 | 0 |
| hs.hsadmin.ng PR#302 at 6de8bd6 (2) | 1 | 0 | 1 |
| hs.hsadmin.ng PR#305 at aed0d4d (4) | too large | 0 | 0 |

The selection never found less than the fixed window;
on PR#302 it gave Sonnet the bypassable BIC check, which the fixed window missed and the full files showed.
It cost 0.002 to 0.005 USD and 45 to 70 seconds per review.
DeepSeek V4 Flash answered 7 of 10 selections of PR#305 with no JSON and took 296 seconds, so it is no candidate.
The 4 findings of PR#305 contradict the documentation; with 3 findings per review, Sonnet reported code findings instead.
The sample is small, so the default stays the fixed window.

## Additional Changes

- Follow-up of PR#30, which the bot merged at GREEN before its hints were addressed:
  the docstring of the approval gate no longer says that RED requests changes,
  the review event has one branch less, and a test no longer repeats an assertion.

## Prerequisite PRs

- PR#29: larger full files and excerpts.
- PR#30: a RED review no longer blocks later commits.