# PR#46: Jev judges each changed file by the risk categories of a template

> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

## The Problem

werkrevisor shall separate the PRs that need a human from the others,
so that the others merge without a human; finding defects is secondary.
With `[path_judge] scope = "all"`, Jev asks two questions per changed file:
does it weaken the checks, and does it touch security.
On the 25 merged hs.hsadmin.ng PRs #281 to #316, these found 7 of the 9 PRs that needed a human, 8 in the run of this PR.

Two questions can miss risks outside security,
e.g. a changed transaction boundary, a migration of existing data, or a request to a third party in a frontend.
The reason "touches security" also does not tell the developer what to check or how to lower the risk.
And which risks matter depends on the ecosystem of the repository: a Spring Boot backend has other ones than a React frontend.

The design of 2026-10-08 defines 15 risk categories for JVM backends and 15 for TypeScript frontends.
Later each repository keeps its categories in its `.werkrevisor.toml`; until then werkrevisor keeps them as templates.

## Non-Goals

- No change summary of an economy model and no comparison of the PR document with it; that is the next PR.
- No PR-wide classification over all files, and no RED from Jev.
- No categories in `.werkrevisor.toml` yet; the host config selects the template.
- No grouping of related changes across files: a unit is still one file with its diff hunks.
- No change of the questions for judged build paths, of `weakens_checks` and `security`, or of the PR document questions.
- No change of a deployed host config: the reviewed repositories on git.javagil.de are Python, and hs.hsadmin.ng waits.

## The Scenarios

### Feature: Jev judges each changed file by the risk categories of a template

#### Background

- **Given** `[path_judge] judge = "jev"` and `scope = "all"`
  - **and** `[path_judge] risk_template` names a template of werkrevisor

#### Scenario#46.01: a likely risk gives YELLOW and names the category

So that the developer knows what a human checks, and can revert a change that the task does not need.

- **Given** a category with `action = "yellow"`
- **When** Jev rates a changed file in this category at or above its `yellow_min`
- **Then** the review is YELLOW
  - **and** the reason names the category, the file and the value
  - **and** the reason says to revert the change if the task does not need it

##### Verified by

- [test_risk: test_a_likely_risk_gives_yellow_and_names_the_category](../../tests/test_risk.py)

#### Scenario#46.02: a quality category gives only a hint

So that categories like performance or accessibility inform the author without holding the merge for a human.

- **Given** a category with `action = "hint"`
- **When** Jev rates a changed file in this category at or above its `yellow_min`
- **Then** the category is a hint with the file and the value
  - **and** it alone keeps the review GREEN

##### Verified by

- [test_risk: test_a_quality_category_gives_only_a_hint](../../tests/test_risk.py)

#### Scenario#46.03: a very likely risk gives ORANGE

So that the author lowers the risk with further commits before a human spends time on the PR.

- **Given** a category with `action = "yellow"` and an `orange_min`
- **When** Jev rates a changed file in this category at or above its `orange_min`
- **Then** the review is ORANGE
  - **and** the reason names the category, the file and the value

##### Verified by

- [test_risk: test_a_very_likely_risk_gives_orange](../../tests/test_risk.py)

#### Scenario#46.04: a test file gets only the categories for tests

So that a test that names a role does not count as a change of the access rule.

- **Given** a category without `test` in its `kinds`
- **When** a PR changes a test file
- **Then** Jev does not answer this category for the test file

##### Verified by

- [test_risk: test_a_test_file_gets_only_the_categories_for_tests](../../tests/test_risk.py)

#### Scenario#46.05: the question about security stays besides the categories

So that the triage finds at least the PRs it found before: the narrower categories alone missed 2 critical PRs.

- **When** Jev rates a main-code file at or above `yellow_security_min` for security
- **Then** the review is YELLOW with the reason `files_touch_security`, as before

##### Verified by

- [test_risk: test_the_security_question_stays_besides_the_categories](../../tests/test_risk.py)

#### Scenario#46.06: without a template, the two measured questions stay

So that the repositories without a fitting template, e.g. Python ones, keep their calibrated triage.

- **Given** no `[path_judge] risk_template`
- **When** a PR changes a main code file
- **Then** Jev answers `weakens_checks` and `security`, as before

##### Verified by

- [test_risk: test_without_a_template_the_two_measured_questions_stay](../../tests/test_risk.py)

#### Scenario#46.07: an unknown template is a config error

So that a typo in the config does not silently disable the categories.

- **Given** `[path_judge] risk_template = "no-such-template"`
- **When** werkrevisor loads the config
- **Then** the run ends with a config error that names the known templates

##### Verified by

- [test_risk: test_an_unknown_template_is_a_config_error](../../tests/test_risk.py)

#### Scenario#46.08: the usage record logs each category value

So that the thresholds can be calibrated on real PRs.

- **Given** Jev judged a changed file by the categories of a template
- **When** werkrevisor writes the usage record
- **Then** the record holds the value of each category for this file

##### Verified by

- [test_risk: test_the_usage_record_logs_each_category_value](../../tests/test_risk.py)

## The Solution

- A new host-only key `[path_judge] risk_template`; it needs `scope = "all"`.
- The templates are TOML files in `werkrevisor/risk/`: `jvm-spring-backend` and `typescript-frontend`, 15 categories each.
- Each category has `name`, `title`, a `noul` question, `kinds` (`main`, `test`, default `main`), `yellow_min`,
  an optional `orange_min`, and `action` (`yellow` or `hint`).
- The categories come on top of `weakens_checks` and `security`; the reason `risk_<category>` names category, files and values.
- REST API compatibility, performance and architecture, and in the frontend also accessibility, start with `action = "hint"`:
  agreed changes, e.g. a renamed API property, need no human.
- All categories start with `yellow_min = 0.95` and, unless a hint, `orange_min = 0.99`.
- Jev answers all questions of a file in one request, so the cost depends on the diff size, not on the number of categories.
- werkrevisor 0.22.0.

## Measurement

On 2026-10-08, Jev judged each changed file outside the docs of the 25 merged hs.hsadmin.ng PRs #281 to #316,
with the questions of this PR; a file below `src/test/` counted as a test.
The labels are those of 2026-10-06: 9 PRs need a human, #313 is borderline, 15 do not.
Each run cost about 0.03 USD for 650,000 input tokens.

| questions | critical found | others YELLOW or worse |
|---|---|---|
| `weakens_checks` and the categories, without `security` | 6 of 9 | 0 of 15 |
| `weakens_checks` and `security` | 8 of 9 | 1 of 15 (#302) |
| with the categories from 0.93 | 8 of 9 | 3 of 15 |
| with the categories from 0.95 | 8 of 9 | 1 of 15 |

Without `security`, #281 and #288 were lost: authorization reached only 0.92 and 0.58 there.
From 0.95 on, the categories find no further PR, but they name the risk:
#283 authentication and concurrency, #289 also authorization, #306 authentication and authorization,
and #310 authorization and schema migration.
They also mark the borderline #313 for error handling.
#282 stays missed; its highest category was concurrency with 0.82.
#302 reached 0.93 for security in this run, 0.92 on 2026-10-06.
No category reached 0.99, so `orange_min` is not calibrated yet.

## Follow-up PRs

- An economy model summarizes the diff without the PR document; Jev compares the PR document with this summary.
  Its contradictions are the first concrete issues.
- A PR-wide classification over the consolidated signals; ORANGE for a likely high risk or a concrete issue,
  RED for a likely high risk and a concrete issue in the same file or category.
- The categories move into `.werkrevisor.toml` of each repository.

## Prerequisite PRs

- PR#45: werkrevisor 0.21.5, merged.