# PR#37: the usage record never ends a review, and the statistics bound the cost sums

> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

## The Problem

Two follow-ups of PR#21, the statistics service:

- werkrevisor writes the usage record after the review, to stdout and to `build/werkrevisor/usage.jsonl`.
  An error there, e.g. a full disk, ends the finished review with a traceback,
  so the gate check fails although the review itself is done and published.
- The statistics service bounds each cost value of a usage record at 100 USD, but not the sums it stores:
  the LLM cost plus the excerpt cost, and the four Jev costs.
  A record with values below the bound each can store a review cost of up to 400 USD.

## Non-Goals

- No change of the bound of 100 USD, which is far above any real review.
- No retry of a failed write: the usage record is statistics, not part of the review.

## The Scenarios

### Feature: the usage record never ends a review

#### Scenario#37.01: a failed write of the usage record keeps the result of the review

So that a full disk or a missing directory does not fail the gate check of a finished review.

- **Given** the usage file `build/werkrevisor/usage.jsonl` cannot be written
- **When** werkrevisor finishes a GREEN review
- **Then** the review ends with exit code 0
  - **and** werkrevisor still reports the usage record to the statistics service

##### Verified by

- [test_flow: test_a_failed_write_of_the_usage_record_keeps_the_result](../../tests/test_flow.py)

#### Scenario#37.02: a usage record that cannot be built keeps the result of the review

So that a defect in the statistics code does not fail the gate check of a finished review.

- **Given** the usage record of a run cannot be built
- **When** werkrevisor finishes a GREEN review
- **Then** the review ends with exit code 0
  - **and** werkrevisor neither writes nor reports a usage record

##### Verified by

- [test_flow: test_a_usage_record_that_cannot_be_built_keeps_the_result](../../tests/test_flow.py)

### Feature: the statistics service bounds the stored costs

#### Scenario#37.03: a record whose costs add up beyond the bound is refused

So that one record cannot distort the costs of the dashboard.

- **Given** a usage record with `<costs>`
- **When** werkrevisor reports it to the statistics service
- **Then** the service refuses it with HTTP 400

| costs |
|---|
| `cost_usd` 60 and `excerpt_cost_usd` 60 |
| `classifier_cost_usd`, `finding_judge_cost_usd`, `db_schema_judge_cost_usd` and `path_judge_cost_usd` 30 each |

##### Verified by

- [UsageApiIntegrationTest: a record whose costs add up beyond the bound is refused (Scenario#37.03)](../../statistics/src/test/kotlin/de/hoennig/werkrevisor/statistics/UsageApiIntegrationTest.kt)

## The Solution

- werkrevisor builds and writes the usage record inside a guard: an error goes to stderr,
  and the review keeps its verdict and exit code;
  a record that cannot be built is neither written nor reported, a record that cannot be written is still reported.
- The statistics service checks the sum of `cost_usd` and `excerpt_cost_usd`, and the sum of the four Jev costs,
  against the bound; without `cost_usd`, the review cost stays unknown,
  and `excerpt_cost_usd` is still checked on its own, which it was not before.
- werkrevisor 0.20.6.

## Prerequisite PRs

- PR#36: the result comment holds the findings until the line comments exist.