werkrevisor/docs/prs/2026-10-06-PR#21-stats-service-and-dashboard.md

# PR#21: a statistics service and a dashboard Web Component

> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

## The Problem

werkrevisor is a command-line tool without state of its own.
Each run writes a usage record with its costs, but only as a CI artifact, and Werkator prunes old artifacts:
on 2026-10-06, 7 usage files were left of more than 50 reviews.
Nobody can tell what the reviews cost per day, week or month, or what one pull request costs on average.

werkrevisor shall become an open-source tool that works with any CI and with Gitea and Forgejo.
The long-term target is a service with an API; the command-line tool stays, also for local reviews.
The decisions on language, place, store and Web Component are listed under Open Questions.
This PR takes the first small step towards it: a service that collects the usage records and shows them.

## Non-Goals

- No reviews in the service yet: the command-line tool still runs the review, in the CI or locally.
- No integration into the Werkator GUI yet; the dashboard is a Web Component, so that this can follow later.
- No import of earlier reviews: the statistics start with the first reported run.
- No deployment in this PR; it follows after the merge, see Follow-up PRs.

## The Scenarios

### Feature: the command-line tool reports each run to the service

#### Scenario#21.01: a run reports its usage record when a service is configured

So that every review lands in the statistics, wherever it runs.

- **Given** `WERKREVISOR_STATS_URL` and `WERKREVISOR_STATS_TOKEN` are set
- **When** werkrevisor finishes a review
- **Then** the service has received the usage record of the run
  - **and** the record holds the link to the pull request

##### Verified by

- [test_stats_client.StatsClientTest.test_run_reports_its_usage_record](../../tests/test_stats_client.py)

#### Scenario#21.02: a failed report never changes the review

So that the statistics can never block or break a review.

- **Given** the service `<state>`
- **When** werkrevisor finishes a review
- **Then** the verdict and the exit code are the same as without a service
  - **and** the log names the failed report

| state |
|---|
| is not configured |
| cannot be reached |
| answers HTTP 500 |
| refuses the token |

##### Verified by

- [test_stats_client.StatsClientTest.test_failed_report_never_changes_the_review](../../tests/test_stats_client.py)
- [test_stats_client.HttpClientTest](../../tests/test_stats_client.py)

### Feature: the service collects the usage records

#### Scenario#21.03: only a known source may report

So that nobody can falsify the statistics.

- **Given** the service knows the source token of werkprobe
- **When** a client reports a usage record with `<token>`
- **Then** the service answers `<status>`

| token | status |
|---|---|
| the source token of werkprobe | 201, record stored |
| an unknown token | 401, nothing stored |
| no token | 401, nothing stored |

##### Verified by

- [UsageApiIntegrationTest: only a known source may report](../../server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt)

#### Scenario#21.04: a record reported twice counts once

So that a retried report does not double the costs.

- **Given** the service has stored the record of a run
- **When** a client reports the same record again
- **Then** the statistics count the run once

A second run on the same commit, e.g. a manual rerun, has its own time; it cost money again, so it counts as another review.

##### Verified by

- [UsageApiIntegrationTest: a record reported twice counts once](../../server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt)

### Feature: the statistics answer what the reviews cost

#### Background

- The costs of a run are the model costs plus the costs of the three Jev uses.
- A run with unknown model costs counts as a review, but not in the sum.

#### Scenario#21.05: totals for the last 24 hours, 7 days and 30 days

So that the operator sees the current spending at a glance.

- **Given** the service has stored reviews of the last 40 days
- **When** anyone asks for the statistics
- **Then** they get for 24 hours, 7 days and 30 days the sum in USD, the number of reviews,
  the number of pull requests, the costs per pull request and the number of reviews with unknown costs

##### Verified by

- [UsageApiIntegrationTest: totals for the last 24 hours, 7 days and 30 days](../../server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt)

#### Scenario#21.06: sums per day, week and month

So that the operator sees the development of the costs.

- **Given** the service has stored reviews of several months
- **When** anyone asks for the statistics
- **Then** they get the sums per day of the last 30 days, per week of the last 12 weeks and per month of the last 12 months

##### Verified by

- [UsageApiIntegrationTest: sums per day, week and month](../../server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt)

#### Scenario#21.07: details only with a read token

So that the names of private repositories and their pull requests stay private.

- **Given** the service has stored reviews of the repository `team/app`
- **When** someone asks for the statistics `<with>`
- **Then** the answer `<holds>`

| with | holds |
|---|---|
| a read token | the totals, the sums per repository, model and verdict, and the last 20 reviews with links |
| no token | only the totals and the sums per period |

##### Verified by

- [UsageApiIntegrationTest: details only with a read token](../../server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt)

### Feature: the dashboard is a Web Component

#### Scenario#21.08: the service delivers the dashboard as a Web Component and an app around it

So that the dashboard can be embedded later in other GUIs, e.g. in Werkator or werkregie.

- **Given** the service runs
- **When** a browser opens the start page of the service
- **Then** the page embeds the element `<werkrevisor-dashboard>` from `/werkrevisor-dashboard.js`
  - **and** the page hands a read token, if the user entered one, to the element

##### Verified by

- [UsageApiIntegrationTest: the start page embeds the dashboard Web Component](../../server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt)
- manual check in a browser, see [Measurement](#measurement)

## The Solution

- The service is a Kotlin Spring Boot application in `server/`, built with Gradle, as Werkator is;
  the review core stays Python, because it builds on PR-Agent, and talks to the service only by HTTP and JSON.
  - `POST /api/usage` stores a usage record in SQLite; a source token from the server config is required.
    The service refuses implausible values: no `owner/name` repository, a link that is no http(s) URL,
    texts above 200 or links above 500 characters, an unknown verdict or cost status, a missing commit id,
    and costs that are no number or outside 0 to 100 USD.
    It refuses a body above 16 KB or without a length, and a token without the `Bearer` scheme.
  - `GET /api/stats` returns the statistics; a read token adds the details.
  - `GET /` delivers the app, `GET /werkrevisor-dashboard.js` the Web Component.
- The Web Component `<werkrevisor-dashboard>` is plain JavaScript in one file, a Custom Element with Shadow DOM,
  without a frontend build, so that Werkator, werkregie or any page can embed it.
- The command-line tool posts its usage record to `WERKREVISOR_STATS_URL` with `WERKREVISOR_STATS_TOKEN`;
  a failure is logged and changes nothing else; after 8 seconds the run stops waiting for the report.
  The URL must use https, except on localhost, so that a source token never travels in clear text.
- The usage record gets the field `pr_url`, so that the dashboard can link the pull request.
- The Gradle wrapper is the one of Werkator; `distributionSha256Sum` pins the Gradle distribution.
- werkrevisor 0.18.0.

## Open Questions

- [x] Which language for the service?
  Kotlin with Spring Boot, as Werkator; no more Python in the backend.
- [x] Where does the service live?
  In `server/` of this repository, so that a change of the usage record and of the service go into one PR.
- [x] Which store?
  SQLite: a few thousand records per year, a backup is a file copy.
- [x] How is the Web Component built?
  Plain JavaScript without a build step, as Werkator's own frontend.

## Measurement

On 2026-10-06, the service ran locally with the 180 usage records of the earlier dry runs and reviews.
Without a token, the dashboard showed the totals and the sums per period; with the read token,
also the sums per repository, model and verdict and the 20 recent reviews with their links.

## Follow-up PRs

- Deployment of the service on mih09 and the report from the Werkator builds.
- Integration of the Web Component into the Werkator GUI.
- Hit rates and calibration of Jev in the dashboard.
- Reviews in the service, triggered by webhooks, with a read-only Git mirror for the agent.