werkrevisor 0.18.0: statistics service and dashboard Web Component #21
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mi/werkrevisor#21
Reference in New Issue
Block a user
See docs/prs/2026-10-06-PR#21-stats-service-and-dashboard.md.
🤖 Generated with Claude Code
🟡 werkrevisor YELLOW for
15bf85a, superseded by the result for2322b76🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
- YELLOW risky paths changed: .werkrevisor.toml (built in)
- YELLOW possible secrets in added lines: server/src/test/kotlin/de/hoennig/werkrevisor/server/UsageApiIntegrationTest.kt:154 (generic_secret)
- YELLOW PR too large for an AI-only approval: 1280 changed lines (docs and mechanical files not counted) exceed [size] max_changed_lines = 800; please review it yourself
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM skipped (possible secrets in the diff), duration 0.3 s, no model call, cost 0 USD, config: defaults + host + repository (.werkrevisor.toml at base7aee874).Reviewed head `15bf85ace864733a70786416b896843d984285b8` against base `7aee874ec7ceeb3e564d92d9fb68795b7188a084`, werkrevisor 0.17.0.
🟠 werkrevisor ORANGE for
2322b76, superseded by the result for53584ec🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Untrusted record fields are unvalidated and stored at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:45-70, ORANGE, as a line comment[medium] Stats failure message is not redacted before printing at
werkrevisor/cli.py:543-550, YELLOW, as a line comment[medium] Dedup key uses the timestamp, so retries with a new timestamp double count at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageStore.kt:26, YELLOW, as a line comment1 more finding(s) not shown.
Classifier
15 of 15 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 72638, calls 1, tokens in 72139 / out 1183 (reasoning included in out), duration 18.6 s, cost 0.1561 USD (reported), classifier `jev-1.13.0` calls 15, tokens in 12675, cost 0.0005 USD, finding judge `jev-1.13.0` calls 3, tokens in 4348, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base3 answers ignored because of a confidence below [classifier] min_confidence.
7aee874).Reviewed head `2322b763c030af942b91d366009dc3112bc3e628` against base `7aee874ec7ceeb3e564d92d9fb68795b7188a084`, werkrevisor 0.17.0.
werkrevisor: ORANGE / FINDINGS for
2322b763c030af942b91d366009dc3112bc3e628.The result comment of werkrevisor holds the details.
@@ -26,3 +26,3 @@# These modules hold the trust boundary: tokens, redaction, the HOME guard and the Gitea writes."werkrevisor/isolation.py", "werkrevisor/redact.py", "werkrevisor/gitea.py", "werkrevisor/config.py","pyproject.toml", "requirements*.txt", "**/*secret*", "**/*.pem", "**/*.key", ".env*","pyproject.toml", "requirements*.txt", "server/*.gradle.kts", "server/gradle/**", "server/gradlew*", "**/*secret*", "**/*.pem", "**/*.key", ".env*",[low] hard_yellow glob for the server build does not cover subdirectories (werkrevisor)
hint
server/*.gradle.ktsmatches only the top-level build files. If the glob semantics are fnmatch-like, this is fine forserver/build.gradle.ktsandserver/settings.gradle.kts.server/gradle/wrapper/gradle-wrapper.jaris covered byserver/gradle/**. The wrapper jar itself is not shown in the diff, so it is unclear whether it is committed. If it is, it is a binary in the trust boundary and should be verified.@@ -0,0 +42,4 @@private fun bearer(authorization: String?) = authorization?.removePrefix("Bearer ")?.trim()private fun toRecord(body: JsonNode): UsageRecord {[medium] Untrusted record fields are unvalidated and stored (werkrevisor)
ORANGE; Jev: real 0.90, severity 3.0 of 3
toRecordaccepts anyrepo,pr_urlandverdictstrings from any source token holder and stores them. There is no length limit and no URL scheme check onpr_url. The dashboard filters the link scheme client-side, but other embedders ofGET /api/statsget raw values. AlsoasDouble()on a non-numericcost_usdyields 0.0, so a malformed record is silently stored as free. A negative cost is also accepted and lowers the sums. Validate scheme, length and cost range server-side.@@ -0,0 +23,4 @@val totalCostUsd get() = (costUsd ?: 0.0) + jevCostUsdval runKey get() = "$repo#$pr@${headSha ?: ""}@$timestamp"[medium] Dedup key uses the timestamp, so retries with a new timestamp double count (werkrevisor)
YELLOW; Jev: real 0.56, severity 2.8 of 3
runKeyincludestimestamp. Scenario 21.04 says a retried report counts once, which holds only for the identical record. A rerun of the same head commit, for example a manual re-trigger, produces a different timestamp and is counted again. That is arguably correct because each run costs money, but the PR document's wording suggests deduplication by run. Clarify the intent, because it determines whether costs are over- or under-counted.@@ -539,3 +541,3 @@print(report.render_comment(result, pr, pr.base_sha, config.review.max_findings, redactor),file=deps.out, flush=True)report.write_usage(report.usage_record(result, pr, _now()), redactor, deps.out, deps.usage_path)record = report.usage_record(result, pr, _now())[medium] Stats failure message is not redacted before printing (werkrevisor)
YELLOW; Jev: real 0.47, severity 2.6 of 3
The stats client builds its error message from an exception, and
redactor(...)is applied to the whole line, which is fine. But theStatsErrorraised inmainfor a missing token or an invalid URL is printed withoutredactor.WERKREVISOR_STATS_URLcan contain userinfo or a secret query string, but the message only contains fixed text, so there is no leak today. The real risk is thatexecute()now runs the stats report after the usage record is written, with no timeout guard beyond the 5 s client timeout. A slow or unreachable service adds up to 5 s to every run, and a DNS stall can exceed it because urllib's timeout does not cover name resolution. Consider whether this latency is acceptable for the merge gate.🟡 werkrevisor YELLOW for
53584ec, superseded by the result forcc63405🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[high] Usage record written before stats report, but exit code can still change at
werkrevisor/cli.py:543-550, hint, still open in an earlier line comment[medium] Unbounded request body and non-finite numbers at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:54-97, hint, as a line comment[medium] Reads all records of 12 months on every stats request at
server/src/main/kotlin/de/hoennig/werkrevisor/server/Statistics.kt:77-123, hint, as a line comment1 more finding(s) not shown.
Classifier
14 of 15 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 75095, calls 1, tokens in 74449 / out 987 (reasoning included in out), duration 16.0 s, cost 0.1588 USD (reported), classifier `jev-1.13.0` calls 15, tokens in 12839, cost 0.0005 USD, finding judge `jev-1.13.0` calls 3, tokens in 5434, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
7aee874).Reviewed head `53584ecdb19ac15f194f8d70a2a1000b6885e4e9` against base `7aee874ec7ceeb3e564d92d9fb68795b7188a084`, werkrevisor 0.17.0.
werkrevisor: YELLOW / HUMAN_REVIEW for
53584ecdb19ac15f194f8d70a2a1000b6885e4e9.The result comment of werkrevisor holds the details.
@@ -30,3 +30,3 @@secret_scan = true# Test data holds fake passwords; private keys and the real key formats stay flagged there too.secret_scan_exclude = ["tests/**"]secret_scan_exclude = ["tests/**", "server/src/test/**"][low] Test secret scan exclusion widened for a new directory (werkrevisor)
hint
secret_scan_excludenow also coversserver/src/test/**. This is consistent withtests/**, but real secrets in Kotlin tests would no longer be flagged. The change is intentional, so only a reviewer note.@@ -0,0 +74,4 @@private val store: UsageStore,private val clock: Clock,) {fun statistics(withDetails: Boolean): Statistics {[medium] Reads all records of 12 months on every stats request (werkrevisor)
hint; Jev: real 0.67, severity 2.3 of 3
statistics()loads every record since the start of the month 11 months back into memory and filters it several times, on an unauthenticated endpoint. A client that polls/api/statsrepeatedly costs one full table scan per request. The volume of a few thousand records per year makes this tolerable today, but the endpoint has no caching or rate limit.@@ -0,0 +51,4 @@private fun bearer(authorization: String?) = authorization?.removePrefix("Bearer ")?.trim()// Every embedder of /api/stats shows these values, so the service accepts only what a real run can report.private fun toRecord(body: JsonNode): UsageRecord {[medium] Unbounded request body and non-finite numbers (werkrevisor)
hint; Jev: real 0.57, severity 2.5 of 3
POST /api/usageaccepts aJsonNodeof any size from any holder of a source token, and storesrun_keyfromrepo,pr,head_shaandtimestamp.head_shais limited to 200 characters, butcost_statusandmodelare stored without a value check.cost_statusvalues other than the known ones (reported,unknown,none) are stored as is and makecostKnownfalse for acostUsdof null only. This is a limited defect.🟠 werkrevisor ORANGE for
cc63405, superseded by the result for0a55535🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Dedup key allows duplicates when
head_shais absent atserver/src/main/kotlin/de/hoennig/werkrevisor/server/UsageStore.kt:26, ORANGE, still open in an earlier line comment[medium] Stats report failure can crash the run at
werkrevisor/cli.py:543-550, hint, still open in an earlier line comment[medium] NaN or infinite cost handling is untested at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:93, hint, as a line commentClassifier
15 of 15 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 75079, calls 1, tokens in 74438 / out 878 (reasoning included in out), duration 16.0 s, cost 0.1577 USD (reported), classifier `jev-1.13.0` calls 15, tokens in 12800, cost 0.0005 USD, finding judge `jev-1.13.0` calls 3, tokens in 3976, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base2 answers ignored because of a confidence below [classifier] min_confidence.
7aee874).Reviewed head `cc63405f7cb7a24e0356b15de72dc193b1265ed0` against base `7aee874ec7ceeb3e564d92d9fb68795b7188a084`, werkrevisor 0.17.0.
werkrevisor: ORANGE / FINDINGS for
cc63405f7cb7a24e0356b15de72dc193b1265ed0.The result comment of werkrevisor holds the details.
@@ -0,0 +90,4 @@verdict = verdict,model = text("model"),costUsd = number("cost_usd"),costStatus = text("cost_status"),[medium] NaN or infinite cost handling is untested (werkrevisor)
hint; Jev: real 0.74, severity 2.3 of 3
number()checksit in 0.0..MAX_COST_USD, which rejects NaN. ThecostStatustext is not validated against a known set. A source with a valid token can send arbitrarycost_statusstrings, andcostKnowntreats anything other than "unknown" as known whencostUsdis present. This is minor, but unrecognized values silently count toward cost sums.🟠 werkrevisor ORANGE for
0a55535, superseded by the result for6c141ca🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[high] Stats report can fail the review at
werkrevisor/cli.py:543-550, ORANGE, still open in an earlier line comment[medium] Unlimited request body size and sloppy bearer parsing at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:53, ORANGE, still open in an earlier line comment[medium] Dedup key allows double counting at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:26, hint, as a line commentClassifier
15 of 15 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 75647, calls 1, tokens in 75002 / out 886 (reasoning included in out), duration 16.6 s, cost 0.1589 USD (reported), classifier `jev-1.13.0` calls 15, tokens in 12809, cost 0.0005 USD, finding judge `jev-1.13.0` calls 3, tokens in 4539, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
7aee874).Reviewed head `0a55535718b7a40c158df1b90ab270de4cb8489b` against base `7aee874ec7ceeb3e564d92d9fb68795b7188a084`, werkrevisor 0.17.0.
werkrevisor: ORANGE / FINDINGS for
0a55535718b7a40c158df1b90ab270de4cb8489b.The result comment of werkrevisor holds the details.
@@ -0,0 +23,4 @@@RestControllerclass UsageApiController(private val properties: ServerProperties,[medium] Dedup key allows double counting (werkrevisor)
hint; Jev: real 0.35, severity 2.3 of 3
The
runKeyincludes the recordtimestamp, which is set per run by the client. A retried report does produce the same timestamp, but the docs say a second run on the same commit counts again. The key relies on exact string equality ofInstant.toString(). Two equal instants with different fraction precision in the JSON input, such as...:00Zand...:00.000Z, parse to the same Instant, so this is fine. However, the uniqueness constraint is only on the key, and thesince()query parsestsback withInstant.parse. This is acceptable, but a missing test for the retry with a changedhead_shais a gap.🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[medium] Stats failure before the report can crash the run at
werkrevisor/cli.py:544-547, ORANGE, still open in an earlier line comment[medium] Non-finite or oversized numbers cause 500 instead of 400 at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:99-102, ORANGE, as a line comment[medium] Gradle wrapper jar not covered by hard_yellow at
.werkrevisor.toml:28, YELLOW, still open in an earlier line commentClassifier
15 of 15 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 78530, calls 1, tokens in 78092 / out 788 (reasoning included in out), duration 16.1 s, cost 0.1641 USD (reported), classifier `jev-1.13.0` calls 15, tokens in 12846, cost 0.0005 USD, finding judge `jev-1.13.0` calls 3, tokens in 4087, cost 0.0002 USD, config: defaults + host + repository (.werkrevisor.toml at base3 answers ignored because of a confidence below [classifier] min_confidence.
7aee874).Reviewed head `6c141ca5eed6afc917957d1726c61b9e5a5c1a25` against base `7aee874ec7ceeb3e564d92d9fb68795b7188a084`, werkrevisor 0.17.0.
werkrevisor: ORANGE / FINDINGS for
6c141ca5eed6afc917957d1726c61b9e5a5c1a25.The result comment of werkrevisor holds the details.
@@ -0,0 +96,4 @@headSha = headSha,verdict = verdict,model = text("model"),costUsd = number("cost_usd"),[medium] Non-finite or oversized numbers cause 500 instead of 400 (werkrevisor)
ORANGE; Jev: real 0.78, severity 3.0 of 3
pris checked withcanConvertToInt, buttimestampparsing catches onlyDateTimeParseException.store.addcan also throw on malformed input, e.g. arepoormodelvalue containing odd characters is fine, but a JSON body that is not an object (an array or string) makesbody.get(name)return null andpris reported as "missing" only. A non-JSON body yields Spring's default 400. The main gap:jevCostUsdis the sum of three values each up to 100 andcostUsdis checked on its own, so total cost per record can reach 400 USD, contradicting the documented 0 to 100 USD bound.edgecases later