werkrevisor 0.18.2: input and output tokens per model in the dashboard #23
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mi/werkrevisor#23
Reference in New Issue
Block a user
See docs/prs/2026-10-06-PR#23-token-statistics.md.
🤖 Generated with Claude Code
🟡 werkrevisor YELLOW for
78ffc42, superseded by the result for912d505🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Jev token sum can mask partial data at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:100-118, hint, as a line comment[medium] Document contradicts the code on the usage record at
docs/prs/2026-10-06-PR#23-token-statistics.md:17-20, hint, as a line comment[low] Measurement section is unclear and unverifiable at
docs/prs/2026-10-06-PR#23-token-statistics.md:70-74, hint, as a line commentClassifier
8 of 8 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 37129, calls 1, tokens in 36323 / out 716 (reasoning included in out), duration 12.4 s, cost 0.0798 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6260, cost 0.0003 USD, finding judge `jev-1.13.0` calls 2, tokens in 2762, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
d7a8ffe).Reviewed head `78ffc42065a72877d6788a336ee65b4500c8fd3f` against base `d7a8ffe825bfbba751916460486f905af1ad519a`, werkrevisor 0.18.1.
werkrevisor: YELLOW / HUMAN_REVIEW for
78ffc42065a72877d6788a336ee65b4500c8fd3f.The result comment of werkrevisor holds the details.
@@ -0,0 +14,4 @@which Werkator prunes after a while.The usage record of every run already holds the token counts.## Non-Goals[medium] Document contradicts the code on the usage record (werkrevisor)
hint; Jev: real 0.88, severity 1.4 of 3
The Non-Goals state "No change of the command-line tool or of the usage record", but the PR changes the
UsageRecorddata class and its stored columns (llm_input_tokens,llm_output_tokens,jev_input_tokens). The statement is misleading; it should say the report format sent by the tool is unchanged.@@ -0,0 +67,4 @@the Web Component shows them in the table per model, with the ratio.- werkrevisor 0.18.2.## Measurement[low] Measurement section is unclear and unverifiable (werkrevisor)
hint
The claim that the model table "showed 113 : 1 ... each in one line without wrapping" refers to a local run with 187 records that nobody can reproduce, and Scenario#23.02 is verified only by it. The ratios (e.g. "1 : 1" for DeepSeek) and the "about 94 %" claim in the Problem section have no derivation shown.
@@ -88,6 +97,12 @@ class UsageApiController(require(SHA.matches(headSha)) { "head_sha must be a hex commit id" }val costStatus = text("cost_status")require(costStatus == null || costStatus in COST_STATUSES) { "cost_status must be one of $COST_STATUSES" }val jevTokens =[medium] Jev token sum can mask partial data (werkrevisor)
hint; Jev: real 0.55, severity 2.1 of 3
jevTokensis the sum of the three Jev token fields, and is null only if all are absent. A run reporting only some of them is stored as a complete sum with no distinction. The sum stays unbounded byMAX_TOKENS, though each part is limited; the total is only up to 30M, which fits in a Long, so this is a minor accuracy concern.🟡 werkrevisor: YELLOW / HUMAN_REVIEW
Change kind:
mixed.Reasons
Findings
[medium] Token sum can overflow Long at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageApiController.kt:100-118, hint, still open in an earlier line comment[low] Fragile schema migration with a concurrent start at
server/src/main/kotlin/de/hoennig/werkrevisor/server/UsageStore.kt:61-64, hint, as a line comment[low] Measurement section is hard to verify at
docs/prs/2026-10-06-PR#23-token-statistics.md:72-76, hint, still open in an earlier line commentClassifier
8 of 8 units passed all questions (1 PR document(s)).
Usage: model `openrouter/anthropic/claude-sonnet-5.5`, LLM done, tokens counted before the call 37216, calls 1, tokens in 36400 / out 676 (reasoning included in out), duration 11.7 s, cost 0.0796 USD (reported), classifier `jev-1.13.0` calls 8, tokens in 6323, cost 0.0003 USD, finding judge `jev-1.13.0` calls 1, tokens in 1505, cost 0.0001 USD, config: defaults + host + repository (.werkrevisor.toml at base1 answer ignored because of a confidence below [classifier] min_confidence.
d7a8ffe).Reviewed head `912d505b174eca93f0cd4465eb88ddc8133ba90a` against base `d7a8ffe825bfbba751916460486f905af1ad519a`, werkrevisor 0.18.1.
werkrevisor: YELLOW / HUMAN_REVIEW for
912d505b174eca93f0cd4465eb88ddc8133ba90a.The result comment of werkrevisor holds the details.
@@ -53,2 +58,4 @@)jdbc.execute("create index if not exists usage_ts on usage (ts)")// A database of werkrevisor 0.18.1 or earlier lacks these columns; its records keep null token counts.val columns = jdbc.queryForList("select name from pragma_table_info('usage')", String::class.java).toSet()[low] Fragile schema migration with a concurrent start (werkrevisor)
hint
The columns are checked first and then added with
alter table. If two instances start against the same SQLite file, one can hit "duplicate column name" and fail at startup. This is a narrow scenario.Assessment of the review hints that were not implemented (head
912d505):UsageApiController.kt:100-118): not a defect. Each of the three Jev token counts is limited to 10 million, so their sum is at most 30 million; a Long holds about 9.2 × 10^18.Also from the first round, Jev token sum can mask partial data: intended. A run without a DB schema judgment has no tokens there; the sum covers what ran.
UsageStore.kt:61-64): not relevant in operation. The service runs as one systemd unit, and a restart stops the old process first; two instances never share the database.Implemented in this PR: the derivation of the input share and the wording on the usage record in the PR document.
🤖 Generated with Claude Code