Files
werkrevisor/docs/prs/2026-10-07-PR#40-economy-model.md
claude-code 4808090fdc
werkator build succeeded after 00:09
werkator/werkrevisor-statistics build succeeded after 00:26
werkrevisor 0.21.1: MiniMax M3 advises in the reviews on git.javagil.de, a failed advising model leaves the verdict to Jev (#40)
Co-authored-by: claude-code <claude-code@javagil.de>
2026-10-07 09:51:57 +02:00

3.3 KiB

PR#40: MiniMax M3 advises in the reviews on git.javagil.de

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

The Problem

Since werkrevisor 0.21.0, Jev decides whether a PR of werkrevisor or Werkator needs a human, and the model only advises. Sonnet 5.5 still advises there, and costs 0.14 to 0.39 USD per review. On 4 heads with 10 known findings, MiniMax M3 with reasoning_effort = "medium" found as many as Sonnet 5.5, 2 of 10, for 0.02 to 0.05 USD per review, but took 181 to 242 s instead of 24 to 63 s.

In a first local review of werkrevisor PR#37 with MiniMax M3, both calls hit the timeout of 300 s, and the review ended as ERROR, although Jev had judged every file and the model only advises. The second review of the same PR took 51 s and cost 0.02 USD.

The docs and configs call such a model a "cheap model". The user prefers the pair "economy model" and "frontier model".

Non-Goals

  • No change of config/hsadmin-ng.toml: there the model decides, and the team chooses it.
  • No change of the settings in config/werkprobe.toml: the tests build on them.

The Scenarios

Feature: an advising model never ends a review

Scenario#40.01: a failed advising model leaves the verdict to Jev

So that a slow economy model does not hold a PR that Jev found harmless.

  • Given [engine] model_role = "advises" and [path_judge] scope = "all"
  • When the model call fails or times out, and Jev <judged>
  • Then the failure is a reason of verdict <verdict>
judged verdict
every changed file and found nothing GREEN, a hint
not every changed file YELLOW

The reason model_failed is YELLOW and becomes a hint only when Jev judged every changed file. With model_role = "decides", the failure stays an ERROR. With either role, a newer head ends the review.

Verified by

The Solution

  • config/werkator.toml and config/werkrevisor.toml use openrouter/minimax/minimax-m3 with reasoning_effort = "medium":
    • prices of 0.30 and 1.20 USD per 1M input and output tokens;
    • max_output_tokens = 32000, since MiniMax M3 thinks up to 30000 tokens;
    • timeout_seconds = 300, since a call took up to 242 s;
    • token_count_factor = 1.05, since MiniMax reported at most 2 % more tokens than o200k_base counted;
    • max_usd_per_review = 0.20.
  • With model_role = "advises", a failed or timed-out model call gives the reason model_failed, which is a hint when Jev judged every changed file, and YELLOW otherwise. PR-Agent reports no usage for a failed call, so the usage record holds the remaining key limit after it.
  • Docs, configs and comments say "economy model" instead of "cheap model".
  • At the deployment, the machine configs of werkrevisor and Werkator on mih09 get the same [model] and [budget], and the Werkator build ai-review gets a timeout of 15 minutes for 2 calls of up to 300 s each.
  • werkrevisor 0.21.1.

Prerequisite PRs

  • PR#39: werkrevisor 0.21.0, Jev decides and the model advises, merged.