werkrevisor/docs/prs/2026-10-09-PR#65-prompt-out.md

PR#65: A Dry Run Writes the Model Prompt to a File

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

The Problem

Many PRs need several review rounds, each for new small hints of the model, e.g. werkprobe PR#12 with 5 rounds. To find a rule that keeps the useful hints and drops the others, a frontier model rates the hints of past PRs once. For a fair rating, it needs the context in which the economy model wrote each hint: the diff, the full files or excerpts, AGENTS.md, the review instructions and Jev's hints. werkrevisor does not keep this prompt, and a dry run cannot write it without calling the model.

Non-Goals

• No change to the reviews on the review machine.
• No dry run of an earlier head: PR-Agent reads the current state of a PR through the Gitea API. The measurement reviews earlier heads as throwaway PRs in mi/werkrevisor-bench.
• The rating and the rule for the hints come in a later PR.

The Scenarios

Feature: a dry run writes the prompt of the model review

Scenario#65.01: the prompt goes to a file instead of the model

So that a later measurement can give another model the same context.

• Given the options --no-publish and --prompt-out prompt.json
• When werkrevisor reviews a PR
• Then prompt.json holds the system prompt and the user prompt of the model review as JSON
• And no model is called

Verified by

• test_prompt_out.PromptOutTest.test_the_prompt_goes_to_the_file_instead_of_the_model

Scenario#65.02: the written prompt is the one after the size checks

So that the file holds the prompt that the model would get, not one that the budget refuses.

• Given a dry run with --prompt-out
• And the first prompt with the full files is too large
• When werkrevisor shortens the prompt to the excerpts or the diff
• Then the file holds the shortened prompt

Verified by

• test_prompt_out.PromptOutTest.test_the_file_holds_the_prompt_after_the_size_checks

Scenario#65.03: only a dry run writes the prompt

So that the option never changes a review on the review machine.

• Given --prompt-out without --no-publish
• When werkrevisor starts
• Then it stops with a configuration error

Verified by

• test_prompt_out.PromptOutTest.test_only_a_dry_run_writes_the_prompt

Scenario#65.04: no prompt without the model review

So that a run with --prompt-out always writes a prompt; with --no-llm, werkrevisor never builds one.

• Given --prompt-out together with --no-llm
• When werkrevisor starts
• Then it stops with a configuration error

Verified by

• test_prompt_out.PromptOutTest.test_only_a_dry_run_writes_the_prompt

The Solution

• werkrevisor --prompt-out FILE and tools/dry-run … --prompt-out build/FILE: the prompt that passes the size and budget checks goes to the file, and no model is called. Jev and the excerpt selection still run, since their results are part of the prompt.
• The README describes the option; the Werkbaum plans the rating by a frontier model under #model.rounds and records that a rating by the user did not work.
• Version 0.30.2.