werkrevisor/docs/prs/2026-10-05-PR#6-deterministic-writing-rules-and-jev-eval-layers.md

PR#6: Deterministic writing rules and config layers for jev-eval

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.

The Problem

The writing style of AGENTS.md has three countable rules: one sentence per line, at most 128 characters per line, and at most 5 sentences per paragraph. Today only the model and the Jev question writing_rules_violated check them. In the measurement with tools/jev-eval, that question flagged all 34 sections, so it does not tell good from bad text.

tools/jev-eval reads only the host config. The review puts .werkrevisor.toml on top of it, so the measurement can use other questions than the review.

Non-Goals

• No check of the other style rules, e.g. active voice or simple words; they stay with the model.
• No RED for a writing rule: a style rule must not block hard.
• No .werkrevisor.toml for hsadmin-ng.

The Scenarios

Feature: deterministic writing rules for changed docs

Background

• The config is config/werkprobe.toml with [writing] max_line_length = 128 and max_sentences_per_paragraph = 5.
• The PR changes the doc file doc/guide.md.

Scenario#6.01: two sentences on one changed line give ORANGE

So that a human sees the violation, while the model still reviews the PR.

• Given a changed line The gate stops. A human decides.
• When werkrevisor reviews the PR
• Then the verdict is ORANGE with the reason writing rules violated in the changed docs: doc/guide.md:1: more than one sentence on a line
  • and the model reviews the PR

Verified by

• test_writing.WritingFlowTest.test_violation_is_orange_and_the_model_still_reviews
• test_writing.LineRuleTest.test_more_than_one_sentence_on_a_line

Scenario#6.02: abbreviations, numbers, file names, code spans and URLs do not end a sentence

So that the rule gives no false finding.

• Given a changed line <line>
• When werkrevisor checks the writing rules
• Then the line has one sentence

line
Use a glob, e.g. Docs or Tests, i.e. A pattern.
Das gilt z. B. Für Tests, d. h. Nicht für Doku, usw. Und mehr.
See Nr. 5 vs. Version 1.5 and werkrevisor 0.9.0 in config.py.
Call `x. Y` or https://example.org/a. B with a mark.

Verified by

• test_writing.SentenceSplitTest.test_abbreviations_numbers_and_file_names_do_not_split
• test_writing.SentenceSplitTest.test_code_spans_and_urls_do_not_split

Scenario#6.03: a changed line longer than 128 characters is a violation

So that a reader does not scroll and a diff stays readable.

• Given a changed prose line of 129 characters
• When werkrevisor checks the writing rules
• Then the violation is doc/guide.md:1: line longer than 128 characters (129)

Verified by

• test_writing.LineRuleTest.test_long_line_is_a_violation

Scenario#6.04: some long lines are allowed

So that tables, headings, links and code stay as they are.

• Given a changed line of 200 characters that is <kind>
• When werkrevisor checks the writing rules
• Then there is no violation

kind
a table row
a heading
a line with only a link or a URL
in a fenced code block
in an HTML comment
in the front matter

Verified by

• test_writing.LineRuleTest.test_long_line_exceptions

Scenario#6.05: a changed paragraph with more than 5 sentences is a violation

So that a paragraph stays short.

• Given a paragraph of 6 sentences on 6 lines, and one of them changed
• When werkrevisor checks the writing rules
• Then the violation is doc/guide.md:1: paragraph with 6 sentences, more than 5
  • and the items of a list count as separate paragraphs

Verified by

• test_writing.ParagraphRuleTest.test_paragraph_with_more_than_five_sentences
• test_writing.ParagraphRuleTest.test_list_items_are_separate_paragraphs

Scenario#6.06: only added or changed lines count

So that old text does not block a new PR.

• Given a file with violations in lines that the PR does not change
• When werkrevisor checks the writing rules
• Then there is no violation
  • and a paragraph counts only if the PR changes one of its lines

Verified by

• test_writing.ChangedLinesTest.test_untouched_lines_and_paragraphs_do_not_count
• test_writing.ChangedLinesTest.test_a_changed_line_makes_its_paragraph_count

Scenario#6.07: one reason with at most 3 examples

So that the reason stays one line.

• Given 5 violations
• When werkrevisor reviews the PR
• Then one reason lists 3 violations as file:line: rule and and 2 more

Verified by

• test_writing.ReasonTest.test_one_reason_with_three_examples_and_the_count_of_the_rest

Scenario#6.08: the docs category and the PR documents are checked

So that every Markdown doc follows the same writing style.

• Given [writing] globs is <globs>
• When a PR changes <file>
• Then werkrevisor checks it: <checked>

globs	file	checked
[]	doc/guide.md (docs category)	yes
[]	src/main/Foo.java	no
["docs/prs/*.md"]	doc/guide.md	no
["doc/*.txt"]	doc/guide.txt	no, not Markdown
[] with [pr_doc] globs = ["notes/*.md"]	notes/pr.md	yes
[] with enabled = false	doc/guide.md	no

Verified by

• test_writing.SelectionTest.test_which_files_are_checked

Feature: the [writing] config section

Scenario#6.09: the repository config may set [writing]

So that a repository keeps its own writing rules next to its code.

• Given the host config sets [writing] max_line_length = 128
  • and .werkrevisor.toml sets [writing] max_line_length = 100
• When werkrevisor builds the config
• Then max_line_length is 100
  • and a config without [writing] in any layer is an error

Verified by

• test_config_layers.LayeringTest.test_repository_sets_the_writing_rules
• test_writing.ConfigTest.test_writing_section_is_required_and_validated

Feature: no Jev question about the writing rules

Scenario#6.10: the example configs ask Jev no writing-rules question

So that Jev asks only questions it can answer.

• Given the example configs and .werkrevisor.toml
• When werkrevisor builds the config
• Then the section questions are hard_to_understand and slop

Verified by

• test_classifier.ConfigTest.test_examples_have_default_questions_and_keep_llm_mode

Feature: tools/jev-eval uses the config layers of the review

Scenario#6.11: --repo-config puts a repository config on top of the host config

So that the measurement asks the same questions as the review.

• Given the host config config/werkprobe.toml
  • and --repo-config names a file with one section question only_this
• When tools/jev-eval judges a PR document
• Then Jev gets only the question only_this for the sections

Verified by

• test_classifier.JevEvalTest.test_repo_config_is_layered_on_top_of_the_host_config

Scenario#6.12: .werkrevisor.toml in the current directory is the default

So that a measurement in a repository uses its review rules without an option.

• Given the current directory has a .werkrevisor.toml
• When tools/jev-eval runs without --repo-config
• Then it uses that file
  • and with --no-repo-config it uses the host config alone

Verified by

• test_classifier.JevEvalTest.test_werkrevisor_toml_in_the_current_directory_is_the_default

Scenario#6.13: the repository config of jev-eval follows the rules of the review

So that a measurement cannot use a config that the review would refuse.

• Given --repo-config names a file that sets [classifier] endpoint
• When tools/jev-eval runs
• Then it ends with exit code 30
  • and the message is must not set [classifier] endpoint: only the host config sets it

Verified by

• test_classifier.JevEvalTest.test_repo_config_follows_the_rules_of_the_review

The Solution

• werkrevisor/writing.py checks the changed Markdown files of the docs category and the PR documents. It reads the new version at the head commit and takes the added line numbers from the diff.
• A line counts if the PR adds or changes it; a paragraph counts if it contains such a line. Each list item is a paragraph of its own.
• Table rows, headings, link-only lines, fenced code blocks, HTML comments and the front matter have no line limit.
• The sentence split is conservative: a sentence ends at . or ! before whitespace and a capital letter. Abbreviations, initials, numbers, code spans, URLs and link targets end no sentence. A question mark ends no sentence, because the PR documents answer a question on the same line.
• A violation gives ORANGE with one reason: at most 3 violations as file:line: rule, then and N more. The check runs before the model call and does not skip it.
• The new section [writing] (enabled, max_line_length, max_sentences_per_paragraph, globs) is required. The host config and .werkrevisor.toml may set all of its keys. config/werkprobe.toml, config/hsadmin-ng.toml and .werkrevisor.toml set it; config/werkrevisor.toml keeps only the host part.
• The example configs and .werkrevisor.toml no longer ask Jev the question writing_rules_violated.
• tools/jev-eval builds its config like the review: defaults, host config, repository config. --repo-config FILE names the repository config; the default is .werkrevisor.toml in the current directory. --no-repo-config uses the host config alone, e.g. to measure hsadmin-ng from the werkrevisor checkout.
• werkrevisor 0.9.0; the README, the classifier concept and the operations guide describe the change.

Open Questions

• Does a question mark end a sentence? Yes: an answer goes on an indented continuation line of the same list item. The open-questions rule counts the whole item as checked, not its source lines.
• Does a number before a period end a sentence? No: it is mostly an ordinal or a list number, e.g. 3. Oktober.
• Does config/werkrevisor.toml get a [writing] section? No: it holds only the host part; .werkrevisor.toml sets [writing] for werkrevisor.
• Does a long code span make a line exempt? No: only the kinds of Scenario#6.04 are exempt.

Additional Changes

• Two existing tests no longer depend on the removed question or on a one-line document of 500 characters.