62 lines
2.9 KiB
Markdown
62 lines
2.9 KiB
Markdown
# PR#41: werkrevisor reviews werkcrappit
|
|
|
|
> **WARNING:** This document describes only the change applied in this PR.
|
|
> It may already be outdated once the next PR is merged.
|
|
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.
|
|
|
|
## The Problem
|
|
|
|
werkcrappit, the Mutation-CRAP plugin for PIT and Gradle, is the most active repository on git.javagil.de without werkrevisor:
|
|
43 merged PRs since 2026-10-01.
|
|
Like hsadmin-ng, it is built with Gradle in Java, so it is a rehearsal for the rollout on dev.hostsharing.net.
|
|
|
|
## Non-Goals
|
|
|
|
- No change of the code of werkrevisor.
|
|
- No change of werkcrappit in this PR; its `AGENTS.md` gets the pull-request workflow in a PR of werkcrappit.
|
|
|
|
## The Solution
|
|
|
|
- `config/werkcrappit.toml`, a complete host config derived from `config/werkator.toml`:
|
|
- MiniMax M3 advises, Jev decides with `[path_judge] scope = "all"`;
|
|
- only secrets are hard paths;
|
|
- Jev judges the build, CI and publication files as judged paths:
|
|
`.gitea/**`, `.github/**`, `.werkator.yml`, `tools/**`, the root and module build files, `gradle/**` and `gradle.properties`;
|
|
- a leading `/` anchors the root build files, so the build files of the test fixtures under `src/functionalTest/resources`
|
|
count as tests;
|
|
- tests in `**/src/test/**` and `**/src/functionalTest/**`, main code in `**/src/main/**`;
|
|
- PR documents `docs/prs/20*-PR#*.md` with the heading `Open Questions`.
|
|
- At the deployment, the machine config of werkcrappit on mih09 gets the build `ai-review` with this config,
|
|
and the branch protection of `main` the werkrevisor approval.
|
|
|
|
## Measurement
|
|
|
|
On 2026-10-07, this config reviewed the 43 merged werkcrappit PRs with Jev alone (`[engine] mode = "classifier"`);
|
|
Jev cost 0.002 USD for all 43 PRs together.
|
|
The gate reasons self-approval and "never marked ready" are left out:
|
|
the PRs predate the workflow with a `WIP: ` title, and their author is the human user.
|
|
|
|
| result | PRs |
|
|
|---|---|
|
|
| GREEN | 25 |
|
|
| YELLOW | 12 |
|
|
| ORANGE | 1 |
|
|
| RED | 5 |
|
|
|
|
Of the 18 PRs that are not GREEN, several have more than one cause:
|
|
|
|
- 5 PRs only have findings in their PR document, which the author can fix in the PR:
|
|
open items under `Open Questions` (RED) and scenarios that Jev rates as unclear or with more than one aspect.
|
|
- 6 PRs exceed `[size] max_changed_lines = 800` or `max_files = 30`.
|
|
- 8 PRs change the build, the CI or the publication in a way Jev rates as risky, e.g. #8, the publication to the registry,
|
|
which Jev rates as touching security.
|
|
- Two of these 8, #32 and #33, lowered the ratchet `maxMutationCrap`, which tightens the check;
|
|
Jev still rated this as weakening the checks (0.71 and 0.43), so the PRs get YELLOW.
|
|
- 3 early PRs deleted a `.gitkeep` in a test directory, which counts as a lost test.
|
|
|
|
Before the fixture build files were anchored, Jev rated them as risky build changes, and only 22 PRs were GREEN.
|
|
|
|
## Prerequisite PRs
|
|
|
|
- PR#40: werkrevisor 0.21.1, MiniMax M3 advises, merged.
|