miandmhoennig
cb24f0fb01
A timed-out mutant counts as killed by default, as in PIT's own mutation score; `strict-v1` stays unchanged and selectable (`werkcrappit { scorePolicy = "strict-v1" }` / `werkcrappit.scorePolicy`). MEMORY_ERROR and RUN_ERROR stay inconclusive.
Why: mutants that turn a loop endless cannot be avoided; under strict-v1 every real project has methods without score and a strict gate that never passes. The hsadmin-ng probe run had 4 such methods.
Deviation from the specification's starting default documented in ADR 0006; every report names its policy.
CI on werkator.javagil.de: green (81 s).
PR-doc: `docs/prs/2026-10-01-PR#9-timeouts-killed-policy.md`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #9
2.0 KiB
ADR 0006: Default Status Policy timeouts-killed-v1
- Status: accepted
- Date: 2026-10-01
- PR: #9
Context
The specification starts with the status policy strict-v1, under which one TIMED_OUT result makes a method INCONCLUSIVE (5.2).
It allows a later timeout policy under an identifier of its own and forbids changing strict-v1 unnoticed.
The first probe run on hsadmin-ng showed that timeouts are a normal result of mutation testing: a mutant that turns a loop endless makes the tests not finish.
Under strict-v1 every such method had no score, and a strict gate failed with an analysis error on every real run.
Nobody can make these mutants go away.
Decision
A second policy timeouts-killed-v1 is strict-v1 except that TIMED_OUT counts as killed — in the numerator and the denominator.
It becomes the default; strict-v1 stays unchanged and selectable.
- Gradle plugin:
werkcrappit { scorePolicy = "strict-v1" }. - PIT plugin alone:
pluginConfigurationkeywerkcrappit.scorePolicy. - An unknown policy stops the run.
Why killed and not neutral: a timeout shows that the tests reacted to the change, and PIT's own mutation score counts it as detected; werkcrappit's mutation score then equals PIT's for the same counts.
Why MEMORY_ERROR and RUN_ERROR stay inconclusive: they may as well come from a broken setup, and counting them as kills would turn a broken setup into good test strength.
Deviation From the Specification
The default changes from strict-v1 to timeouts-killed-v1, which section 2.3 calls a starting decision, not a fixed one.
strict-v1 itself, the formula and the non-numeric states are unchanged.
Every report names its policy in definition.scorePolicyId, so reports of different policies are never mistaken for each other.
Consequences
- A timeout from a genuinely hanging setup counts as a kill; the raw count
TIMED_OUTstays visible per method. - Reports made before 0.3.0 used
strict-v1; comparing scores across that boundary needs the same policy.