138 lines
7.0 KiB
Markdown
138 lines
7.0 KiB
Markdown
# Guide: werkcrappit in hsadmin-ng
|
|
|
|
hsadmin-ng is werkcrappit's pilot project.
|
|
This guide adds werkcrappit to its build, records what a probe run showed, and how hsadmin-ng turned on the gate (its PR#317).
|
|
|
|
## What hsadmin-ng Already Has
|
|
|
|
hsadmin-ng runs PIT through `info.solidsoft.pitest` 1.19.0 with PIT 1.30.0 and the JUnit 5 plugin 1.2.3, with `timestampedReports` off and its `outputFormats` on the `pitest` extension.
|
|
That is exactly what werkcrappit needs; nothing of it changes.
|
|
`lombok.config` sets `lombok.addLombokGeneratedAnnotation = true`, so Lombok's methods are recognized as generated.
|
|
|
|
## Integration
|
|
|
|
`settings.gradle`: the registry for the Gradle plugin.
|
|
|
|
```groovy
|
|
pluginManagement {
|
|
repositories {
|
|
maven { url = uri("https://git.javagil.de/api/packages/mi/maven") }
|
|
// … the existing repositories
|
|
}
|
|
}
|
|
```
|
|
|
|
`build.gradle.kts`: the plugin, the registry for `werkcrappit-pit`, and the `werkcrappit` block.
|
|
|
|
```kotlin
|
|
plugins {
|
|
id("info.solidsoft.pitest") version "1.19.0"
|
|
id("de.javagil.werkcrappit") version "0.22.1"
|
|
}
|
|
|
|
repositories {
|
|
mavenCentral()
|
|
maven { url = uri("https://git.javagil.de/api/packages/mi/maven") }
|
|
}
|
|
|
|
werkcrappit {
|
|
maxMutationCrap = 10
|
|
// ratchets at today's highest values: lower them as classes improve, never raise them
|
|
maxLcom4 = 4
|
|
maxEfferentCoupling = 34
|
|
codeCharta {
|
|
excludeClasses = listOf("**.generated.**")
|
|
sourceLinkTemplate = "https://dev.hostsharing.net/hostsharing/hs.hsadmin.ng/src/commit/{commit}/{path}#L{line}"
|
|
}
|
|
}
|
|
```
|
|
|
|
The plugin adds `werkcrappit-pit` of its own version to PIT's tool classpath, adds its output format next to `XML` and `HTML`, and names the analysis `:pitest`.
|
|
`pitestPartial` and `pitestFull` both run the `pitest` task, so both write the reports.
|
|
|
|
## What It Produces
|
|
|
|
Next to PIT's reports in `build/reports/pitest/`:
|
|
|
|
- `mutation-crap.md` — scored methods by descending Mutation-CRAP, the others with their reasons;
|
|
- `mutation-crap.json` — the same as a versioned contract;
|
|
- a few lines in the build log, starting with `werkcrappit:`.
|
|
|
|
Werkator already collects `build/reports` as artifacts, so both files appear with every build.
|
|
|
|
## Probe Run, 2026-10-01
|
|
|
|
A full `pitest` on master `c8955b2e5`, with werkcrappit 0.1.0 and the policy `strict-v1`:
|
|
|
|
| | |
|
|
| --- | ---: |
|
|
| PIT run | 28 min 10 s |
|
|
| selected classes | 294, all read (scope complete) |
|
|
| methods | 2735 |
|
|
| `SCORED` | 1158 |
|
|
| `EXCLUDED` | 1275: Lombok-generated 987, bridge methods 164, other JaCoCo filters 124 |
|
|
| `NO_MUTATION_RESULTS` | 298 |
|
|
| `INCONCLUSIVE` | 4, all from `TIMED_OUT` |
|
|
|
|
Mutation-CRAP of the scored methods: median 1.0, 90th percentile 3.07, highest 12.72; none above 15.
|
|
|
|
| Mutation-CRAP | C | MS | Method |
|
|
| ---: | ---: | ---: | --- |
|
|
| 12.72 | 6 | 0.43 | `rbac.context.Context.maskProperties` |
|
|
| 12.41 | 5 | 0.33 | `hs.validation.IntegerProperty.deferredInit` |
|
|
| 12.00 | 12 | 1.00 | `hs.hosting.asset.validators.HsDomainDnsSetupHostingAssetValidator.toZonefileString` |
|
|
| 12.00 | 3 | 0.00 | `hs.office.partner.HsOfficePartnerController.optionallyCreateExPartnerRelation` |
|
|
| 12.00 | 3 | 0.00 | `repr.Stringify.<init>` |
|
|
| 11.60 | 7 | 0.55 | `hs.office.debitor.HsOfficeDebitorController.postNewDebitor` |
|
|
|
|
The first and the fourth row show what the metric separates: `toZonefileString` scores 12 by its complexity alone, every mutant is killed; `optionallyCreateExPartnerRelation` is simple, but no mutant is noticed.
|
|
|
|
Findings:
|
|
|
|
- **Java 25** bytecode is analysed without a gap.
|
|
- **Incremental analysis** works: a second run with `.pitest/history.bin` took 1 min 3 s, PIT reused 2297 mutations from the history, and the report was identical method by method.
|
|
- **Timeouts** made 4 methods inconclusive under `strict-v1`; since 0.3.0 the default policy `timeouts-killed-v1` scores them (ADR 0006).
|
|
|
|
## Turning On the Gate
|
|
|
|
No threshold of this metric is proven in general; for hsadmin-ng the target is 10.
|
|
The gate fails when a method's value is greater than the limit, so `maxMutationCrap = 10` means "at most 10".
|
|
|
|
In the probe run six methods were above 10; each needed one more killed mutant, except one that needed to be split.
|
|
"After" is the value of a full run once hsadmin-ng's PR#317 had added the tests and split the method:
|
|
|
|
| Before | Method | C | Killed | What was missing | After |
|
|
| ---: | --- | ---: | ---: | --- | ---: |
|
|
| 12.72 | `rbac.context.Context.maskProperties` | 6 | 3/7 | a field to mask in a nested object and in an array | 6.00 |
|
|
| 12.41 | `hs.validation.IntegerProperty.deferredInit` | 5 | 2/6 | `min` with `minFrom`, `max` with `maxFrom`: the exception | 5.00 |
|
|
| 12.00 | `repr.Stringify.<init>` | 3 | 0/2 | a `Stringify` without explicit name, with and without `@DisplayAs` | 3.00 |
|
|
| 12.00 | `hs.office.partner.HsOfficePartnerController.optionallyCreateExPartnerRelation` | 3 | 0/2 | a changed partner person saves an `EX_PARTNER` relation, an unchanged one does not | 3.00 |
|
|
| 11.60 | `hs.office.debitor.HsOfficeDebitorController.postNewDebitor` | 7 | 6/11 | `debitorRel` and `debitorRelUuid` both or neither, or `debitorRel.mark`: 400 | 7.04 |
|
|
| 12.00 | `hs.hosting.asset.validators.HsDomainDnsSetupHostingAssetValidator.toZonefileString` | 12 | 12/12 | nothing to test: split the method | 1.00 |
|
|
|
|
Mutation-CRAP is never below the complexity, so `toZonefileString` with complexity 12 stays at 12 however well it is tested.
|
|
Tests that cover the controller methods may exist as integration tests already; they do not count, since PIT in hsadmin-ng runs only `*UnitTest` and `*RestTest`.
|
|
|
|
Then the gate runs in every build:
|
|
|
|
```bash
|
|
./gradlew pitestPartial mutationCrapCodeCharta mutationCrapCheck
|
|
```
|
|
|
|
The export runs before the gate, so a build that exceeds the limit still has the code city of exactly the methods to look at.
|
|
|
|
`mutationCrapCheck` depends on `pitest` and reuses the run of `pitestPartial`; it is not part of `check`.
|
|
It fails when a method exceeds the limit, and with an analysis error when the analysis cannot be judged — for example a `MEMORY_ERROR` or `RUN_ERROR` mutant.
|
|
PIT's own thresholds (`mutationThreshold`, `coverageThreshold`, `testStrengthThreshold`) stay in force; when they fail `pitest`, the gate does not run.
|
|
|
|
The metric shows little of hsadmin-ng's remaining gaps: 161 methods have no killed mutant at all, but 144 of them have complexity 1, and a method of complexity 1 cannot score above 2.
|
|
PIT's own report shows them better than a Mutation-CRAP ranking.
|
|
|
|
## State After hsadmin-ng's PR#317
|
|
|
|
- Branch builds run `pitestPartial`, the nightly `release` build of `master` runs `pitestFull`; both run the export before the gate.
|
|
- `pitestPartial` reuses older results from `.pitest/history.bin`, so a branch build can show a higher value than a full run.
|
|
- Werkator publishes the code city of `master@release` without a token although the repository is private (`publicArtifacts` in `.werkator.yml`); all other artifacts stay hidden.
|
|
The viewer at `https://javagil.de/codecharta/` loads it from there.
|
|
- 332 methods have `NO_MUTATION_RESULTS`, all of complexity 1 or 2, so none of them can exceed the limit; their causes are still open.
|