miandmhoennig
a4c0e52265
Groundwork for mutation testing with PIT and werkcrappit. - Every spec ends in `*UnitTest` (no Spring, no real git/process), `*RestTest` (`@WebMvcTest` slice) or `*IntegrationTest` (full context, real git, processes, WireMock, containers): 55 / 9 / 14; 77 files renamed, file and class. - Gradle tasks `unitTest` (UnitTest + RestTest, 16 s test time) and `integrationTest` (40.5 s); `test` and CI unchanged. - `TestNamingConventionUnitTest` fails for a spec without a suffix. - AGENTS.md and the writing-tests skill describe the convention. Found on the way: PIT's timeouts on Werkator came from its 4 s limit, not from the integration tests (323 → 2 timeouts at 20 s); that goes into the next PR together with PIT and werkcrappit. CI on werkator.javagil.de: green (141 s). PR-doc: `docs/prs/2026-10-01-PR#91-separate-unit-and-integration-tests.md` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: mhoennig <michael@hoennig.de> Reviewed-on: #91
60 lines
2.9 KiB
Markdown
60 lines
2.9 KiB
Markdown
# PR#91: Separate Unit Tests From Integration Tests
|
|
|
|
> This document describes the change of this pull request at the time it was made.
|
|
> Later pull requests may change the behaviour without updating it.
|
|
|
|
## The Problem
|
|
|
|
Werkator's 78 specs had no way to tell a cheap test from an expensive one.
|
|
Mutation testing is to come next (PIT with werkcrappit), and PIT runs the selected tests once per mutant in its own process.
|
|
A spec that starts the full Spring context, creates real git repositories or runs processes costs seconds every time.
|
|
Locally there is also no way to run just the fast half while working.
|
|
|
|
A probe run of PIT on the package `metrics` on 2026-10-01 showed how much the selection matters, and that it is not all:
|
|
with only the seven unit specs of the package, 323 of 787 mutants timed out at PIT's default limit of 4 s, two at 20 s.
|
|
The limit is the next PR's business; this one makes the selection possible.
|
|
|
|
## Non-Goals
|
|
|
|
- PIT itself, werkcrappit and the timeout setting: the next PR.
|
|
- Changing what a test does: only names, tasks and documentation change.
|
|
- Separate source sets: the suffix is enough for Gradle's filter and for PIT's `targetTests`.
|
|
|
|
## The Scenarios
|
|
|
|
#### Scenario#91.01: every spec says what it needs
|
|
|
|
- **Given** the specs under `src/test/kotlin`
|
|
- **Then** each name ends in `UnitTest`, `RestTest` or `IntegrationTest`
|
|
- **And** a spec without one of them fails the build, since it would run in neither task
|
|
|
|
##### Verified by
|
|
|
|
- [TestNamingConventionUnitTest](../../src/test/kotlin/de/hoennig/werkator/TestNamingConventionUnitTest.kt)
|
|
|
|
#### Scenario#91.02: the cheap and the expensive half run on their own
|
|
|
|
- **When** `./gradlew unitTest` runs
|
|
- **Then** the 55 `*UnitTest` and 9 `*RestTest` specs run, plus the naming test, 16 s of test time
|
|
- **When** `./gradlew integrationTest` runs
|
|
- **Then** the 14 `*IntegrationTest` specs run, 40.5 s of test time
|
|
- **And** `./gradlew test` and CI still run all of them
|
|
|
|
##### Verified by
|
|
|
|
- local runs of both tasks on 2026-10-01; the CI build of this branch runs `test`
|
|
|
|
## The Solution
|
|
|
|
The suffixes follow hsadmin-ng:
|
|
|
|
- `*UnitTest` — no Spring context, no real git, `sh` or other process; temporary files and MockK are fine.
|
|
- `*RestTest` — a `@WebMvcTest` slice; it counts as cheap, so that mutation testing covers the controllers.
|
|
- `*IntegrationTest` — the full context, real git, processes, WireMock, Testcontainers or real waiting.
|
|
|
|
The 77 specs without a suffix were renamed, file and class; `BuildExecutorArtifactIntegrationTest` already had one.
|
|
The classification was derived from what each spec uses and checked by hand; two first guesses were wrong
|
|
(`InitCommandTest` and `NginxProxyManagerTest` only mock processes and waiting).
|
|
`build.gradle.kts` registers `unitTest` and `integrationTest` with Gradle's test filter.
|
|
AGENTS.md and the writing-tests skill describe the convention; maintained references to renamed tests were updated, historic PR-docs were not.
|