werkator/docs/prs/2026-10-10-PR#139-pit-specs-stop-at-first-failure.md

PR#139: PIT Specs Stop at Their First Failure

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.

The Problem

For a mutant, PIT runs the whole Kotest spec, every test in it, even after the first test has failed. One failure already kills the mutant, so the rest of the spec is wasted time.

It is worst where tests wait for something. A mutant that never ends a build lets every test of BuildExecutorUnitTest wait up to five seconds. The spec then runs past PIT's timeout: the mutant ends TIMED_OUT after more than 20 s, and PIT starts a new test process.

Non-Goals

• No change to normal test runs: they still run every test and show every failure.
• The integration specs that wait up to 30 s for a build still run into the timeout with their first test; they follow in their own PR.

The Scenarios

Scenario#139.01: A normal test run runs every test

• Given a spec with a failing test
• When it runs outside the mutation build
• Then its other tests still run
• So that a developer sees every failure at once.

Verified by

• FailFastUnitTest

Scenario#139.02: In the mutation build a spec stops at its first failure

• Given a mutant that makes a test fail
• When PIT runs the spec for it
• Then the spec skips its remaining tests
• So that the mutant is killed at once and does not wait into PIT's timeout.

Verified by

• The local PIT run below; Kotest's own fail-fast is not tested again here.

The Solution

• ProjectConfig.failfast is read from the system property werkator.failfast.
• The pitest block passes -Dwerkator.failfast=true to PIT's test processes only.

A local PIT run of BuildExecutor alone, two minions, without history, with the tests of PR#138:

Run	Duration	Mutants timed out	Mutants killed
Without the PR	23.7 min	35	91
With fail-fast	18.6 min	25	102

Prerequisite PRs

• PR#136, which changes the same ProjectConfig.