# PR#140: Executor Integration Specs out of the Mutation Build

> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.

## The Problem

For each mutant, PIT runs the covering specs fastest first.
For `BuildExecutor` the two fastest are `PullRequestTargetBuildIntegrationTest` (0.7 s) and `BuildExecutorArtifactIntegrationTest` (1.3 s), both with real git and real processes.
They wait up to 30 s for a build to end.
A mutant that never ends a build therefore runs into PIT's timeout in them, before `BuildExecutorUnitTest` could kill it within a second.

In a local PIT run of `BuildExecutor` alone, with PR#138 and PR#139, 25 of 194 mutants timed out, and the run took 18.6 min.

## Non-Goals

- The two specs stay in the normal build: they test the executor with real processes, which no unit test does.
- No change to the executor itself.

## The Scenarios

#### Scenario#140.01: A finished build hands its result, its logs and its workspace to the artifact store

- **Given** a build that succeeded
- **When** it is settled
- **Then** the artifact store gets its result, its staging directory with the logs, and its workspace
- **So that** the reports of the build can be stored.

##### Verified by

- [BuildExecutorUnitTest](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorUnitTest.kt)

#### Scenario#140.02: A build whose artifacts leave the workspace ends failed

- **Given** a build whose commands succeed
- **And** the artifact store refuses its artifacts because they point out of the workspace
- **When** it is settled
- **Then** its status is taken back to failed, and its live log says why
- **So that** a build whose result cannot be collected never counts as green.

##### Verified by

- [BuildExecutorUnitTest](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorUnitTest.kt)

Two scenarios of PR#117 get a unit test beside their integration test:
Scenario#117.03 (a review is started as one and learns its pull request and the forge) and Scenario#117.01 (a head already queued or running is not queued twice).

## The Solution

- `BuildExecutorUnitTest` gets the four tests above; its fake runner records whether a command was started as a pull-request target build, and its artifact store is a mock it can check.
- The `pitest` block leaves out `PullRequestTargetBuildIntegrationTest` and `BuildExecutorArtifactIntegrationTest`.

A local PIT run of `BuildExecutor` alone, two minions, without history, with PR#138 and PR#139:

| Run                                     | Duration | Mutants timed out |
|-----------------------------------------|---------:|------------------:|
| With the two specs                      | 18.6 min |                25 |
| Without them, before the new unit tests |  9.3 min |                 2 |
| Without them, with the new unit tests   |  9.9 min |                 2 |

Without the two specs, 8 mutants survived that they had killed; the new unit tests kill all of them, and 3 more.
In `CommandProcess`, `RunningBuild` and `PullRequestTarget`, which the two specs cover as well, a PIT run without them kills every mutant the nightly killed, except two that depend on timing (a removed `Thread.sleep` and `Thread.join`), and the timeouts there drop from 8 to 2.

## Additional Changes

- `FailFastUnitTest` from PR#139 is left out of the mutation build as well.
  It checks that a normal test run does not fail fast, and PIT's test processes do, so under PIT it fails without a mutant.
  PIT then aborts the whole run already in its coverage phase: the nightly would fail without measuring anything.