3ca2e66d95
A nightly build an hour after `docker system prune -af` first rebuilt its image (233 s on vm4006) and ran into the build timeout. The cleanup now keeps images labelled org.werkator.build-inputs-sha256 on six nights and takes them in the night to Sunday, so the base image is at most a week old. A replaced build image loses its tag but keeps the label, so `docker image prune -f` follows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mhoennig <michael@hoennig.de> Reviewed-on: #83
62 lines
3.2 KiB
Markdown
62 lines
3.2 KiB
Markdown
> **WARNING:** This document describes only the change applied in this PR.
|
|
> It may already be outdated once the next PR is merged.
|
|
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
|
|
|
## The Problem
|
|
|
|
On vm4006 the nightly `master@release` build of hs.hsadmin.ng timed out on 2026-09-29 at 30 minutes.
|
|
The cleanup timer runs `docker system prune -af` at 02:00 host time, which is 00:00 UTC.
|
|
It removes `hsadmin-ng-build-env`, because at that time no container uses the image.
|
|
The nightly build at 01:00 UTC then first rebuilt the image, in 233 seconds.
|
|
With `pitestFull`, the build itself takes about 28.5 minutes; together they did not fit.
|
|
|
|
## Non-Goals
|
|
|
|
- No change to the build timeout; vm4006 raised its `executor.buildTimeout` separately.
|
|
- No keeping of other images, e.g. the ones Testcontainers pulls; they carry no label Werkator knows.
|
|
- No `--pull` when Werkator builds an image.
|
|
|
|
## The Scenarios
|
|
|
|
### Feature: the nightly cleanup keeps Werkator's build images
|
|
|
|
#### Scenario#83.01: Build images survive six nights, the night to Sunday takes them
|
|
|
|
So that a nightly build does not spend its time on a rebuild, and the build environment is still at most a week old.
|
|
|
|
- **Given** a build image Werkator built, labelled `org.werkator.build-inputs-sha256`, and no container using it
|
|
- **When** the cleanup runs on Monday to Saturday, **then** the image stays
|
|
- **When** the cleanup runs on Sunday, **then** it is removed like any other unused image
|
|
- **And** unused images without the label are removed every night, as before
|
|
|
|
##### Verified by
|
|
|
|
- [SystemdServiceFilesTest](../../src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt) — the command runs with a faked `date` and `docker` for every weekday
|
|
- By hand: the generated `ExecStart` line in a real systemd user unit ran the filtered prune on a Tuesday, so systemd's `$$` and `%%` escapes arrive as intended.
|
|
|
|
#### Scenario#83.02: A replaced build image does not pile up
|
|
|
|
So that a changed Dockerfile does not leave a 4 GB image behind every time.
|
|
|
|
- **Given** a build image that was rebuilt, so its old image lost its tag but kept the label
|
|
- **When** the cleanup runs, **then** the untagged image is removed
|
|
|
|
##### Verified by
|
|
|
|
- [SystemdServiceFilesTest](../../src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt) — `docker image prune -f` follows the main step
|
|
- By hand with Docker 27.3.1: `docker image prune -af --filter label!=…` keeps a labelled untagged image, and `docker image prune -f` removes it.
|
|
|
|
## The Solution
|
|
|
|
The prune service runs one `sh -c` line: on Sunday (`date +%u` is 7) the plain `docker system prune -af`, on other days the same with `--filter label!=org.werkator.build-inputs-sha256`.
|
|
The label is the one `DockerBuildRunner` already sets to detect stale images, so no new label is needed.
|
|
A second step, `docker image prune -f`, removes dangling images, which the label filter would keep.
|
|
The anonymous-volume step (PR#80) stays unchanged after these two.
|
|
|
|
Once a week is a trade-off: `docker build` runs without `--pull`, so a kept image keeps its base image and packages until it is removed.
|
|
|
|
## Additional Changes
|
|
|
|
- `docs/deployment.md` describes the new behaviour.
|
|
- Version 1.23.2 with its release note.
|