Follow-up builds #23

Merged
mi merged 8 commits from mi/follow-up-builds AGit into main 2026-09-05 20:02:32 +02:00
3 changed files with 52 additions and 6 deletions
Showing only changes of commit fc8dbf1c91 - Show all commits
+3 -3
View File
@@ -40,9 +40,9 @@ All production code lives under `de.hoennig.werkator`, with sub-packages `comman
- Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly. Server routes carry the repository as `/repos/<name>/…` and `/api/repos/<name>/…`, with the unscoped form permanently meaning the served repository; the pages stay per repository and a drop-down in the page title switches between them. - Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly. Server routes carry the repository as `/repos/<name>/…` and `/api/repos/<name>/…`, with the unscoped form permanently meaning the served repository; the pages stay per repository and a drop-down in the page title switches between them.
- When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`. - When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
- Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config. - Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and werkdock (`werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`) sandbox policies, and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone. - A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and werkdock (`werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`) sandbox policies, the trust gate (`requirePullRequest`), and the trigger of a follow-up build (`trigger.afterSuccessOf` everywhere, and the whole `trigger` block of a definition the host defines as a follow-up). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, bypass its own pull-request gate, or deploy itself; a branch's definitions apply to that branch alone.
- A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins. - A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `afterSuccessOf`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `afterSuccessOf` makes a definition the follow-up of another one — a deployment is a build that follows a green build (PR#23): it runs at the predecessor's commit after every green run of it, whoever started that run, and the `FollowUpTrigger` that enqueues it is armed only by `Watcher.start()`. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins.
- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary`. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called `bwrap` until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets `ConfigVersions.FORMAT_BROKE_IN`. - The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`, and the trigger of a follow-up build. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called `bwrap` until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets `ConfigVersions.FORMAT_BROKE_IN`.
- `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version. - `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version.
- Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats. - Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats.
- Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK. - Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK.
+48 -3
View File
@@ -101,7 +101,9 @@ single branch may decide it:
- the container sandbox policy: `docker.enabled`/`docker.network` and - the container sandbox policy: `docker.enabled`/`docker.network` and
`werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` — host-pinned as `werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` — host-pinned as
long as only the host's configuration sets them, master-pinned once the committed long as only the host's configuration sets them, master-pinned once the committed
configuration does. configuration does;
- the trigger of a [follow-up build](#follow-up-builds): `trigger.afterSuccessOf` in every definition, and the whole `trigger` block of a definition the host defines as a follow-up.
A branch may say what its deployment does, never that — or for which branches — it happens.
The distinction is documentary. The distinction is documentary.
Werkator applies one rule: every pinned key is stripped from the branch layer, and the Werkator applies one rule: every pinned key is stripped from the branch layer, and the
@@ -110,7 +112,7 @@ The names say where a key is meant to live, not how it is enforced.
This keeps a branch from reaching credentials, reporting statuses to another repository, This keeps a branch from reaching credentials, reporting statuses to another repository,
raising the global concurrency, disabling its own build container, changing its network raising the global concurrency, disabling its own build container, changing its network
mode, or bypassing its own pull-request gate. Everything else is the branch's to decide — mode, bypassing its own pull-request gate, or deploying itself. Everything else is the branch's to decide —
it can already run any command through `buildCommand`. it can already run any command through `buildCommand`.
The pinned settings are stripped wherever they appear, in a build definition as well as in The pinned settings are stripped wherever they appear, in a build definition as well as in
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
@@ -252,6 +254,7 @@ builds:
# branches: ["*", "!master"] # names or globs; a "!" pattern excludes; default: all # branches: ["*", "!master"] # names or globs; a "!" pattern excludes; default: all
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05) # atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
# activeWithin: 24h # only branches with commits in the last 24h # activeWithin: 24h # only branches with commits in the last 24h
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
# run before each build # run before each build
cleanCommand: rm -rf build cleanCommand: rm -rf build
# shell command for each build # shell command for each build
@@ -397,9 +400,50 @@ Writing any of its keys outside the block is refused with a message naming the d
Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC). Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC).
A slot may also be written as `??:MM` — that minute of every hour, expanded to its 24 slots, so the build runs hourly. A slot may also be written as `??:MM` — that minute of every hour, expanded to its 24 slots, so the build runs hourly.
Only the latest due slot of a day triggers, so slots missed while the server was down are skipped instead of piling up, and a slot whose pool is still building is retried on the next poll cycle until it succeeds. Only the latest due slot of a day triggers, so slots missed while the server was down are skipped instead of piling up, and a slot whose pool is still building is retried on the next poll cycle until it succeeds.
A definition may have both; one with neither never triggers automatically — which is how `builds.default` is written when it is meant as a settings base only. A definition may combine them; one with none of the three triggers (`onPush`, `atTimes`, `afterSuccessOf`) never runs automatically — which is how `builds.default` is written when it is meant as a settings base only.
Werkator logs a warning once when no definition has a trigger at all, because such an instance never builds anything on its own. Werkator logs a warning once when no definition has a trigger at all, because such an instance never builds anything on its own.
#### Follow-up builds
`afterSuccessOf: <name>` makes a definition a *follow-up* of another one, its *predecessor*: it runs on the predecessor's branch at the predecessor's commit whenever a run of the predecessor ends green.
This is how a deployment is configured: a deployment is a build that follows a green build, and it gets everything a build has — its own row in History with log and duration, its own Gitea check under its own `statusContext`, restart, cancel, artifacts, and the per-branch serialization.
Every green run counts, whatever started it — the push watcher, an `atTimes` slot, a UI restart, `werkator retry`, or the startup recovery — and a repeated green run of the same commit triggers the follow-up again.
The follow-up builds the commit that was tested, not the branch's current origin head.
A failed, cancelled, or interrupted run triggers nothing.
The follow-up runs in the branch's worktree, after its predecessor, in the branch's sandbox or container — so a deployment tool like the Docker CLI is provided the way a compiler is.
It inherits `builds.default` like every definition, so a deployment that wants the predecessor's output has to set `cleanCommand: ""` itself; a deployment command should rather be self-contained and rebuild what it ships, because a restart of the follow-up alone runs without its predecessor.
The pull-request gate is not consulted for a follow-up: the predecessor passed it for the same commit, and the follow-up's `branches` selector is its own gate.
The trigger of a follow-up is pinned: a branch's committed config can neither add `afterSuccessOf` to a definition nor change the `trigger` block of a definition the host defines as a follow-up, so a branch cannot deploy itself.
What the deployment *does* comes with the repository, like every build command; where and when it happens, and with which credentials, is the host's.
The host's part lives in `.git/werkator/.werkator.yml`; credentials reach the sandbox like any build setting, through `werkdock.env`/`docker.env`, and a file such as an SSH key through the werkdock sandbox's persistent toolchain home, `.git/werkator/buildenv/home/` on the host, which the sandbox mounts as `/root`.
```yaml
# .git/werkator/.werkator.yml — the host's part: when, for which branches, with what
builds:
deploy:
trigger:
afterSuccessOf: frontend
branches: ["main"]
statusContext: werkator/deploy
werkdock:
env:
DEPLOY_TARGET: user@host:~/doms/example.org/htdocs-ssl
```
```yaml
# .werkator.yml — the repository's part: what
builds:
deploy:
cleanCommand: ""
buildCommand: scripts/deploy-prod.sh -y "$DEPLOY_TARGET"
```
A follow-up whose predecessor no definition has, and a cycle of follow-ups, refuse the start with a message naming the definition — a deployment that silently never runs is the failure the flat-key refusal exists to prevent.
A branch whose committed config drops or renames the predecessor only loses its follow-up, with a warning naming the branch.
A one-shot `werkator build` runs no follow-ups — its process ends with its build — and says which ones the server would have run.
Selector: `trigger.branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches. Selector: `trigger.branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches.
A pattern prefixed with `!` excludes instead, and an exclusion always wins regardless of order — `["*", "!master"]` is every branch but master. A pattern prefixed with `!` excludes instead, and an exclusion always wins regardless of order — `["*", "!master"]` is every branch but master.
That is how a branch gets a build of its own without being built by the default one as well. That is how a branch gets a build of its own without being built by the default one as well.
@@ -409,6 +453,7 @@ Both parts combine as an intersection.
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `werkdock` with all their keys. Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `werkdock` with all their keys.
A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults. A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults.
`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config. `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
So is the trigger of a [follow-up build](#follow-up-builds).
Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited. Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited.
Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only. Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only.
Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of. Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of.
@@ -222,6 +222,7 @@ class InitCommand(
# branches: ["*", "!master"] # names or globs; "!" excludes; default: all # branches: ["*", "!master"] # names or globs; "!" excludes; default: all
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05) # atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
# activeWithin: 24h # only branches with recent commits # activeWithin: 24h # only branches with recent commits
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
# run before each build # run before each build
cleanCommand: rm -rf build cleanCommand: rm -rf build
# shell command for each build # shell command for each build