6 Commits
Author SHA1 Message Date
mhoennigandClaude Opus 5 3ffc1a8eed docs(prs): PR#19 — the sandbox config section is werkdock
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:27:16 +02:00
mhoennigandClaude Opus 5 34af8ba1bf feat(config): the sandbox section is werkdock, not bwrap
`bwrap` named the mechanism one layer below the tool that actually runs it: since
v1.0.0 Werkator does not invoke bwrap at all, it shells out to the werkdock CLI —
which made `bwrap.werkdock` a key naming its own executor.

The section is `werkdock` now and that key is `werkdock.binary`; BwrapConfig,
BwrapOverrides and BwrapBuildRunner follow the name. A file still writing `bwrap`
is read as before and warned about once per file, in `renameLegacySandbox` on the
raw map of every layer before merging — so nothing downstream knows two names, and
the old name is not a way around the pinning either. Renaming rather than refusing,
because the section lives in the machine configuration of every webspace instance,
which no repository tracks; the hard refusal belongs to the release that sets
ConfigVersions.FORMAT_BROKE_IN, where a file declaring no version can be caught
by name at all.

WERKATOR_SANDBOX in tools/remote follows, and still accepts `bwrap`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:27:16 +02:00
mhoennigandClaude Opus 5 95564587d3 docs(prs): PR#18 — tools/remote drives any host layout
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:27:11 +02:00
mhoennigandClaude Opus 5 618a2acb9f docs(deployment): tools/remote drives any host layout
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:27:11 +02:00
mhoennigandClaude Opus 5 2ffc14a20c fix(remote): upload before stopping the service, and verify the transfer
instance-update stopped the unit and only then started the upload, so a transfer
that dies mid-way leaves the host with no running Werkator and nothing to start
again. That is not theoretical: deploying to vm4006 on 2026-09-03 failed with
"scp: Connection closed" with the service already stopped.

The upload now happens before the stop, and each artifact is transferred to a
.part file whose sha256 is compared with the local one before it is moved into
place, retrying twice. A truncated archive would otherwise unpack into a broken
runtime, which is worse than the failed transfer it came from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:27:11 +02:00
mhoennigandClaude Opus 5 b5df75f844 feat(remote): the host layout is configurable, not the mih convention
tools/remote assumed the layout instance-install creates: the watched repository
in $WERKATOR_PATH/werkator, the runtime in $WERKATOR_PATH/.werkator, a werkdock
binary and a rootfs beside it, and the unit hardcoded as werkator-werkator.service.
An installation that predates the script — vm4006, a docker host with the repository
in ~/hs.hsadmin.ng and the runtime in ~/opt — could not be deployed with it at all.

WERKATOR_REPO_DIR, WERKATOR_INSTALL_DIR and WERKATOR_SANDBOX name the three values
that actually differ; their defaults are what instance-install writes, so the
existing env files resolve to exactly the same paths as before. The unit name is
derived from the repository directory the way SystemdServiceFiles.unitName does it,
instead of being spelled out. With WERKATOR_SANDBOX=docker the werkdock binary and
the rootfs archive are neither built nor uploaded — a docker host has no sandbox to
install, and check-prerequisites asks the docker daemon instead of werkdock doctor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:27:11 +02:00
26 changed files with 65 additions and 1456 deletions
+3 -3
View File
@@ -40,9 +40,9 @@ All production code lives under `de.hoennig.werkator`, with sub-packages `comman
- Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly. Server routes carry the repository as `/repos/<name>/…` and `/api/repos/<name>/…`, with the unscoped form permanently meaning the served repository; the pages stay per repository and a drop-down in the page title switches between them. - Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly. Server routes carry the repository as `/repos/<name>/…` and `/api/repos/<name>/…`, with the unscoped form permanently meaning the served repository; the pages stay per repository and a drop-down in the page title switches between them.
- When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`. - When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
- Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config. - Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and werkdock (`werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`) sandbox policies, the trust gate (`requirePullRequest`), and the trigger of a follow-up build (`trigger.afterSuccessOf` everywhere, and the whole `trigger` block of a definition the host defines as a follow-up). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, bypass its own pull-request gate, or deploy itself; a branch's definitions apply to that branch alone. - A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and werkdock (`werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`) sandbox policies, and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone.
- A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `afterSuccessOf`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `afterSuccessOf` makes a definition the follow-up of another one — a deployment is a build that follows a green build (PR#23): it runs at the predecessor's commit after every green run of it, whoever started that run, and the `FollowUpTrigger` that enqueues it is armed only by `Watcher.start()`. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins. - A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins.
- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`, and the trigger of a follow-up build. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called `bwrap` until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets `ConfigVersions.FORMAT_BROKE_IN`. - The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary`. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called `bwrap` until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets `ConfigVersions.FORMAT_BROKE_IN`.
- `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version. - `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version.
- Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats. - Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats.
- Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK. - Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK.
-22
View File
@@ -8,28 +8,6 @@ Lightweight, declarative and highly opinionated software build system (CI/CD).
- [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init` - [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init`
- [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy - [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy
## Adding a Gitea Repository
One instance serves several repositories (`docs/deployment.md`, ADR 0009).
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
It prints the registry entry: add it to `~/.werkator.yml` under `repositories:`, then restart the service.
The optional `[<name>]` overrides the directory basename: it becomes the route segment (`/repos/<name>/…`) and the UI switcher entry, so it must be unique.
Needed only when the clone directory name is wrong or collides — e.g. `michael.hoennig.de.git` checked out as `michael.hoennig.de`, or two forges serving a repo of the same name.
A **public** repository needs nothing else: the clone runs anonymously.
A **private** repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning:
```yaml
defaults:
git:
account: <gitea-user>
token: <token> # Gitea → Settings → Applications → Generate Token, scope read:repository
```
## Developer Setup ## Developer Setup
Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`: Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`:
+3 -48
View File
@@ -101,9 +101,7 @@ single branch may decide it:
- the container sandbox policy: `docker.enabled`/`docker.network` and - the container sandbox policy: `docker.enabled`/`docker.network` and
`werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` — host-pinned as `werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` — host-pinned as
long as only the host's configuration sets them, master-pinned once the committed long as only the host's configuration sets them, master-pinned once the committed
configuration does; configuration does.
- the trigger of a [follow-up build](#follow-up-builds): `trigger.afterSuccessOf` in every definition, and the whole `trigger` block of a definition the host defines as a follow-up.
A branch may say what its deployment does, never that — or for which branches — it happens.
The distinction is documentary. The distinction is documentary.
Werkator applies one rule: every pinned key is stripped from the branch layer, and the Werkator applies one rule: every pinned key is stripped from the branch layer, and the
@@ -112,7 +110,7 @@ The names say where a key is meant to live, not how it is enforced.
This keeps a branch from reaching credentials, reporting statuses to another repository, This keeps a branch from reaching credentials, reporting statuses to another repository,
raising the global concurrency, disabling its own build container, changing its network raising the global concurrency, disabling its own build container, changing its network
mode, bypassing its own pull-request gate, or deploying itself. Everything else is the branch's to decide — mode, or bypassing its own pull-request gate. Everything else is the branch's to decide —
it can already run any command through `buildCommand`. it can already run any command through `buildCommand`.
The pinned settings are stripped wherever they appear, in a build definition as well as in The pinned settings are stripped wherever they appear, in a build definition as well as in
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
@@ -254,7 +252,6 @@ builds:
# branches: ["*", "!master"] # names or globs; a "!" pattern excludes; default: all # branches: ["*", "!master"] # names or globs; a "!" pattern excludes; default: all
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05) # atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
# activeWithin: 24h # only branches with commits in the last 24h # activeWithin: 24h # only branches with commits in the last 24h
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
# run before each build # run before each build
cleanCommand: rm -rf build cleanCommand: rm -rf build
# shell command for each build # shell command for each build
@@ -400,50 +397,9 @@ Writing any of its keys outside the block is refused with a message naming the d
Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC). Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC).
A slot may also be written as `??:MM` — that minute of every hour, expanded to its 24 slots, so the build runs hourly. A slot may also be written as `??:MM` — that minute of every hour, expanded to its 24 slots, so the build runs hourly.
Only the latest due slot of a day triggers, so slots missed while the server was down are skipped instead of piling up, and a slot whose pool is still building is retried on the next poll cycle until it succeeds. Only the latest due slot of a day triggers, so slots missed while the server was down are skipped instead of piling up, and a slot whose pool is still building is retried on the next poll cycle until it succeeds.
A definition may combine them; one with none of the three triggers (`onPush`, `atTimes`, `afterSuccessOf`) never runs automatically — which is how `builds.default` is written when it is meant as a settings base only. A definition may have both; one with neither never triggers automatically — which is how `builds.default` is written when it is meant as a settings base only.
Werkator logs a warning once when no definition has a trigger at all, because such an instance never builds anything on its own. Werkator logs a warning once when no definition has a trigger at all, because such an instance never builds anything on its own.
#### Follow-up builds
`afterSuccessOf: <name>` makes a definition a *follow-up* of another one, its *predecessor*: it runs on the predecessor's branch at the predecessor's commit whenever a run of the predecessor ends green.
This is how a deployment is configured: a deployment is a build that follows a green build, and it gets everything a build has — its own row in History with log and duration, its own Gitea check under its own `statusContext`, restart, cancel, artifacts, and the per-branch serialization.
Every green run counts, whatever started it — the push watcher, an `atTimes` slot, a UI restart, `werkator retry`, or the startup recovery — and a repeated green run of the same commit triggers the follow-up again.
The follow-up builds the commit that was tested, not the branch's current origin head.
A failed, cancelled, or interrupted run triggers nothing.
The follow-up runs in the branch's worktree, after its predecessor, in the branch's sandbox or container — so a deployment tool like the Docker CLI is provided the way a compiler is.
It inherits `builds.default` like every definition, so a deployment that wants the predecessor's output has to set `cleanCommand: ""` itself; a deployment command should rather be self-contained and rebuild what it ships, because a restart of the follow-up alone runs without its predecessor.
The pull-request gate is not consulted for a follow-up: the predecessor passed it for the same commit, and the follow-up's `branches` selector is its own gate.
The trigger of a follow-up is pinned: a branch's committed config can neither add `afterSuccessOf` to a definition nor change the `trigger` block of a definition the host defines as a follow-up, so a branch cannot deploy itself.
What the deployment *does* comes with the repository, like every build command; where and when it happens, and with which credentials, is the host's.
The host's part lives in `.git/werkator/.werkator.yml`; credentials reach the sandbox like any build setting, through `werkdock.env`/`docker.env`, and a file such as an SSH key through the werkdock sandbox's persistent toolchain home, `.git/werkator/buildenv/home/` on the host, which the sandbox mounts as `/root`.
```yaml
# .git/werkator/.werkator.yml — the host's part: when, for which branches, with what
builds:
deploy:
trigger:
afterSuccessOf: frontend
branches: ["main"]
statusContext: werkator/deploy
werkdock:
env:
DEPLOY_TARGET: user@host:~/doms/example.org/htdocs-ssl
```
```yaml
# .werkator.yml — the repository's part: what
builds:
deploy:
cleanCommand: ""
buildCommand: scripts/deploy-prod.sh -y "$DEPLOY_TARGET"
```
A follow-up whose predecessor no definition has, and a cycle of follow-ups, refuse the start with a message naming the definition — a deployment that silently never runs is the failure the flat-key refusal exists to prevent.
A branch whose committed config drops or renames the predecessor only loses its follow-up, with a warning naming the branch.
A one-shot `werkator build` runs no follow-ups — its process ends with its build — and says which ones the server would have run.
Selector: `trigger.branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches. Selector: `trigger.branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches.
A pattern prefixed with `!` excludes instead, and an exclusion always wins regardless of order — `["*", "!master"]` is every branch but master. A pattern prefixed with `!` excludes instead, and an exclusion always wins regardless of order — `["*", "!master"]` is every branch but master.
That is how a branch gets a build of its own without being built by the default one as well. That is how a branch gets a build of its own without being built by the default one as well.
@@ -453,7 +409,6 @@ Both parts combine as an intersection.
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `werkdock` with all their keys. Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `werkdock` with all their keys.
A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults. A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults.
`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config. `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
So is the trigger of a [follow-up build](#follow-up-builds).
Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited. Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited.
Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only. Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only.
Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of. Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of.
+3 -4
View File
@@ -120,10 +120,9 @@ Adding a repository is editing a registry entry — never a data migration, beca
tools/remote --env-file .env.<instance> werkator repo-add https://github.com/<owner>/<repo>.git [<name>] tools/remote --env-file .env.<instance> werkator repo-add https://github.com/<owner>/<repo>.git [<name>]
``` ```
It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry. It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry.
It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back. It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back.
Cloning and initialising is mechanical; registering is a decision. Cloning and initialising is mechanical; registering is a decision.
A private `https` origin authenticates with the shared `defaults.git.account`/`defaults.git.token` of `~/.werkator.yml` (the token travels via a one-shot `GIT_ASKPASS` on the host, never in a URL or process list); enter those once before cloning a private repository — without them only public origins clone.
4. **Restart** the service; startup recovery re-enqueues what was in flight: 4. **Restart** the service; startup recovery re-enqueues what was in flight:
@@ -1,42 +0,0 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
The build for commit `7028ca8` on `main` failed on the mih09 production instance with
`BuildExecutorTest > with maxConcurrent 1 a second branch stays PENDING until the first finished`,
while a retry of the very same commit passed, and the test passes locally.
The test was timing-dependent.
It started a build for `branch-a` whose build command was `sleep 1`, immediately started a second build for `branch-b`,
and then asserted — without any synchronization at all — that `branch-b` was still `PENDING`.
That assertion only held as long as the test thread reached it within the one second `branch-a` slept.
On a shared host under CPU contention the executor can get through `branch-a` entirely (queued, running, slept, succeeded) first,
and `branch-b` is then already `RUNNING` or `SUCCESS` when the assertion runs.
The failure therefore says nothing about the executor; it is pure scheduling noise that costs a build and a retry every time it hits.
## Non-Goals
- No change to production code — `BuildExecutor` is not touched, its queueing behaviour is unchanged.
- No sweep of the other timing-sensitive tests in the suite; only the one that actually flaked is fixed.
## The Solution
`branch-a` no longer sleeps for a fixed time, it blocks until the test says so:
its build command is `until [ -f gate ]; do sleep 0.05; done`, and the build workspace is the test's working directory.
The test now
1. waits (via `eventually`) until `branch-a` is `RUNNING`, so the single executor slot is provably occupied,
2. asserts that `branch-b` is `PENDING` — which cannot race anything, because `branch-a` cannot finish before the gate file exists,
3. creates the gate file, and only then awaits both builds' `SUCCESS`.
The assertion on the event transitions (`branch-b` goes `RUNNING` only after `branch-a` reached `SUCCESS`) is unchanged.
A blocking gate was chosen over a mocked `BuildRunner` because it keeps the test on the real `ProcessBuildRunner`,
so it still covers the actual process handling rather than only the executor's bookkeeping.
Verified by running `BuildExecutorTest` five times on an idle machine and three more times with twice `nproc` busy-loops saturating the CPU,
which is the condition that produced the original failure.
- [BuildExecutorTest](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorTest.kt)
@@ -1,30 +0,0 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
The repository's home is `https://git.javagil.de/mi/werkator.git` since the move to the own Gitea instance,
but `tools/remote` still defaulted `WERKATOR_REPO_URL` to the GitHub mirror.
The mih09 production instance was cloned from that mirror and consequently watched a `main` that nobody pushes to any more:
it kept reporting the last GitHub state as green while three merged pull requests sat unbuilt on the real `main`.
The failure that started this — a flaky test already fixed on Gitea's `main` — could not be re-verified live,
because the instance had no way to see the fix.
## Non-Goals
- The existing clone on mih09 is not touched by this change; its remote was repointed by hand (`git remote set-url`), and `repo-init` skips an existing clone.
- The generic placeholder URLs in `docs/deployment.md` stay as they are — they describe cloning *any* watched repository, not werkator's own.
- No decision about mirroring to GitHub; the mirror simply stops being the source an instance builds from.
## The Solution
`REPO_URL` in [tools/remote](../../tools/remote) defaults to the Gitea URL, and the usage comment says so.
The value stays overridable via `WERKATOR_REPO_URL` in the instance's env file,
so an installation that deliberately watches a different remote is unaffected.
Anonymous HTTPS works against Gitea exactly as it did against GitHub, so no deploy key or token is involved.
Note that pull requests opened via AGit-Flow create no branch in Gitea,
so an instance watching this remote sees `main` only — branch builds require pushing real branches.
@@ -1,90 +0,0 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
`init --systemd` generates the host integration — the systemd unit's resource limits, the Apache `.htaccess` and the maintenance page — from the effective configuration.
It read that configuration through a helper with two independent defects, both of which fail silently.
**It swallowed every error.**
The load sat in `try { … } catch (_: Exception) { ServerConfig() }`.
Any configuration error at all — a missing required field, a malformed layer, a version floor violation — produced a default `ServerConfig` with a blank `publicBaseUrl`.
A repository with a broken `.werkator.yml` then looked exactly like one that simply has no public base URL configured:
the `.htaccess` and the maintenance page were skipped without a word.
This surfaced while verifying [PR#17](2026-09-03-PR%2317-maintenance-page.md) on mih09, where the missing files looked like an unconfigured `publicBaseUrl` and were in fact an unrelated validation error.
**It read from the wrong directory.**
The helper called `configLoader.load(Paths.get("."))` — the process's current directory — while everything else in the command works off the git top level resolved by `GitService.getTopLevel`.
`ConfigLoader.loadRaw` resolves the layers directly under the directory it is given and does not walk up to the repository root, so the two agree only when `init` happens to be invoked from the root itself.
From a subdirectory the command read another repository's configuration, or none.
That also broke `--apply`: the fragment is installed into the repository root deliberately before the systemd files are written, so that its port and limits reach the generated unit, and a current-directory read does not see it.
`init --systemd` runs during initial deployment setup, which is exactly when a silent wrong answer is most expensive.
## Non-Goals
- The fallback itself is kept: a configuration that cannot be loaded is not fatal for `init`, the units are still generated with the defaults.
During the very first bootstrap there is legitimately nothing to load yet.
- No change to `ConfigLoader`, to the configuration schema, or to any other command.
- No sweep for catch-all exception handlers elsewhere in the code base;
the two other `catch` blocks in `InitCommand` already print an `Error:` and abort, so they were only checked, not changed.
## The Scenarios
### Feature: init reports what it read and where it read it from
#### Background
- The *repository root* is the git top level as resolved by `GitService.getTopLevel`, the directory holding `.werkator.yml`, `.git/werkator/.werkator.yml` and an applied fragment.
- The *current directory* is the process working directory, which is the repository root only when `init` is invoked there.
#### Scenario#22.01: A broken configuration is named, not defaulted over
So that a validation error during deployment setup is not mistaken for an unconfigured installation.
- **Given** a repository whose effective configuration cannot be loaded
- **When** `init --systemd` runs
- **Then** the exception message is printed as a warning
- **and** the unit files are still generated with the default settings
- **and** the warning appears exactly once, although three settings are read from the configuration
##### Verified by
- [InitCommandTest: `--systemd warns once when the effective configuration cannot be loaded`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
#### Scenario#22.02: The configuration is read from the repository root
So that the generated host integration reflects the repository being initialized, whatever directory `init` was invoked from.
- **Given** a repository whose root configuration sets `server.publicBaseUrl` and `server.port`
- **and** a current directory that is not that repository root
- **When** `init --systemd` runs
- **Then** the `.htaccess` and the maintenance page are generated
- **and** the `.htaccess` proxies to the port from the root configuration
##### Verified by
- [InitCommandTest: `--systemd reads the configuration from the repository root, not the current directory`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
## The Solution
The catch-all now prints the exception message before falling back:
```
Warning: the effective configuration could not be loaded (<message>)
continuing with default server settings — check the generated unit and host files
```
The configuration is read three times while the systemd files are written (`memoryMax`, `tasksMax`, `publicBaseUrl`), which would repeat the warning three times.
It is therefore loaded once per run and cached in the command, and the cache is reset at the top of `run()` so a reused instance — the command is a Spring singleton — re-reads.
The repository root is passed down into the two accessors instead of `Paths.get(".")`.
This matches every other caller of `ConfigLoader.load` in the code base, all of which pass an explicit working directory;
`InitCommand` was the only one relying on the process's current directory.
Both fixes are the same failure in two forms — the command answered from a configuration it never actually read — which is why they are in one PR.
## Additional Changes
- None.
@@ -1,199 +0,0 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- [ADR 0007 — build definitions](../adrs/0007-2026-08-31.build-definitions.md): the `builds` section and its `trigger` block this PR extends.
- [configuration.md — build definitions](../configuration.md#build-definitions): the reference this PR updates.
- Werkbaum's `scripts/deploy-prod.sh`: the first deployment meant to run this way.
## The Problem
Werkator builds and reports, but it cannot deploy.
Werkbaum's production deployment is still a script run by hand from a developer machine, after looking at the Werkator status.
The step "if the build is green, run this" is exactly what a CI system is for, and the hand-off between the two is where releases go wrong: the wrong commit gets deployed, or a red commit, or nothing.
Three ways to add deployments were considered:
- A `deployCommand` next to `buildCommand`, run after a green `buildCommand` inside the same run.
Simple, but it creates a second command path inside one build — one log, one status, one duration for two different things — and a failed deployment would turn a green build red.
- A separate `deploy` section with its own executor path, statuses, and cancellation.
A second execution path next to `buildCommand`, with everything the first one has to be built again.
- **A deployment is a build that follows a green build.** Chosen.
A build definition may declare that it runs whenever another definition of the same branch turns green.
Everything a build has comes for free: its own row in History with log and duration, its own Gitea check under its own `statusContext`, restart without rebuilding, cancel, artifacts, and the per-branch serialization.
## Non-Goals
- A native execution path for deployments.
A follow-up build runs where every build of its branch runs — in the werkdock sandbox or the Docker container — so a deployment tool such as the Docker CLI is provided the same way a compiler is.
- Waiting for *several* builds to be green.
`afterSuccessOf` names one build; an `all of` form is a follow-up PR if a repository ever needs it.
- A separate secrets mechanism.
Deployment credentials reach the sandbox the way any build setting does — the host's `werkdock.env`/`docker.env` for the definition, and files placed in the sandbox's persistent toolchain home — and the pinning below keeps them on the branches the host names.
- Handing the predecessor's artifacts to the follow-up.
The follow-up runs in the same worktree, so the predecessor's output is *usually* still there, but a deployment command must not rely on it (see Open Questions).
- Recovering a follow-up whose enqueueing was lost to a server restart between the two builds.
The operator restarts the predecessor, which triggers the follow-up again.
## The Scenarios
### Feature: a build that follows a green build
#### Background
- A *follow-up build* is a build definition whose `trigger` block carries `afterSuccessOf: <name>`, naming another definition of the same configuration — its *predecessor*.
- The follow-up runs on the predecessor's branch at the predecessor's commit, never at the branch's current origin head, so a deployment always ships the commit that was tested.
- Like every non-default build it records under its own pool `<branch>@<name>`, and it should carry its own `statusContext` so that Gitea shows the deployment as a check of its own.
- *Green* is every run of the predecessor that ends with `SUCCESS`, whatever started it: the push watcher, an `atTimes` slot, a UI restart, `werkator retry`, or the startup recovery.
- The key belongs to the `trigger` block: it says *when* the build runs, so it is never inherited from `builds.default`, and writing it flat is refused like every other trigger key.
#### Scenario#23.01: A green predecessor triggers the follow-up on the same commit
So that a deployment ships exactly the commit that was just tested, with its own log, duration, and Gitea check.
- **Given** a definition `deploy` with `trigger.afterSuccessOf: frontend` and `statusContext: werkator/deploy`
- **and** the build `frontend` of branch `main` at commit `c1` is running
- **When** that build finishes with `SUCCESS`
- **Then** a build `deploy` of branch `main` at commit `c1` is enqueued
- **and** it is recorded under the pool `main@deploy`
- **and** its Gitea status is posted under the context `werkator/deploy`
- **and** the origin head of `main` having moved on to `c2` meanwhile changes nothing about that
##### Verified by
- [FollowUpTriggerTest — "a green predecessor enqueues the follow-up at the predecessor's commit"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
#### Scenario#23.02: Every green run triggers again, including a repeated run of the same commit
So that a deployment can be repeated by rerunning the build — and so that no run of a green build is silently *not* deployed, which would be more confusing than a redundant deployment.
- **Given** the build `frontend` of `main` at `c1` was green and `deploy` ran for it
- **When** `frontend` at `c1` is restarted from the UI, retried from the CLI, or rebuilt by an `atTimes` slot, and turns green again
- **Then** `deploy` is enqueued for `c1` again
##### Verified by
- [FollowUpTriggerTest — "every green run of the predecessor triggers the follow-up again"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
#### Scenario#23.03: A run that is not green triggers nothing
So that nothing is ever deployed from a failed, cancelled, or interrupted build.
- **Given** the same definitions
- **When** the build `frontend` ends as `FAILED`, `CANCELLED`, or `INTERRUPTED`
- **or** a build other than `frontend` ends as `SUCCESS`
- **Then** no `deploy` build is enqueued
##### Verified by
- [FollowUpTriggerTest — "only a SUCCESS of the named predecessor triggers"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
#### Scenario#23.04: The trigger of a follow-up is host-pinned
So that a branch can never deploy itself: a follow-up build is the host's way to hand real-world effects — targets, credentials — to a commit, and the host alone decides *when* and *for which branches* that happens.
- **Given** the host configuration defines `deploy` with `trigger: {afterSuccessOf: frontend, branches: [main]}`
- **When** a branch's committed `.werkator.yml` sets `builds.deploy.trigger.branches: ["*"]`
- **or** adds `afterSuccessOf` to any definition's trigger
- **Then** the host's trigger block of `deploy` is used unchanged, and the branch's `afterSuccessOf` is dropped with a warning naming the branch
- **and** a branch the host's selector does not name never runs `deploy`, however green its own builds are
##### Verified by
- [ConfigLoaderTest — "a follow-up trigger is pinned to the host, a branch cannot add or widen one"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#23.05: What the follow-up does comes with the repository
So that the deployment command is versioned with the code it deploys, like every other build command — while the host keeps the targets and credentials.
- **Given** the host defines `deploy` with its trigger and `werkdock.env: {DEPLOY_TARGET: …}`
- **and** the committed `.werkator.yml` of `main` defines `deploy` with `cleanCommand: ""` and `buildCommand: scripts/deploy-prod.sh -y "$DEPLOY_TARGET"`
- **When** `deploy` runs for `main`
- **Then** it runs the committed command with the host's environment, in the branch's sandbox, in the branch's worktree, after its predecessor and serialized with the branch's other builds
##### Verified by
- [BuildExecutorTest — "a follow-up build runs in its branch's worktree after its predecessor"](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorTest.kt)
- [ConfigLoaderTest — "a branch supplies the command of a host-triggered follow-up"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#23.06: A follow-up that could never fire is refused at start
So that a deployment that silently never runs cannot exist — the same reasoning that refuses a flat trigger key.
- **Given** a configuration whose `afterSuccessOf` names a definition that does not exist
- **or** whose follow-ups form a cycle (`a` after `b`, `b` after `a`)
- **or** which writes `afterSuccessOf` outside the `trigger` block
- **When** the configuration is loaded
- **Then** loading fails with a message naming the definition and the reason
- **and** a definition with only `afterSuccessOf` counts as triggered, so the "no build triggered" warning is not raised for it
##### Verified by
- [ConfigLoaderTest — "afterSuccessOf must name an existing definition and must not form a cycle"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
- [ConfigLoaderTest — "afterSuccessOf written flat is refused like every trigger key"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#23.07: Follow-ups fire in server mode only
So that the invariant "nothing is scheduled during CLI runs or tests" holds: a CLI `build` ends when its build ends, and a follow-up enqueued into a process that is about to exit would only ever be recorded as interrupted.
- **Given** `werkator build main` runs from the CLI and turns green
- **When** the command finishes
- **Then** no follow-up was enqueued, and the CLI says which follow-up the server would have run
##### Verified by
- [FollowUpTriggerTest — "the trigger listens only while the watcher runs"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
- [WatcherTest — "start arms the follow-up trigger before the recovery, stop disarms it"](../../src/test/kotlin/de/hoennig/werkator/watcher/WatcherTest.kt)
- [BuildCommandTest — "a green CLI build names the follow-up builds the server would run, and runs none"](../../src/test/kotlin/de/hoennig/werkator/commands/BuildCommandTest.kt)
## The Solution
`TriggerConfig` gets a fourth key, `afterSuccessOf: String` (empty: none).
It sits inside the `trigger` block on purpose, next to `onPush` and `atTimes`: it answers "when does this build run", so the structural rule of ADR 0007 makes it non-inheritable without touching any list, and `checkTriggerBlocks` refuses it written flat by adding it to `FLAT_TRIGGER_KEYS`.
`isTriggered` counts it, so a definition with nothing but a predecessor is not reported as never triggering.
A definition may itself be followed; `ConfigLoader` refuses an unknown predecessor and a cycle when it validates the merged configuration.
A new `FollowUpTrigger` in the `watcher` package listens to `BuildStatusChangedEvent`.
On a `SUCCESS` it resolves the definitions of the result's branch at the result's commit — the same `definitionsFor` the watcher uses for the branch layer — and enqueues, through `BuildExecutor.startBuild(repo, branch, commit, name)`, every definition whose `afterSuccessOf` names the finished build and whose selector selects the branch.
Passing the commit explicitly is what makes Scenario#23.01's last line true; the executor's duplicate guard folds a follow-up that is already queued for the same commit.
The listener is armed by the watcher's `start()` and disarmed by `stop()`, which is how it stays silent in CLI runs and tests (Scenario#23.07).
The pinning extends `stripPinned`: `afterSuccessOf` is removed from every trigger of the branch layer, and for every definition whose host trigger carries `afterSuccessOf` the branch layer's whole `trigger` block is dropped.
That is the smallest rule that makes Scenario#23.04 hold: a branch keeps the right to describe what its deployment does, and loses only the right to decide that — or where — it happens.
The pull-request gate is not consulted for a follow-up: the predecessor passed it already for the same commit, and the host's selector is the follow-up's own gate.
The follow-up runs like any other build of its branch — same worktree, same sandbox, serialized behind its predecessor.
It inherits `builds.default`, so a deployment that wants the predecessor's output has to set `cleanCommand: ""` itself.
Three places stay in sync with the new key, as the invariant demands: the data classes, the `init` templates (a commented `afterSuccessOf` line under the `trigger` block), and `docs/configuration.md`, which gets a "Follow-up builds" paragraph under build definitions and the new pinning in the branch-layer section.
`AGENTS.md` names the key in the trigger and pinning invariants.
## Open Questions
- **Credentials inside the sandbox.**
Werkdock clears the environment and mounts the repository's persistent toolchain home as `/root`, so an SSH key placed under `.git/werkator/buildenv/home/.ssh/` on the host is `/root/.ssh/` inside the sandbox, and `werkdock.env` carries the target.
For Docker there is no equivalent mount today; a key passed through `docker.env` works but is visible in `docker inspect`.
Implemented: nothing new — the PR documents the werkdock path in `configuration.md` and leaves a Docker mount to a follow-up if vm4006 ever deploys.
- **Restarting the follow-up alone.**
Restart re-runs `deploy` without its predecessor, in a worktree that may have been cleaned by a later build of the branch.
Implemented: allowed, and the reference recommends a self-contained deployment command that rebuilds what it ships (Werkbaum's `deploy-prod.sh` does).
- **A predecessor that exists only on a branch.**
The host's `afterSuccessOf: frontend` refers to a name the branch layer may rename.
Implemented: refused at start for the primary configuration; for a branch whose layer lacks the name, a warning once per branch and commit, and no follow-up
(verified by [ConfigLoaderTest — "a branch whose layer lacks the predecessor loses only its follow-up"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)).
An instance fragment checked by `init --apply` is not checked for its predecessor at all: the build it names may well live in the project config it is merged with, and the merged configuration is checked on every load anyway.
## Additional Changes
- `BuildStatusChangedEvent` now carries the `RepoContext` the result belongs to: a `BuildResult` does not know its repository, and the listener has to enqueue into the right one.
- `ConfigLoader.loadWithBranchLayer` and `loadForWorktree` take an optional branch name, used only to name the branch in the pinning warnings; the watcher passes it.
- The follow-up check distinguishes three loads: the primary configuration refuses a missing predecessor, a branch layer warns, and an instance fragment (`init --apply`) skips the check — its predecessor may live in the project config it is merged with.
- The "no build triggered" warning now names `afterSuccessOf` as the third trigger.
## Follow-up PRs
- Werkbaum: commit the `deploy` definition to its `.werkator.yml`, add the host part on mih09, and retire the manual `deploy-prod.sh` invocation from the README.
- `afterSuccessOf` as a list (all green), if a repository ever needs a deployment gated on more than one build.
- A Docker bind mount for credential files, if a Docker host deploys.
@@ -1,92 +0,0 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- ADR 0009 — multi-repo instance: the `defaults` block carries shared repository-level keys, read by `ConfigLoader`, never directly by consumers.
- `docs/deployment.md` — registry setup: `repo-add` clones, initialises, and prints the registry entry.
## The Problem
`tools/remote werkator repo-add <private-https-url>` fails on the host with `could not read Username`.
The clone runs anonymously, but the credentials exist only in the instance file's `defaults.git` block — which the script never consults.
Registering a private repository therefore needs a manual SSH session today.
## Non-Goals
The script still does not write `~/.werkator.yml`: registering stays the operator's decision.
No SSH-URL support: the forge is reached over `https` from the host.
No new config keys: `defaults.git.account`/`defaults.git.token` already exist.
## The Scenarios
### Feature: authenticated clone for private origins
#### Background
- The shared credentials live in `~/.werkator.yml` under `defaults.git` (ADR 0009).
- Public origins and instances without shared credentials must keep cloning anonymously.
#### Scenario#000.01: Private https origin clones with shared credentials
- **Given** `defaults.git.account`/`defaults.git.token` in `~/.werkator.yml` on the host
- **When** `tools/remote werkator repo-add <private-https-url>` runs
- **Then** the clone authenticates with those credentials and succeeds.
##### Verified by
- Manual stub-`git` test: URL carries `account@`, askpass answers the token, `GIT_TERMINAL_PROMPT=0`.
#### Scenario#000.02: Public origin clones anonymously
- **Given** no shared credentials (or a public repository)
- **When** `repo-add` or `repo-init` runs
- **Then** the clone runs exactly as before, without authentication.
##### Verified by
- Local helper test against a `file://` origin with and without an instance file.
#### Scenario#000.03: Token never leaks locally
- **Given** an authenticated clone
- **When** the command runs from the workstation
- **Then** the token appears in neither the local process list nor a repository config.
##### Verified by
- Code inspection: the token is read on the host and passed via a one-shot `GIT_ASKPASS` script.
## The Solution
`tools/remote` gained a `clone_repo` helper used by both `repo-init` and `repo-add`.
For `https://` URLs it ships a small Python helper (base64-encoded, so no `$` is expanded locally) to the host.
The helper reads `defaults.git.account`/`defaults.git.token` from `~/.werkator.yml`, puts the account into the URL, and hands the token to git via a one-shot `0700` `GIT_ASKPASS` script deleted in `finally`.
Without credentials it falls back to the plain anonymous clone; non-`https` URLs clone unchanged.
`docs/deployment.md` documents that the shared credentials must exist before cloning a private repository.
## Open Questions
- None.
## Attachments
### Adding a Gitea repository — example
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
Add the printed entry to `~/.werkator.yml` under `repositories:`, then restart the service.
A public repository needs nothing else: the clone runs anonymously.
A private repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning (token: Gitea → Settings → Applications → Generate Token, scope `read:repository`):
```yaml
defaults:
git:
account: <gitea-user>
token: <token>
```
@@ -119,7 +119,7 @@ class BuildExecutor(
artifactKey = runningBuild.artifactKey, artifactKey = runningBuild.artifactKey,
) )
repo.results.append(pending) repo.results.append(pending)
eventPublisher.publishEvent(BuildStatusChangedEvent(pending, repo)) eventPublisher.publishEvent(BuildStatusChangedEvent(pending))
val activeBuild = ActiveBuild(runningBuild, repo) val activeBuild = ActiveBuild(runningBuild, repo)
builds[runningBuild.artifactKey] = activeBuild builds[runningBuild.artifactKey] = activeBuild
publishGiteaStatus(activeBuild, BuildStatus.PENDING, duration = null) publishGiteaStatus(activeBuild, BuildStatus.PENDING, duration = null)
@@ -384,7 +384,7 @@ class BuildExecutor(
duration = duration, duration = duration,
artifactKey = runningBuild.artifactKey, artifactKey = runningBuild.artifactKey,
).also { build.repo.results.append(it) } ).also { build.repo.results.append(it) }
eventPublisher.publishEvent(BuildStatusChangedEvent(updated, build.repo)) eventPublisher.publishEvent(BuildStatusChangedEvent(updated))
publishGiteaStatus(build, status, duration) publishGiteaStatus(build, status, duration)
return updated return updated
} }
@@ -35,12 +35,7 @@ data class RunningBuild(
var runningSince: Instant? = null var runningSince: Instant? = null
} }
/** /** Published via Spring's `ApplicationEventPublisher` on every persisted status transition. */
* Published via Spring's `ApplicationEventPublisher` on every persisted status transition.
* Carries the repository because a [BuildResult] does not: a listener that reacts to the
* transition — the follow-up trigger — has to act on that repository.
*/
data class BuildStatusChangedEvent( data class BuildStatusChangedEvent(
val result: BuildResult, val result: BuildResult,
val repo: RepoContext,
) )
@@ -1,11 +1,9 @@
package de.hoennig.werkator.commands package de.hoennig.werkator.commands
import de.hoennig.werkator.build.BuildStatus import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.config.BuildDefinition
import de.hoennig.werkator.git.GitService import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext import de.hoennig.werkator.repo.RepoContext
import de.hoennig.werkator.repo.RepoRegistry import de.hoennig.werkator.repo.RepoRegistry
import de.hoennig.werkator.watcher.FollowUpTrigger
import org.springframework.stereotype.Component import org.springframework.stereotype.Component
import picocli.CommandLine.Command import picocli.CommandLine.Command
import picocli.CommandLine.ExitCode import picocli.CommandLine.ExitCode
@@ -29,7 +27,6 @@ class BuildCommand(
private val gitService: GitService, private val gitService: GitService,
private val consoleBuildRunner: ConsoleBuildRunner, private val consoleBuildRunner: ConsoleBuildRunner,
private val registry: RepoRegistry, private val registry: RepoRegistry,
private val followUpTrigger: FollowUpTrigger,
) : Callable<Int> { ) : Callable<Int> {
@Mixin @Mixin
var repoOption = RepoOption() var repoOption = RepoOption()
@@ -61,33 +58,9 @@ class BuildCommand(
} }
println("building branch $branch at commit ${commit.take(12)}") println("building branch $branch at commit ${commit.take(12)}")
val status = consoleBuildRunner.buildAndStream(repo, branch, commit) val status = consoleBuildRunner.buildAndStream(repo, branch, commit)
if (status == BuildStatus.SUCCESS) {
reportSkippedFollowUps(branch, commit)
}
return if (status == BuildStatus.SUCCESS) ExitCode.OK else ExitCode.SOFTWARE return if (status == BuildStatus.SUCCESS) ExitCode.OK else ExitCode.SOFTWARE
} }
/**
* A one-shot build ends with its process, so it never runs the follow-up builds the
* server would enqueue after a green run (PR#23) it says which ones instead of
* leaving the operator to wonder why nothing was deployed.
*/
private fun reportSkippedFollowUps(
branch: String,
commit: String,
) {
val followUps =
try {
followUpTrigger.followUpsOf(repo, branch, commit, BuildDefinition.DEFAULT)
} catch (e: Exception) {
System.err.println("warning: could not determine the follow-up builds (${e.message})")
return
}
if (followUps.isNotEmpty()) {
println("note: the server would now run the follow-up build(s) ${followUps.joinToString(", ")}; a CLI build does not")
}
}
/** A one-shot build should still work offline, from the last fetched origin state. */ /** A one-shot build should still work offline, from the last fetched origin state. */
private fun fetchBestEffort() { private fun fetchBestEffort() {
try { try {
@@ -48,7 +48,6 @@ class InitCommand(
internal var javaExecutableResolver: () -> Path = { Paths.get(System.getProperty("java.home"), "bin", "java") } internal var javaExecutableResolver: () -> Path = { Paths.get(System.getProperty("java.home"), "bin", "java") }
override fun run() { override fun run() {
cachedServerConfig = null
val normalizedWorkingDir = workingDir.toAbsolutePath().normalize() val normalizedWorkingDir = workingDir.toAbsolutePath().normalize()
val root = val root =
try { try {
@@ -89,7 +88,7 @@ class InitCommand(
if (url == null) return DetectedValues() if (url == null) return DetectedValues()
if (url.startsWith("http")) { if (url.startsWith("http")) {
val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/(.+?)(?:\.git)?$""") val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/([^/.]+)(?:\.git)?""")
val match = regex.find(url) val match = regex.find(url)
if (match != null) { if (match != null) {
val (user, host, owner, repo) = match.destructured val (user, host, owner, repo) = match.destructured
@@ -102,7 +101,7 @@ class InitCommand(
} }
} else if (url.contains("@") && url.contains(":")) { } else if (url.contains("@") && url.contains(":")) {
// Assume SSH: git@host:owner/repo.git // Assume SSH: git@host:owner/repo.git
val regex = Regex("""([^@]+)@([^:]+):([^/]+)/(.+?)(?:\.git)?$""") val regex = Regex("""([^@]+)@([^:]+):([^/]+)/([^/.]+)(?:\.git)?""")
val match = regex.find(url) val match = regex.find(url)
if (match != null) { if (match != null) {
val (_, host, owner, repo) = match.destructured val (_, host, owner, repo) = match.destructured
@@ -223,7 +222,6 @@ class InitCommand(
# branches: ["*", "!master"] # names or globs; "!" excludes; default: all # branches: ["*", "!master"] # names or globs; "!" excludes; default: all
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05) # atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
# activeWithin: 24h # only branches with recent commits # activeWithin: 24h # only branches with recent commits
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
# run before each build # run before each build
cleanCommand: rm -rf build cleanCommand: rm -rf build
# shell command for each build # shell command for each build
@@ -298,31 +296,14 @@ class InitCommand(
* already loadable (re-running `init --systemd` on an installed instance); during * already loadable (re-running `init --systemd` on an installed instance); during
* the very first bootstrap they stay unset and the defaults (no directives) apply. * the very first bootstrap they stay unset and the defaults (no directives) apply.
*/ */
private fun loadedSystemdConfig(root: Path): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig(root).systemd private fun loadedSystemdConfig(): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig().systemd
/** Loaded once per run, so a broken configuration is reported once and not per caller. */ private fun loadedServerConfig(): de.hoennig.werkator.config.ServerConfig =
private var cachedServerConfig: de.hoennig.werkator.config.ServerConfig? = null try {
configLoader.load(Paths.get(".")).server
/** } catch (_: Exception) {
* Read from the repository root like every other file this command touches the de.hoennig.werkator.config
* layers sit there, not in whatever directory the process happens to run in, and .ServerConfig()
* an applied fragment must reach the generated unit even when `init` is invoked
* from a subdirectory.
*
* A configuration error here is not fatal the units are still generated with defaults
* but it must not pass for "nothing configured": without the warning a broken `.werkator.yml`
* looks exactly like an unset `publicBaseUrl` and the host integration is skipped silently.
*/
private fun loadedServerConfig(root: Path): de.hoennig.werkator.config.ServerConfig =
cachedServerConfig ?: run {
try {
configLoader.load(root).server
} catch (e: Exception) {
println("Warning: the effective configuration could not be loaded (${e.message})")
println(" continuing with default server settings — check the generated unit and host files")
de.hoennig.werkator.config
.ServerConfig()
}.also { cachedServerConfig = it }
} }
private fun createSystemdFiles( private fun createSystemdFiles(
@@ -346,8 +327,8 @@ class InitCommand(
javaExecutable = javaExecutableResolver(), javaExecutable = javaExecutableResolver(),
jarPath = jarPath, jarPath = jarPath,
envFile = envFile, envFile = envFile,
memoryMax = loadedSystemdConfig(root).memoryMax, memoryMax = loadedSystemdConfig().memoryMax,
tasksMax = loadedSystemdConfig(root).tasksMax, tasksMax = loadedSystemdConfig().tasksMax,
), ),
) )
println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}") println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}")
@@ -370,7 +351,7 @@ class InitCommand(
// generated host integration like the units: only meaningful behind a web // generated host integration like the units: only meaningful behind a web
// frontend, so it needs a public base URL; unused elsewhere and harmless // frontend, so it needs a public base URL; unused elsewhere and harmless
val server = loadedServerConfig(root) val server = loadedServerConfig()
if (server.publicBaseUrl.isNotBlank()) { if (server.publicBaseUrl.isNotBlank()) {
val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME) val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME)
htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port)) htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port))
@@ -89,9 +89,8 @@ data class BuildDefinition(
* When a build runs and for which branches the `trigger` block of a build definition, * When a build runs and for which branches the `trigger` block of a build definition,
* and the one part of it that is never inherited from `builds.default`. * and the one part of it that is never inherited from `builds.default`.
* *
* A definition with neither [onPush] nor [atTimes] nor [afterSuccessOf] never triggers * A definition with neither [onPush] nor [atTimes] never triggers automatically; that is
* automatically; that is how `builds.default` is written when it is meant as a settings * how `builds.default` is written when it is meant as a settings base only.
* base only.
*/ */
data class TriggerConfig( data class TriggerConfig(
/** Build every new commit of the selected branches. */ /** Build every new commit of the selected branches. */
@@ -113,19 +112,7 @@ data class TriggerConfig(
* empty applies no age filter. Combines with [branches] as an intersection. * empty applies no age filter. Combines with [branches] as an intersection.
*/ */
val activeWithin: String = "", val activeWithin: String = "",
/**
* Name of another definition of this configuration the *predecessor*: this build
* runs on the predecessor's branch at the predecessor's commit whenever a run of it
* ends with `SUCCESS`, whatever started that run (PR#23). Empty means none. Sits in
* the trigger block because it says *when* this build runs, so it is never inherited;
* and it is host-pinned, because a follow-up build is the host's way to hand
* real-world effects deployment targets, credentials to a green commit.
*/
val afterSuccessOf: String = "",
) { ) {
/** True when this build follows another one, see [afterSuccessOf]. */
fun isFollowUp(): Boolean = afterSuccessOf.isNotBlank()
/** True when [branch] matches the [branches] patterns (or none are configured) and none excludes it. */ /** True when [branch] matches the [branches] patterns (or none are configured) and none excludes it. */
fun selectsByName(branch: String): Boolean { fun selectsByName(branch: String): Boolean {
val (excluding, including) = branches.partition { it.startsWith(EXCLUDE_PREFIX) } val (excluding, including) = branches.partition { it.startsWith(EXCLUDE_PREFIX) }
@@ -90,8 +90,7 @@ class ConfigLoader(
fun loadForWorktree( fun loadForWorktree(
workingDir: Path, workingDir: Path,
worktreeDir: Path, worktreeDir: Path,
branch: String? = null, ): WerkatorConfig = withBranchLayer(workingDir, loadFile(worktreeDir.resolve(ConfigFiles.firstExisting(worktreeDir)).toFile()))
): WerkatorConfig = withBranchLayer(workingDir, loadFile(worktreeDir.resolve(ConfigFiles.firstExisting(worktreeDir)).toFile()), branch)
/** /**
* The primary/`.git` config with the committed `.werkator.yml` of one branch * The primary/`.git` config with the committed `.werkator.yml` of one branch
@@ -108,53 +107,30 @@ class ConfigLoader(
* (`requirePullRequest`, which decides whether the branch is built at all). * (`requirePullRequest`, which decides whether the branch is built at all).
* They are stripped from the branch layer before merging, so a branch can neither * They are stripped from the branch layer before merging, so a branch can neither
* escape its container, nor bypass its own pull-request gate, nor raise the global * escape its container, nor bypass its own pull-request gate, nor raise the global
* concurrency, nor reach the credentials. The trigger of a follow-up build is pinned * concurrency, nor reach the credentials.
* the same way (PR#23): a branch may say what its deployment does, never that or
* where it happens. [branch] only names the branch in the warnings.
*/ */
fun loadWithBranchLayer( fun loadWithBranchLayer(
workingDir: Path, workingDir: Path,
branchConfigYaml: String?, branchConfigYaml: String?,
branch: String? = null, ): WerkatorConfig = withBranchLayer(workingDir, parseYaml(branchConfigYaml))
): WerkatorConfig = withBranchLayer(workingDir, parseYaml(branchConfigYaml), branch)
private fun withBranchLayer( private fun withBranchLayer(
workingDir: Path, workingDir: Path,
branchLayer: Map<String, Any?>, branchLayer: Map<String, Any?>,
branch: String?,
): WerkatorConfig { ): WerkatorConfig {
// scoped to this branch: an incompatible branch config fails its own builds and // scoped to this branch: an incompatible branch config fails its own builds and
// must never stop the server or hold up the branches that are fine // must never stop the server or hold up the branches that are fine
checkVersion(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT) checkVersion(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
checkTriggerBlocks(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT) checkTriggerBlocks(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
val primary = loadRaw(workingDir) return toConfig(deepMerge(loadRaw(workingDir), stripPinned(branchLayer)))
return toConfig(deepMerge(primary, stripPinned(branchLayer, primary, branch)), MissingPredecessor.WARN)
} }
/** private fun toConfig(raw: Map<String, Any?>): WerkatorConfig {
* What a follow-up whose predecessor no definition has means for this load, see
* [checkFollowUps].
*/
private enum class MissingPredecessor {
/** The primary configuration: a deployment that could never fire refuses the start. */
REFUSE,
/** A branch layer on top: the branch renamed or dropped the build the host's trigger names, and only loses its follow-up. */
WARN,
/** A fragment checked on its own: the predecessor may well live in the project config it is merged with later. */
SKIP,
}
private fun toConfig(
raw: Map<String, Any?>,
missingPredecessor: MissingPredecessor = MissingPredecessor.REFUSE,
): WerkatorConfig {
val config = val config =
if (raw.isEmpty()) { if (raw.isEmpty()) {
WerkatorConfig() WerkatorConfig()
} else { } else {
yaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw), missingPredecessor), WerkatorConfig::class.java) yaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw)), WerkatorConfig::class.java)
} }
return defaultPublicBaseUrl(config) return defaultPublicBaseUrl(config)
} }
@@ -202,11 +178,7 @@ class ConfigLoader(
* second as soon as the committed configuration carries them. * second as soon as the committed configuration carries them.
*/ */
@Suppress("UNCHECKED_CAST") @Suppress("UNCHECKED_CAST")
private fun stripPinned( private fun stripPinned(branchLayer: Map<String, Any?>): Map<String, Any?> {
branchLayer: Map<String, Any?>,
primary: Map<String, Any?>,
branch: String?,
): Map<String, Any?> {
if (branchLayer.isEmpty()) { if (branchLayer.isEmpty()) {
return branchLayer return branchLayer
} }
@@ -216,55 +188,9 @@ class ConfigLoader(
val entries = result[section] as? Map<String, Any?> ?: continue val entries = result[section] as? Map<String, Any?> ?: continue
result[section] = entries.mapValues { (_, value) -> stripPinnedSettings(value) } result[section] = entries.mapValues { (_, value) -> stripPinnedSettings(value) }
} }
(result["builds"] as? Map<String, Any?>)?.let { builds ->
val hostBuilds = primary["builds"] as? Map<String, Any?> ?: emptyMap()
result["builds"] = builds.mapValues { (name, value) -> stripPinnedTrigger(name, value, hostBuilds[name], branch) }
}
return result return result
} }
/**
* The follow-up part of the pinning (PR#23): a branch's definition loses its
* `afterSuccessOf`, and where the host's definition of the same name is a follow-up,
* the branch's whole `trigger` block otherwise a branch could widen the host's
* selector to include itself, and deploy itself with the host's credentials. Said
* out loud, because a trigger the branch wrote and does not see in effect is a
* question it would otherwise ask the log in vain.
*/
@Suppress("UNCHECKED_CAST")
private fun stripPinnedTrigger(
name: String,
value: Any?,
hostDefinition: Any?,
branch: String?,
): Any? {
val definition = value as? Map<String, Any?> ?: return value
val trigger = definition["trigger"] as? Map<String, Any?> ?: return value
val where = branch?.let { "branch '$it'" } ?: "this branch"
if (predecessorOf(hostDefinition) != null) {
log.warn(
"ignoring the trigger block of builds.{} in the committed {} of {}: the host defines that build as a follow-up, " +
"and when and where a follow-up runs is the host's decision alone",
name,
ConfigFiles.COMMITTED,
where,
)
return definition - "trigger"
}
if (predecessorOf(definition) == null) {
return value
}
log.warn(
"ignoring builds.{}.trigger.afterSuccessOf in the committed {} of {}: a branch cannot make a build follow another, " +
"only the host can",
name,
ConfigFiles.COMMITTED,
where,
)
val stripped = trigger - "afterSuccessOf"
return if (stripped.isEmpty()) definition - "trigger" else definition + ("trigger" to stripped)
}
@Suppress("UNCHECKED_CAST") @Suppress("UNCHECKED_CAST")
private fun stripPinnedSettings(value: Any?): Any? { private fun stripPinnedSettings(value: Any?): Any? {
val entry = value as? Map<String, Any?> ?: return value val entry = value as? Map<String, Any?> ?: return value
@@ -299,16 +225,12 @@ class ConfigLoader(
* an empty docker policy and run natively on the host, which is exactly the escape * an empty docker policy and run natively on the host, which is exactly the escape
* the pinned keys exist to prevent. * the pinned keys exist to prevent.
*/ */
private fun resolveBuildSections( private fun resolveBuildSections(raw: Map<String, Any?>): Map<String, Any?> {
raw: Map<String, Any?>,
missingPredecessor: MissingPredecessor,
): Map<String, Any?> {
@Suppress("UNCHECKED_CAST") @Suppress("UNCHECKED_CAST")
val definitions = raw["builds"] as? Map<String, Any?> ?: emptyMap() val definitions = raw["builds"] as? Map<String, Any?> ?: emptyMap()
if (definitions.isEmpty()) { if (definitions.isEmpty()) {
return mergeBranchDefaults(raw) return mergeBranchDefaults(raw)
} }
checkFollowUps(definitions, missingPredecessor)
if (raw.containsKey("branches") && warnedSections.add(LEGACY_BRANCHES_WARNING)) { if (raw.containsKey("branches") && warnedSections.add(LEGACY_BRANCHES_WARNING)) {
log.warn( log.warn(
"ignoring the branches section: this configuration defines builds, and a build definition " + "ignoring the branches section: this configuration defines builds, and a build definition " +
@@ -330,59 +252,13 @@ class ConfigLoader(
return return
} }
if (warnedSections.add(NO_TRIGGER_WARNING)) { if (warnedSections.add(NO_TRIGGER_WARNING)) {
log.warn("no build defines onPush, atTimes, or afterSuccessOf; the watcher will never start a build on its own") log.warn("no build defines onPush or atTimes; the watcher will never start a build on its own")
} }
} }
private fun isTriggered(definition: Any?): Boolean { private fun isTriggered(definition: Any?): Boolean {
val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return false val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return false
return trigger["onPush"] == true || return trigger["onPush"] == true || (trigger["atTimes"] as? List<*>)?.isNotEmpty() == true
(trigger["atTimes"] as? List<*>)?.isNotEmpty() == true ||
predecessorOf(definition) != null
}
private fun predecessorOf(definition: Any?): String? {
val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return null
return (trigger["afterSuccessOf"] as? String)?.takeIf { it.isNotBlank() }
}
/**
* A follow-up build that could never fire must not exist, for the same reason a flat
* trigger key is refused: a deployment that silently never runs is worse than a
* configuration that refuses to load (PR#23). Refused are a predecessor no definition
* has, and a cycle of follow-ups (a build following itself included) a cycle is a
* configuration error whoever wrote it, while a missing predecessor depends on what
* is being loaded ([MissingPredecessor]).
*/
private fun checkFollowUps(
definitions: Map<String, Any?>,
missingPredecessor: MissingPredecessor,
) {
val predecessors = definitions.mapValues { (_, definition) -> predecessorOf(definition) }
val effective = definitions.keys + BuildDefinition.DEFAULT
for ((name, predecessor) in predecessors) {
if (predecessor == null || predecessor in effective) continue
val message = "builds.$name follows '$predecessor' (trigger.afterSuccessOf), but no build of that name is defined"
when (missingPredecessor) {
MissingPredecessor.REFUSE -> throw ConfigFormatException("$message. Name an existing build, or remove the follow-up.")
MissingPredecessor.WARN -> log.warn("$message on this branch; the follow-up will not run for it")
MissingPredecessor.SKIP -> {}
}
}
for (start in predecessors.keys) {
val path = mutableListOf(start)
var current = predecessors[start]
while (current != null && current !in path) {
path += current
current = predecessors[current]
}
if (current == start) {
throw ConfigFormatException(
"builds.$start follows itself through trigger.afterSuccessOf (${path.joinToString(" -> ")} -> $start); " +
"a follow-up build cannot wait for its own success.",
)
}
}
} }
/** /**
@@ -550,10 +426,7 @@ class ConfigLoader(
checkVersion(raw, fragment.toString(), ROLLBACK_HINT) checkVersion(raw, fragment.toString(), ROLLBACK_HINT)
checkTriggerBlocks(raw, fragment.toString(), ROLLBACK_HINT) checkTriggerBlocks(raw, fragment.toString(), ROLLBACK_HINT)
try { try {
strictYaml.convertValue( strictYaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw)), WerkatorConfig::class.java)
resolveBuildSections(dropNonDefinitionBuilds(raw), MissingPredecessor.SKIP),
WerkatorConfig::class.java,
)
} catch (e: IllegalArgumentException) { } catch (e: IllegalArgumentException) {
throw IllegalArgumentException( throw IllegalArgumentException(
"instance fragment $fragment does not match the configuration schema: ${e.message}", "instance fragment $fragment does not match the configuration schema: ${e.message}",
@@ -737,7 +610,7 @@ class ConfigLoader(
private val TRIGGER_KEYS = setOf("trigger") private val TRIGGER_KEYS = setOf("trigger")
/** The keys that moved into [TRIGGER_KEYS]; still writing them flat is refused, not ignored. */ /** The keys that moved into [TRIGGER_KEYS]; still writing them flat is refused, not ignored. */
private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin", "afterSuccessOf") private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin")
/** /**
* Top-level sections owned by the instance once a home config exists (ADR 0009): * Top-level sections owned by the instance once a home config exists (ADR 0009):
@@ -1,134 +0,0 @@
package de.hoennig.werkator.watcher
import de.hoennig.werkator.build.BuildExecutor
import de.hoennig.werkator.build.BuildResult
import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.build.BuildStatusChangedEvent
import de.hoennig.werkator.config.BuildDefinition
import de.hoennig.werkator.config.ConfigFiles
import de.hoennig.werkator.config.ConfigLoader
import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext
import org.slf4j.LoggerFactory
import org.springframework.context.event.EventListener
import org.springframework.stereotype.Component
import java.time.Clock
import java.util.concurrent.atomic.AtomicBoolean
/**
* Runs the follow-up builds (PR#23): whenever a build ends with `SUCCESS`, every
* definition of that branch whose `trigger.afterSuccessOf` names the finished build
* and whose selector selects the branch is enqueued on the same branch at the *same
* commit*, never at the branch's current origin head, so a deployment always ships the
* commit that was tested. Every green run counts, whatever started it, and a repeated
* green run of the same commit triggers again: a run of a green build that is silently
* not deployed would be more confusing than a redundant deployment.
*
* The definitions are resolved with the branch's committed config at the finished
* build's commit the same layering the watcher applies so the follow-up's command
* comes with the repository while its trigger stays the host's (pinned by
* [ConfigLoader]). The pull-request gate is not consulted: the predecessor passed it for
* this very commit, and the host's selector is the follow-up's own gate.
*
* Armed by [Watcher.start] and disarmed by [Watcher.stop], so a CLI `build` whose
* process ends with its build never enqueues a follow-up into a JVM that is about to
* exit; the CLI names the follow-ups the server would have run instead.
*/
@Component
class FollowUpTrigger(
private val gitService: GitService,
private val configLoader: ConfigLoader,
private val buildExecutor: BuildExecutor,
private val clock: Clock,
) {
private val log = LoggerFactory.getLogger(FollowUpTrigger::class.java)
private val armed = AtomicBoolean(false)
fun arm() {
armed.set(true)
}
fun disarm() {
armed.set(false)
}
fun isArmed(): Boolean = armed.get()
@EventListener
fun onBuildStatusChanged(event: BuildStatusChangedEvent) {
if (!armed.get() || event.result.status != BuildStatus.SUCCESS) {
return
}
val result = event.result
try {
for (name in followUpsOf(event.repo, result)) {
log.info(
"[{}] enqueueing follow-up build {} of branch {} at commit {}, after {}",
event.repo.name,
name,
result.branch,
result.commit,
result.build,
)
buildExecutor.startBuild(event.repo, result.branch, result.commit, name)
}
} catch (e: Exception) {
log.error("[{}] could not enqueue the follow-ups of {} at {}", event.repo.name, result.name, result.commit, e)
}
}
/** The names of the builds that follow a green [result], in the order of their definitions; nothing is enqueued. */
fun followUpsOf(
repo: RepoContext,
result: BuildResult,
): List<String> = followUpsOf(repo, result.branch, result.commit, result.build)
/** The names of the builds that follow a green run of [build] on [branch] at [commit]; nothing is enqueued. */
fun followUpsOf(
repo: RepoContext,
branch: String,
commit: String,
build: String,
): List<String> {
val workingDir = repo.workingDir
val definitions = definitionsAt(repo, branch, commit)
val headCommittedAt = lazy { gitService.originBranchCommitTimes(workingDir)[branch] }
return definitions
.filter { (_, definition) -> definition.trigger.afterSuccessOf == build }
.filter { (_, definition) -> definition.trigger.selects(branch, { headCommittedAt.value }, clock.instant()) }
.keys
.toList()
}
/**
* The branch's definitions at [commit] not at its head, which may have moved on
* since the predecessor started. An unreadable branch config falls back to the
* primary definitions, like the watcher does.
*/
private fun definitionsAt(
repo: RepoContext,
branch: String,
commit: String,
): Map<String, BuildDefinition> {
val workingDir = repo.workingDir
return try {
configLoader
.loadWithBranchLayer(
workingDir,
ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) },
branch,
).effectiveBuildDefinitions()
} catch (e: Exception) {
log.warn(
"[{}] ignoring the committed {} of branch {} at {} for its follow-ups: {}",
repo.name,
ConfigFiles.COMMITTED,
branch,
commit,
e.message ?: e.javaClass.simpleName,
)
configLoader.load(workingDir).effectiveBuildDefinitions()
}
}
}
@@ -41,7 +41,6 @@ class Watcher(
private val buildExecutor: BuildExecutor, private val buildExecutor: BuildExecutor,
private val configLoader: ConfigLoader, private val configLoader: ConfigLoader,
private val clock: Clock, private val clock: Clock,
private val followUpTrigger: FollowUpTrigger,
) { ) {
private val log = LoggerFactory.getLogger(Watcher::class.java) private val log = LoggerFactory.getLogger(Watcher::class.java)
@@ -84,14 +83,11 @@ class Watcher(
* Runs the startup recovery of every repository and schedules the poll loop with the * Runs the startup recovery of every repository and schedules the poll loop with the
* fixed delay `watcher.pollInterval` one loop, one delay: the instance's setting, * fixed delay `watcher.pollInterval` one loop, one delay: the instance's setting,
* which every repository's effective config carries; the first poll runs immediately. * which every repository's effective config carries; the first poll runs immediately.
* Arms the [FollowUpTrigger] first, so the recovery's re-enqueued builds get their
* follow-ups too.
*/ */
@Synchronized @Synchronized
fun start(repos: List<RepoContext>) { fun start(repos: List<RepoContext>) {
check(scheduler == null) { "watcher is already running" } check(scheduler == null) { "watcher is already running" }
require(repos.isNotEmpty()) { "no repository to watch" } require(repos.isNotEmpty()) { "no repository to watch" }
followUpTrigger.arm()
repos.forEach { recoverSafely(it) } repos.forEach { recoverSafely(it) }
val interval = DurationParser.parse(configLoader.load(repos.first().workingDir).watcher.pollInterval) val interval = DurationParser.parse(configLoader.load(repos.first().workingDir).watcher.pollInterval)
scheduler = scheduler =
@@ -115,7 +111,6 @@ class Watcher(
@Synchronized @Synchronized
fun stop() { fun stop() {
followUpTrigger.disarm()
scheduler?.shutdownNow() scheduler?.shutdownNow()
scheduler = null scheduler = null
state = state.copy(running = false) state = state.copy(running = false)
@@ -329,7 +324,6 @@ class Watcher(
.loadWithBranchLayer( .loadWithBranchLayer(
workingDir, workingDir,
ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) }, ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) },
branch,
).effectiveBuildDefinitions() ).effectiveBuildDefinitions()
} catch (e: Exception) { } catch (e: Exception) {
log.warn( log.warn(
@@ -292,42 +292,6 @@ class BuildExecutorTest : FunSpec() {
.build shouldBe "default" .build shouldBe "default"
} }
test("a follow-up build runs in its branch's worktree after its predecessor") {
val h =
Harness(
"""
executor:
maxConcurrent: 2
builds:
default:
trigger:
onPush: true
cleanCommand: ""
buildCommand: "sleep 1; echo built > output.txt"
deploy:
trigger:
afterSuccessOf: default
buildCommand: "cat output.txt"
""".trimIndent(),
)
h.executor.startBuild(h.repo, "main", "c1", "default")
h.executor.startBuild(h.repo, "main", "c1", "deploy")
awaitStatus(h, "main@deploy", BuildStatus.SUCCESS)
awaitIdle(h)
// same branch, same commit: the same worktree, and the follow-up saw the predecessor's output
h.workspaceCalls shouldContainExactly listOf("main" to "c1", "main" to "c1")
val predecessor = h.repository.latestFor("main").shouldNotBeNull()
val followUp = h.repository.latestFor("main@deploy").shouldNotBeNull()
predecessor.status shouldBe BuildStatus.SUCCESS
followUp.runningSince
.shouldNotBeNull()
.isBefore(predecessor.runningSince.shouldNotBeNull())
.shouldBeFalse()
Files.readString(h.workingDir.resolve("output.txt")).trim() shouldBe "built"
}
test("a build whose definition was removed from the config falls back to the branch's settings") { test("a build whose definition was removed from the config falls back to the branch's settings") {
val h = harness(buildCommand = "echo regular-\$branch") val h = harness(buildCommand = "echo regular-\$branch")
@@ -541,8 +505,6 @@ class BuildExecutorTest : FunSpec() {
} }
test("with maxConcurrent 1 a second branch stays PENDING until the first finished") { test("with maxConcurrent 1 a second branch stays PENDING until the first finished") {
// branch-a blocks on a gate file the test creates, so the PENDING assertion
// below cannot race the first build finishing on a loaded machine
val h = val h =
Harness( Harness(
""" """
@@ -550,7 +512,7 @@ class BuildExecutorTest : FunSpec() {
maxConcurrent: 1 maxConcurrent: 1
branches: branches:
branch-a: branch-a:
buildCommand: "until [ -f gate ]; do sleep 0.05; done" buildCommand: "sleep 1"
cleanCommand: "" cleanCommand: ""
branch-b: branch-b:
buildCommand: "echo ok" buildCommand: "echo ok"
@@ -561,13 +523,8 @@ class BuildExecutorTest : FunSpec() {
h.executor.startBuild(h.repo, "branch-a", "sha-a") h.executor.startBuild(h.repo, "branch-a", "sha-a")
h.executor.startBuild(h.repo, "branch-b", "sha-b") h.executor.startBuild(h.repo, "branch-b", "sha-b")
eventually(30.seconds) {
h.repository.latestFor("branch-a")?.status shouldBe BuildStatus.RUNNING
}
h.repository.latestFor("branch-b")?.status shouldBe BuildStatus.PENDING h.repository.latestFor("branch-b")?.status shouldBe BuildStatus.PENDING
Files.createFile(h.workingDir.resolve("gate"))
awaitStatus(h, "branch-b", BuildStatus.SUCCESS) awaitStatus(h, "branch-b", BuildStatus.SUCCESS)
awaitStatus(h, "branch-a", BuildStatus.SUCCESS) awaitStatus(h, "branch-a", BuildStatus.SUCCESS)
val transitions = h.events.map { it.result.branch to it.result.status } val transitions = h.events.map { it.result.branch to it.result.status }
@@ -4,7 +4,6 @@ import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.git.GitService import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext import de.hoennig.werkator.repo.RepoContext
import de.hoennig.werkator.repo.RepoRegistry import de.hoennig.werkator.repo.RepoRegistry
import de.hoennig.werkator.watcher.FollowUpTrigger
import io.kotest.core.spec.style.FunSpec import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.shouldBe import io.kotest.matchers.shouldBe
import io.kotest.matchers.string.shouldContain import io.kotest.matchers.string.shouldContain
@@ -23,31 +22,16 @@ class BuildCommandTest : FunSpec() {
private val dir: Path = Paths.get(".") private val dir: Path = Paths.get(".")
private val repo = RepoContext("test", dir, mockk(), mockk()) private val repo = RepoContext("test", dir, mockk(), mockk())
private val registry = mockk<RepoRegistry>().also { every { it.current() } returns repo } private val registry = mockk<RepoRegistry>().also { every { it.current() } returns repo }
private val followUpTrigger = mockk<FollowUpTrigger>()
private fun command(fragment: String? = null) = private fun command(fragment: String? = null) =
BuildCommand(gitService, consoleBuildRunner, registry, followUpTrigger).apply { BuildCommand(gitService, consoleBuildRunner, registry).apply {
branchFragment = fragment branchFragment = fragment
} }
init { init {
beforeEach { beforeEach {
clearMocks(gitService, consoleBuildRunner, followUpTrigger) clearMocks(gitService, consoleBuildRunner)
justRun { gitService.fetchOrigin(dir) } justRun { gitService.fetchOrigin(dir) }
every { followUpTrigger.followUpsOf(any(), any(), any(), any()) } returns emptyList()
}
test("a green CLI build names the follow-up builds the server would run, and runs none") {
every { gitService.currentBranch(dir) } returns "main"
every { gitService.localHeadCommit("main", dir) } returns "local-head"
every { gitService.hasNewCommits("main", dir) } returns false
every { consoleBuildRunner.buildAndStream(repo, "main", "local-head") } returns BuildStatus.SUCCESS
every { followUpTrigger.followUpsOf(repo, "main", "local-head", "default") } returns listOf("deploy")
val console = captureConsole { command().call() }
console.stdout.shouldContain("follow-up build(s) deploy")
verify(exactly = 1) { consoleBuildRunner.buildAndStream(repo, "main", "local-head") }
} }
test("builds the current branch at its local head when no branch is given") { test("builds the current branch at its local head when no branch is given") {
@@ -109,21 +109,6 @@ class InitCommandTest : FunSpec() {
projectContent shouldContain "repo: my-repo" projectContent shouldContain "repo: my-repo"
} }
test("keeps dots in repository names") {
val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/mi/michael.hoennig.de.git"
initCommand.run()
val projectConfig = tempDir.resolve(".werkator.yml")
val projectContent = projectConfig.toFile().readText()
projectContent shouldContain "owner: mi"
projectContent shouldContain "repo: michael.hoennig.de"
}
test("does not overwrite existing files") { test("does not overwrite existing files") {
val tempDir = Files.createTempDirectory("werkator-init-test") val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir initCommand.workingDir = tempDir
@@ -267,50 +252,5 @@ class InitCommandTest : FunSpec() {
// This should not throw IllegalArgumentException // This should not throw IllegalArgumentException
initCommand.run() initCommand.run()
} }
test("--systemd reads the configuration from the repository root, not the current directory") {
val tempDir = Files.createTempDirectory("werkator-init-test")
// written before the run, so `init` keeps it instead of creating a template
tempDir.resolve(".werkator.yml").toFile().writeText(
"server:\n publicBaseUrl: \"https://werkator.example.org/\"\n port: 18099\n",
)
initCommand.workingDir = tempDir
initCommand.systemd = true
initCommand.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
initCommand.javaExecutableResolver = { Paths.get("/usr/bin/java") }
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
initCommand.run()
// the host integration is generated only when the root's config has a public base URL
val htaccess = tempDir.resolve(".git/werkator/${SystemdServiceFiles.HTACCESS_NAME}")
htaccess.toFile().shouldExist()
htaccess.toFile().readText() shouldContain "18099"
tempDir.resolve(".git/werkator/${SystemdServiceFiles.MAINTENANCE_PAGE_NAME}").toFile().shouldExist()
}
test("--systemd warns once when the effective configuration cannot be loaded") {
val tempDir = Files.createTempDirectory("werkator-init-test")
val brokenLoader = mockk<de.hoennig.werkator.config.ConfigLoader>()
every { brokenLoader.load(any()) } throws IllegalStateException("gitea.owner is required")
val command = InitCommand(gitService, brokenLoader)
command.workingDir = tempDir
command.systemd = true
command.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
command.javaExecutableResolver = { Paths.get("/usr/bin/java") }
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
val console = captureConsole { command.run() }
console.stdout shouldContain "gitea.owner is required"
// the three readers of the configuration must not repeat the warning
console.stdout.windowed("Warning:".length).count { it == "Warning:" } shouldBe 1
// the units are still written with the defaults
tempDir.resolve(".git/werkator/${SystemdServiceFiles.unitName(tempDir)}").toFile().shouldExist()
}
} }
} }
@@ -739,190 +739,6 @@ class ConfigLoaderTest : FunSpec() {
.shouldBeTrue() .shouldBeTrue()
} }
test("afterSuccessOf must name an existing definition and must not form a cycle") {
val dir = Files.createTempDirectory("werkator-test")
val project = dir.resolve(".werkator.yml").toFile()
project.writeText(
"""
builds:
default:
trigger:
onPush: true
deploy:
trigger:
afterSuccessOf: default
branches: ["main"]
buildCommand: scripts/deploy.sh
""".trimIndent(),
)
// a follow-up with nothing but its predecessor is a triggered build, and it binds
val deploy = loader.load(dir).buildDefinitions.getValue("deploy")
deploy.trigger.afterSuccessOf shouldBe "default"
deploy.trigger.isFollowUp().shouldBeTrue()
deploy.trigger.onPush.shouldBeFalse()
project.writeText(
"""
builds:
deploy:
trigger:
afterSuccessOf: frontend
""".trimIndent(),
)
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.let {
it.shouldContain("builds.deploy")
it.shouldContain("frontend")
}
project.writeText(
"""
builds:
a:
trigger:
afterSuccessOf: b
b:
trigger:
afterSuccessOf: a
""".trimIndent(),
)
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.shouldContain("a -> b -> a")
project.writeText(
"""
builds:
a:
trigger:
afterSuccessOf: a
""".trimIndent(),
)
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.shouldContain("builds.a follows itself")
}
test("a branch whose layer lacks the predecessor loses only its follow-up") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
frontend:
trigger:
onPush: true
deploy:
trigger:
afterSuccessOf: frontend
""".trimIndent(),
)
// the branch renamed the predecessor: a warning, not a failed load — the branch's
// own builds must keep running, its follow-up simply never fires for it
val config =
loader.loadWithBranchLayer(
dir,
"""
builds:
frontend: null
ui:
trigger:
onPush: true
""".trimIndent(),
)
config.buildDefinitions
.getValue("ui")
.trigger.onPush
.shouldBeTrue()
config.buildDefinitions
.getValue("deploy")
.trigger.afterSuccessOf shouldBe "frontend"
}
test("afterSuccessOf written flat is refused like every trigger key") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
default:
trigger:
onPush: true
deploy:
afterSuccessOf: default
""".trimIndent(),
)
val thrown = shouldThrow<ConfigFormatException> { loader.load(dir) }
thrown.message.shouldContain("builds.deploy: afterSuccessOf")
thrown.message.shouldContain("trigger:")
}
test("a follow-up trigger is pinned to the host, a branch cannot add or widen one") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
frontend:
trigger:
onPush: true
deploy:
trigger:
afterSuccessOf: frontend
branches: ["main"]
""".trimIndent(),
)
val config =
loader.loadWithBranchLayer(
dir,
"""
builds:
deploy:
trigger:
afterSuccessOf: frontend
branches: ["*"]
nightly:
trigger:
atTimes: ["01:00"]
afterSuccessOf: frontend
""".trimIndent(),
"feature/x",
)
// the host's trigger block of the follow-up is used unchanged
val deploy = config.buildDefinitions.getValue("deploy").trigger
deploy.afterSuccessOf shouldBe "frontend"
deploy.branches shouldBe listOf("main")
deploy.selectsByName("feature/x").shouldBeFalse()
// and a branch cannot make any build of its own a follow-up
val nightly = config.buildDefinitions.getValue("nightly").trigger
nightly.afterSuccessOf shouldBe ""
nightly.atTimes shouldBe listOf("01:00")
}
test("a branch supplies the command of a host-triggered follow-up") {
val dir = Files.createTempDirectory("werkator-test")
Files.createDirectories(dir.resolve(".git/werkator"))
dir.resolve(".git/werkator/.werkator.yml").toFile().writeText(
"""
builds:
deploy:
trigger:
afterSuccessOf: default
branches: ["main"]
werkdock:
env:
DEPLOY_TARGET: host:/srv/www
""".trimIndent(),
)
val worktree = Files.createTempDirectory("werkator-test-worktree")
worktree.resolve(".werkator.yml").toFile().writeText(
"""
builds:
deploy:
cleanCommand: ""
buildCommand: scripts/deploy-prod.sh -y "${'$'}DEPLOY_TARGET"
""".trimIndent(),
)
val settings = loader.loadForWorktree(dir, worktree, "main").buildSettings("main", "deploy")
settings.buildCommand shouldBe "scripts/deploy-prod.sh -y \"${'$'}DEPLOY_TARGET\""
settings.cleanCommand shouldBe ""
settings.werkdock.env shouldBe mapOf("DEPLOY_TARGET" to "host:/srv/www")
}
test("builds.default is the base of every other build, but never its trigger") { test("builds.default is the base of every other build, but never its trigger") {
val dir = Files.createTempDirectory("werkator-test") val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText( dir.resolve(".werkator.yml").toFile().writeText(
@@ -113,7 +113,7 @@ class PermanentBranchRoutesTest : FunSpec() {
every { registry.byName(any()) } returns null every { registry.byName(any()) } returns null
every { registry.byName("test") } returns repo every { registry.byName("test") } returns repo
every { configLoader.load(any()) } returns WerkatorConfig() every { configLoader.load(any()) } returns WerkatorConfig()
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns WerkatorConfig() every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns WerkatorConfig()
every { gitService.showFileAtCommit(any(), any(), any()) } returns null every { gitService.showFileAtCommit(any(), any(), any()) } returns null
every { controlTokens.token() } returns "test-token" every { controlTokens.token() } returns "test-token"
every { branchListing.branches(any()) } returns emptyList() every { branchListing.branches(any()) } returns emptyList()
@@ -142,7 +142,7 @@ class UiControllerTest : FunSpec() {
server = ServerConfig(impressumUrl = "https://example.org/imprint"), server = ServerConfig(impressumUrl = "https://example.org/imprint"),
gitea = GiteaConfig(baseUrl = "https://git.example.org", owner = "acme", repo = "widget"), gitea = GiteaConfig(baseUrl = "https://git.example.org", owner = "acme", repo = "widget"),
) )
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns WerkatorConfig() every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns WerkatorConfig()
every { gitService.showFileAtCommit(any(), any(), any()) } returns null every { gitService.showFileAtCommit(any(), any(), any()) } returns null
every { controlTokens.token() } returns "test-token" every { controlTokens.token() } returns "test-token"
every { repository.latestGreenFor(any()) } returns null every { repository.latestGreenFor(any()) } returns null
@@ -385,7 +385,7 @@ class UiControllerTest : FunSpec() {
) )
every { repository.history() } returns listOf(pitestResult) every { repository.history() } returns listOf(pitestResult)
every { artifactStore.artifactDir("main-pitest-key") } returns null every { artifactStore.artifactDir("main-pitest-key") } returns null
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns
WerkatorConfig( WerkatorConfig(
branches = mapOf("default" to BranchConfig(buildCommand = "./gradlew quick-check")), branches = mapOf("default" to BranchConfig(buildCommand = "./gradlew quick-check")),
buildDefinitions = mapOf("pitest" to BuildDefinition(buildCommand = "./gradlew pitestFull")), buildDefinitions = mapOf("pitest" to BuildDefinition(buildCommand = "./gradlew pitestFull")),
@@ -1,173 +0,0 @@
package de.hoennig.werkator.watcher
import de.hoennig.werkator.build.ArtifactKeys
import de.hoennig.werkator.build.BuildExecutor
import de.hoennig.werkator.build.BuildResult
import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.build.BuildStatusChangedEvent
import de.hoennig.werkator.build.RunningBuild
import de.hoennig.werkator.config.BuildDefinition
import de.hoennig.werkator.config.ConfigLoader
import de.hoennig.werkator.config.TriggerConfig
import de.hoennig.werkator.config.WerkatorConfig
import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext
import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.collections.shouldBeEmpty
import io.kotest.matchers.collections.shouldContainExactly
import io.mockk.every
import io.mockk.mockk
import java.nio.file.Files
import java.time.Clock
import java.time.Instant
import java.time.ZoneOffset
import java.util.concurrent.CopyOnWriteArrayList
class FollowUpTriggerTest : FunSpec() {
private val noon = Instant.parse("2026-09-04T12:00:00Z")
/** Which build was started on which branch at which commit. */
private data class Started(
val branch: String,
val commit: String,
val build: String,
)
private inner class Harness(
config: WerkatorConfig,
) {
val workingDir = Files.createTempDirectory("werkator-followup-test")
val gitService = mockk<GitService>()
val configLoader = mockk<ConfigLoader>()
val buildExecutor = mockk<BuildExecutor>()
val repo = RepoContext("test", workingDir, mockk(), mockk())
val started = CopyOnWriteArrayList<Started>()
val trigger = FollowUpTrigger(gitService, configLoader, buildExecutor, Clock.fixed(noon, ZoneOffset.UTC))
init {
every { configLoader.load(any()) } returns config
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns config
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
// the branch moved on since the predecessor started
every { gitService.originHeadCommit(any(), any()) } returns "c2"
every { gitService.originBranchCommitTimes(any()) } returns mapOf("main" to noon.minusSeconds(60))
every { buildExecutor.startBuild(any(), any(), any(), any()) } answers {
val branch = secondArg<String>()
val commit = thirdArg<String>()
val build = arg<String>(3)
started += Started(branch, commit, build)
val staging = Files.createTempDirectory("werkator-followup-staging")
RunningBuild(
repo = repo,
branch = branch,
build = build,
commit = commit,
artifactKey = ArtifactKeys.buildKey(BuildDefinition.poolName(branch, build), noon),
startedAt = noon,
stagingDir = staging,
liveLogFile = staging.resolve("build.log"),
)
}
}
fun finished(
build: String,
status: BuildStatus,
branch: String = "main",
commit: String = "c1",
) {
val result =
BuildResult(
branch = branch,
build = build,
commit = commit,
status = status,
startedAt = noon,
artifactKey = ArtifactKeys.buildKey(BuildDefinition.poolName(branch, build), noon),
)
trigger.onBuildStatusChanged(BuildStatusChangedEvent(result, repo))
}
}
private fun deployAfter(
predecessor: String,
branches: List<String> = emptyList(),
): WerkatorConfig =
WerkatorConfig(
buildDefinitions =
mapOf(
"frontend" to BuildDefinition(trigger = TriggerConfig(onPush = true)),
"backend" to BuildDefinition(trigger = TriggerConfig(onPush = true)),
"deploy" to
BuildDefinition(
trigger = TriggerConfig(afterSuccessOf = predecessor, branches = branches),
buildCommand = "scripts/deploy.sh",
),
),
)
init {
test("a green predecessor enqueues the follow-up at the predecessor's commit") {
val h = Harness(deployAfter("frontend", branches = listOf("main")))
h.trigger.arm()
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
// c1, not the origin head c2 the branch has moved on to
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
}
test("every green run of the predecessor triggers the follow-up again") {
val h = Harness(deployAfter("frontend"))
h.trigger.arm()
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
h.started shouldContainExactly
listOf(
Started("main", "c1", "deploy"),
Started("main", "c1", "deploy"),
)
}
test("only a SUCCESS of the named predecessor triggers") {
val h = Harness(deployAfter("frontend", branches = listOf("main")))
h.trigger.arm()
h.finished("frontend", BuildStatus.FAILED)
h.finished("frontend", BuildStatus.CANCELLED)
h.finished("frontend", BuildStatus.INTERRUPTED)
h.finished("frontend", BuildStatus.PENDING)
h.finished("frontend", BuildStatus.RUNNING)
h.finished("backend", BuildStatus.SUCCESS)
// a branch the host's selector does not name never runs the follow-up
h.finished("frontend", BuildStatus.SUCCESS, branch = "feature/x")
h.started.shouldBeEmpty()
}
test("the trigger listens only while the watcher runs") {
val h = Harness(deployAfter("frontend"))
h.finished("frontend", BuildStatus.SUCCESS)
h.started.shouldBeEmpty()
h.trigger.arm()
h.finished("frontend", BuildStatus.SUCCESS)
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
h.trigger.disarm()
h.finished("frontend", BuildStatus.SUCCESS)
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
}
test("followUpsOf names the follow-ups without enqueueing anything") {
val h = Harness(deployAfter("default"))
h.trigger.followUpsOf(h.repo, "main", "c1", "default") shouldContainExactly listOf("deploy")
h.trigger.followUpsOf(h.repo, "main", "c1", "frontend").shouldBeEmpty()
h.started.shouldBeEmpty()
}
}
}
@@ -63,7 +63,6 @@ class WatcherTest : FunSpec() {
val artifactStore = mockk<ArtifactStore>() val artifactStore = mockk<ArtifactStore>()
val startedBuilds = CopyOnWriteArrayList<Pair<String, String>>() val startedBuilds = CopyOnWriteArrayList<Pair<String, String>>()
val configLoader = mockk<ConfigLoader>() val configLoader = mockk<ConfigLoader>()
val followUpTrigger = mockk<FollowUpTrigger>(relaxed = true)
val repo = RepoContext("test", workingDir, repository, artifactStore) val repo = RepoContext("test", workingDir, repository, artifactStore)
val watcher = val watcher =
Watcher( Watcher(
@@ -71,7 +70,6 @@ class WatcherTest : FunSpec() {
buildExecutor = buildExecutor, buildExecutor = buildExecutor,
configLoader = configLoader, configLoader = configLoader,
clock = Clock.fixed(noon, ZoneOffset.UTC), clock = Clock.fixed(noon, ZoneOffset.UTC),
followUpTrigger = followUpTrigger,
) )
private var seedCounter = 0L private var seedCounter = 0L
@@ -87,7 +85,7 @@ class WatcherTest : FunSpec() {
every { gitService.originBranchCommitTimes(any()) } returns emptyMap() every { gitService.originBranchCommitTimes(any()) } returns emptyMap()
every { gitService.originBranchHeads(any()) } returns emptyMap() every { gitService.originBranchHeads(any()) } returns emptyMap()
every { gitService.showFileAtCommit(any(), any(), any()) } returns null every { gitService.showFileAtCommit(any(), any(), any()) } returns null
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns config every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns config
every { gitService.pullRequestHeads(any()) } returns emptySet() every { gitService.pullRequestHeads(any()) } returns emptySet()
every { gitService.worktreePrune(any()) } returns Unit every { gitService.worktreePrune(any()) } returns Unit
every { gitService.fastForwardLocalBranches(any()) } returns emptyList() every { gitService.fastForwardLocalBranches(any()) } returns emptyList()
@@ -159,19 +157,6 @@ class WatcherTest : FunSpec() {
) )
init { init {
test("start arms the follow-up trigger before the recovery, stop disarms it") {
val harness = Harness()
harness.watcher.start(listOf(harness.repo))
try {
verify(exactly = 1) { harness.followUpTrigger.arm() }
verify(exactly = 0) { harness.followUpTrigger.disarm() }
} finally {
harness.watcher.stop()
}
verify(exactly = 1) { harness.followUpTrigger.disarm() }
}
test("a fetch failure is exposed in the state and only retried next cycle") { test("a fetch failure is exposed in the state and only retried next cycle") {
val harness = Harness() val harness = Harness()
every { harness.gitService.fetchOrigin(any()) } throws RuntimeException("origin unreachable") every { harness.gitService.fetchOrigin(any()) } throws RuntimeException("origin unreachable")
@@ -541,7 +526,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.originBranchHeads(any()) } returns every { harness.gitService.originBranchHeads(any()) } returns
mapOf("main" to "commit-main", "experiment" to "commit-exp") mapOf("main" to "commit-main", "experiment" to "commit-exp")
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml" every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp" every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main" every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
@@ -564,7 +549,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.originBranches(any()) } returns listOf("experiment") every { harness.gitService.originBranches(any()) } returns listOf("experiment")
every { harness.gitService.originBranchHeads(any()) } returns mapOf("experiment" to "commit-exp") every { harness.gitService.originBranchHeads(any()) } returns mapOf("experiment" to "commit-exp")
every { harness.gitService.showFileAtCommit("commit-exp", ".gittally.yml", any()) } returns "branch-yaml" every { harness.gitService.showFileAtCommit("commit-exp", ".gittally.yml", any()) } returns "branch-yaml"
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp" every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
harness.watcher.poll(harness.repo) harness.watcher.poll(harness.repo)
@@ -583,7 +568,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.originBranchHeads(any()) } returns every { harness.gitService.originBranchHeads(any()) } returns
mapOf("main" to "commit-main", "experiment" to "commit-exp") mapOf("main" to "commit-main", "experiment" to "commit-exp")
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml" every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
every { harness.gitService.originHeadCommit(any(), any()) } returns "commit-any" every { harness.gitService.originHeadCommit(any(), any()) } returns "commit-any"
harness.watcher.poll(harness.repo) harness.watcher.poll(harness.repo)
@@ -623,7 +608,7 @@ class WatcherTest : FunSpec() {
buildDefinitions = mapOf("nightly" to BuildDefinition(trigger = TriggerConfig(atTimes = listOf("11:00")))), buildDefinitions = mapOf("nightly" to BuildDefinition(trigger = TriggerConfig(atTimes = listOf("11:00")))),
) )
every { harness.configLoader.load(any()) } returns edited every { harness.configLoader.load(any()) } returns edited
every { harness.configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns edited every { harness.configLoader.loadWithBranchLayer(any(), anyNullable()) } returns edited
harness.watcher.poll(harness.repo) harness.watcher.poll(harness.repo)
@@ -637,7 +622,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.hasNewCommits("main", any()) } returns true every { harness.gitService.hasNewCommits("main", any()) } returns true
every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-main") every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-main")
every { harness.gitService.showFileAtCommit("commit-main", Watcher.CONFIG_FILE, any()) } returns "broken" every { harness.gitService.showFileAtCommit("commit-main", Watcher.CONFIG_FILE, any()) } returns "broken"
every { harness.configLoader.loadWithBranchLayer(any(), "broken", anyNullable()) } throws every { harness.configLoader.loadWithBranchLayer(any(), "broken") } throws
RuntimeException("mapping problem") RuntimeException("mapping problem")
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main" every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
+15 -63
View File
@@ -44,7 +44,7 @@
# Optional in the env file: # Optional in the env file:
# WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start) # WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start)
# WERKATOR_REPO_URL https clone URL of the watched repository # WERKATOR_REPO_URL https clone URL of the watched repository
# (default: https://git.javagil.de/mi/werkator.git) # (default: https://github.com/mhoennig/werkator.git)
# WERKATOR_REPO_DIR directory of the watched repository, absolute or relative to # WERKATOR_REPO_DIR directory of the watched repository, absolute or relative to
# WERKATOR_PATH (default: werkator); it also names the systemd # WERKATOR_PATH (default: werkator); it also names the systemd
# unit, exactly as `init --systemd` derives it # unit, exactly as `init --systemd` derives it
@@ -122,7 +122,7 @@ require_env WERKATOR_REMOTE WERKATOR_PATH
HOST="$WERKATOR_REMOTE" HOST="$WERKATOR_REMOTE"
TARGET_DIR="$WERKATOR_PATH" TARGET_DIR="$WERKATOR_PATH"
ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}" ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}"
REPO_URL="${WERKATOR_REPO_URL:-https://git.javagil.de/mi/werkator.git}" REPO_URL="${WERKATOR_REPO_URL:-https://github.com/mhoennig/werkator.git}"
# The host layout is three values, not one convention: an installation that grew # The host layout is three values, not one convention: an installation that grew
# before this script existed puts them elsewhere, and the defaults are exactly what # before this script existed puts them elsewhere, and the defaults are exactly what
@@ -332,65 +332,9 @@ instance_update() {
echo "==> Instance updated." echo "==> Instance updated."
} }
# Clones one URL into one directory on the host. # Sets up the WATCHED repository: an anonymous https clone (a private origin
# A private https origin authenticates with the shared `defaults.git.account` / # gets its credentials via git.account/git.token in the machine config that
# `defaults.git.token` of `~/.werkator.yml` (ADR 0009). The whole authenticated # `werkator init` creates), the werkator init with the instance fragment
# clone runs in one remote python script: the token is read from the instance
# file on the host and passed to git via a one-shot GIT_ASKPASS script, so it
# appears in neither the local process list nor a repository config.
# Public origins (or an instance without shared credentials) clone anonymously.
clone_repo() {
local url="$1" dest="$2"
if ssh "$HOST" "test -d '$dest/.git'"; then
echo " (already cloned, skipping)"
return 0
fi
case "$url" in
https://*)
# The python helper travels base64-encoded: the clone command itself
# stays a plain `ssh` line, so no `$` inside the script is ever
# expanded by the local shell, and the token never leaves the host.
local helper_b64
helper_b64="$(python3 -c 'import base64,sys; print(base64.b64encode(sys.stdin.read().encode()).decode())' <<'PYEOF_CLONE'
import os, stat, subprocess, sys, tempfile, urllib.parse
url, dest = sys.argv[1], sys.argv[2]
try:
import yaml
cfg = yaml.safe_load(open(os.path.expanduser("~/.werkator.yml"))) or {}
except (FileNotFoundError, ImportError):
cfg = {}
d = (cfg.get("defaults") or {}).get("git") or {}
account, token = d.get("account"), d.get("token")
env = dict(os.environ, GIT_TERMINAL_PROMPT="0")
ask = None
if account and token:
parts = urllib.parse.urlsplit(url)
host = parts.netloc.rsplit("@", 1)[-1]
url = urllib.parse.urlunsplit(parts._replace(netloc=account + "@" + host))
ask = tempfile.NamedTemporaryFile(mode="w", prefix="werkator-clone-askpass-",
suffix=".sh", delete=False)
ask.write("#!/bin/sh\nexec echo \"$WERKATOR_CLONE_TOKEN\"\n")
ask.close()
os.chmod(ask.name, stat.S_IRWXU)
env.update(GIT_ASKPASS=ask.name, WERKATOR_CLONE_TOKEN=token)
try:
subprocess.run(["git", "clone", url, dest], env=env, check=True)
finally:
if ask is not None:
os.unlink(ask.name)
PYEOF_CLONE
)"
ssh "$HOST" "echo '$helper_b64' | base64 -d | python3 - '$url' '$dest'"
;;
*)
ssh "$HOST" "git clone '$url' '$dest'"
;;
esac
}
# Sets up the WATCHED repository: an https clone (a private origin
# authenticates with the shared `defaults.git.*` credentials of
# `~/.werkator.yml`; see `clone_repo`), the werkator init with the instance
# applied, and the rootfs archive for the sandbox builds. All configuration # applied, and the rootfs archive for the sandbox builds. All configuration
# writing is init's — this script transports and invokes (step 23). # writing is init's — this script transports and invokes (step 23).
repo_init() { repo_init() {
@@ -400,7 +344,11 @@ repo_init() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning the watched repository" echo "==> Cloning the watched repository"
clone_repo "$REPO_URL" "$REPO_DIR" if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
fi
if [ "$SANDBOX" = "docker" ]; then if [ "$SANDBOX" = "docker" ]; then
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile" echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
@@ -453,7 +401,11 @@ repo_add() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning $url as '$name'" echo "==> Cloning $url as '$name'"
clone_repo "$url" "$SIBLING_DIR/$name" if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
fi
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock # The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
# binary). Without it a watched repository builds on the bare host, where the # binary). Without it a watched repository builds on the bare host, where the