Author SHA1 Message Date
mhoennigandClaude Fable 5.1 79a26dbf10 docs(prs): PR#23 records where the implementation went beyond the spec (point 7)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:22:06 +02:00
mhoennigandClaude Fable 5.1 fc8dbf1c91 docs: follow-up builds in the reference, the init template, and the invariants (PR#23, point 5)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:20:51 +02:00
mhoennigandClaude Fable 5.1 776defd391 docs(prs): PR#23 names every test that verifies it (point 4)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:19:39 +02:00
mhoennigandClaude Fable 5.1 bbb4969fb9 Run follow-up builds after a green predecessor (PR#23, point 3)
FollowUpTrigger listens to build status transitions and, on a SUCCESS,
enqueues every definition of that branch whose afterSuccessOf names the
finished build — at the finished build's commit, not the branch's
current head, so a deployment ships the commit that was tested. Every
green run triggers, a repeated one again. The trigger is armed by
Watcher.start() and disarmed by stop(), so a CLI build never enqueues
into a JVM about to exit; the CLI names the follow-ups the server
would run instead. The event now carries the repository the result
belongs to, which the listener has to act on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:19:10 +02:00
mhoennigandClaude Fable 5.1 ab522bc68d Pin the trigger of a follow-up build to the host (PR#23, point 2)
A branch's committed config loses afterSuccessOf from every trigger, and
where the host defines the same build as a follow-up, the branch's whole
trigger block: a follow-up is the host's way to hand deployment targets
and credentials to a green commit, so a branch may say what its
deployment does, never that or where it happens — otherwise it could
widen the host's selector to include itself. Both cases are logged
naming the branch, which the watcher now passes along.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:15:41 +02:00
mhoennigandClaude Fable 5.1 06eafc8010 Add trigger.afterSuccessOf, the follow-up build's predecessor (PR#23, point 1)
A build definition may name the build it follows. The key lives in the
trigger block, so it is never inherited and refused when written flat;
a definition with only a predecessor counts as triggered. An unknown
predecessor refuses the start of the primary configuration — a
deployment that silently never runs is the failure the flat-key check
exists to prevent — while a branch layer that dropped the name only
loses its follow-up with a warning, and an instance fragment is not
checked alone since its predecessor may live in the project config.
A cycle is refused wherever it appears.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:12:47 +02:00
mhoennigandClaude Fable 5.1 e4b935c684 docs(prs): PR#23 got its number
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:02:58 +02:00
mhoennigandClaude Fable 5.1 0f57549b4f docs(prs): the follow-up-builds PR-doc
A build definition may declare afterSuccessOf, running whenever the named
build of the same branch turns green — the deployment path chosen over a
deployCommand or a separate deploy section.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 18:01:38 +02:00
092183ca30 Clone the watched repository from Gitea by default (#21)
The repository moved to git.javagil.de, but repo-init still defaulted to
the GitHub mirror, which is no longer updated.
A freshly cloned instance therefore watched a stale main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #21
2026-09-04 13:33:08 +02:00
mhoennig 0761a274db Merge main into the deflake-maxconcurrent-test branch 2026-09-04 12:46:25 +02:00
mhoennigandClaude Opus 5 5c73c4cc21 docs(prs): the deflake-maxconcurrent-test PR-doc
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 08:53:00 +02:00
16543f038b the sandbox config section is werkdock, not bwrap (#19)
`bwrap` named the mechanism one layer below the tool that actually runs it: since
v1.0.0 Werkator does not invoke bwrap at all, it shells out to the werkdock CLI —
which made `bwrap.werkdock` a key naming its own executor.

The section is `werkdock` now and that key is `werkdock.binary`; BwrapConfig,
BwrapOverrides and BwrapBuildRunner follow the name. A file still writing `bwrap`
is read as before and warned about once per file, in `renameLegacySandbox` on the
raw map of every layer before merging — so nothing downstream knows two names, and
the old name is not a way around the pinning either. Renaming rather than refusing,
because the section lives in the machine configuration of every webspace instance,
which no repository tracks; the hard refusal belongs to the release that sets
ConfigVersions.FORMAT_BROKE_IN, where a file declaring no version can be caught
by name at all.

WERKATOR_SANDBOX in tools/remote follows, and still accepts `bwrap`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #19
2026-09-03 20:41:18 +02:00
3ccc901d1b tools/remote drives any host layout (#18)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #18
2026-09-03 20:37:58 +02:00
mhoennigandClaude Fable 5.1 11f6f9bd26 docs(rfcs): RFC 0001 proposes the Instrument Panel web UI redesign
Six explored directions, the chosen one (E) worked out in a light and a dark
teal palette, with a repository strip previewing every served repository,
the ten-repository and the phone case, and the backend it needs (/api/repos).
Renderings of all artboards live next to the RFC; AGENTS.md registers docs/rfcs/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:15:04 +02:00
mhoennigandClaude Opus 5 9221550a2a test(build): gate the maxConcurrent-1 test instead of racing a sleep
The build of branch-a slept one second while the test asserted, without
any synchronization, that branch-b was still PENDING.
Under CPU contention on the mih09 host the sleep could elapse first, so
branch-b was already RUNNING or SUCCESS when the assertion ran.
branch-a now blocks until the test creates a gate file, and the test
first waits for branch-a to be RUNNING; the PENDING assertion no longer
depends on timing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 19:32:22 +02:00
mhoennigandClaude Opus 5 3b095fd04a docs(prs): PR#17 verified live on mih09
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 18:00:07 +02:00
mhoennigandClaude Sonnet 5 6521c47092 Bump version to 1.1.2 for the maintenance-page deployment
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-03 17:54:23 +02:00
mhoennigandClaude Sonnet 5 ed89911871 docs(prs): the maintenance-page PR-doc
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-03 17:49:21 +02:00
45 changed files with 1932 additions and 204 deletions
+4 -4
View File
@@ -1,6 +1,6 @@
---
name: architecture
description: Detailed Werkator subsystem architecture — CLI wiring and exit codes, server mode, web UI, configuration system, git access, build execution (native, Docker, and bwrap), watcher poll cycle, and system metrics. Use when designing or modifying code in the commands, config, git, gitea, build, artifacts, watcher, metrics, or server packages, or when a question goes beyond the overview in AGENTS.md.
description: Detailed Werkator subsystem architecture — CLI wiring and exit codes, server mode, web UI, configuration system, git access, build execution (native, Docker, and the werkdock sandbox), watcher poll cycle, and system metrics. Use when designing or modifying code in the commands, config, git, gitea, build, artifacts, watcher, metrics, or server packages, or when a question goes beyond the overview in AGENTS.md.
---
# Werkator Architecture
@@ -49,7 +49,7 @@ Werkator is configured by three YAML files, deep-merged by `ConfigLoader` (later
Every lookup falls back to the pre-rename name (`ConfigFiles`): `.gittally.yml`, and `.git/gittally/.gittally.yml` for the machine layer. Current name first, and where both exist the old one is ignored rather than merged — a missing config is not an error, so an un-renamed installation would otherwise start on defaults without a single failure.
On top of those comes the **branch layer**: the `.werkator.yml` committed on a branch, applied by `loadWithBranchLayer` (the watcher passes the content read via `git show`, `loadForWorktree` the file in the build worktree). A branch describes its own CI and wins over both layers — the whole `builds` section — so a configuration can be tried out on a branch without touching other branches' builds. `stripPinned` removes what is not a description of this branch's build: `git`, `server`, `gitea`, `executor`, `watcher`, and — inside every `builds` definition as well as every legacy `branches` entry — `requirePullRequest`, `statusContext`, `docker.enabled`/`docker.network`, and `bwrap.enabled`/`bwrap.rootfs`/`bwrap.werkdock`.
On top of those comes the **branch layer**: the `.werkator.yml` committed on a branch, applied by `loadWithBranchLayer` (the watcher passes the content read via `git show`, `loadForWorktree` the file in the build worktree). A branch describes its own CI and wins over both layers — the whole `builds` section — so a configuration can be tried out on a branch without touching other branches' builds. `stripPinned` removes what is not a description of this branch's build: `git`, `server`, `gitea`, `executor`, `watcher`, and — inside every `builds` definition as well as every legacy `branches` entry — `requirePullRequest`, `statusContext`, `docker.enabled`/`docker.network`, and `werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` (the section was called `bwrap` until v1.2.0; `renameLegacySandbox` maps the old name onto the new one on every raw layer, before merging).
Each file is version-checked before merging (`werkator.version.since`/`below`, `ConfigVersions.verdict`), so the message can name the file to fix: `since` is hard in both directions — too old a Werkator, or a file written before `ConfigVersions.FORMAT_BROKE_IN` and read after it — while `below` only warns. There is no format version (`apiVersion`) on purpose: only one configuration generation is supported, and the declared version exists to make the incompatibility nameable.
@@ -71,9 +71,9 @@ Everything repository-scoped goes through a `RepoContext` (`repo` package, ADR 0
On context close (e.g. systemd SIGTERM), a `ContextClosedEvent` listener in `BuildExecutor` terminates the process trees of all executing builds and waits (bounded) until their results are persisted as INTERRUPTED — a shutdown is never recorded as FAILED. Builds still queued stay PENDING and start no process. Both are re-enqueued by the watcher's startup recovery; INTERRUPTED therefore publishes as Gitea state `pending`, not `failure` (`GiteaStateMapping`).
The runtime is selected per build behind the `BuildRunner` interface: `DispatchingBuildRunner` (`@Primary`) routes to native `ProcessBuildRunner` (the default), to `DockerBuildRunner` when `docker.enabled`, or to `BwrapBuildRunner` when `bwrap.enabled` — docker and bwrap are mutually exclusive per build and rejected in `buildSettings`, never picked silently. The Docker runner shells out to the `docker` CLI (no SDK): it (re)builds the configured image when the Dockerfile inputs changed (tracked via the `org.werkator.build-inputs-sha256` image label), maintains a per-repo Gradle cache volume, mounts the worktree and the Docker socket into a labelled (`org.hoennig.werkator`) `--rm --init` container, and repairs workspace ownership in-container after each command (under a rootless daemon the container runs as root, which is the host user, and the repair degenerates to `0:0`). Git works inside the container: the primary `.git` is mounted read-only with `.git/werkator/` masked by an empty tmpfs (credential isolation) and the worktree's admin dir mounted read-write (`gitMetadataMounts`). The returned `Process` is the attached `docker run` client, so log streaming and termination work exactly like native builds.
The runtime is selected per build behind the `BuildRunner` interface: `DispatchingBuildRunner` (`@Primary`) routes to native `ProcessBuildRunner` (the default), to `DockerBuildRunner` when `docker.enabled`, or to `WerkdockBuildRunner` when `werkdock.enabled` — docker and werkdock are mutually exclusive per build and rejected in `buildSettings`, never picked silently. The Docker runner shells out to the `docker` CLI (no SDK): it (re)builds the configured image when the Dockerfile inputs changed (tracked via the `org.werkator.build-inputs-sha256` image label), maintains a per-repo Gradle cache volume, mounts the worktree and the Docker socket into a labelled (`org.hoennig.werkator`) `--rm --init` container, and repairs workspace ownership in-container after each command (under a rootless daemon the container runs as root, which is the host user, and the repair degenerates to `0:0`). Git works inside the container: the primary `.git` is mounted read-only with `.git/werkator/` masked by an empty tmpfs (credential isolation) and the worktree's admin dir mounted read-write (`gitMetadataMounts`). The returned `Process` is the attached `docker run` client, so log streaming and termination work exactly like native builds.
`BwrapBuildRunner` (ADR 0008) is the third runtime, for hosts without root and without Docker — Hostsharing Managed Webspaces. It shells out to the `bwrap` CLI (no library): a prepared rootfs archive (`bwrap.rootfs`, built by `tools/build-bwrap-rootfs.sh`) is unpacked on demand into `.git/werkator/buildenv/<envKey>/rootfs` and bound read-only at `/`, with uid 0 inside mapped to the calling user; isolation is filesystem-only — network, uid, `/proc`, `/dev` are the host's by contract. It reuses the Docker runner's `gitMetadataMounts`; mount order matters (repo dir read-write before the metadata mounts and the workspace), and bind mountpoints missing from the rootfs are pre-created there, since the rootfs is a plain host directory while bwrap cannot mkdir against the read-only sandbox root. `bwrap.enabled`/`bwrap.rootfs` are pinned like the docker sandbox policy. The returned `Process` is the attached `bwrap` process, so streaming and cancellation are unchanged. The generic sandbox machinery is the standalone tool [Werkdock](https://git.javagil.de/mi/werkdock) (plan step 21: grown in `werkdock/`, consumed via the CLI since session C, its own repository since session E); the runner delegates to the `werkdock` CLI and this repository no longer carries its source.
`WerkdockBuildRunner` (ADR 0008) is the third runtime, for hosts without root and without Docker — Hostsharing Managed Webspaces. It shells out to the `bwrap` CLI (no library): a prepared rootfs archive (`werkdock.rootfs`, built by `tools/build-bwrap-rootfs.sh`) is unpacked on demand into `.git/werkator/buildenv/<envKey>/rootfs` and bound read-only at `/`, with uid 0 inside mapped to the calling user; isolation is filesystem-only — network, uid, `/proc`, `/dev` are the host's by contract. It reuses the Docker runner's `gitMetadataMounts`; mount order matters (repo dir read-write before the metadata mounts and the workspace), and bind mountpoints missing from the rootfs are pre-created there, since the rootfs is a plain host directory while bwrap cannot mkdir against the read-only sandbox root. `werkdock.enabled`/`werkdock.rootfs` are pinned like the docker sandbox policy. The returned `Process` is the attached `bwrap` process, so streaming and cancellation are unchanged. The generic sandbox machinery is the standalone tool [Werkdock](https://git.javagil.de/mi/werkdock) (plan step 21: grown in `werkdock/`, consumed via the CLI since session C, its own repository since session E); the runner delegates to the `werkdock` CLI and this repository no longer carries its source.
## Watcher
+4 -3
View File
@@ -40,9 +40,9 @@ All production code lives under `de.hoennig.werkator`, with sub-packages `comman
- Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly. Server routes carry the repository as `/repos/<name>/…` and `/api/repos/<name>/…`, with the unscoped form permanently meaning the served repository; the pages stay per repository and a drop-down in the page title switches between them.
- When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
- Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and bubblewrap (`bwrap.enabled`, `bwrap.rootfs`, `bwrap.werkdock`) sandbox policies, and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone.
- A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins.
- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `bwrap.enabled`, `bwrap.rootfs`, and `bwrap.werkdock`. Docker and bwrap are mutually exclusive per branch — enabling both is rejected at start.
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and werkdock (`werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`) sandbox policies, the trust gate (`requirePullRequest`), and the trigger of a follow-up build (`trigger.afterSuccessOf` everywhere, and the whole `trigger` block of a definition the host defines as a follow-up). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, bypass its own pull-request gate, or deploy itself; a branch's definitions apply to that branch alone.
- A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `afterSuccessOf`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `afterSuccessOf` makes a definition the follow-up of another one — a deployment is a build that follows a green build (PR#23): it runs at the predecessor's commit after every green run of it, whoever started that run, and the `FollowUpTrigger` that enqueues it is armed only by `Watcher.start()`. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins.
- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`, and the trigger of a follow-up build. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called `bwrap` until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets `ConfigVersions.FORMAT_BROKE_IN`.
- `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version.
- Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats.
- Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK.
@@ -68,6 +68,7 @@ Keep sentences short.
- `docs/deployment.md` — running Werkator as a systemd user service behind an existing reverse proxy (`init --systemd` generates the unit).
- `docs/werkator-migrationsplan.md` — renaming a running installation from GitTally to Werkator: what the name fallback covers and what has to be moved by hand.
- `docs/plan/` — the step-by-step rewrite plan; `docs/plan/README.md` explains how to execute a step, `docs/plan/00-legacy-analysis.md` summarizes the legacy bash script.
- `docs/rfcs/` — requests for comments: proposals that are larger than one PR and not yet a decision (an accepted RFC becomes an ADR or a plan step).
- `docs/prs/` — one document per pull request; every PR needs one. IMPORTANT: Before opening or finishing a pull request, load the [pr-doc skill](.claude/skills/pr-doc/SKILL.md) and write the PR-doc.
## Key Architectural Decisions
+1 -1
View File
@@ -13,7 +13,7 @@ group = "de.hoennig"
// so the UI footer (BuildProperties), --version and the release notes identify what is
// actually running; a deployment bundles whatever was committed since the last one.
// ReleaseVersionConsistencyTest fails the build if this and the top releases.html entry disagree.
version = "1.1.1"
version = "1.2.0"
java {
toolchain {
+59 -13
View File
@@ -71,7 +71,7 @@ above, giving the precedence **branch > repo install > project**. It takes prece
everything that describes how this branch is built: the whole `builds` section — its own
definitions and its overrides of the definitions from the project config, with
`buildCommand`, `cleanCommand`, `artifactDirs`, log file names, and
`docker.image`/`dockerfile`/`context`/`env` and `bwrap.env` inside them. That is how a new configuration is tried out: change it on a branch, and
`docker.image`/`dockerfile`/`context`/`env` and `werkdock.env` inside them. That is how a new configuration is tried out: change it on a branch, and
no other branch's builds are affected.
The branch layer is used in both places where it matters: the watcher reads the committed
@@ -99,9 +99,11 @@ single branch may decide it:
- the repository-side settings: the whole `gitea`, `executor`, and `watcher` sections;
- the trust gate: `requirePullRequest`, and the Gitea status context: `statusContext`;
- the container sandbox policy: `docker.enabled`/`docker.network` and
`bwrap.enabled`/`bwrap.rootfs`/`bwrap.werkdock` — host-pinned as
`werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` — host-pinned as
long as only the host's configuration sets them, master-pinned once the committed
configuration does.
configuration does;
- the trigger of a [follow-up build](#follow-up-builds): `trigger.afterSuccessOf` in every definition, and the whole `trigger` block of a definition the host defines as a follow-up.
A branch may say what its deployment does, never that — or for which branches — it happens.
The distinction is documentary.
Werkator applies one rule: every pinned key is stripped from the branch layer, and the
@@ -110,7 +112,7 @@ The names say where a key is meant to live, not how it is enforced.
This keeps a branch from reaching credentials, reporting statuses to another repository,
raising the global concurrency, disabling its own build container, changing its network
mode, or bypassing its own pull-request gate. Everything else is the branch's to decide —
mode, bypassing its own pull-request gate, or deploying itself. Everything else is the branch's to decide —
it can already run any command through `buildCommand`.
The pinned settings are stripped wherever they appear, in a build definition as well as in
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
@@ -252,6 +254,7 @@ builds:
# branches: ["*", "!master"] # names or globs; a "!" pattern excludes; default: all
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
# activeWithin: 24h # only branches with commits in the last 24h
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
# run before each build
cleanCommand: rm -rf build
# shell command for each build
@@ -397,18 +400,60 @@ Writing any of its keys outside the block is refused with a message naming the d
Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC).
A slot may also be written as `??:MM` — that minute of every hour, expanded to its 24 slots, so the build runs hourly.
Only the latest due slot of a day triggers, so slots missed while the server was down are skipped instead of piling up, and a slot whose pool is still building is retried on the next poll cycle until it succeeds.
A definition may have both; one with neither never triggers automatically — which is how `builds.default` is written when it is meant as a settings base only.
A definition may combine them; one with none of the three triggers (`onPush`, `atTimes`, `afterSuccessOf`) never runs automatically — which is how `builds.default` is written when it is meant as a settings base only.
Werkator logs a warning once when no definition has a trigger at all, because such an instance never builds anything on its own.
#### Follow-up builds
`afterSuccessOf: <name>` makes a definition a *follow-up* of another one, its *predecessor*: it runs on the predecessor's branch at the predecessor's commit whenever a run of the predecessor ends green.
This is how a deployment is configured: a deployment is a build that follows a green build, and it gets everything a build has — its own row in History with log and duration, its own Gitea check under its own `statusContext`, restart, cancel, artifacts, and the per-branch serialization.
Every green run counts, whatever started it — the push watcher, an `atTimes` slot, a UI restart, `werkator retry`, or the startup recovery — and a repeated green run of the same commit triggers the follow-up again.
The follow-up builds the commit that was tested, not the branch's current origin head.
A failed, cancelled, or interrupted run triggers nothing.
The follow-up runs in the branch's worktree, after its predecessor, in the branch's sandbox or container — so a deployment tool like the Docker CLI is provided the way a compiler is.
It inherits `builds.default` like every definition, so a deployment that wants the predecessor's output has to set `cleanCommand: ""` itself; a deployment command should rather be self-contained and rebuild what it ships, because a restart of the follow-up alone runs without its predecessor.
The pull-request gate is not consulted for a follow-up: the predecessor passed it for the same commit, and the follow-up's `branches` selector is its own gate.
The trigger of a follow-up is pinned: a branch's committed config can neither add `afterSuccessOf` to a definition nor change the `trigger` block of a definition the host defines as a follow-up, so a branch cannot deploy itself.
What the deployment *does* comes with the repository, like every build command; where and when it happens, and with which credentials, is the host's.
The host's part lives in `.git/werkator/.werkator.yml`; credentials reach the sandbox like any build setting, through `werkdock.env`/`docker.env`, and a file such as an SSH key through the werkdock sandbox's persistent toolchain home, `.git/werkator/buildenv/home/` on the host, which the sandbox mounts as `/root`.
```yaml
# .git/werkator/.werkator.yml — the host's part: when, for which branches, with what
builds:
deploy:
trigger:
afterSuccessOf: frontend
branches: ["main"]
statusContext: werkator/deploy
werkdock:
env:
DEPLOY_TARGET: user@host:~/doms/example.org/htdocs-ssl
```
```yaml
# .werkator.yml — the repository's part: what
builds:
deploy:
cleanCommand: ""
buildCommand: scripts/deploy-prod.sh -y "$DEPLOY_TARGET"
```
A follow-up whose predecessor no definition has, and a cycle of follow-ups, refuse the start with a message naming the definition — a deployment that silently never runs is the failure the flat-key refusal exists to prevent.
A branch whose committed config drops or renames the predecessor only loses its follow-up, with a warning naming the branch.
A one-shot `werkator build` runs no follow-ups — its process ends with its build — and says which ones the server would have run.
Selector: `trigger.branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches.
A pattern prefixed with `!` excludes instead, and an exclusion always wins regardless of order — `["*", "!master"]` is every branch but master.
That is how a branch gets a build of its own without being built by the default one as well.
`activeWithin` (e.g. `24h`) additionally keeps only branches whose origin head commit is younger than the duration — useful to run a nightly deep check over all recently active branches.
Both parts combine as an intersection.
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `bwrap` with all their keys.
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `werkdock` with all their keys.
A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults.
`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `bwrap.enabled`, `bwrap.rootfs`, and `bwrap.werkdock` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
So is the trigger of a [follow-up build](#follow-up-builds).
Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited.
Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only.
Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of.
@@ -464,22 +509,23 @@ Note that the rest of `.git` — including `.git/config` — is visible to build
The Docker socket is mounted into the container and `DOCKER_HOST`/`TESTCONTAINERS_*` variables are set, so Testcontainers-based builds work inside the container.
All Werkator containers carry `org.hoennig.werkator` labels; stale build containers of the repository are removed before the first Docker build after a restart.
### Notes on `builds.<name>.bwrap`
### Notes on `builds.<name>.werkdock`
With `bwrap.enabled`, Werkator runs the build in a bubblewrap sandbox instead of native execution.
With `werkdock.enabled`, Werkator runs the build in a bubblewrap sandbox instead of native execution.
This is the third runtime, for hosts without root and without a Docker daemon (e.g. Hostsharing managed webspaces); see `docs/plan/17-bwrap-build-runtime.md` and ADR 0008.
Since step 21 session C the sandbox is executed by the `werkdock` CLI (`bwrap.werkdock`, default: resolved via `PATH`) — Werkator no longer invokes `bwrap` itself; `bwrap` must be installed for werkdock.
Since step 21 session C the sandbox is executed by the `werkdock` CLI (`werkdock.binary`, default: resolved via `PATH`) — Werkator no longer invokes `bwrap` itself; `bwrap` must be installed for werkdock.
The section was called `bwrap` and its binary key `bwrap.werkdock` until v1.2.0; both are still read, with a warning naming the file, so an installation can be migrated at its next configuration edit rather than at the next update.
`werkdock doctor` checks the host's capability (it replaced the retired `tools/werkator-build-prerequisites.sh` in step 23).
`bwrap.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
`werkdock.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`.
Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker.
The archive is loaded once per source as the werkdock image `werkator-buildenv-<hash>` into werkdock's store (`$WERKDOCK_HOME`, default `~/.werkdock`) — shared by every repository of this OS user; the hash derives from the source string, so a changed `rootfs` loads a fresh image and stale ones can be removed from the store.
Per-repo Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`.
`bwrap.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in.
`werkdock.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in.
Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user.
`docker` and `bwrap` are mutually exclusive per branch: enabling both is rejected at start, not silently picked.
`docker` and `werkdock` are mutually exclusive per branch: enabling both is rejected at start, not silently picked.
Git works inside the sandbox exactly as inside the Docker container: the primary `.git` is mounted read-only with `.git/werkator/` masked, so builds can run read-only git commands but never reach the machine config or the control token.
## `.git/werkator/.werkator.yml` (not committed)
+7 -1
View File
@@ -184,6 +184,11 @@ ssh <user>@<host>
The tarball unpacks to a `werkator/` directory, so it must not be extracted over `~/opt` directly — unpack it in `/tmp` and move it into place, as above.
Rollback is the reverse: stop, remove the new directory (or jar), move `.bak` back, start.
`tools/remote --env-file .env.<instance> werkator instance-update` does the same sequence for any host, not only the webspace layout it was written for.
Three optional keys in the env file name what differs (see the script's header): `WERKATOR_REPO_DIR` (directory of the watched repository, which also names the systemd unit), `WERKATOR_INSTALL_DIR` (where the runtime bundle is unpacked), and `WERKATOR_SANDBOX` (`werkdock`, the default, or `docker` — a Docker host has no werkdock binary and no rootfs archive to upload).
Their defaults are the layout `instance-install` creates, so an env file that names none of them behaves exactly as before.
The upload happens before the service is stopped and every artifact is checksum-verified after the transfer, so a dropped connection costs the transfer and not the running service.
Then check `https://<public-url>/` for the new version in the footer, and `journalctl --user -u werkator-<repo-name>.service -n 50` for a clean start.
Config file changes are not needed for an update; new keys take their defaults.
@@ -324,7 +329,7 @@ Werkator runs as a systemd *user* service on the assigned localhost port ("eigen
Werkator is never built on the webspace: the runtime bundle and the werkdock binary are built locally and uploaded (ADR 0006).
All steps are driven by `tools/remote`; commands name their role — `instance-*` manages the installed Werkator, `repo-*` the repository it watches.
Each instance is a pair of files (step 23): a transport env file selected with `--env-file` (default `.env`), and a YAML fragment in the configuration schema, named by its `WERKATOR_INIT_CONFIG` key and installed remotely via `werkator init --apply` — e.g. `.env.mih34` + `.env.mih34.yml`, both gitignored.
The fragment carries the Werkator configuration (`server.port`, `publicBaseUrl`, systemd limits, `builds.default.bwrap.*`); the env file only says where and how to reach the host.
The fragment carries the Werkator configuration (`server.port`, `publicBaseUrl`, systemd limits, `builds.default.werkdock.*`); the env file only says where and how to reach the host.
```bash
tools/remote --env-file .env.mih34 werkator check-prerequisites # uploads werkdock, runs its doctor
@@ -335,6 +340,7 @@ tools/remote --env-file .env.mih34 port-forward start # browser tunne
```
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation).
The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
@@ -0,0 +1,91 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
`instance-update` restarts the systemd unit: for a few seconds nothing listens on Werkator's port at all.
On the Hostsharing Managed Webspace deployment (see [deployment.md](../deployment.md#hostsharing-managed-webspace)) the platform's Apache sits in front and proxies via `.htaccess` (`SystemdServiceFiles.htaccessContent`); with the backend refusing connections, Apache answers with its own bare 502 error page, or the request just hangs until it times out.
Either way the instance looks dead rather than "back in a moment", which is confusing during a deploy that is otherwise routine and fast (the restart itself is under a second; the rest of `instance-update`'s time is upload).
## Non-Goals
- A live progress indicator ("2 of 10 seconds left") — the maintenance page is a static file with no way to know how far the restart has gotten.
- Zero-downtime / blue-green deployment — the restart gap itself is not eliminated, only made to look intentional instead of broken.
- The Docker-host and bwrap-webspace-without-a-domain deployment variants, which do not go through Apache/`.htaccess` at all.
## The Scenarios
### Feature: a refused connection shows a maintenance page instead of a bare error
#### Background
- `init --systemd` writes the `.htaccess` only when `server.publicBaseUrl` is set — unchanged; the maintenance page is generated under the same condition, next to it.
- The generated `.htaccess` already proxies every request to `http://127.0.0.1:<port>` via `mod_rewrite [P]`; a refused connection surfaces as Apache's own 502 (and, depending on Apache's timeout handling, 503/504).
#### Scenario#17.01: A refused backend connection serves the maintenance page
So that a deployment's restart window reads "please retry" instead of a bare error or a hang.
- **Given** the generated `.htaccess` and `werkator-maintenance.html` are in a domain's docroot
- **When** the proxied backend refuses the connection (Apache's 502/503/504)
- **Then** Apache serves `werkator-maintenance.html` instead of its default error page
##### Verified by
- [SystemdServiceFilesTest — "the htaccess maps a refused connection to the static maintenance page"](../../src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt)
- Manual on `mih09` (2026-09-03): with the service stopped, `https://werkator.javagil.de/` answered **HTTP 503 with the maintenance page in 0.14 s** — Apache reaches the error path immediately, it does not wait for a timeout. With the service running, the same URL is 200 as before, and `/werkator-maintenance.html` is directly reachable (the `RewriteCond` keeps it out of the proxy).
#### Scenario#17.02: The maintenance page itself is never proxied
So that `ErrorDocument`'s internal sub-request for the page does not loop back into the same rewrite rule (which would proxy it to the — still down — backend and fail again).
- **Given** the generated `.htaccess`
- **When** Apache serves `werkator-maintenance.html` as an `ErrorDocument`
- **Then** the `RewriteCond` excludes exactly that path from the proxy `RewriteRule`
##### Verified by
- [SystemdServiceFilesTest — "the htaccess maps a refused connection to the static maintenance page"](../../src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt)
#### Scenario#17.03: A host without a configured public base URL is unaffected
So that a developer machine or a Docker-host deployment, which never went through `.htaccess`, sees no new file and no behavior change.
- **Given** `server.publicBaseUrl` is blank
- **When** `init --systemd` runs
- **Then** neither `.htaccess` nor `werkator-maintenance.html` is written — unchanged from before this PR
##### Verified by
- Existing `InitCommand` behavior (`server.publicBaseUrl.isNotBlank()` gate); not separately tested, as before this PR.
## The Solution
`SystemdServiceFiles.htaccessContent` gains three `ErrorDocument` lines (502/503/504) pointing at a new static `werkator-maintenance.html`, plus one `RewriteCond` excluding that file from the catch-all proxy rule.
`SystemdServiceFiles.maintenancePageContent()` is a small, self-contained HTML page ("Werkator is restarting — please retry in a few minutes") — no external CSS/fonts/images, since nothing would be there to serve them while Werkator itself is the thing that is down.
`InitCommand.createSystemdFiles` writes it under the same `server.publicBaseUrl.isNotBlank()` gate as the `.htaccess`, next to it in `.git/werkator/`.
`tools/remote instance-start` copies both files into the domain docroot in the same step (it already copied the `.htaccess` there; the maintenance page rides along).
No config key was added: like the `.htaccess` itself, the maintenance page is generated whenever a `publicBaseUrl` is configured, not behind a separate toggle nobody is expected to turn off.
## Open Questions
- **Does Apache actually reach the error path, or does the client just time out first?** Answered by the live test above: on `mih09` a refused connection surfaces as **503 within 0.14 s**, not a hang — a connection *refused* is immediate, unlike a connection that is accepted and then stalls. The `ErrorDocument` covers 502/503/504 so the exact code Apache picks does not matter.
## Additional Changes
- None beyond the feature itself.
## Follow-up work discovered while deploying this
- `tools/remote instance-start` places the `.htaccess` (and now the maintenance page) into `doms/<domain>/subs/www/`, but the live `mih09` instance serves from `doms/<domain>/htdocs-ssl/` — a pre-existing path mismatch, unrelated to this PR. Both files were therefore copied into `htdocs-ssl/` by hand for this deployment; `instance-start` still needs fixing separately.
- `InitCommand.loadedServerConfig()` swallows every config-loading exception and falls back to a blank `ServerConfig`, so any unrelated config error makes `init --systemd` silently skip both files as if no `publicBaseUrl` were set. Noticed while smoke-testing this change; worth a warning on that path.
## Prerequisite PRs
- None; builds on the existing `.htaccess` generation (`SystemdServiceFiles`, `InitCommand`), unchanged in shape.
## Follow-up PRs
- An explicit Apache `ProxyTimeout` if the client-side hang (see Open Questions) turns out to matter in practice.
@@ -0,0 +1,98 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
`tools/remote` grew with the Hostsharing Managed Webspace rollout (steps 21 and 23) and encoded that rollout's layout as if it were the only one.
The watched repository had to be `$WERKATOR_PATH/werkator`, the runtime `$WERKATOR_PATH/.werkator/werkator`, and the systemd unit was spelled out as `werkator-werkator.service`; a werkdock binary and a bwrap rootfs archive were built and uploaded unconditionally.
`vm4006`, the Docker host that has been running Werkator since long before the script existed, matches none of that: its watched repository is `~/hs.hsadmin.ng` (so its unit is `werkator-hs.hsadmin.ng.service`), its runtime lives in `~/opt/werkator`, and it has neither werkdock nor a rootfs because its builds run in Docker.
So that host could only be deployed by hand, and had drifted nine releases behind — the question that started this PR was whether the Werkdock and multi-repo work had broken it, which it had not: what was broken was the deployment tooling.
The second problem surfaced while deploying: `instance-update` stopped the systemd unit and *then* started the upload.
The 66 MB transfer to `vm4006` died with `scp: Connection closed`, leaving the host with no running Werkator and nothing new to start.
## Non-Goals
- `instance-start` for a Docker host: it places a Hostsharing `.htaccess` into a `doms/<domain>/` docroot, which only exists on a Managed Webspace. `vm4006` uses the managed nginx container instead, and keeps its existing units.
- `repo-init` for `vm4006` — the repository has been cloned and configured there for months; only `instance-update` was needed.
- Making the machine configuration of `vm4006` current: it still carries the pre-rename `gitTally:` meta key, which is read by nothing today. Harmless while `ConfigVersions.FORMAT_BROKE_IN` is empty, a trap on the day it is not.
## The Scenarios
### Feature: one deployment command for every host layout
#### Background
- The env file carries transport values only; the three new keys describe *where* things are on the host, not what Werkator does.
- The defaults are exactly the layout `instance-install` creates, so an env file naming none of them resolves as before.
#### Scenario#18.01: A host that predates the script can be deployed with it
So that an installation is not condemned to hand-typed `scp` sequences because it was set up before the tooling existed.
- **Given** an env file with `WERKATOR_REPO_DIR=hs.hsadmin.ng`, `WERKATOR_INSTALL_DIR=/home/tallyman/opt` and `WERKATOR_SANDBOX=docker`
- **When** `tools/remote --env-file .env.vm4006 werkator instance-update` runs
- **Then** it addresses `werkator-hs.hsadmin.ng.service`, unpacks into `~/opt`, and uploads neither a werkdock binary nor a rootfs archive
##### Verified by
- Live on `vm4006` (2026-09-03): `check-prerequisites` reported the Docker daemon instead of running `werkdock doctor`; the update swapped `~/opt/werkator` from v1.0.1 to **v1.1.2**, the unit came up `active`, `/api/watcher` polls without errors, and `/api/system` still reports real disk figures (`diskSource.kind: volume`) rather than the quota shape from PR#16.
#### Scenario#18.02: The webspace hosts are unaffected
So that making the script layout-aware does not break the deployment path that is actually in production.
- **Given** `.env.mih09` and `.env.mih34`, neither naming any of the new keys
- **When** the layout is resolved
- **Then** repository directory, install directory, unit name and sandbox are identical to the hardcoded values they replace
##### Verified by
- Resolution measured for both env files (2026-09-03): `REPO_DIR=$WERKATOR_PATH/werkator`, `INSTALL_DIR=$WERKATOR_PATH/.werkator`, `UNIT=werkator-werkator.service`, `SANDBOX=bwrap`.
- Live on `mih09` (2026-09-03): a full `instance-update` ran through the bwrap path — werkdock uploaded, `werkdock 0.1.0-dev` reported after the swap, service `active`, `https://werkator.javagil.de/` answering 200.
#### Scenario#18.03: A failed transfer does not take the service down
So that a dropped connection costs the upload and nothing else.
- **Given** an instance whose service is running
- **When** the runtime bundle cannot be transferred
- **Then** the service is still running, because the upload happens before the stop, and a partially transferred file is never moved into place
##### Verified by
- The failure itself on `vm4006` (2026-09-03), which is what this scenario is written from: with the old order, `scp: Connection closed` left the unit stopped and the host without Werkator.
- Live on `mih09` (2026-09-03): the bundle already on the host was recognised by its sha256 and skipped, so the upload step cost nothing and the stop followed only after it.
## The Solution
Three optional env keys replace three hardcoded assumptions.
`WERKATOR_REPO_DIR` and `WERKATOR_INSTALL_DIR` are resolved absolute-or-relative-to-`WERKATOR_PATH`, and the unit name is now *derived* from the repository directory the way `SystemdServiceFiles.unitName` derives it (basename, every character outside `[A-Za-z0-9_.-]` replaced by a dash) instead of being spelled out — one rule, in two places, with the script naming the Kotlin function it mirrors.
`WERKATOR_SANDBOX=docker` skips everything bwrap-shaped: no werkdock build, no werkdock upload, no rootfs archive, and `check-prerequisites` asks the Docker daemon instead of running `werkdock doctor`.
`repo-add` clones beside the watched repository rather than into `WERKATOR_PATH`, which is the same directory whenever the default layout is used.
`deploy_instance` is split into `upload_instance_artifacts` and `swap_instance_runtime`, and `instance_update` calls the first *before* stopping the unit.
Every artifact is uploaded to `<name>.part`, compared by sha256 with the local file, and only then moved into place; three attempts, and an unchanged artifact is skipped entirely.
The checksum is not belt-and-braces: a truncated tarball would unpack into a runtime that starts and misbehaves, which is far worse than the failed transfer it came from.
## Open Questions
- **Should `instance-start` learn the Docker-host shape too?** Not answered here. `vm4006` keeps its existing units and its managed nginx; the day it needs regenerating, `init --systemd` on the host is the documented path.
## Additional Changes
- None beyond the feature itself.
## Follow-up work discovered while deploying this
- `https://vm4006.hostsharing.net:8443/` is not reachable from outside the host, while `https://127.0.0.1:8443/` answers 200 and the `werkator-nginx-hs.hsadmin.ng` container publishes both ports. Pre-existing and unrelated to this PR — the host's own firewall, not Werkator.
- The machine configuration on `vm4006` still declares `gitTally: version: since: "0.9.20"`; the current code reads only `werkator:`, so the file's version claim is silently ignored (`werkator.version.since` prints empty). It costs nothing while `ConfigVersions.FORMAT_BROKE_IN` is `""`, and stops protecting that host the moment plan step 18 sets it.
## Prerequisite PRs
- None; it changes only `tools/remote` and the documentation.
## Follow-up PRs
- None planned.
@@ -0,0 +1,98 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
The build sandbox for hosts without Docker was configured as `bwrap`, named after the mechanism rather than after the thing Werkator runs.
Since step 21 session C (v1.0.0) Werkator does not invoke `bwrap` at all: it shells out to the [werkdock](https://git.javagil.de/mi/werkdock) CLI, which assembles the bubblewrap invocation and owns the image store.
The name outlived its truth, and the clearest symptom was the key `bwrap.werkdock` — a section naming its own executor.
It also leaked outward.
`tools/remote` gained a `WERKATOR_SANDBOX` key in PR#18 whose value had to be `bwrap` while the very thing it switches on is *uploading the werkdock binary*, and the question that prompted this PR — "is there also `WERKATOR_SANDBOX=werkdock`?" — is one nobody would ask about a name that matched.
## Non-Goals
- Refusing the old name. That belongs to the release which sets `ConfigVersions.FORMAT_BROKE_IN` (plan step 18): only there can a file that declares no version be caught by name at all, and only there is one migration asked of the operator instead of two.
- Renaming `tools/build-bwrap-rootfs.sh` or `docs/plan/17-bwrap-build-runtime.md`. The script really does build a bubblewrap rootfs, and plan documents are historic records.
- Touching ADR 0008, which decided the *runtime* and is a snapshot of that decision.
## The Scenarios
### Feature: the sandbox is named after the tool that runs it
#### Background
- `builds.<name>.werkdock` replaces `builds.<name>.bwrap`, and `werkdock.binary` replaces `bwrap.werkdock`.
- The pinned set is unchanged in meaning: `enabled`, `rootfs` and the binary stay host-pinned, under their new names.
#### Scenario#19.01: A configuration written for the old name keeps working
So that no installation has to be edited before it can be updated — the section lives in machine configurations that no repository tracks.
- **Given** a configuration writing `builds.default.bwrap` with `enabled`, `rootfs`, `werkdock` and `env`
- **When** the configuration is loaded
- **Then** the settings appear as `werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary` and `werkdock.env`, and the file is named once in a warning
##### Verified by
- [ConfigLoaderTest — "the legacy bwrap section is read as werkdock, its werkdock key as binary"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#19.02: The old name is not a way around the pinning
So that a branch cannot escape its sandbox by writing the section a branch is not allowed to write under its previous name.
- **Given** a host configuration with `werkdock.enabled: true` and a rootfs
- **When** a branch's committed config sets `bwrap.enabled: false` with a foreign rootfs
- **Then** the sandbox stays enabled and the host's rootfs is used
##### Verified by
- [ConfigLoaderTest — "a legacy bwrap section on a branch is pinned exactly like the new name"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#19.03: The new name behaves exactly as the old one did
So that the rename is a rename, not a change of behavior.
- **Given** configurations using `werkdock` throughout
- **When** builds are dispatched, pinned keys stripped, and both sandboxes enabled at once
- **Then** the werkdock runner is selected, the branch cannot override the pinned keys, and enabling docker and werkdock together is rejected naming both
##### Verified by
- [DispatchingBuildRunnerTest — "runs in the werkdock sandbox when the branch enables it (and not Docker)"](../../src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt)
- [ConfigLoaderTest — "a branch cannot disable its werkdock sandbox …"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt) and "enabling both docker and werkdock on a build is rejected, not picked silently"
- [WerkdockBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt), unchanged in substance and renamed with the runner
## The Solution
`BwrapConfig``WerkdockConfig` (field `werkdock``binary`), `BwrapOverrides``WerkdockOverrides`, `BranchConfig.bwrap``.werkdock`, `BwrapBuildRunner``WerkdockBuildRunner`, and `PINNED_BWRAP_KEYS``PINNED_WERKDOCK_KEYS` with `binary` in place of `werkdock`.
The compatibility lives in exactly one function, `ConfigLoader.renameLegacySandbox`, applied in `loadFile` and `parseYaml` — the two places a raw layer enters — so it runs before merging, before pinning and before binding, and every consumer downstream knows one name.
That placement is what makes Scenario#19.02 hold without a second thought: the branch layer is normalised *before* `stripPinned` reads it, so the old name cannot smuggle a pinned key past a check that looks for the new one.
Where a file writes both sections, the explicit `werkdock` one wins, because it is the name that is meant.
The warning is emitted once per file (`warnedSections`), like the other section-level warnings, since the config is re-read on every poll cycle.
The version is bumped to 1.2.0 with a release note: the minor, because this changes the configuration schema, and a deployment must be identifiable as the one that introduced it.
## Open Questions
- **Should `WERKATOR_SANDBOX` keep accepting `bwrap`?** It does today, normalised on read and documented as the former name. The env files are local and gitignored, so this alias costs one line and can go whenever the config alias does.
## Additional Changes
- None beyond the rename and its documentation.
## Deployment note
The order matters, and only in one direction: deploy v1.2.0 to a host *before* rewriting its init fragment (`.env.<instance>.yml`) to the new key names.
A fragment carrying `werkdock:` applied by an older Werkator fails the fragment's strict schema validation — which is the safe outcome, but a failed `repo-init` nonetheless.
The reverse never breaks: v1.2.0 reads every existing `bwrap:` fragment and machine config as before.
## Prerequisite PRs
- [PR#18](2026-09-03-PR%2318-remote-host-layout.md) introduced `WERKATOR_SANDBOX`, whose value this PR renames.
## Follow-up PRs
- Plan step 18 (removing the legacy `branches` section) sets `ConfigVersions.FORMAT_BROKE_IN`; the `bwrap` alias should be dropped in the same release, refusing the key by name.
@@ -0,0 +1,42 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
The build for commit `7028ca8` on `main` failed on the mih09 production instance with
`BuildExecutorTest > with maxConcurrent 1 a second branch stays PENDING until the first finished`,
while a retry of the very same commit passed, and the test passes locally.
The test was timing-dependent.
It started a build for `branch-a` whose build command was `sleep 1`, immediately started a second build for `branch-b`,
and then asserted — without any synchronization at all — that `branch-b` was still `PENDING`.
That assertion only held as long as the test thread reached it within the one second `branch-a` slept.
On a shared host under CPU contention the executor can get through `branch-a` entirely (queued, running, slept, succeeded) first,
and `branch-b` is then already `RUNNING` or `SUCCESS` when the assertion runs.
The failure therefore says nothing about the executor; it is pure scheduling noise that costs a build and a retry every time it hits.
## Non-Goals
- No change to production code — `BuildExecutor` is not touched, its queueing behaviour is unchanged.
- No sweep of the other timing-sensitive tests in the suite; only the one that actually flaked is fixed.
## The Solution
`branch-a` no longer sleeps for a fixed time, it blocks until the test says so:
its build command is `until [ -f gate ]; do sleep 0.05; done`, and the build workspace is the test's working directory.
The test now
1. waits (via `eventually`) until `branch-a` is `RUNNING`, so the single executor slot is provably occupied,
2. asserts that `branch-b` is `PENDING` — which cannot race anything, because `branch-a` cannot finish before the gate file exists,
3. creates the gate file, and only then awaits both builds' `SUCCESS`.
The assertion on the event transitions (`branch-b` goes `RUNNING` only after `branch-a` reached `SUCCESS`) is unchanged.
A blocking gate was chosen over a mocked `BuildRunner` because it keeps the test on the real `ProcessBuildRunner`,
so it still covers the actual process handling rather than only the executor's bookkeeping.
Verified by running `BuildExecutorTest` five times on an idle machine and three more times with twice `nproc` busy-loops saturating the CPU,
which is the condition that produced the original failure.
- [BuildExecutorTest](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorTest.kt)
@@ -0,0 +1,30 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
The repository's home is `https://git.javagil.de/mi/werkator.git` since the move to the own Gitea instance,
but `tools/remote` still defaulted `WERKATOR_REPO_URL` to the GitHub mirror.
The mih09 production instance was cloned from that mirror and consequently watched a `main` that nobody pushes to any more:
it kept reporting the last GitHub state as green while three merged pull requests sat unbuilt on the real `main`.
The failure that started this — a flaky test already fixed on Gitea's `main` — could not be re-verified live,
because the instance had no way to see the fix.
## Non-Goals
- The existing clone on mih09 is not touched by this change; its remote was repointed by hand (`git remote set-url`), and `repo-init` skips an existing clone.
- The generic placeholder URLs in `docs/deployment.md` stay as they are — they describe cloning *any* watched repository, not werkator's own.
- No decision about mirroring to GitHub; the mirror simply stops being the source an instance builds from.
## The Solution
`REPO_URL` in [tools/remote](../../tools/remote) defaults to the Gitea URL, and the usage comment says so.
The value stays overridable via `WERKATOR_REPO_URL` in the instance's env file,
so an installation that deliberately watches a different remote is unaffected.
Anonymous HTTPS works against Gitea exactly as it did against GitHub, so no deploy key or token is involved.
Note that pull requests opened via AGit-Flow create no branch in Gitea,
so an instance watching this remote sees `main` only — branch builds require pushing real branches.
@@ -0,0 +1,199 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- [ADR 0007 — build definitions](../adrs/0007-2026-08-31.build-definitions.md): the `builds` section and its `trigger` block this PR extends.
- [configuration.md — build definitions](../configuration.md#build-definitions): the reference this PR updates.
- Werkbaum's `scripts/deploy-prod.sh`: the first deployment meant to run this way.
## The Problem
Werkator builds and reports, but it cannot deploy.
Werkbaum's production deployment is still a script run by hand from a developer machine, after looking at the Werkator status.
The step "if the build is green, run this" is exactly what a CI system is for, and the hand-off between the two is where releases go wrong: the wrong commit gets deployed, or a red commit, or nothing.
Three ways to add deployments were considered:
- A `deployCommand` next to `buildCommand`, run after a green `buildCommand` inside the same run.
Simple, but it creates a second command path inside one build — one log, one status, one duration for two different things — and a failed deployment would turn a green build red.
- A separate `deploy` section with its own executor path, statuses, and cancellation.
A second execution path next to `buildCommand`, with everything the first one has to be built again.
- **A deployment is a build that follows a green build.** Chosen.
A build definition may declare that it runs whenever another definition of the same branch turns green.
Everything a build has comes for free: its own row in History with log and duration, its own Gitea check under its own `statusContext`, restart without rebuilding, cancel, artifacts, and the per-branch serialization.
## Non-Goals
- A native execution path for deployments.
A follow-up build runs where every build of its branch runs — in the werkdock sandbox or the Docker container — so a deployment tool such as the Docker CLI is provided the same way a compiler is.
- Waiting for *several* builds to be green.
`afterSuccessOf` names one build; an `all of` form is a follow-up PR if a repository ever needs it.
- A separate secrets mechanism.
Deployment credentials reach the sandbox the way any build setting does — the host's `werkdock.env`/`docker.env` for the definition, and files placed in the sandbox's persistent toolchain home — and the pinning below keeps them on the branches the host names.
- Handing the predecessor's artifacts to the follow-up.
The follow-up runs in the same worktree, so the predecessor's output is *usually* still there, but a deployment command must not rely on it (see Open Questions).
- Recovering a follow-up whose enqueueing was lost to a server restart between the two builds.
The operator restarts the predecessor, which triggers the follow-up again.
## The Scenarios
### Feature: a build that follows a green build
#### Background
- A *follow-up build* is a build definition whose `trigger` block carries `afterSuccessOf: <name>`, naming another definition of the same configuration — its *predecessor*.
- The follow-up runs on the predecessor's branch at the predecessor's commit, never at the branch's current origin head, so a deployment always ships the commit that was tested.
- Like every non-default build it records under its own pool `<branch>@<name>`, and it should carry its own `statusContext` so that Gitea shows the deployment as a check of its own.
- *Green* is every run of the predecessor that ends with `SUCCESS`, whatever started it: the push watcher, an `atTimes` slot, a UI restart, `werkator retry`, or the startup recovery.
- The key belongs to the `trigger` block: it says *when* the build runs, so it is never inherited from `builds.default`, and writing it flat is refused like every other trigger key.
#### Scenario#23.01: A green predecessor triggers the follow-up on the same commit
So that a deployment ships exactly the commit that was just tested, with its own log, duration, and Gitea check.
- **Given** a definition `deploy` with `trigger.afterSuccessOf: frontend` and `statusContext: werkator/deploy`
- **and** the build `frontend` of branch `main` at commit `c1` is running
- **When** that build finishes with `SUCCESS`
- **Then** a build `deploy` of branch `main` at commit `c1` is enqueued
- **and** it is recorded under the pool `main@deploy`
- **and** its Gitea status is posted under the context `werkator/deploy`
- **and** the origin head of `main` having moved on to `c2` meanwhile changes nothing about that
##### Verified by
- [FollowUpTriggerTest — "a green predecessor enqueues the follow-up at the predecessor's commit"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
#### Scenario#23.02: Every green run triggers again, including a repeated run of the same commit
So that a deployment can be repeated by rerunning the build — and so that no run of a green build is silently *not* deployed, which would be more confusing than a redundant deployment.
- **Given** the build `frontend` of `main` at `c1` was green and `deploy` ran for it
- **When** `frontend` at `c1` is restarted from the UI, retried from the CLI, or rebuilt by an `atTimes` slot, and turns green again
- **Then** `deploy` is enqueued for `c1` again
##### Verified by
- [FollowUpTriggerTest — "every green run of the predecessor triggers the follow-up again"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
#### Scenario#23.03: A run that is not green triggers nothing
So that nothing is ever deployed from a failed, cancelled, or interrupted build.
- **Given** the same definitions
- **When** the build `frontend` ends as `FAILED`, `CANCELLED`, or `INTERRUPTED`
- **or** a build other than `frontend` ends as `SUCCESS`
- **Then** no `deploy` build is enqueued
##### Verified by
- [FollowUpTriggerTest — "only a SUCCESS of the named predecessor triggers"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
#### Scenario#23.04: The trigger of a follow-up is host-pinned
So that a branch can never deploy itself: a follow-up build is the host's way to hand real-world effects — targets, credentials — to a commit, and the host alone decides *when* and *for which branches* that happens.
- **Given** the host configuration defines `deploy` with `trigger: {afterSuccessOf: frontend, branches: [main]}`
- **When** a branch's committed `.werkator.yml` sets `builds.deploy.trigger.branches: ["*"]`
- **or** adds `afterSuccessOf` to any definition's trigger
- **Then** the host's trigger block of `deploy` is used unchanged, and the branch's `afterSuccessOf` is dropped with a warning naming the branch
- **and** a branch the host's selector does not name never runs `deploy`, however green its own builds are
##### Verified by
- [ConfigLoaderTest — "a follow-up trigger is pinned to the host, a branch cannot add or widen one"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#23.05: What the follow-up does comes with the repository
So that the deployment command is versioned with the code it deploys, like every other build command — while the host keeps the targets and credentials.
- **Given** the host defines `deploy` with its trigger and `werkdock.env: {DEPLOY_TARGET: …}`
- **and** the committed `.werkator.yml` of `main` defines `deploy` with `cleanCommand: ""` and `buildCommand: scripts/deploy-prod.sh -y "$DEPLOY_TARGET"`
- **When** `deploy` runs for `main`
- **Then** it runs the committed command with the host's environment, in the branch's sandbox, in the branch's worktree, after its predecessor and serialized with the branch's other builds
##### Verified by
- [BuildExecutorTest — "a follow-up build runs in its branch's worktree after its predecessor"](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorTest.kt)
- [ConfigLoaderTest — "a branch supplies the command of a host-triggered follow-up"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#23.06: A follow-up that could never fire is refused at start
So that a deployment that silently never runs cannot exist — the same reasoning that refuses a flat trigger key.
- **Given** a configuration whose `afterSuccessOf` names a definition that does not exist
- **or** whose follow-ups form a cycle (`a` after `b`, `b` after `a`)
- **or** which writes `afterSuccessOf` outside the `trigger` block
- **When** the configuration is loaded
- **Then** loading fails with a message naming the definition and the reason
- **and** a definition with only `afterSuccessOf` counts as triggered, so the "no build triggered" warning is not raised for it
##### Verified by
- [ConfigLoaderTest — "afterSuccessOf must name an existing definition and must not form a cycle"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
- [ConfigLoaderTest — "afterSuccessOf written flat is refused like every trigger key"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#23.07: Follow-ups fire in server mode only
So that the invariant "nothing is scheduled during CLI runs or tests" holds: a CLI `build` ends when its build ends, and a follow-up enqueued into a process that is about to exit would only ever be recorded as interrupted.
- **Given** `werkator build main` runs from the CLI and turns green
- **When** the command finishes
- **Then** no follow-up was enqueued, and the CLI says which follow-up the server would have run
##### Verified by
- [FollowUpTriggerTest — "the trigger listens only while the watcher runs"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
- [WatcherTest — "start arms the follow-up trigger before the recovery, stop disarms it"](../../src/test/kotlin/de/hoennig/werkator/watcher/WatcherTest.kt)
- [BuildCommandTest — "a green CLI build names the follow-up builds the server would run, and runs none"](../../src/test/kotlin/de/hoennig/werkator/commands/BuildCommandTest.kt)
## The Solution
`TriggerConfig` gets a fourth key, `afterSuccessOf: String` (empty: none).
It sits inside the `trigger` block on purpose, next to `onPush` and `atTimes`: it answers "when does this build run", so the structural rule of ADR 0007 makes it non-inheritable without touching any list, and `checkTriggerBlocks` refuses it written flat by adding it to `FLAT_TRIGGER_KEYS`.
`isTriggered` counts it, so a definition with nothing but a predecessor is not reported as never triggering.
A definition may itself be followed; `ConfigLoader` refuses an unknown predecessor and a cycle when it validates the merged configuration.
A new `FollowUpTrigger` in the `watcher` package listens to `BuildStatusChangedEvent`.
On a `SUCCESS` it resolves the definitions of the result's branch at the result's commit — the same `definitionsFor` the watcher uses for the branch layer — and enqueues, through `BuildExecutor.startBuild(repo, branch, commit, name)`, every definition whose `afterSuccessOf` names the finished build and whose selector selects the branch.
Passing the commit explicitly is what makes Scenario#23.01's last line true; the executor's duplicate guard folds a follow-up that is already queued for the same commit.
The listener is armed by the watcher's `start()` and disarmed by `stop()`, which is how it stays silent in CLI runs and tests (Scenario#23.07).
The pinning extends `stripPinned`: `afterSuccessOf` is removed from every trigger of the branch layer, and for every definition whose host trigger carries `afterSuccessOf` the branch layer's whole `trigger` block is dropped.
That is the smallest rule that makes Scenario#23.04 hold: a branch keeps the right to describe what its deployment does, and loses only the right to decide that — or where — it happens.
The pull-request gate is not consulted for a follow-up: the predecessor passed it already for the same commit, and the host's selector is the follow-up's own gate.
The follow-up runs like any other build of its branch — same worktree, same sandbox, serialized behind its predecessor.
It inherits `builds.default`, so a deployment that wants the predecessor's output has to set `cleanCommand: ""` itself.
Three places stay in sync with the new key, as the invariant demands: the data classes, the `init` templates (a commented `afterSuccessOf` line under the `trigger` block), and `docs/configuration.md`, which gets a "Follow-up builds" paragraph under build definitions and the new pinning in the branch-layer section.
`AGENTS.md` names the key in the trigger and pinning invariants.
## Open Questions
- **Credentials inside the sandbox.**
Werkdock clears the environment and mounts the repository's persistent toolchain home as `/root`, so an SSH key placed under `.git/werkator/buildenv/home/.ssh/` on the host is `/root/.ssh/` inside the sandbox, and `werkdock.env` carries the target.
For Docker there is no equivalent mount today; a key passed through `docker.env` works but is visible in `docker inspect`.
Implemented: nothing new — the PR documents the werkdock path in `configuration.md` and leaves a Docker mount to a follow-up if vm4006 ever deploys.
- **Restarting the follow-up alone.**
Restart re-runs `deploy` without its predecessor, in a worktree that may have been cleaned by a later build of the branch.
Implemented: allowed, and the reference recommends a self-contained deployment command that rebuilds what it ships (Werkbaum's `deploy-prod.sh` does).
- **A predecessor that exists only on a branch.**
The host's `afterSuccessOf: frontend` refers to a name the branch layer may rename.
Implemented: refused at start for the primary configuration; for a branch whose layer lacks the name, a warning once per branch and commit, and no follow-up
(verified by [ConfigLoaderTest — "a branch whose layer lacks the predecessor loses only its follow-up"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)).
An instance fragment checked by `init --apply` is not checked for its predecessor at all: the build it names may well live in the project config it is merged with, and the merged configuration is checked on every load anyway.
## Additional Changes
- `BuildStatusChangedEvent` now carries the `RepoContext` the result belongs to: a `BuildResult` does not know its repository, and the listener has to enqueue into the right one.
- `ConfigLoader.loadWithBranchLayer` and `loadForWorktree` take an optional branch name, used only to name the branch in the pinning warnings; the watcher passes it.
- The follow-up check distinguishes three loads: the primary configuration refuses a missing predecessor, a branch layer warns, and an instance fragment (`init --apply`) skips the check — its predecessor may live in the project config it is merged with.
- The "no build triggered" warning now names `afterSuccessOf` as the third trigger.
## Follow-up PRs
- Werkbaum: commit the `deploy` definition to its `.werkator.yml`, add the host part on mih09, and retire the manual `deploy-prod.sh` invocation from the README.
- `afterSuccessOf` as a list (all green), if a repository ever needs a deployment gated on more than one build.
- A Docker bind mount for credential files, if a Docker host deploys.
+151
View File
@@ -0,0 +1,151 @@
# RFC 0001: Web UI Redesign — the Instrument Panel
**Status:**
- proposed: 2026-09-03
- accepted: -
- rejected: -
**Proposal:** The Werkator web UI adopts the **Instrument Panel** direction: a teal palette in a light and a dark mode, IBM Plex typography, a repository strip that previews the state of every served repository, a title hierarchy that names the view first and explains it second, and a tab bar at the foot that becomes the mobile navigation.
The architecture does not change: server-rendered Thymeleaf, one `werkator.css`, one hand-written `werkator.js`, JSON polling, no framework, no frontend build pipeline.
## Context and Problem Statement
The current UI is a functional port of the legacy generated pages: a table per view, pill badges, system font, blue links.
It is correct and calm, but it looks like every other CI page and gives no hint of the other repositories an instance serves (ADR 0009).
The brief for this RFC was "fancy, but serious and trustworthy", with two references from the same author for visual kinship:
- [werkbaum.javagil.de](https://werkbaum.javagil.de/) — light paper with a fine grid, IBM Plex, a petrol accent, panel labels in small caps.
- [javagil.de/vibe-engineering](https://javagil.de/vibe-engineering) — a dark instrument panel: ink and petrol, clay for warnings, monospaced spaced labels, a tab bar at the foot.
A hard constraint of this RFC is honesty towards the data.
The mockups show only what the API delivers today; nothing is invented to make a screen look richer.
### What the UI Has to Work With
Per build row (`BuildRowView`, `BuildResultDto`): status, branch name, commit (12-character abbreviation, full id for copying), started at (`yyyy-MM-dd HH:mm`), duration (`m:ss`; a pending build shows its wait time in italics), artifact key with the artifact, permalink and live-log links, and the actions restart and delete (history has no restart).
Statuses: `pending`, `running`, `success`, `failed`, `interrupted`, `cancelled`, plus `unknown` for a never-built branch and the client-side `finished` on a card whose build has left the current list.
Views: Latest (one build per name), Branches (every origin branch and its latest build), History (all stored builds), Current (running builds with their live log), System (seven metric rows with current/min/max/avg, warn from 80 %, critical from 90 %), the artifact page, and the release notes.
Live state: the indicator is `static`, `live` or `error`; the watcher banner reports `watcher stopped`, `origin unreachable` or `poll cycle failed`.
Multi-repo: the repository switcher is a server-rendered `<select>` of names; `WatcherState.repositories` already carries a per-repository watcher state that the UI does not show.
What does **not** exist, and therefore appears in no mockup: a commit subject line, a typical or expected duration, an ETA, a per-branch build history, test counts on a row, and any cross-repository status summary in the API.
## Considered Options
Six directions were sketched on a shared design canvas, two rounds of three, all with the same sample rows.
| Option | Idea | Why | Tradeoff |
|---|---|---|---|
| A · Quiet Console | Today's design refined: top bar, dot-plus-word statuses, hover actions | Smallest step, everything stays valid | Least distinctive |
| B · Mission Board | Health tiles, one card per branch with a history strip, running build with progress | Answers "is everything fine?" at a glance | Needs data the API does not have (history, typical duration) |
| C · Ledger | Warm paper, serif masthead, hairline rules, typographic status marks | The most "serious"; reads like a signed record | Leaves the system font, needs its own dark theme |
| D · Paper Rail | Werkbaum's paper and grid, a repository rail on the left with per-branch dots | Family resemblance to Werkbaum; other repositories visible | 250 px of table width lost; empty with one repository |
| **E · Instrument Panel** | Vibe-Engineering's dark panel, repositories as tiles, tab bar at the foot | Reads like a control room; failures in clay stay serious without alarm | Dark-only as drawn; needs a light palette |
| F · Fleet Overview | A new landing page with one panel per repository, ledger typography on paper | One page answers the question for the whole instance | Becomes a list beyond five repositories |
Round one (AC) still contained invented data; it is kept on the canvas for the visual ideas only.
**E was chosen**, and round three worked out what it lacked: the light mode, the ten-repository case, the title hierarchy, and the phone layout.
## The Design
### Palette
Both modes are CSS custom properties on `:root`, switched by `prefers-color-scheme` as today (`color-scheme: light dark`).
Failures use clay, not red, so they stay serious without shouting; the accent is teal in both modes.
| Token | Dark | Light | Used for |
|---|---|---|---|
| bg | `#061C1F` | `#EAF4F2` | page ground |
| panel | `#0A2A2E` | `#FFFFFF` | tables, cards, chips |
| panel-2 | `#0F3A3D` | `#D6ECE8` | the current repository, the active tab |
| line | `#17474B` | `#C9DFDB` | borders and rules |
| text | `#E4EEEC` | `#0B2B2E` | body text |
| text-2 | `#B4CBC8` | `#35595B` | timestamps |
| muted | `#7DA19E` | `#5E8583` | labels, footers |
| accent | `#5FD3C7` | `#0E8079` | links, success, running, the live indicator |
| accent-2 | `#1E9A93` | `#149A90` | underlines, the current repository's border |
| clay | `#E09070` | `#B0563B` | failed, error, delete, watcher warnings |
| clay-2 | `#C4664A` | `#C4664A` | the border of a failing repository chip |
| ghost | `#4A7370` | `#BFD4D1` | cancelled, interrupted, unknown |
Tinted rows: a running row gets 16 % (dark) or 10 % (light) of accent-2 as background, a failed row 12 % or 10 % of clay-2.
The reference's background grid was tried and dropped: it competes with the table, especially in light mode.
### Typography
IBM Plex Sans for text, IBM Plex Mono for commits, timestamps, durations and every label.
Labels are 10 px Mono, uppercase, letter-spaced 0.12 em, in `muted`; statuses are 11 px Mono uppercase in their status color, each preceded by an 8 px dot (outlined for pending, pulsing for running).
Fallback stacks: `"IBM Plex Sans", "Segoe UI", system-ui, sans-serif` and `"IBM Plex Mono", ui-monospace, Consolas, monospace`.
Whether Plex is bundled under `static/` or the fallback stack is accepted is an open question below.
### Anatomy of a Page (desktop)
1. **Header**, 52 px: logo, `Werkator` with the Gitea repository name in accent, a small label `updated HH:mm:ss`; right: the live indicator as an outlined chip with a pulsing dot, the reload button.
2. **Repository strip**: see below.
3. **Panel** with the view's title: the view name at 22 px semibold with a 2 px accent-2 underline, followed by a one-line label that explains it (`Latest``one build per branch, newest first`; `Branches``every origin branch and its latest build`; `History``all stored builds, newest first`; `System``instance metrics since first start`); on the right a Mono line with the row count, the last poll and the watcher state.
4. **Table**, columns as today (Status, Branch, Commit, Started, Duration, Artifacts, Actions), rows 9 px padding on a 1 px `line` rule; copy buttons as outlined 13 px icons; artifact links and actions as stroke icons (no emoji).
5. **Footer**: version and copyright left, the navigation as a Mono tab bar in the middle (Latest, Branches, History, System with icons; the active tab in panel-2 with an accent underline), Impressum and Privacy right.
### The Repository Strip
The `<select>` switcher is replaced by a strip below the header that shows every served repository with its state, so a failure elsewhere is visible without leaving the page.
- Up to about three repositories: **tiles** (220 px), each with `current` or `repo` label, the name, one dot per branch in the branch's latest status, a summary line (`6 builds · 1 failed · 1 running`), and the watcher warning in clay when that repository's watcher reports an error.
- More repositories: **chips** (30 px), each with one dot for the worst status in the repository, the name, an optional short finding (`1 failed`, `main`, `never built`), and a warning triangle when the watcher reports an error; the current repository has an accent-2 border on panel-2, a failing one a clay-2 border on the clay tint.
- Order is *failing first*: the current repository, then failing, running, then green; a summary line above (`10 served · 2 failing · 1 unreachable · 2 running`) and a sort control on the right.
- The strip **scrolls**: horizontally on the phone, and on the desktop it wraps to a second row up to about ten repositories and becomes a horizontally scrollable band beyond that, with the failing chips pinned at the front so they never scroll out of view.
- Beyond roughly twenty repositories the strip shows only the conspicuous chips (failing, running, unreachable) plus a search field for the rest.
- With a single served repository the strip is omitted, as the switcher is today.
### Phone (below 680 px)
The existing breakpoint behaviour is kept and restyled: rows become cards with the `data-label` captions, the live indicator collapses to a dot.
The header stacks `Werkator` over the repository name; the repository strip scrolls horizontally under its summary line; the panel title keeps its hierarchy; each card carries the status line with the branch, then commit, started and duration, then the artifact icons and the actions as 44 px targets.
The footer tab bar becomes a fixed bottom tab bar with icons — the same four entries as on the desktop.
No painted status bar or keyboard; the device provides those.
### What Is Deliberately Not in the Proposal
- The `DE` language button in the mockups is a leftover of the reference; the UI stays English-only.
- No commit subjects, typical durations, ETAs or history strips: they need data the server does not have, and each would be its own RFC with its own storage.
- No manual theme toggle; `prefers-color-scheme` decides, as today.
## Consequences
### Backend
- One new endpoint, `GET /api/repos`: for every served repository its name, its UI root (`/repos/<name>`), whether it is the current one, the latest status per build name (the Latest view's `latestPerName` reduced to counts, plus the worst status), and its `RepoWatcherState` (`lastFetchError`, `lastPollError`, `lastPollAt`).
With one served repository the endpoint returns a list of one and the strip stays hidden.
- `werkator.js` polls it on the table interval (10 s) and renders the strip; every fetch keeps the timeout and the explicit error badge.
- `UiFormats` and `werkator.js` keep producing identical formats; the palette and the title labels are template and CSS only.
### Rollout, One Concern per Pull Request
1. Palette, typography and the title hierarchy in `werkator.css` and the fragments — no data change, both modes.
2. Header and footer tab bar, including the phone tab bar.
3. `GET /api/repos` and the repository strip, replacing the `<select>`.
4. Card refinements on the phone and the System and artifact pages in the new vocabulary.
Each step leaves the UI usable, and the tests in `server` that assert on markup are adjusted with the step that changes it.
## Open Questions
- **Fonts:** bundle IBM Plex Sans and Mono under `static/fonts/` (about 100150 KB in WOFF2 for the four faces), or accept the fallback stack on hosts without the font; the reference sites load Plex from a CDN, which the deployment behind a strict reverse proxy may not want.
- **Current view:** it is reachable today only from a running row's live icon; the tab bar has room for it as a fifth entry with a count badge, or it stays a link from the row.
- **Instance pages:** `/system` and `/releases` are instance-level; in the tab bar they sit next to the per-repository views, which the strip makes visible enough, or they move to the footer's right side.
## Design Sources
The design canvas with all eleven artboards (rounds one to three, desktop and phone) is a private Claude artifact of the author; its renderings live next to this RFC under `0001-web-ui-instrument-panel/`.
The sample rows are real field shapes with invented values; the repositories other than `werkator` are invented.
The proposal:
- [E · dark, desktop](0001-web-ui-instrument-panel/e-dark-desktop.png) · [E · light, desktop](0001-web-ui-instrument-panel/e-light-desktop.png)
- [E · dark, ten repositories](0001-web-ui-instrument-panel/e-dark-10-repos.png) · [E · light, ten repositories](0001-web-ui-instrument-panel/e-light-10-repos.png)
- [E · dark, phone](0001-web-ui-instrument-panel/e-dark-phone.png) · [E · light, phone](0001-web-ui-instrument-panel/e-light-phone.png)
The alternatives, for the record:
- [A · Quiet Console](0001-web-ui-instrument-panel/a-quiet-console.png), [B · Mission Board](0001-web-ui-instrument-panel/b-mission-board.png), [C · Ledger](0001-web-ui-instrument-panel/c-ledger.png) — round one, still with invented data.
- [D · Paper Rail](0001-web-ui-instrument-panel/d-paper-rail.png), [F · Fleet Overview](0001-web-ui-instrument-panel/f-fleet-overview.png) — round two.
Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 171 KiB

@@ -119,7 +119,7 @@ class BuildExecutor(
artifactKey = runningBuild.artifactKey,
)
repo.results.append(pending)
eventPublisher.publishEvent(BuildStatusChangedEvent(pending))
eventPublisher.publishEvent(BuildStatusChangedEvent(pending, repo))
val activeBuild = ActiveBuild(runningBuild, repo)
builds[runningBuild.artifactKey] = activeBuild
publishGiteaStatus(activeBuild, BuildStatus.PENDING, duration = null)
@@ -384,7 +384,7 @@ class BuildExecutor(
duration = duration,
artifactKey = runningBuild.artifactKey,
).also { build.repo.results.append(it) }
eventPublisher.publishEvent(BuildStatusChangedEvent(updated))
eventPublisher.publishEvent(BuildStatusChangedEvent(updated, build.repo))
publishGiteaStatus(build, status, duration)
return updated
}
@@ -43,17 +43,17 @@ class ProcessBuildRunner : BuildRunner {
}
/**
* Selects the runtime per branch: Docker when `branches.<name>.docker.enabled`,
* bubblewrap when `branches.<name>.bwrap.enabled`, native shell execution otherwise
* (the unchanged default). Docker and bwrap are mutually exclusive per branch and are
* rejected together at config load, so the branch order here never has to "pick".
* Selects the runtime per branch: Docker when `builds.<name>.docker.enabled`, the
* werkdock sandbox when `builds.<name>.werkdock.enabled`, native shell execution
* otherwise (the unchanged default). Docker and werkdock are mutually exclusive per
* branch and are rejected together at config load, so the order here never has to "pick".
*/
@Primary
@Component
class DispatchingBuildRunner(
private val processBuildRunner: ProcessBuildRunner,
private val dockerBuildRunner: DockerBuildRunner,
private val bwrapBuildRunner: BwrapBuildRunner,
private val werkdockBuildRunner: WerkdockBuildRunner,
) : BuildRunner {
override fun start(
command: String,
@@ -66,7 +66,7 @@ class DispatchingBuildRunner(
val runner =
when {
branchConfig.docker.enabled -> dockerBuildRunner
branchConfig.bwrap.enabled -> bwrapBuildRunner
branchConfig.werkdock.enabled -> werkdockBuildRunner
else -> processBuildRunner
}
return runner.start(command, workingDir, environment, repoDir, branchConfig, onAuxProcess)
@@ -35,7 +35,12 @@ data class RunningBuild(
var runningSince: Instant? = null
}
/** Published via Spring's `ApplicationEventPublisher` on every persisted status transition. */
/**
* Published via Spring's `ApplicationEventPublisher` on every persisted status transition.
* Carries the repository because a [BuildResult] does not: a listener that reacts to the
* transition the follow-up trigger has to act on that repository.
*/
data class BuildStatusChangedEvent(
val result: BuildResult,
val repo: RepoContext,
)
@@ -1,7 +1,7 @@
package de.hoennig.werkator.build
import de.hoennig.werkator.config.BranchConfig
import de.hoennig.werkator.config.BwrapConfig
import de.hoennig.werkator.config.WerkdockConfig
import de.hoennig.werkator.git.GitCommandRunner
import org.slf4j.LoggerFactory
import org.springframework.stereotype.Component
@@ -13,7 +13,7 @@ import java.security.MessageDigest
* Runs build commands inside a bubblewrap user-namespace sandbox (Step 17 / ADR 0008),
* for hosts without root and without a Docker daemon (e.g. Hostsharing managed
* webspaces). Since step 21 session C it no longer assembles the raw `bwrap` argv:
* it shells out to the `werkdock` CLI (`bwrap.werkdock`, default via PATH) the same
* it shells out to the `werkdock` CLI (`werkdock.binary`, default via PATH) the same
* pattern as git and docker, CLI, no library.
*
* The rootfs archive becomes a werkdock *image*, loaded once per source
@@ -36,10 +36,10 @@ import java.security.MessageDigest
* native builds.
*/
@Component
class BwrapBuildRunner(
class WerkdockBuildRunner(
private val commandRunner: GitCommandRunner,
) : BuildRunner {
private val log = LoggerFactory.getLogger(BwrapBuildRunner::class.java)
private val log = LoggerFactory.getLogger(WerkdockBuildRunner::class.java)
/** Replaceable process launcher so unit tests can capture the assembled `werkdock` argv. */
internal var processStarter: (List<String>, Path) -> Process = { command, dir ->
@@ -54,14 +54,14 @@ class BwrapBuildRunner(
branchConfig: BranchConfig,
onAuxProcess: (Process) -> Unit,
): Process {
val bwrap = branchConfig.bwrap
require(bwrap.rootfs.isNotBlank()) { "branches.<name>.bwrap.rootfs must be set when bwrap.enabled is true" }
val werkdock = bwrap.werkdock.ifBlank { "werkdock" }
val image = imageName(bwrap.rootfs)
ensureImage(werkdock, image, bwrap, repoDir, onAuxProcess)
val sandbox = branchConfig.werkdock
require(sandbox.rootfs.isNotBlank()) { "builds.<name>.werkdock.rootfs must be set when werkdock.enabled is true" }
val werkdock = sandbox.binary.ifBlank { "werkdock" }
val image = imageName(sandbox.rootfs)
ensureImage(werkdock, image, sandbox, repoDir, onAuxProcess)
val homeDir = repoDir.resolve(BUILDENV_DIR).resolve(HOME_DIR)
Files.createDirectories(homeDir)
val args = invocation(command, workingDir, environment, repoDir, bwrap, werkdock, image, homeDir)
val args = invocation(command, workingDir, environment, repoDir, sandbox, werkdock, image, homeDir)
return processStarter(args, repoDir)
}
@@ -73,7 +73,7 @@ class BwrapBuildRunner(
private fun ensureImage(
werkdock: String,
image: String,
bwrap: BwrapConfig,
sandbox: WerkdockConfig,
repoDir: Path,
onAuxProcess: (Process) -> Unit,
) {
@@ -81,10 +81,10 @@ class BwrapBuildRunner(
if (image in loaded) {
return
}
val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(bwrap.rootfs))
val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(sandbox.rootfs))
Files.createDirectories(envDir)
val archive = localArchive(bwrap.rootfs, envDir, repoDir, onAuxProcess)
log.info("loading build environment {} as werkdock image {}", bwrap.rootfs, image)
val archive = localArchive(sandbox.rootfs, envDir, repoDir, onAuxProcess)
log.info("loading build environment {} as werkdock image {}", sandbox.rootfs, image)
commandRunner.runOrThrow(
listOf(werkdock, "load", "-i", archive, "--name", image),
repoDir,
@@ -93,7 +93,7 @@ class BwrapBuildRunner(
}
/**
* Resolves [BwrapConfig.rootfs] to a local archive path: a bare or `file:` path is
* Resolves [WerkdockConfig.rootfs] to a local archive path: a bare or `file:` path is
* used as-is; an `http(s)` URL is downloaded once into the buildenv cache.
*/
private fun localArchive(
@@ -123,7 +123,7 @@ class BwrapBuildRunner(
workspace: Path,
environment: Map<String, String>,
repoDir: Path,
bwrap: BwrapConfig,
sandbox: WerkdockConfig,
werkdock: String,
image: String,
homeDir: Path,
@@ -148,7 +148,7 @@ class BwrapBuildRunner(
for ((key, value) in environment) {
args += listOf("-e", "$key=$value")
}
for ((key, value) in bwrap.env) {
for ((key, value) in sandbox.env) {
args += listOf("-e", "$key=$value")
}
args += listOf("-w", "$workspaceAbs")
@@ -1,9 +1,11 @@
package de.hoennig.werkator.commands
import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.config.BuildDefinition
import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext
import de.hoennig.werkator.repo.RepoRegistry
import de.hoennig.werkator.watcher.FollowUpTrigger
import org.springframework.stereotype.Component
import picocli.CommandLine.Command
import picocli.CommandLine.ExitCode
@@ -27,6 +29,7 @@ class BuildCommand(
private val gitService: GitService,
private val consoleBuildRunner: ConsoleBuildRunner,
private val registry: RepoRegistry,
private val followUpTrigger: FollowUpTrigger,
) : Callable<Int> {
@Mixin
var repoOption = RepoOption()
@@ -58,9 +61,33 @@ class BuildCommand(
}
println("building branch $branch at commit ${commit.take(12)}")
val status = consoleBuildRunner.buildAndStream(repo, branch, commit)
if (status == BuildStatus.SUCCESS) {
reportSkippedFollowUps(branch, commit)
}
return if (status == BuildStatus.SUCCESS) ExitCode.OK else ExitCode.SOFTWARE
}
/**
* A one-shot build ends with its process, so it never runs the follow-up builds the
* server would enqueue after a green run (PR#23) it says which ones instead of
* leaving the operator to wonder why nothing was deployed.
*/
private fun reportSkippedFollowUps(
branch: String,
commit: String,
) {
val followUps =
try {
followUpTrigger.followUpsOf(repo, branch, commit, BuildDefinition.DEFAULT)
} catch (e: Exception) {
System.err.println("warning: could not determine the follow-up builds (${e.message})")
return
}
if (followUps.isNotEmpty()) {
println("note: the server would now run the follow-up build(s) ${followUps.joinToString(", ")}; a CLI build does not")
}
}
/** A one-shot build should still work offline, from the last fetched origin state. */
private fun fetchBestEffort() {
try {
@@ -222,6 +222,7 @@ class InitCommand(
# branches: ["*", "!master"] # names or globs; "!" excludes; default: all
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
# activeWithin: 24h # only branches with recent commits
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
# run before each build
cleanCommand: rm -rf build
# shell command for each build
@@ -242,12 +243,13 @@ class InitCommand(
context: "." # Docker build context used with dockerfile
network: "" # Docker network mode for the build container; empty = Docker default (pinned)
env: {} # additional environment variables set inside the build container
# bubblewrap user-namespace sandbox for hosts without root and without a
# Docker daemon (e.g. Hostsharing managed webspaces). Mutually exclusive with docker.
bwrap:
enabled: false # run clean/build in a bwrap sandbox instead of natively (pinned)
# werkdock sandbox (bubblewrap user namespace) for hosts without root and
# without a Docker daemon (e.g. Hostsharing managed webspaces). Mutually
# exclusive with docker. Called bwrap before v1.2.0, still read under that name.
werkdock:
enabled: false # run clean/build in the sandbox instead of natively (pinned)
rootfs: "" # prepared rootfs archive (path or URL); required when enabled (pinned)
werkdock: werkdock # the werkdock CLI executing the sandbox; default resolves via PATH (pinned)
binary: werkdock # the werkdock CLI executing the sandbox; default resolves via PATH (pinned)
env: {} # additional environment variables set inside the sandbox
# Gitea check this build reports as; empty uses gitea.statusContext.
# Two builds of one commit under the same context overwrite each other.
@@ -42,8 +42,8 @@ data class BuildDefinition(
val statusContext: String? = null,
/** Overrides of the docker settings; null inherits them. */
val docker: DockerOverrides? = null,
/** Overrides of the bwrap settings; null inherits them. */
val bwrap: BwrapOverrides? = null,
/** Overrides of the werkdock settings; null inherits them. */
val werkdock: WerkdockOverrides? = null,
) {
/** The settings this build runs with: [branchConfig] with this definition applied; unset values fall through. */
fun applyTo(branchConfig: BranchConfig): BranchConfig =
@@ -64,12 +64,12 @@ data class BuildDefinition(
network = docker?.network ?: branchConfig.docker.network,
env = docker?.env ?: branchConfig.docker.env,
),
bwrap =
branchConfig.bwrap.copy(
enabled = bwrap?.enabled ?: branchConfig.bwrap.enabled,
rootfs = bwrap?.rootfs ?: branchConfig.bwrap.rootfs,
werkdock = bwrap?.werkdock ?: branchConfig.bwrap.werkdock,
env = bwrap?.env ?: branchConfig.bwrap.env,
werkdock =
branchConfig.werkdock.copy(
enabled = werkdock?.enabled ?: branchConfig.werkdock.enabled,
rootfs = werkdock?.rootfs ?: branchConfig.werkdock.rootfs,
binary = werkdock?.binary ?: branchConfig.werkdock.binary,
env = werkdock?.env ?: branchConfig.werkdock.env,
),
)
@@ -89,8 +89,9 @@ data class BuildDefinition(
* When a build runs and for which branches the `trigger` block of a build definition,
* and the one part of it that is never inherited from `builds.default`.
*
* A definition with neither [onPush] nor [atTimes] never triggers automatically; that is
* how `builds.default` is written when it is meant as a settings base only.
* A definition with neither [onPush] nor [atTimes] nor [afterSuccessOf] never triggers
* automatically; that is how `builds.default` is written when it is meant as a settings
* base only.
*/
data class TriggerConfig(
/** Build every new commit of the selected branches. */
@@ -112,7 +113,19 @@ data class TriggerConfig(
* empty applies no age filter. Combines with [branches] as an intersection.
*/
val activeWithin: String = "",
/**
* Name of another definition of this configuration the *predecessor*: this build
* runs on the predecessor's branch at the predecessor's commit whenever a run of it
* ends with `SUCCESS`, whatever started that run (PR#23). Empty means none. Sits in
* the trigger block because it says *when* this build runs, so it is never inherited;
* and it is host-pinned, because a follow-up build is the host's way to hand
* real-world effects deployment targets, credentials to a green commit.
*/
val afterSuccessOf: String = "",
) {
/** True when this build follows another one, see [afterSuccessOf]. */
fun isFollowUp(): Boolean = afterSuccessOf.isNotBlank()
/** True when [branch] matches the [branches] patterns (or none are configured) and none excludes it. */
fun selectsByName(branch: String): Boolean {
val (excluding, including) = branches.partition { it.startsWith(EXCLUDE_PREFIX) }
@@ -169,13 +182,13 @@ data class DockerOverrides(
val env: Map<String, String>? = null,
)
/** Nullable bubblewrap overrides of a [BuildDefinition]; null values inherit the branch's setting. */
data class BwrapOverrides(
/** Run the build in the bwrap sandbox instead of natively. Pinned — a branch must not escape its sandbox. */
/** Nullable werkdock overrides of a [BuildDefinition]; null values inherit the branch's setting. */
data class WerkdockOverrides(
/** Run the build in the sandbox instead of natively. Pinned — a branch must not escape its sandbox. */
val enabled: Boolean? = null,
/** Rootfs archive source. Pinned — a branch must not substitute a foreign rootfs. */
val rootfs: String? = null,
/** The werkdock CLI executing the sandbox. Pinned — a branch must not substitute the executing binary. */
val werkdock: String? = null,
val binary: String? = null,
val env: Map<String, String>? = null,
)
@@ -90,7 +90,8 @@ class ConfigLoader(
fun loadForWorktree(
workingDir: Path,
worktreeDir: Path,
): WerkatorConfig = withBranchLayer(workingDir, loadFile(worktreeDir.resolve(ConfigFiles.firstExisting(worktreeDir)).toFile()))
branch: String? = null,
): WerkatorConfig = withBranchLayer(workingDir, loadFile(worktreeDir.resolve(ConfigFiles.firstExisting(worktreeDir)).toFile()), branch)
/**
* The primary/`.git` config with the committed `.werkator.yml` of one branch
@@ -107,30 +108,53 @@ class ConfigLoader(
* (`requirePullRequest`, which decides whether the branch is built at all).
* They are stripped from the branch layer before merging, so a branch can neither
* escape its container, nor bypass its own pull-request gate, nor raise the global
* concurrency, nor reach the credentials.
* concurrency, nor reach the credentials. The trigger of a follow-up build is pinned
* the same way (PR#23): a branch may say what its deployment does, never that or
* where it happens. [branch] only names the branch in the warnings.
*/
fun loadWithBranchLayer(
workingDir: Path,
branchConfigYaml: String?,
): WerkatorConfig = withBranchLayer(workingDir, parseYaml(branchConfigYaml))
branch: String? = null,
): WerkatorConfig = withBranchLayer(workingDir, parseYaml(branchConfigYaml), branch)
private fun withBranchLayer(
workingDir: Path,
branchLayer: Map<String, Any?>,
branch: String?,
): WerkatorConfig {
// scoped to this branch: an incompatible branch config fails its own builds and
// must never stop the server or hold up the branches that are fine
checkVersion(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
checkTriggerBlocks(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
return toConfig(deepMerge(loadRaw(workingDir), stripPinned(branchLayer)))
val primary = loadRaw(workingDir)
return toConfig(deepMerge(primary, stripPinned(branchLayer, primary, branch)), MissingPredecessor.WARN)
}
private fun toConfig(raw: Map<String, Any?>): WerkatorConfig {
/**
* What a follow-up whose predecessor no definition has means for this load, see
* [checkFollowUps].
*/
private enum class MissingPredecessor {
/** The primary configuration: a deployment that could never fire refuses the start. */
REFUSE,
/** A branch layer on top: the branch renamed or dropped the build the host's trigger names, and only loses its follow-up. */
WARN,
/** A fragment checked on its own: the predecessor may well live in the project config it is merged with later. */
SKIP,
}
private fun toConfig(
raw: Map<String, Any?>,
missingPredecessor: MissingPredecessor = MissingPredecessor.REFUSE,
): WerkatorConfig {
val config =
if (raw.isEmpty()) {
WerkatorConfig()
} else {
yaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw)), WerkatorConfig::class.java)
yaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw), missingPredecessor), WerkatorConfig::class.java)
}
return defaultPublicBaseUrl(config)
}
@@ -178,7 +202,11 @@ class ConfigLoader(
* second as soon as the committed configuration carries them.
*/
@Suppress("UNCHECKED_CAST")
private fun stripPinned(branchLayer: Map<String, Any?>): Map<String, Any?> {
private fun stripPinned(
branchLayer: Map<String, Any?>,
primary: Map<String, Any?>,
branch: String?,
): Map<String, Any?> {
if (branchLayer.isEmpty()) {
return branchLayer
}
@@ -188,9 +216,55 @@ class ConfigLoader(
val entries = result[section] as? Map<String, Any?> ?: continue
result[section] = entries.mapValues { (_, value) -> stripPinnedSettings(value) }
}
(result["builds"] as? Map<String, Any?>)?.let { builds ->
val hostBuilds = primary["builds"] as? Map<String, Any?> ?: emptyMap()
result["builds"] = builds.mapValues { (name, value) -> stripPinnedTrigger(name, value, hostBuilds[name], branch) }
}
return result
}
/**
* The follow-up part of the pinning (PR#23): a branch's definition loses its
* `afterSuccessOf`, and where the host's definition of the same name is a follow-up,
* the branch's whole `trigger` block otherwise a branch could widen the host's
* selector to include itself, and deploy itself with the host's credentials. Said
* out loud, because a trigger the branch wrote and does not see in effect is a
* question it would otherwise ask the log in vain.
*/
@Suppress("UNCHECKED_CAST")
private fun stripPinnedTrigger(
name: String,
value: Any?,
hostDefinition: Any?,
branch: String?,
): Any? {
val definition = value as? Map<String, Any?> ?: return value
val trigger = definition["trigger"] as? Map<String, Any?> ?: return value
val where = branch?.let { "branch '$it'" } ?: "this branch"
if (predecessorOf(hostDefinition) != null) {
log.warn(
"ignoring the trigger block of builds.{} in the committed {} of {}: the host defines that build as a follow-up, " +
"and when and where a follow-up runs is the host's decision alone",
name,
ConfigFiles.COMMITTED,
where,
)
return definition - "trigger"
}
if (predecessorOf(definition) == null) {
return value
}
log.warn(
"ignoring builds.{}.trigger.afterSuccessOf in the committed {} of {}: a branch cannot make a build follow another, " +
"only the host can",
name,
ConfigFiles.COMMITTED,
where,
)
val stripped = trigger - "afterSuccessOf"
return if (stripped.isEmpty()) definition - "trigger" else definition + ("trigger" to stripped)
}
@Suppress("UNCHECKED_CAST")
private fun stripPinnedSettings(value: Any?): Any? {
val entry = value as? Map<String, Any?> ?: return value
@@ -201,10 +275,10 @@ class ConfigLoader(
val strippedDocker = docker.toMutableMap().apply { PINNED_DOCKER_KEYS.forEach { remove(it) } }
if (strippedDocker.isEmpty()) result.remove("docker") else result["docker"] = strippedDocker
}
val bwrap = entry["bwrap"] as? Map<String, Any?>
if (bwrap != null) {
val strippedBwrap = bwrap.toMutableMap().apply { PINNED_BWRAP_KEYS.forEach { remove(it) } }
if (strippedBwrap.isEmpty()) result.remove("bwrap") else result["bwrap"] = strippedBwrap
val werkdock = entry["werkdock"] as? Map<String, Any?>
if (werkdock != null) {
val strippedWerkdock = werkdock.toMutableMap().apply { PINNED_WERKDOCK_KEYS.forEach { remove(it) } }
if (strippedWerkdock.isEmpty()) result.remove("werkdock") else result["werkdock"] = strippedWerkdock
}
return result
}
@@ -225,12 +299,16 @@ class ConfigLoader(
* an empty docker policy and run natively on the host, which is exactly the escape
* the pinned keys exist to prevent.
*/
private fun resolveBuildSections(raw: Map<String, Any?>): Map<String, Any?> {
private fun resolveBuildSections(
raw: Map<String, Any?>,
missingPredecessor: MissingPredecessor,
): Map<String, Any?> {
@Suppress("UNCHECKED_CAST")
val definitions = raw["builds"] as? Map<String, Any?> ?: emptyMap()
if (definitions.isEmpty()) {
return mergeBranchDefaults(raw)
}
checkFollowUps(definitions, missingPredecessor)
if (raw.containsKey("branches") && warnedSections.add(LEGACY_BRANCHES_WARNING)) {
log.warn(
"ignoring the branches section: this configuration defines builds, and a build definition " +
@@ -252,13 +330,59 @@ class ConfigLoader(
return
}
if (warnedSections.add(NO_TRIGGER_WARNING)) {
log.warn("no build defines onPush or atTimes; the watcher will never start a build on its own")
log.warn("no build defines onPush, atTimes, or afterSuccessOf; the watcher will never start a build on its own")
}
}
private fun isTriggered(definition: Any?): Boolean {
val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return false
return trigger["onPush"] == true || (trigger["atTimes"] as? List<*>)?.isNotEmpty() == true
return trigger["onPush"] == true ||
(trigger["atTimes"] as? List<*>)?.isNotEmpty() == true ||
predecessorOf(definition) != null
}
private fun predecessorOf(definition: Any?): String? {
val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return null
return (trigger["afterSuccessOf"] as? String)?.takeIf { it.isNotBlank() }
}
/**
* A follow-up build that could never fire must not exist, for the same reason a flat
* trigger key is refused: a deployment that silently never runs is worse than a
* configuration that refuses to load (PR#23). Refused are a predecessor no definition
* has, and a cycle of follow-ups (a build following itself included) a cycle is a
* configuration error whoever wrote it, while a missing predecessor depends on what
* is being loaded ([MissingPredecessor]).
*/
private fun checkFollowUps(
definitions: Map<String, Any?>,
missingPredecessor: MissingPredecessor,
) {
val predecessors = definitions.mapValues { (_, definition) -> predecessorOf(definition) }
val effective = definitions.keys + BuildDefinition.DEFAULT
for ((name, predecessor) in predecessors) {
if (predecessor == null || predecessor in effective) continue
val message = "builds.$name follows '$predecessor' (trigger.afterSuccessOf), but no build of that name is defined"
when (missingPredecessor) {
MissingPredecessor.REFUSE -> throw ConfigFormatException("$message. Name an existing build, or remove the follow-up.")
MissingPredecessor.WARN -> log.warn("$message on this branch; the follow-up will not run for it")
MissingPredecessor.SKIP -> {}
}
}
for (start in predecessors.keys) {
val path = mutableListOf(start)
var current = predecessors[start]
while (current != null && current !in path) {
path += current
current = predecessors[current]
}
if (current == start) {
throw ConfigFormatException(
"builds.$start follows itself through trigger.afterSuccessOf (${path.joinToString(" -> ")} -> $start); " +
"a follow-up build cannot wait for its own success.",
)
}
}
}
/**
@@ -426,7 +550,10 @@ class ConfigLoader(
checkVersion(raw, fragment.toString(), ROLLBACK_HINT)
checkTriggerBlocks(raw, fragment.toString(), ROLLBACK_HINT)
try {
strictYaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw)), WerkatorConfig::class.java)
strictYaml.convertValue(
resolveBuildSections(dropNonDefinitionBuilds(raw), MissingPredecessor.SKIP),
WerkatorConfig::class.java,
)
} catch (e: IllegalArgumentException) {
throw IllegalArgumentException(
"instance fragment $fragment does not match the configuration schema: ${e.message}",
@@ -481,14 +608,62 @@ class ConfigLoader(
private fun loadFile(file: File): Map<String, Any?> {
if (!file.exists()) return emptyMap()
@Suppress("UNCHECKED_CAST")
return yaml.readValue(file, Map::class.java) as Map<String, Any?>
return renameLegacySandbox(yaml.readValue(file, Map::class.java) as Map<String, Any?>, file.toString())
}
/** Parses a `.werkator.yml` read from git (not from disk); blank or null yields no layer. */
private fun parseYaml(text: String?): Map<String, Any?> {
if (text.isNullOrBlank()) return emptyMap()
@Suppress("UNCHECKED_CAST")
return yaml.readValue(text, Map::class.java) as? Map<String, Any?> ?: emptyMap()
val raw = yaml.readValue(text, Map::class.java) as? Map<String, Any?> ?: emptyMap()
return renameLegacySandbox(raw, "the branch configuration")
}
/**
* Reads the pre-PR#19 `bwrap` section under its new name `werkdock`, including its
* `werkdock` key which is `binary` now. Done on the raw map of every layer, before
* merging, so nothing downstream merging, pinning, binding knows two names.
*
* Renaming rather than rejecting: the section is written in the machine configuration
* of every webspace instance, which no repository tracks. The warning is what makes
* the old name go away; the hard refusal belongs to the release that sets
* [ConfigVersions.FORMAT_BROKE_IN], where a file declaring no version can be caught
* by name at all.
*/
@Suppress("UNCHECKED_CAST")
private fun renameLegacySandbox(
raw: Map<String, Any?>,
source: String,
): Map<String, Any?> {
var renamed = false
val result =
raw.mapValues { (section, value) ->
if (section != "builds" && section != "branches") {
return@mapValues value
}
val entries = value as? Map<String, Any?> ?: return@mapValues value
entries.mapValues inner@{ (_, entry) ->
val settings = entry as? Map<String, Any?> ?: return@inner entry
val legacy = settings["bwrap"] as? Map<String, Any?> ?: return@inner entry
renamed = true
val moved =
legacy.mapKeys { (key, _) -> if (key == "werkdock") "binary" else key }
val existing = settings["werkdock"] as? Map<String, Any?> ?: emptyMap()
settings.toMutableMap().apply {
remove("bwrap")
// an explicit werkdock section wins: the new name is the one meant
put("werkdock", moved + existing)
}
}
}
if (renamed && warnedSections.add("bwrap-renamed:$source")) {
log.warn(
"reading the 'bwrap' section of {} as 'werkdock' (and 'bwrap.werkdock' as 'werkdock.binary'); " +
"rename it — the old name goes away with the next breaking configuration change",
source,
)
}
return result
}
@Suppress("UNCHECKED_CAST")
@@ -550,8 +725,8 @@ class ConfigLoader(
/** `docker` keys a branch must never override: the sandbox policy. */
private val PINNED_DOCKER_KEYS = setOf("enabled", "network")
/** `bwrap` keys a branch must never override: the sandbox policy (Step 17) and its executing binary. */
private val PINNED_BWRAP_KEYS = setOf("enabled", "rootfs", "werkdock")
/** `werkdock` keys a branch must never override: the sandbox policy (Step 17) and its executing binary. */
private val PINNED_WERKDOCK_KEYS = setOf("enabled", "rootfs", "binary")
/**
* The one key of a build definition that says *when* and *for which branches* it
@@ -562,7 +737,7 @@ class ConfigLoader(
private val TRIGGER_KEYS = setOf("trigger")
/** The keys that moved into [TRIGGER_KEYS]; still writing them flat is refused, not ignored. */
private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin")
private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin", "afterSuccessOf")
/**
* Top-level sections owned by the instance once a home config exists (ADR 0009):
@@ -38,9 +38,9 @@ data class WerkatorConfig(
): BranchConfig {
val branchConfig = branches[branch] ?: branches["default"] ?: BranchConfig()
val settings = effectiveBuildDefinitions()[build]?.applyTo(branchConfig) ?: branchConfig
if (settings.docker.enabled && settings.bwrap.enabled) {
if (settings.docker.enabled && settings.werkdock.enabled) {
throw IllegalArgumentException(
"builds.$build on '$branch' enables both docker and bwrap; a build runs in exactly one sandbox. " +
"builds.$build on '$branch' enables both docker and werkdock; a build runs in exactly one sandbox. " +
"Disable one of them.",
)
}
@@ -179,17 +179,22 @@ data class BranchConfig(
val statusContext: String = "",
val autoBuild: AutoBuildConfig = AutoBuildConfig(),
val docker: DockerConfig = DockerConfig(),
/** bubblewrap user-namespace sandbox; mutually exclusive with [docker]. */
val bwrap: BwrapConfig = BwrapConfig(),
/** werkdock sandbox (bubblewrap user namespace); mutually exclusive with [docker]. */
val werkdock: WerkdockConfig = WerkdockConfig(),
)
/**
* bubblewrap build sandbox (Step 17): runs the build in an unprivileged user namespace
* with a prepared Debian root filesystem. For hosts without root and without a Docker
* daemon (e.g. Hostsharing managed webspaces); see `docs/plan/17-bwrap-build-runtime.md`.
* The werkdock build sandbox (Step 17, executed by the werkdock CLI since step 21):
* runs the build in an unprivileged bubblewrap user namespace over a prepared Debian
* root filesystem. For hosts without root and without a Docker daemon (e.g. Hostsharing
* managed webspaces); see `docs/plan/17-bwrap-build-runtime.md`.
*
* The section was called `bwrap` until PR#19 and is still read under that name, with a
* warning: `bwrap` named the mechanism one layer below the tool that actually runs it,
* which made `bwrap.werkdock` the key naming its own executor.
*/
data class BwrapConfig(
/** Run the clean and build commands in a bwrap sandbox instead of natively. */
data class WerkdockConfig(
/** Run the clean and build commands in the sandbox instead of natively. */
val enabled: Boolean = false,
/**
* Path or URL of the prepared rootfs archive (e.g. `werkator-buildenv-trixie-java21.tar.zst`),
@@ -200,8 +205,9 @@ data class BwrapConfig(
/**
* The werkdock CLI executing the sandbox (step 21 session C); empty or the default
* resolves via PATH. Pinned a branch must not substitute the executing binary.
* Was `bwrap.werkdock` until PR#19.
*/
val werkdock: String = "werkdock",
val binary: String = "werkdock",
/** Additional environment variables set inside the sandbox. */
val env: Map<String, String> = emptyMap(),
)
@@ -0,0 +1,134 @@
package de.hoennig.werkator.watcher
import de.hoennig.werkator.build.BuildExecutor
import de.hoennig.werkator.build.BuildResult
import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.build.BuildStatusChangedEvent
import de.hoennig.werkator.config.BuildDefinition
import de.hoennig.werkator.config.ConfigFiles
import de.hoennig.werkator.config.ConfigLoader
import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext
import org.slf4j.LoggerFactory
import org.springframework.context.event.EventListener
import org.springframework.stereotype.Component
import java.time.Clock
import java.util.concurrent.atomic.AtomicBoolean
/**
* Runs the follow-up builds (PR#23): whenever a build ends with `SUCCESS`, every
* definition of that branch whose `trigger.afterSuccessOf` names the finished build
* and whose selector selects the branch is enqueued on the same branch at the *same
* commit*, never at the branch's current origin head, so a deployment always ships the
* commit that was tested. Every green run counts, whatever started it, and a repeated
* green run of the same commit triggers again: a run of a green build that is silently
* not deployed would be more confusing than a redundant deployment.
*
* The definitions are resolved with the branch's committed config at the finished
* build's commit the same layering the watcher applies so the follow-up's command
* comes with the repository while its trigger stays the host's (pinned by
* [ConfigLoader]). The pull-request gate is not consulted: the predecessor passed it for
* this very commit, and the host's selector is the follow-up's own gate.
*
* Armed by [Watcher.start] and disarmed by [Watcher.stop], so a CLI `build` whose
* process ends with its build never enqueues a follow-up into a JVM that is about to
* exit; the CLI names the follow-ups the server would have run instead.
*/
@Component
class FollowUpTrigger(
private val gitService: GitService,
private val configLoader: ConfigLoader,
private val buildExecutor: BuildExecutor,
private val clock: Clock,
) {
private val log = LoggerFactory.getLogger(FollowUpTrigger::class.java)
private val armed = AtomicBoolean(false)
fun arm() {
armed.set(true)
}
fun disarm() {
armed.set(false)
}
fun isArmed(): Boolean = armed.get()
@EventListener
fun onBuildStatusChanged(event: BuildStatusChangedEvent) {
if (!armed.get() || event.result.status != BuildStatus.SUCCESS) {
return
}
val result = event.result
try {
for (name in followUpsOf(event.repo, result)) {
log.info(
"[{}] enqueueing follow-up build {} of branch {} at commit {}, after {}",
event.repo.name,
name,
result.branch,
result.commit,
result.build,
)
buildExecutor.startBuild(event.repo, result.branch, result.commit, name)
}
} catch (e: Exception) {
log.error("[{}] could not enqueue the follow-ups of {} at {}", event.repo.name, result.name, result.commit, e)
}
}
/** The names of the builds that follow a green [result], in the order of their definitions; nothing is enqueued. */
fun followUpsOf(
repo: RepoContext,
result: BuildResult,
): List<String> = followUpsOf(repo, result.branch, result.commit, result.build)
/** The names of the builds that follow a green run of [build] on [branch] at [commit]; nothing is enqueued. */
fun followUpsOf(
repo: RepoContext,
branch: String,
commit: String,
build: String,
): List<String> {
val workingDir = repo.workingDir
val definitions = definitionsAt(repo, branch, commit)
val headCommittedAt = lazy { gitService.originBranchCommitTimes(workingDir)[branch] }
return definitions
.filter { (_, definition) -> definition.trigger.afterSuccessOf == build }
.filter { (_, definition) -> definition.trigger.selects(branch, { headCommittedAt.value }, clock.instant()) }
.keys
.toList()
}
/**
* The branch's definitions at [commit] not at its head, which may have moved on
* since the predecessor started. An unreadable branch config falls back to the
* primary definitions, like the watcher does.
*/
private fun definitionsAt(
repo: RepoContext,
branch: String,
commit: String,
): Map<String, BuildDefinition> {
val workingDir = repo.workingDir
return try {
configLoader
.loadWithBranchLayer(
workingDir,
ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) },
branch,
).effectiveBuildDefinitions()
} catch (e: Exception) {
log.warn(
"[{}] ignoring the committed {} of branch {} at {} for its follow-ups: {}",
repo.name,
ConfigFiles.COMMITTED,
branch,
commit,
e.message ?: e.javaClass.simpleName,
)
configLoader.load(workingDir).effectiveBuildDefinitions()
}
}
}
@@ -41,6 +41,7 @@ class Watcher(
private val buildExecutor: BuildExecutor,
private val configLoader: ConfigLoader,
private val clock: Clock,
private val followUpTrigger: FollowUpTrigger,
) {
private val log = LoggerFactory.getLogger(Watcher::class.java)
@@ -83,11 +84,14 @@ class Watcher(
* Runs the startup recovery of every repository and schedules the poll loop with the
* fixed delay `watcher.pollInterval` one loop, one delay: the instance's setting,
* which every repository's effective config carries; the first poll runs immediately.
* Arms the [FollowUpTrigger] first, so the recovery's re-enqueued builds get their
* follow-ups too.
*/
@Synchronized
fun start(repos: List<RepoContext>) {
check(scheduler == null) { "watcher is already running" }
require(repos.isNotEmpty()) { "no repository to watch" }
followUpTrigger.arm()
repos.forEach { recoverSafely(it) }
val interval = DurationParser.parse(configLoader.load(repos.first().workingDir).watcher.pollInterval)
scheduler =
@@ -111,6 +115,7 @@ class Watcher(
@Synchronized
fun stop() {
followUpTrigger.disarm()
scheduler?.shutdownNow()
scheduler = null
state = state.copy(running = false)
@@ -324,6 +329,7 @@ class Watcher(
.loadWithBranchLayer(
workingDir,
ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) },
branch,
).effectiveBuildDefinitions()
} catch (e: Exception) {
log.warn(
@@ -7,6 +7,26 @@
<div th:replace="~{fragments :: nav(${view})}"></div>
<div class="panel release-notes">
<h2>v1.2.0 <span class="muted">— 2026-09-03</span></h2>
<ul>
<li>The build sandbox for hosts without Docker is configured as <code>werkdock</code> now,
not <code>bwrap</code> (PR#19), and its <code>bwrap.werkdock</code> key — which named its
own executor — is <code>werkdock.binary</code>. The old section is still read, with a
warning naming the file, so no installation has to be changed before its next
configuration edit. <code>bwrap</code> named the mechanism one layer below the tool that
actually runs it: builds have been executed by the werkdock CLI since v1.0.0.</li>
</ul>
<h2>v1.1.2 <span class="muted">— 2026-09-03</span></h2>
<ul>
<li>On a Hostsharing Managed Webspace, an <code>instance-update</code> restart no longer looks
dead while nothing is listening on the port for a moment: the generated
<code>.htaccess</code> (PR#17) now maps a refused connection to a static
"Werkator is restarting — please retry in a few minutes" page instead of Apache's default
error page. Generated only where a <code>server.publicBaseUrl</code> is configured, next to
the existing <code>.htaccess</code>; nothing changes for a Docker-host deployment.</li>
</ul>
<h2>v1.1.1 <span class="muted">— 2026-09-03</span></h2>
<ul>
<li>The system page's disk metric is now quota-aware (PR#16): it shows the tightest of the
@@ -292,6 +292,42 @@ class BuildExecutorTest : FunSpec() {
.build shouldBe "default"
}
test("a follow-up build runs in its branch's worktree after its predecessor") {
val h =
Harness(
"""
executor:
maxConcurrent: 2
builds:
default:
trigger:
onPush: true
cleanCommand: ""
buildCommand: "sleep 1; echo built > output.txt"
deploy:
trigger:
afterSuccessOf: default
buildCommand: "cat output.txt"
""".trimIndent(),
)
h.executor.startBuild(h.repo, "main", "c1", "default")
h.executor.startBuild(h.repo, "main", "c1", "deploy")
awaitStatus(h, "main@deploy", BuildStatus.SUCCESS)
awaitIdle(h)
// same branch, same commit: the same worktree, and the follow-up saw the predecessor's output
h.workspaceCalls shouldContainExactly listOf("main" to "c1", "main" to "c1")
val predecessor = h.repository.latestFor("main").shouldNotBeNull()
val followUp = h.repository.latestFor("main@deploy").shouldNotBeNull()
predecessor.status shouldBe BuildStatus.SUCCESS
followUp.runningSince
.shouldNotBeNull()
.isBefore(predecessor.runningSince.shouldNotBeNull())
.shouldBeFalse()
Files.readString(h.workingDir.resolve("output.txt")).trim() shouldBe "built"
}
test("a build whose definition was removed from the config falls back to the branch's settings") {
val h = harness(buildCommand = "echo regular-\$branch")
@@ -505,6 +541,8 @@ class BuildExecutorTest : FunSpec() {
}
test("with maxConcurrent 1 a second branch stays PENDING until the first finished") {
// branch-a blocks on a gate file the test creates, so the PENDING assertion
// below cannot race the first build finishing on a loaded machine
val h =
Harness(
"""
@@ -512,7 +550,7 @@ class BuildExecutorTest : FunSpec() {
maxConcurrent: 1
branches:
branch-a:
buildCommand: "sleep 1"
buildCommand: "until [ -f gate ]; do sleep 0.05; done"
cleanCommand: ""
branch-b:
buildCommand: "echo ok"
@@ -523,8 +561,13 @@ class BuildExecutorTest : FunSpec() {
h.executor.startBuild(h.repo, "branch-a", "sha-a")
h.executor.startBuild(h.repo, "branch-b", "sha-b")
eventually(30.seconds) {
h.repository.latestFor("branch-a")?.status shouldBe BuildStatus.RUNNING
}
h.repository.latestFor("branch-b")?.status shouldBe BuildStatus.PENDING
Files.createFile(h.workingDir.resolve("gate"))
awaitStatus(h, "branch-b", BuildStatus.SUCCESS)
awaitStatus(h, "branch-a", BuildStatus.SUCCESS)
val transitions = h.events.map { it.result.branch to it.result.status }
@@ -1,8 +1,8 @@
package de.hoennig.werkator.build
import de.hoennig.werkator.config.BranchConfig
import de.hoennig.werkator.config.BwrapConfig
import de.hoennig.werkator.config.DockerConfig
import de.hoennig.werkator.config.WerkdockConfig
import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.shouldBe
import io.mockk.Called
@@ -15,13 +15,13 @@ import java.nio.file.Paths
class DispatchingBuildRunnerTest : FunSpec() {
private val processBuildRunner = mockk<ProcessBuildRunner>()
private val dockerBuildRunner = mockk<DockerBuildRunner>()
private val bwrapBuildRunner = mockk<BwrapBuildRunner>()
private val dispatcher = DispatchingBuildRunner(processBuildRunner, dockerBuildRunner, bwrapBuildRunner)
private val werkdockBuildRunner = mockk<WerkdockBuildRunner>()
private val dispatcher = DispatchingBuildRunner(processBuildRunner, dockerBuildRunner, werkdockBuildRunner)
private val process = mockk<Process>()
private val dir = Paths.get(".")
init {
beforeEach { clearMocks(processBuildRunner, dockerBuildRunner, bwrapBuildRunner) }
beforeEach { clearMocks(processBuildRunner, dockerBuildRunner, werkdockBuildRunner) }
test("runs natively by default") {
val branchConfig = BranchConfig()
@@ -30,7 +30,7 @@ class DispatchingBuildRunnerTest : FunSpec() {
dispatcher.start("cmd", dir, emptyMap(), dir, branchConfig) shouldBe process
verify { dockerBuildRunner wasNot Called }
verify { bwrapBuildRunner wasNot Called }
verify { werkdockBuildRunner wasNot Called }
}
test("runs in Docker when the branch enables it") {
@@ -40,12 +40,12 @@ class DispatchingBuildRunnerTest : FunSpec() {
dispatcher.start("cmd", dir, emptyMap(), dir, branchConfig) shouldBe process
verify { processBuildRunner wasNot Called }
verify { bwrapBuildRunner wasNot Called }
verify { werkdockBuildRunner wasNot Called }
}
test("runs in bwrap when the branch enables it (and not Docker)") {
val branchConfig = BranchConfig(bwrap = BwrapConfig(enabled = true, rootfs = "/srv/buildenv.tar.zst"))
every { bwrapBuildRunner.start("cmd", dir, emptyMap(), dir, branchConfig) } returns process
test("runs in the werkdock sandbox when the branch enables it (and not Docker)") {
val branchConfig = BranchConfig(werkdock = WerkdockConfig(enabled = true, rootfs = "/srv/buildenv.tar.zst"))
every { werkdockBuildRunner.start("cmd", dir, emptyMap(), dir, branchConfig) } returns process
dispatcher.start("cmd", dir, emptyMap(), dir, branchConfig) shouldBe process
@@ -1,7 +1,7 @@
package de.hoennig.werkator.build
import de.hoennig.werkator.config.BranchConfig
import de.hoennig.werkator.config.BwrapConfig
import de.hoennig.werkator.config.WerkdockConfig
import de.hoennig.werkator.git.GitCommandResult
import de.hoennig.werkator.git.GitCommandRunner
import io.kotest.assertions.throwables.shouldThrow
@@ -15,20 +15,20 @@ import io.mockk.verify
import java.nio.file.Files
import java.nio.file.Path
class BwrapBuildRunnerTest : FunSpec() {
class WerkdockBuildRunnerTest : FunSpec() {
private val commandRunner = mockk<GitCommandRunner>()
private lateinit var runner: BwrapBuildRunner
private lateinit var runner: WerkdockBuildRunner
private lateinit var repoDir: Path
private lateinit var workspace: Path
private val captured = mutableListOf<List<String>>()
private fun bwrapBranchConfig(
private fun werkdockBranchConfig(
rootfs: String = "/srv/buildenv.tar.zst",
env: Map<String, String> = emptyMap(),
): BranchConfig =
BranchConfig(
bwrap =
BwrapConfig(
werkdock =
WerkdockConfig(
enabled = true,
rootfs = rootfs,
env = env,
@@ -52,9 +52,9 @@ class BwrapBuildRunnerTest : FunSpec() {
beforeEach {
clearMocks(commandRunner)
captured.clear()
repoDir = Files.createTempDirectory("werkator-bwrap-runner")
repoDir = Files.createTempDirectory("werkator-werkdock-runner")
workspace = repoDir.resolve("workspace")
runner = BwrapBuildRunner(commandRunner)
runner = WerkdockBuildRunner(commandRunner)
runner.processStarter = { command, _ ->
captured += command
ProcessBuilder("true").start()
@@ -64,7 +64,7 @@ class BwrapBuildRunnerTest : FunSpec() {
test("assembles the exact werkdock run command for a loaded image") {
givenImageLoaded()
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig())
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
captured.single() shouldBe
listOf(
@@ -99,7 +99,7 @@ class BwrapBuildRunnerTest : FunSpec() {
)
} returns GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig())
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify {
commandRunner.runOrThrow(
@@ -114,7 +114,7 @@ class BwrapBuildRunnerTest : FunSpec() {
test("does not load an image werkdock already has") {
givenImageLoaded()
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig())
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
}
@@ -124,7 +124,7 @@ class BwrapBuildRunnerTest : FunSpec() {
GitCommandResult(0, imageName() + "\n", "")
val branchConfig =
BranchConfig(
bwrap = BwrapConfig(enabled = true, rootfs = "/srv/buildenv.tar.zst", werkdock = "/opt/bin/werkdock"),
werkdock = WerkdockConfig(enabled = true, rootfs = "/srv/buildenv.tar.zst", binary = "/opt/bin/werkdock"),
)
runner.start("./gradlew test", workspace, emptyMap(), repoDir, branchConfig)
@@ -136,7 +136,7 @@ class BwrapBuildRunnerTest : FunSpec() {
givenImageLoaded()
val relativeWorkspace = repoDir.relativize(workspace)
runner.start("./gradlew test", relativeWorkspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig())
runner.start("./gradlew test", relativeWorkspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
val args = captured.single()
val absolute = workspace.toAbsolutePath().normalize().toString()
@@ -145,7 +145,7 @@ class BwrapBuildRunnerTest : FunSpec() {
args.count { it == "$absolute:$absolute" } shouldBe 1
}
test("adds bwrap env after the branch environment") {
test("adds the sandbox env after the branch environment") {
givenImageLoaded()
runner.start(
@@ -153,7 +153,7 @@ class BwrapBuildRunnerTest : FunSpec() {
workspace,
mapOf("branch" to "main"),
repoDir,
bwrapBranchConfig(env = mapOf("FOO" to "bar")),
werkdockBranchConfig(env = mapOf("FOO" to "bar")),
)
val args = captured.single()
@@ -171,7 +171,7 @@ class BwrapBuildRunnerTest : FunSpec() {
Files.writeString(workspace.resolve(".git"), "gitdir: $adminDir\n")
givenImageLoaded()
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig())
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
val args = captured.single()
args[args.indexOf("$gitDir:$gitDir:ro") - 1] shouldBe "-v"
@@ -190,7 +190,7 @@ class BwrapBuildRunnerTest : FunSpec() {
givenImageLoaded()
Files.createDirectories(workspace)
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig())
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
val args = captured.single()
val gitDir = repoDir.resolve(".git")
@@ -199,21 +199,21 @@ class BwrapBuildRunnerTest : FunSpec() {
}
test("fails without a configured rootfs") {
val branchConfig = BranchConfig(bwrap = BwrapConfig(enabled = true))
val branchConfig = BranchConfig(werkdock = WerkdockConfig(enabled = true))
val exception =
shouldThrow<IllegalArgumentException> {
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, branchConfig)
}
exception.message shouldContain "bwrap.rootfs"
exception.message shouldContain "werkdock.rootfs"
}
test("downloads a URL rootfs once before loading it") {
val url = "https://example.test/buildenv.tar.zst"
val downloadTarget =
repoDir
.resolve(BwrapBuildRunner.BUILDENV_DIR)
.resolve(WerkdockBuildRunner.BUILDENV_DIR)
.resolve(url.sha12())
.resolve("buildenv.tar.zst")
givenImageMissing()
@@ -222,7 +222,7 @@ class BwrapBuildRunnerTest : FunSpec() {
every { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) } returns
GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig(rootfs = url))
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig(rootfs = url))
verify {
commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any())
@@ -4,6 +4,7 @@ import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext
import de.hoennig.werkator.repo.RepoRegistry
import de.hoennig.werkator.watcher.FollowUpTrigger
import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.shouldBe
import io.kotest.matchers.string.shouldContain
@@ -22,16 +23,31 @@ class BuildCommandTest : FunSpec() {
private val dir: Path = Paths.get(".")
private val repo = RepoContext("test", dir, mockk(), mockk())
private val registry = mockk<RepoRegistry>().also { every { it.current() } returns repo }
private val followUpTrigger = mockk<FollowUpTrigger>()
private fun command(fragment: String? = null) =
BuildCommand(gitService, consoleBuildRunner, registry).apply {
BuildCommand(gitService, consoleBuildRunner, registry, followUpTrigger).apply {
branchFragment = fragment
}
init {
beforeEach {
clearMocks(gitService, consoleBuildRunner)
clearMocks(gitService, consoleBuildRunner, followUpTrigger)
justRun { gitService.fetchOrigin(dir) }
every { followUpTrigger.followUpsOf(any(), any(), any(), any()) } returns emptyList()
}
test("a green CLI build names the follow-up builds the server would run, and runs none") {
every { gitService.currentBranch(dir) } returns "main"
every { gitService.localHeadCommit("main", dir) } returns "local-head"
every { gitService.hasNewCommits("main", dir) } returns false
every { consoleBuildRunner.buildAndStream(repo, "main", "local-head") } returns BuildStatus.SUCCESS
every { followUpTrigger.followUpsOf(repo, "main", "local-head", "default") } returns listOf("deploy")
val console = captureConsole { command().call() }
console.stdout.shouldContain("follow-up build(s) deploy")
verify(exactly = 1) { consoleBuildRunner.buildAndStream(repo, "main", "local-head") }
}
test("builds the current branch at its local head when no branch is given") {
@@ -496,7 +496,7 @@ class ConfigLoaderTest : FunSpec() {
settings.docker.image shouldBe "attacker-image"
}
test("a branch cannot disable its bwrap sandbox or substitute a foreign rootfs through a build definition") {
test("the legacy bwrap section is read as werkdock, its werkdock key as binary") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
@@ -505,6 +505,58 @@ class ConfigLoaderTest : FunSpec() {
bwrap:
enabled: true
rootfs: /host/rootfs.tar.zst
werkdock: /opt/bin/werkdock
env:
FOO: bar
""".trimIndent(),
)
val settings = loader.load(dir).buildSettings("any-branch", "default")
settings.werkdock.enabled shouldBe true
settings.werkdock.rootfs shouldBe "/host/rootfs.tar.zst"
settings.werkdock.binary shouldBe "/opt/bin/werkdock"
settings.werkdock.env shouldBe mapOf("FOO" to "bar")
}
test("a legacy bwrap section on a branch is pinned exactly like the new name") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
default:
werkdock:
enabled: true
rootfs: /host/rootfs.tar.zst
""".trimIndent(),
)
val worktree = Files.createTempDirectory("werkator-test-worktree")
// the old name must not become a way around the pinning
worktree.resolve(".werkator.yml").toFile().writeText(
"""
builds:
default:
bwrap:
enabled: false
rootfs: /attacker/rootfs.tar.zst
""".trimIndent(),
)
val settings = loader.loadForWorktree(dir, worktree).buildSettings("any-branch", "default")
settings.werkdock.enabled shouldBe true
settings.werkdock.rootfs shouldBe "/host/rootfs.tar.zst"
}
test("a branch cannot disable its werkdock sandbox or substitute a foreign rootfs through a build definition") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
default:
werkdock:
enabled: true
rootfs: /host/rootfs.tar.zst
""".trimIndent(),
)
val worktree = Files.createTempDirectory("werkator-test-worktree")
@@ -512,7 +564,7 @@ class ConfigLoaderTest : FunSpec() {
"""
builds:
default:
bwrap:
werkdock:
enabled: false
rootfs: /attacker/rootfs.tar.zst
env:
@@ -523,10 +575,10 @@ class ConfigLoaderTest : FunSpec() {
val settings = loader.loadForWorktree(dir, worktree).buildSettings("any-branch", "default")
// pinned: the sandbox can neither be switched off nor pointed at a foreign rootfs
settings.bwrap.enabled shouldBe true
settings.bwrap.rootfs shouldBe "/host/rootfs.tar.zst"
settings.werkdock.enabled shouldBe true
settings.werkdock.rootfs shouldBe "/host/rootfs.tar.zst"
// everything that describes the build itself stays the branch's own business
settings.bwrap.env shouldBe mapOf("FOO" to "from-branch")
settings.werkdock.env shouldBe mapOf("FOO" to "from-branch")
}
test("a build the branch invents inherits the host's sandbox policy") {
@@ -565,7 +617,7 @@ class ConfigLoaderTest : FunSpec() {
settings.requirePullRequest shouldBe true
}
test("enabling both docker and bwrap on a build is rejected, not picked silently") {
test("enabling both docker and werkdock on a build is rejected, not picked silently") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
@@ -574,7 +626,7 @@ class ConfigLoaderTest : FunSpec() {
docker:
enabled: true
image: build-env
bwrap:
werkdock:
enabled: true
rootfs: /srv/rootfs.tar.zst
""".trimIndent(),
@@ -586,7 +638,7 @@ class ConfigLoaderTest : FunSpec() {
config.buildSettings("any-branch", "default")
}
exception.message shouldContain "both docker and bwrap"
exception.message shouldContain "both docker and werkdock"
exception.message shouldContain "builds.default"
}
@@ -687,6 +739,190 @@ class ConfigLoaderTest : FunSpec() {
.shouldBeTrue()
}
test("afterSuccessOf must name an existing definition and must not form a cycle") {
val dir = Files.createTempDirectory("werkator-test")
val project = dir.resolve(".werkator.yml").toFile()
project.writeText(
"""
builds:
default:
trigger:
onPush: true
deploy:
trigger:
afterSuccessOf: default
branches: ["main"]
buildCommand: scripts/deploy.sh
""".trimIndent(),
)
// a follow-up with nothing but its predecessor is a triggered build, and it binds
val deploy = loader.load(dir).buildDefinitions.getValue("deploy")
deploy.trigger.afterSuccessOf shouldBe "default"
deploy.trigger.isFollowUp().shouldBeTrue()
deploy.trigger.onPush.shouldBeFalse()
project.writeText(
"""
builds:
deploy:
trigger:
afterSuccessOf: frontend
""".trimIndent(),
)
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.let {
it.shouldContain("builds.deploy")
it.shouldContain("frontend")
}
project.writeText(
"""
builds:
a:
trigger:
afterSuccessOf: b
b:
trigger:
afterSuccessOf: a
""".trimIndent(),
)
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.shouldContain("a -> b -> a")
project.writeText(
"""
builds:
a:
trigger:
afterSuccessOf: a
""".trimIndent(),
)
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.shouldContain("builds.a follows itself")
}
test("a branch whose layer lacks the predecessor loses only its follow-up") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
frontend:
trigger:
onPush: true
deploy:
trigger:
afterSuccessOf: frontend
""".trimIndent(),
)
// the branch renamed the predecessor: a warning, not a failed load — the branch's
// own builds must keep running, its follow-up simply never fires for it
val config =
loader.loadWithBranchLayer(
dir,
"""
builds:
frontend: null
ui:
trigger:
onPush: true
""".trimIndent(),
)
config.buildDefinitions
.getValue("ui")
.trigger.onPush
.shouldBeTrue()
config.buildDefinitions
.getValue("deploy")
.trigger.afterSuccessOf shouldBe "frontend"
}
test("afterSuccessOf written flat is refused like every trigger key") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
default:
trigger:
onPush: true
deploy:
afterSuccessOf: default
""".trimIndent(),
)
val thrown = shouldThrow<ConfigFormatException> { loader.load(dir) }
thrown.message.shouldContain("builds.deploy: afterSuccessOf")
thrown.message.shouldContain("trigger:")
}
test("a follow-up trigger is pinned to the host, a branch cannot add or widen one") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
"""
builds:
frontend:
trigger:
onPush: true
deploy:
trigger:
afterSuccessOf: frontend
branches: ["main"]
""".trimIndent(),
)
val config =
loader.loadWithBranchLayer(
dir,
"""
builds:
deploy:
trigger:
afterSuccessOf: frontend
branches: ["*"]
nightly:
trigger:
atTimes: ["01:00"]
afterSuccessOf: frontend
""".trimIndent(),
"feature/x",
)
// the host's trigger block of the follow-up is used unchanged
val deploy = config.buildDefinitions.getValue("deploy").trigger
deploy.afterSuccessOf shouldBe "frontend"
deploy.branches shouldBe listOf("main")
deploy.selectsByName("feature/x").shouldBeFalse()
// and a branch cannot make any build of its own a follow-up
val nightly = config.buildDefinitions.getValue("nightly").trigger
nightly.afterSuccessOf shouldBe ""
nightly.atTimes shouldBe listOf("01:00")
}
test("a branch supplies the command of a host-triggered follow-up") {
val dir = Files.createTempDirectory("werkator-test")
Files.createDirectories(dir.resolve(".git/werkator"))
dir.resolve(".git/werkator/.werkator.yml").toFile().writeText(
"""
builds:
deploy:
trigger:
afterSuccessOf: default
branches: ["main"]
werkdock:
env:
DEPLOY_TARGET: host:/srv/www
""".trimIndent(),
)
val worktree = Files.createTempDirectory("werkator-test-worktree")
worktree.resolve(".werkator.yml").toFile().writeText(
"""
builds:
deploy:
cleanCommand: ""
buildCommand: scripts/deploy-prod.sh -y "${'$'}DEPLOY_TARGET"
""".trimIndent(),
)
val settings = loader.loadForWorktree(dir, worktree, "main").buildSettings("main", "deploy")
settings.buildCommand shouldBe "scripts/deploy-prod.sh -y \"${'$'}DEPLOY_TARGET\""
settings.cleanCommand shouldBe ""
settings.werkdock.env shouldBe mapOf("DEPLOY_TARGET" to "host:/srv/www")
}
test("builds.default is the base of every other build, but never its trigger") {
val dir = Files.createTempDirectory("werkator-test")
dir.resolve(".werkator.yml").toFile().writeText(
@@ -113,7 +113,7 @@ class PermanentBranchRoutesTest : FunSpec() {
every { registry.byName(any()) } returns null
every { registry.byName("test") } returns repo
every { configLoader.load(any()) } returns WerkatorConfig()
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns WerkatorConfig()
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns WerkatorConfig()
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
every { controlTokens.token() } returns "test-token"
every { branchListing.branches(any()) } returns emptyList()
@@ -142,7 +142,7 @@ class UiControllerTest : FunSpec() {
server = ServerConfig(impressumUrl = "https://example.org/imprint"),
gitea = GiteaConfig(baseUrl = "https://git.example.org", owner = "acme", repo = "widget"),
)
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns WerkatorConfig()
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns WerkatorConfig()
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
every { controlTokens.token() } returns "test-token"
every { repository.latestGreenFor(any()) } returns null
@@ -385,7 +385,7 @@ class UiControllerTest : FunSpec() {
)
every { repository.history() } returns listOf(pitestResult)
every { artifactStore.artifactDir("main-pitest-key") } returns null
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns
WerkatorConfig(
branches = mapOf("default" to BranchConfig(buildCommand = "./gradlew quick-check")),
buildDefinitions = mapOf("pitest" to BuildDefinition(buildCommand = "./gradlew pitestFull")),
@@ -0,0 +1,173 @@
package de.hoennig.werkator.watcher
import de.hoennig.werkator.build.ArtifactKeys
import de.hoennig.werkator.build.BuildExecutor
import de.hoennig.werkator.build.BuildResult
import de.hoennig.werkator.build.BuildStatus
import de.hoennig.werkator.build.BuildStatusChangedEvent
import de.hoennig.werkator.build.RunningBuild
import de.hoennig.werkator.config.BuildDefinition
import de.hoennig.werkator.config.ConfigLoader
import de.hoennig.werkator.config.TriggerConfig
import de.hoennig.werkator.config.WerkatorConfig
import de.hoennig.werkator.git.GitService
import de.hoennig.werkator.repo.RepoContext
import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.collections.shouldBeEmpty
import io.kotest.matchers.collections.shouldContainExactly
import io.mockk.every
import io.mockk.mockk
import java.nio.file.Files
import java.time.Clock
import java.time.Instant
import java.time.ZoneOffset
import java.util.concurrent.CopyOnWriteArrayList
class FollowUpTriggerTest : FunSpec() {
private val noon = Instant.parse("2026-09-04T12:00:00Z")
/** Which build was started on which branch at which commit. */
private data class Started(
val branch: String,
val commit: String,
val build: String,
)
private inner class Harness(
config: WerkatorConfig,
) {
val workingDir = Files.createTempDirectory("werkator-followup-test")
val gitService = mockk<GitService>()
val configLoader = mockk<ConfigLoader>()
val buildExecutor = mockk<BuildExecutor>()
val repo = RepoContext("test", workingDir, mockk(), mockk())
val started = CopyOnWriteArrayList<Started>()
val trigger = FollowUpTrigger(gitService, configLoader, buildExecutor, Clock.fixed(noon, ZoneOffset.UTC))
init {
every { configLoader.load(any()) } returns config
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns config
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
// the branch moved on since the predecessor started
every { gitService.originHeadCommit(any(), any()) } returns "c2"
every { gitService.originBranchCommitTimes(any()) } returns mapOf("main" to noon.minusSeconds(60))
every { buildExecutor.startBuild(any(), any(), any(), any()) } answers {
val branch = secondArg<String>()
val commit = thirdArg<String>()
val build = arg<String>(3)
started += Started(branch, commit, build)
val staging = Files.createTempDirectory("werkator-followup-staging")
RunningBuild(
repo = repo,
branch = branch,
build = build,
commit = commit,
artifactKey = ArtifactKeys.buildKey(BuildDefinition.poolName(branch, build), noon),
startedAt = noon,
stagingDir = staging,
liveLogFile = staging.resolve("build.log"),
)
}
}
fun finished(
build: String,
status: BuildStatus,
branch: String = "main",
commit: String = "c1",
) {
val result =
BuildResult(
branch = branch,
build = build,
commit = commit,
status = status,
startedAt = noon,
artifactKey = ArtifactKeys.buildKey(BuildDefinition.poolName(branch, build), noon),
)
trigger.onBuildStatusChanged(BuildStatusChangedEvent(result, repo))
}
}
private fun deployAfter(
predecessor: String,
branches: List<String> = emptyList(),
): WerkatorConfig =
WerkatorConfig(
buildDefinitions =
mapOf(
"frontend" to BuildDefinition(trigger = TriggerConfig(onPush = true)),
"backend" to BuildDefinition(trigger = TriggerConfig(onPush = true)),
"deploy" to
BuildDefinition(
trigger = TriggerConfig(afterSuccessOf = predecessor, branches = branches),
buildCommand = "scripts/deploy.sh",
),
),
)
init {
test("a green predecessor enqueues the follow-up at the predecessor's commit") {
val h = Harness(deployAfter("frontend", branches = listOf("main")))
h.trigger.arm()
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
// c1, not the origin head c2 the branch has moved on to
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
}
test("every green run of the predecessor triggers the follow-up again") {
val h = Harness(deployAfter("frontend"))
h.trigger.arm()
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
h.started shouldContainExactly
listOf(
Started("main", "c1", "deploy"),
Started("main", "c1", "deploy"),
)
}
test("only a SUCCESS of the named predecessor triggers") {
val h = Harness(deployAfter("frontend", branches = listOf("main")))
h.trigger.arm()
h.finished("frontend", BuildStatus.FAILED)
h.finished("frontend", BuildStatus.CANCELLED)
h.finished("frontend", BuildStatus.INTERRUPTED)
h.finished("frontend", BuildStatus.PENDING)
h.finished("frontend", BuildStatus.RUNNING)
h.finished("backend", BuildStatus.SUCCESS)
// a branch the host's selector does not name never runs the follow-up
h.finished("frontend", BuildStatus.SUCCESS, branch = "feature/x")
h.started.shouldBeEmpty()
}
test("the trigger listens only while the watcher runs") {
val h = Harness(deployAfter("frontend"))
h.finished("frontend", BuildStatus.SUCCESS)
h.started.shouldBeEmpty()
h.trigger.arm()
h.finished("frontend", BuildStatus.SUCCESS)
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
h.trigger.disarm()
h.finished("frontend", BuildStatus.SUCCESS)
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
}
test("followUpsOf names the follow-ups without enqueueing anything") {
val h = Harness(deployAfter("default"))
h.trigger.followUpsOf(h.repo, "main", "c1", "default") shouldContainExactly listOf("deploy")
h.trigger.followUpsOf(h.repo, "main", "c1", "frontend").shouldBeEmpty()
h.started.shouldBeEmpty()
}
}
}
@@ -63,6 +63,7 @@ class WatcherTest : FunSpec() {
val artifactStore = mockk<ArtifactStore>()
val startedBuilds = CopyOnWriteArrayList<Pair<String, String>>()
val configLoader = mockk<ConfigLoader>()
val followUpTrigger = mockk<FollowUpTrigger>(relaxed = true)
val repo = RepoContext("test", workingDir, repository, artifactStore)
val watcher =
Watcher(
@@ -70,6 +71,7 @@ class WatcherTest : FunSpec() {
buildExecutor = buildExecutor,
configLoader = configLoader,
clock = Clock.fixed(noon, ZoneOffset.UTC),
followUpTrigger = followUpTrigger,
)
private var seedCounter = 0L
@@ -85,7 +87,7 @@ class WatcherTest : FunSpec() {
every { gitService.originBranchCommitTimes(any()) } returns emptyMap()
every { gitService.originBranchHeads(any()) } returns emptyMap()
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns config
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns config
every { gitService.pullRequestHeads(any()) } returns emptySet()
every { gitService.worktreePrune(any()) } returns Unit
every { gitService.fastForwardLocalBranches(any()) } returns emptyList()
@@ -157,6 +159,19 @@ class WatcherTest : FunSpec() {
)
init {
test("start arms the follow-up trigger before the recovery, stop disarms it") {
val harness = Harness()
harness.watcher.start(listOf(harness.repo))
try {
verify(exactly = 1) { harness.followUpTrigger.arm() }
verify(exactly = 0) { harness.followUpTrigger.disarm() }
} finally {
harness.watcher.stop()
}
verify(exactly = 1) { harness.followUpTrigger.disarm() }
}
test("a fetch failure is exposed in the state and only retried next cycle") {
val harness = Harness()
every { harness.gitService.fetchOrigin(any()) } throws RuntimeException("origin unreachable")
@@ -526,7 +541,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.originBranchHeads(any()) } returns
mapOf("main" to "commit-main", "experiment" to "commit-exp")
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
@@ -549,7 +564,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.originBranches(any()) } returns listOf("experiment")
every { harness.gitService.originBranchHeads(any()) } returns mapOf("experiment" to "commit-exp")
every { harness.gitService.showFileAtCommit("commit-exp", ".gittally.yml", any()) } returns "branch-yaml"
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
harness.watcher.poll(harness.repo)
@@ -568,7 +583,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.originBranchHeads(any()) } returns
mapOf("main" to "commit-main", "experiment" to "commit-exp")
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer
every { harness.gitService.originHeadCommit(any(), any()) } returns "commit-any"
harness.watcher.poll(harness.repo)
@@ -608,7 +623,7 @@ class WatcherTest : FunSpec() {
buildDefinitions = mapOf("nightly" to BuildDefinition(trigger = TriggerConfig(atTimes = listOf("11:00")))),
)
every { harness.configLoader.load(any()) } returns edited
every { harness.configLoader.loadWithBranchLayer(any(), anyNullable()) } returns edited
every { harness.configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns edited
harness.watcher.poll(harness.repo)
@@ -622,7 +637,7 @@ class WatcherTest : FunSpec() {
every { harness.gitService.hasNewCommits("main", any()) } returns true
every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-main")
every { harness.gitService.showFileAtCommit("commit-main", Watcher.CONFIG_FILE, any()) } returns "broken"
every { harness.configLoader.loadWithBranchLayer(any(), "broken") } throws
every { harness.configLoader.loadWithBranchLayer(any(), "broken", anyNullable()) } throws
RuntimeException("mapping problem")
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
+142 -47
View File
@@ -44,14 +44,24 @@
# Optional in the env file:
# WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start)
# WERKATOR_REPO_URL https clone URL of the watched repository
# (default: https://github.com/mhoennig/werkator.git)
# (default: https://git.javagil.de/mi/werkator.git)
# WERKATOR_REPO_DIR directory of the watched repository, absolute or relative to
# WERKATOR_PATH (default: werkator); it also names the systemd
# unit, exactly as `init --systemd` derives it
# WERKATOR_INSTALL_DIR directory holding the unpacked runtime bundle, absolute or
# relative to WERKATOR_PATH (default: .werkator)
# WERKATOR_SANDBOX build runtime of the host: werkdock (default, the bubblewrap
# sandbox; `bwrap` is accepted as its former name) or docker —
# a docker host needs neither the werkdock binary nor a rootfs
# WERKDOCK_REPO checkout of the werkdock repository, whose binary the
# instance runs (default: <repo>/../werkdock)
# WERKDOCK_BINARY the built werkdock binary (default: $WERKDOCK_REPO/dist/werkdock)
# WERKATOR_ROOTFS rootfs archive path for repo-init
# (default: <repo>/build/werkator-buildenv-trixie-java-go-node.tar.zst)
#
# Install layout on the host:
# Install layout on the host — the default, which the three keys above bend to an
# installation that predates this script (e.g. the docker host vm4006: the watched
# repository is ~/hs.hsadmin.ng, the runtime lives in ~/opt, there is no werkdock):
# $WERKATOR_PATH/werkator/ the watched repository (clone)
# $WERKATOR_PATH/.werkator/werkator/ the unpacked runtime bundle
# $WERKATOR_PATH/.werkator/bin/ the werkdock binary
@@ -112,10 +122,37 @@ require_env WERKATOR_REMOTE WERKATOR_PATH
HOST="$WERKATOR_REMOTE"
TARGET_DIR="$WERKATOR_PATH"
ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}"
REPO_URL="${WERKATOR_REPO_URL:-https://github.com/mhoennig/werkator.git}"
MACHINE_CONFIG="$TARGET_DIR/werkator/.git/werkator/.werkator.yml"
WERKATOR_BIN="$TARGET_DIR/.werkator/werkator/bin/werkator"
UNIT="werkator-werkator.service"
REPO_URL="${WERKATOR_REPO_URL:-https://git.javagil.de/mi/werkator.git}"
# The host layout is three values, not one convention: an installation that grew
# before this script existed puts them elsewhere, and the defaults are exactly what
# `instance-install` creates, so an env file that names none of them behaves as before.
# Both directories may be absolute; a bare name is taken relative to WERKATOR_PATH.
resolve_dir() {
case "$1" in
/*) echo "$1" ;;
*) echo "$TARGET_DIR/$1" ;;
esac
}
REPO_DIR="$(resolve_dir "${WERKATOR_REPO_DIR:-werkator}")"
INSTALL_DIR="$(resolve_dir "${WERKATOR_INSTALL_DIR:-.werkator}")"
# where `repo-add` puts a further repository of the registry: beside the watched one
SIBLING_DIR="$(dirname "$REPO_DIR")"
SANDBOX="${WERKATOR_SANDBOX:-werkdock}"
# `bwrap` was the name of the config section until Werkator v1.2.0; accepted so an env
# file written for the older script keeps working, normalised so only one name is used.
[ "$SANDBOX" = "bwrap" ] && SANDBOX="werkdock"
case "$SANDBOX" in
werkdock|docker) ;;
*) die "WERKATOR_SANDBOX is 'werkdock' or 'docker', not '$SANDBOX'" ;;
esac
MACHINE_CONFIG="$REPO_DIR/.git/werkator/.werkator.yml"
WERKATOR_BIN="$INSTALL_DIR/werkator/bin/werkator"
# mirrors SystemdServiceFiles.unitName: the repository's directory name, every
# character outside [A-Za-z0-9_.-] replaced by a dash — the unit `init --systemd`
# writes, which is the one this script may stop and start.
UNIT="werkator-$(basename "$REPO_DIR" | sed 's/[^A-Za-z0-9_.-]/-/g').service"
ssh_present() {
ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST" true 2>/dev/null
@@ -133,13 +170,21 @@ ensure_ssh() {
# Werkdock owns the host checks (`werkdock doctor` ports the old prerequisites
# script); the binary is uploaded first, so the check works pre-install.
# On a docker host there is no werkdock and no sandbox to check: the build runtime
# is the docker daemon, so the check is that the daemon answers this user.
check_prerequisites() {
if [ "$SANDBOX" = "docker" ]; then
echo "==> Checking the docker build runtime on $HOST (WERKATOR_SANDBOX=docker)"
ssh "$HOST" "docker info >/dev/null" || die "docker is not usable by this user on $HOST"
ssh "$HOST" "docker --version"
return 0
fi
ensure_werkdock_binary
echo "==> Uploading werkdock and running its doctor on $HOST (target dir: $TARGET_DIR)"
ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator/bin'"
scp -q "$WERKDOCK_BINARY" "$HOST:$TARGET_DIR/.werkator/bin/werkdock.new"
ssh "$HOST" "mv '$TARGET_DIR/.werkator/bin/werkdock.new' '$TARGET_DIR/.werkator/bin/werkdock' && chmod 755 '$TARGET_DIR/.werkator/bin/werkdock'"
if ! ssh "$HOST" "'$TARGET_DIR/.werkator/bin/werkdock' doctor '$TARGET_DIR'"; then
ssh "$HOST" "mkdir -p '$INSTALL_DIR/bin'"
scp -q "$WERKDOCK_BINARY" "$HOST:$INSTALL_DIR/bin/werkdock.new"
ssh "$HOST" "mv '$INSTALL_DIR/bin/werkdock.new' '$INSTALL_DIR/bin/werkdock' && chmod 755 '$INSTALL_DIR/bin/werkdock'"
if ! ssh "$HOST" "'$INSTALL_DIR/bin/werkdock' doctor '$TARGET_DIR'"; then
die "werkdock doctor failed on $HOST — install aborted"
fi
}
@@ -160,7 +205,7 @@ next to this repository, or point WERKDOCK_REPO/WERKDOCK_BINARY at your checkout
upload_fragment() {
[ -n "${WERKATOR_INIT_CONFIG:-}" ] || { echo ""; return 0; }
[ -f "$WERKATOR_INIT_CONFIG" ] || die "init fragment missing: $WERKATOR_INIT_CONFIG"
local remote="$TARGET_DIR/.werkator/$(basename "$WERKATOR_INIT_CONFIG")"
local remote="$INSTALL_DIR/$(basename "$WERKATOR_INIT_CONFIG")"
scp -q "$WERKATOR_INIT_CONFIG" "$HOST:$remote"
echo "$remote"
}
@@ -174,43 +219,85 @@ ensure_instance_artifacts() {
(cd "$REPO_ROOT" && ./gradlew runtimeBundle --console=plain -q)
fi
[ -f "$RUNTIME_BUNDLE" ] || die "runtime bundle missing: $RUNTIME_BUNDLE"
ensure_werkdock_binary
[ "$SANDBOX" = "docker" ] || ensure_werkdock_binary
}
# Uploads and unpacks the instance artifacts. The previous runtime stays as
# werkator.prev for one deployment as the rollback asset.
deploy_instance() {
# Uploads one file and verifies it arrived whole: a transfer that dies mid-way
# (scp: Connection closed) otherwise leaves a truncated archive that unpacks into
# a broken runtime. Retries twice, because a dropped WAN connection is not a reason
# to abort a deployment.
upload_verified() {
local src="$1" dest="$2"
local local_sha remote_sha attempt
local_sha="$(sha256sum "$src" | cut -d' ' -f1)"
remote_sha="$(ssh "$HOST" "sha256sum '$dest' 2>/dev/null | cut -d' ' -f1" || true)"
if [ "$local_sha" = "$remote_sha" ]; then
echo " $(basename "$src"): already on the host, skipping"
return 0
fi
for attempt in 1 2 3; do
if scp -q "$src" "$HOST:$dest.part"; then
remote_sha="$(ssh "$HOST" "sha256sum '$dest.part' 2>/dev/null | cut -d' ' -f1" || true)"
if [ "$local_sha" = "$remote_sha" ]; then
ssh "$HOST" "mv '$dest.part' '$dest'"
return 0
fi
echo " checksum mismatch after transfer $attempt of $(basename "$src")" >&2
else
echo " transfer $attempt of $(basename "$src") failed" >&2
fi
done
ssh "$HOST" "rm -f '$dest.part'" || true
die "cannot upload $src to $HOST:$dest — three attempts failed"
}
# Uploads the instance artifacts, without touching the installed runtime: the
# service keeps running until swap_instance_runtime replaces it, so a failed
# transfer costs nothing but the transfer.
upload_instance_artifacts() {
if [ "$SANDBOX" = "docker" ]; then
echo "==> Uploading runtime bundle"
else
echo "==> Uploading runtime bundle and werkdock binary"
ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator/bin'"
scp -q "$RUNTIME_BUNDLE" "$HOST:$TARGET_DIR/.werkator/"
scp -q "$WERKDOCK_BINARY" "$HOST:$TARGET_DIR/.werkator/bin/werkdock.new"
fi
ssh "$HOST" "mkdir -p '$INSTALL_DIR/bin'"
upload_verified "$RUNTIME_BUNDLE" "$INSTALL_DIR/$(basename "$RUNTIME_BUNDLE")"
if [ "$SANDBOX" != "docker" ]; then
upload_verified "$WERKDOCK_BINARY" "$INSTALL_DIR/bin/werkdock.new"
fi
}
# Swaps in the uploaded artifacts. The previous runtime stays as werkator.prev
# for one deployment as the rollback asset.
swap_instance_runtime() {
echo "==> Unpacking"
ssh "$HOST" "set -e
cd '$TARGET_DIR/.werkator'
mv bin/werkdock.new bin/werkdock && chmod 755 bin/werkdock
cd '$INSTALL_DIR'
[ ! -f bin/werkdock.new ] || { mv bin/werkdock.new bin/werkdock && chmod 755 bin/werkdock; }
rm -rf werkator.prev
[ ! -d werkator ] || mv werkator werkator.prev
tar xzf '$(basename "$RUNTIME_BUNDLE")'
'./werkator/bin/werkator' --version
'./bin/werkdock' version"
[ ! -x bin/werkdock ] || './bin/werkdock' version"
}
instance_install() {
ensure_ssh
check_prerequisites
ensure_instance_artifacts
deploy_instance
upload_instance_artifacts
swap_instance_runtime
echo
echo "==> Instance installed."
echo " Runtime: $WERKATOR_BIN"
echo " werkdock: $TARGET_DIR/.werkator/bin/werkdock"
[ "$SANDBOX" = "docker" ] || echo " werkdock: $INSTALL_DIR/bin/werkdock"
echo " Next: tools/remote werkator repo-init, then instance-start"
}
# Refuse to swap the runtime under a running build; FORCE=1 overrides.
require_idle() {
local port
port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' 2>/dev/null && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"' || true)"
port="$(ssh "$HOST" "cd '$REPO_DIR' 2>/dev/null && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"' || true)"
[ -n "$port" ] || return 0
local current
current="$(ssh "$HOST" "curl -s --max-time 5 http://127.0.0.1:$port/api/builds/current" || true)"
@@ -224,6 +311,9 @@ instance_update() {
ensure_ssh
ensure_instance_artifacts
require_idle
# upload first, stop second: a transfer that fails must not leave the host
# without a running service (measured on vm4006, 2026-09-03)
upload_instance_artifacts
local was_active=0
if ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user is-active --quiet '$UNIT'"; then
was_active=1
@@ -232,7 +322,7 @@ instance_update() {
echo "==> Stopping $UNIT"
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user stop '$UNIT'"
fi
deploy_instance
swap_instance_runtime
if [ "$was_active" = "1" ]; then
echo "==> Starting $UNIT"
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user start '$UNIT' && sleep 3 && systemctl --user is-active '$UNIT'"
@@ -249,18 +339,22 @@ instance_update() {
# writing is init's — this script transports and invokes (step 23).
repo_init() {
ensure_ssh
[ -f "$ROOTFS" ] || die "rootfs archive missing: $ROOTFS — build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS"
[ "$SANDBOX" = "docker" ] || [ -f "$ROOTFS" ] ||
die "rootfs archive missing: $ROOTFS — build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS"
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning the watched repository"
if ssh "$HOST" "test -d '$TARGET_DIR/werkator/.git'"; then
if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$REPO_URL' '$TARGET_DIR/werkator'"
ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
fi
if [ "$SANDBOX" = "docker" ]; then
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
else
echo "==> Uploading the rootfs archive (skipped when unchanged)"
local rootfs_remote="$TARGET_DIR/.werkator/$(basename "$ROOTFS")"
local rootfs_remote="$INSTALL_DIR/$(basename "$ROOTFS")"
local local_sha remote_sha
local_sha="$(sha256sum "$ROOTFS" | cut -d' ' -f1)"
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' 2>/dev/null | cut -d' ' -f1" || true)"
@@ -271,18 +365,19 @@ repo_init() {
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' | cut -d' ' -f1")"
[ "$local_sha" = "$remote_sha" ] || die "rootfs upload checksum mismatch"
fi
fi
echo "==> Running werkator init${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
local fragment_remote
fragment_remote="$(upload_fragment)"
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
echo "==> Verifying the effective configuration"
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null | grep -A4 'bwrap:' | head -5"
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' config:print 2>/dev/null | grep -A4 '$SANDBOX:' | head -5"
echo
echo "==> Repository ready."
echo " Repo: $TARGET_DIR/werkator"
echo " Repo: $REPO_DIR"
echo " Next: fill git.account/git.token in $MACHINE_CONFIG if the origin is private,"
echo " then tools/remote werkator instance-start"
}
@@ -306,10 +401,10 @@ repo_add() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning $url as '$name'"
if ssh "$HOST" "test -d '$TARGET_DIR/$name/.git'"; then
if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$url' '$TARGET_DIR/$name'"
ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
fi
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
@@ -318,26 +413,26 @@ repo_add() {
echo "==> Running werkator init in $name${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
local fragment_remote
fragment_remote="$(upload_fragment)"
ssh "$HOST" "cd '$TARGET_DIR/$name' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
ssh "$HOST" "cd '$SIBLING_DIR/$name' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
echo "==> Checking the registry"
# Grepped locally: the entry may name the path absolute or as ~/<name>, and
# matching both is easier without a second layer of remote shell quoting.
if ssh "$HOST" "cat ~/.werkator.yml 2>/dev/null" |
grep -qE "path: *(~|$TARGET_DIR)/$name[[:space:]]*$"; then
grep -qE "path: *(~|$SIBLING_DIR)/$name[[:space:]]*$"; then
echo " (~/.werkator.yml already names this path)"
else
echo " not registered yet — add this entry to ~/.werkator.yml on $HOST:"
echo
echo " repositories:"
echo " - path: $TARGET_DIR/$name"
echo " - path: $SIBLING_DIR/$name"
echo " name: $name"
echo
fi
echo "==> Repository prepared."
echo " Repo: $TARGET_DIR/$name"
echo " Next: fill git.account/git.token in $TARGET_DIR/$name/.git/werkator/.werkator.yml if the origin is private"
echo " Repo: $SIBLING_DIR/$name"
echo " Next: fill git.account/git.token in $SIBLING_DIR/$name/.git/werkator/.werkator.yml if the origin is private"
echo " (or once for all repositories in the 'defaults' block of ~/.werkator.yml),"
echo " then restart the service — the registry is read at start."
}
@@ -353,11 +448,11 @@ instance_start() {
echo "==> Applying the instance fragment and generating the host integration (init --systemd)"
local fragment_remote
fragment_remote="$(upload_fragment)"
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'} --systemd"
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'} --systemd"
local htaccess_src="$TARGET_DIR/werkator/.git/werkator/werkator.htaccess"
local htaccess_src="$REPO_DIR/.git/werkator/werkator.htaccess"
local htaccess="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/.htaccess"
local maintenance_src="$TARGET_DIR/werkator/.git/werkator/werkator-maintenance.html"
local maintenance_src="$REPO_DIR/.git/werkator/werkator-maintenance.html"
local maintenance="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/werkator-maintenance.html"
if ssh "$HOST" "test -f '$htaccess_src'"; then
echo "==> Placing the generated Apache reverse proxy at $htaccess"
@@ -368,9 +463,9 @@ instance_start() {
echo "==> Linking the units into ~/.config/systemd/user and enabling the service"
ssh "$HOST" "mkdir -p ~/.config/systemd/user && \
ln -sf '$TARGET_DIR/werkator/.git/werkator/$UNIT' ~/.config/systemd/user/ && \
ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.service' ~/.config/systemd/user/ && \
ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.timer' ~/.config/systemd/user/ && \
ln -sf '$REPO_DIR/.git/werkator/$UNIT' ~/.config/systemd/user/ && \
ln -sf '$REPO_DIR/.git/werkator/werkator-docker-prune.service' ~/.config/systemd/user/ && \
ln -sf '$REPO_DIR/.git/werkator/werkator-docker-prune.timer' ~/.config/systemd/user/ && \
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user daemon-reload && \
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user restart '$UNIT' && \
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user status '$UNIT' --no-pager -l | head -12"
@@ -388,7 +483,7 @@ port_forward() {
# the effective port, wherever it is configured (machine config or applied
# fragment) — config:print is the single answer, not this script's parser
local remote_port
remote_port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"')"
remote_port="$(ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"')"
[ -n "$remote_port" ] || die "no server.port configured — run 'tools/remote werkator instance-start' first"
case "$COMMAND" in
@@ -430,7 +525,7 @@ port_forward() {
# CLI owns creation and format (step 23), this script only invokes it.
control_token() {
ensure_ssh
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' control-token"
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' control-token"
}
case "$REPO" in