Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
79a26dbf10 | ||
|
|
fc8dbf1c91 | ||
|
|
776defd391 | ||
|
|
bbb4969fb9 | ||
|
|
ab522bc68d | ||
|
|
06eafc8010 | ||
|
|
e4b935c684 | ||
|
|
0f57549b4f |
@@ -40,9 +40,9 @@ All production code lives under `de.hoennig.werkator`, with sub-packages `comman
|
|||||||
- Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly. Server routes carry the repository as `/repos/<name>/…` and `/api/repos/<name>/…`, with the unscoped form permanently meaning the served repository; the pages stay per repository and a drop-down in the page title switches between them.
|
- Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly. Server routes carry the repository as `/repos/<name>/…` and `/api/repos/<name>/…`, with the unscoped form permanently meaning the served repository; the pages stay per repository and a drop-down in the page title switches between them.
|
||||||
- When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
|
- When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
|
||||||
- Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
|
- Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
|
||||||
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and werkdock (`werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`) sandbox policies, and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone.
|
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and werkdock (`werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`) sandbox policies, the trust gate (`requirePullRequest`), and the trigger of a follow-up build (`trigger.afterSuccessOf` everywhere, and the whole `trigger` block of a definition the host defines as a follow-up). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, bypass its own pull-request gate, or deploy itself; a branch's definitions apply to that branch alone.
|
||||||
- A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins.
|
- A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `afterSuccessOf`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `afterSuccessOf` makes a definition the follow-up of another one — a deployment is a build that follows a green build (PR#23): it runs at the predecessor's commit after every green run of it, whoever started that run, and the `FollowUpTrigger` that enqueues it is armed only by `Watcher.start()`. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins.
|
||||||
- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary`. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called `bwrap` until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets `ConfigVersions.FORMAT_BROKE_IN`.
|
- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary`, and the trigger of a follow-up build. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called `bwrap` until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets `ConfigVersions.FORMAT_BROKE_IN`.
|
||||||
- `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version.
|
- `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version.
|
||||||
- Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats.
|
- Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats.
|
||||||
- Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK.
|
- Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK.
|
||||||
|
|||||||
@@ -8,28 +8,6 @@ Lightweight, declarative and highly opinionated software build system (CI/CD).
|
|||||||
- [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init`
|
- [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init`
|
||||||
- [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy
|
- [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy
|
||||||
|
|
||||||
## Adding a Gitea Repository
|
|
||||||
|
|
||||||
One instance serves several repositories (`docs/deployment.md`, ADR 0009).
|
|
||||||
From the workstation, clone and initialise, then register:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
|
|
||||||
```
|
|
||||||
|
|
||||||
It prints the registry entry: add it to `~/.werkator.yml` under `repositories:`, then restart the service.
|
|
||||||
The optional `[<name>]` overrides the directory basename: it becomes the route segment (`/repos/<name>/…`) and the UI switcher entry, so it must be unique.
|
|
||||||
Needed only when the clone directory name is wrong or collides — e.g. `michael.hoennig.de.git` checked out as `michael.hoennig.de`, or two forges serving a repo of the same name.
|
|
||||||
A **public** repository needs nothing else: the clone runs anonymously.
|
|
||||||
A **private** repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
defaults:
|
|
||||||
git:
|
|
||||||
account: <gitea-user>
|
|
||||||
token: <token> # Gitea → Settings → Applications → Generate Token, scope read:repository
|
|
||||||
```
|
|
||||||
|
|
||||||
## Developer Setup
|
## Developer Setup
|
||||||
|
|
||||||
Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`:
|
Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`:
|
||||||
|
|||||||
+49
-6
@@ -101,7 +101,9 @@ single branch may decide it:
|
|||||||
- the container sandbox policy: `docker.enabled`/`docker.network` and
|
- the container sandbox policy: `docker.enabled`/`docker.network` and
|
||||||
`werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` — host-pinned as
|
`werkdock.enabled`/`werkdock.rootfs`/`werkdock.binary` — host-pinned as
|
||||||
long as only the host's configuration sets them, master-pinned once the committed
|
long as only the host's configuration sets them, master-pinned once the committed
|
||||||
configuration does.
|
configuration does;
|
||||||
|
- the trigger of a [follow-up build](#follow-up-builds): `trigger.afterSuccessOf` in every definition, and the whole `trigger` block of a definition the host defines as a follow-up.
|
||||||
|
A branch may say what its deployment does, never that — or for which branches — it happens.
|
||||||
|
|
||||||
The distinction is documentary.
|
The distinction is documentary.
|
||||||
Werkator applies one rule: every pinned key is stripped from the branch layer, and the
|
Werkator applies one rule: every pinned key is stripped from the branch layer, and the
|
||||||
@@ -110,7 +112,7 @@ The names say where a key is meant to live, not how it is enforced.
|
|||||||
|
|
||||||
This keeps a branch from reaching credentials, reporting statuses to another repository,
|
This keeps a branch from reaching credentials, reporting statuses to another repository,
|
||||||
raising the global concurrency, disabling its own build container, changing its network
|
raising the global concurrency, disabling its own build container, changing its network
|
||||||
mode, or bypassing its own pull-request gate. Everything else is the branch's to decide —
|
mode, bypassing its own pull-request gate, or deploying itself. Everything else is the branch's to decide —
|
||||||
it can already run any command through `buildCommand`.
|
it can already run any command through `buildCommand`.
|
||||||
The pinned settings are stripped wherever they appear, in a build definition as well as in
|
The pinned settings are stripped wherever they appear, in a build definition as well as in
|
||||||
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
|
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
|
||||||
@@ -252,6 +254,7 @@ builds:
|
|||||||
# branches: ["*", "!master"] # names or globs; a "!" pattern excludes; default: all
|
# branches: ["*", "!master"] # names or globs; a "!" pattern excludes; default: all
|
||||||
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
|
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
|
||||||
# activeWithin: 24h # only branches with commits in the last 24h
|
# activeWithin: 24h # only branches with commits in the last 24h
|
||||||
|
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
|
||||||
# run before each build
|
# run before each build
|
||||||
cleanCommand: rm -rf build
|
cleanCommand: rm -rf build
|
||||||
# shell command for each build
|
# shell command for each build
|
||||||
@@ -397,9 +400,50 @@ Writing any of its keys outside the block is refused with a message naming the d
|
|||||||
Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC).
|
Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC).
|
||||||
A slot may also be written as `??:MM` — that minute of every hour, expanded to its 24 slots, so the build runs hourly.
|
A slot may also be written as `??:MM` — that minute of every hour, expanded to its 24 slots, so the build runs hourly.
|
||||||
Only the latest due slot of a day triggers, so slots missed while the server was down are skipped instead of piling up, and a slot whose pool is still building is retried on the next poll cycle until it succeeds.
|
Only the latest due slot of a day triggers, so slots missed while the server was down are skipped instead of piling up, and a slot whose pool is still building is retried on the next poll cycle until it succeeds.
|
||||||
A definition may have both; one with neither never triggers automatically — which is how `builds.default` is written when it is meant as a settings base only.
|
A definition may combine them; one with none of the three triggers (`onPush`, `atTimes`, `afterSuccessOf`) never runs automatically — which is how `builds.default` is written when it is meant as a settings base only.
|
||||||
Werkator logs a warning once when no definition has a trigger at all, because such an instance never builds anything on its own.
|
Werkator logs a warning once when no definition has a trigger at all, because such an instance never builds anything on its own.
|
||||||
|
|
||||||
|
#### Follow-up builds
|
||||||
|
|
||||||
|
`afterSuccessOf: <name>` makes a definition a *follow-up* of another one, its *predecessor*: it runs on the predecessor's branch at the predecessor's commit whenever a run of the predecessor ends green.
|
||||||
|
This is how a deployment is configured: a deployment is a build that follows a green build, and it gets everything a build has — its own row in History with log and duration, its own Gitea check under its own `statusContext`, restart, cancel, artifacts, and the per-branch serialization.
|
||||||
|
|
||||||
|
Every green run counts, whatever started it — the push watcher, an `atTimes` slot, a UI restart, `werkator retry`, or the startup recovery — and a repeated green run of the same commit triggers the follow-up again.
|
||||||
|
The follow-up builds the commit that was tested, not the branch's current origin head.
|
||||||
|
A failed, cancelled, or interrupted run triggers nothing.
|
||||||
|
The follow-up runs in the branch's worktree, after its predecessor, in the branch's sandbox or container — so a deployment tool like the Docker CLI is provided the way a compiler is.
|
||||||
|
It inherits `builds.default` like every definition, so a deployment that wants the predecessor's output has to set `cleanCommand: ""` itself; a deployment command should rather be self-contained and rebuild what it ships, because a restart of the follow-up alone runs without its predecessor.
|
||||||
|
The pull-request gate is not consulted for a follow-up: the predecessor passed it for the same commit, and the follow-up's `branches` selector is its own gate.
|
||||||
|
|
||||||
|
The trigger of a follow-up is pinned: a branch's committed config can neither add `afterSuccessOf` to a definition nor change the `trigger` block of a definition the host defines as a follow-up, so a branch cannot deploy itself.
|
||||||
|
What the deployment *does* comes with the repository, like every build command; where and when it happens, and with which credentials, is the host's.
|
||||||
|
The host's part lives in `.git/werkator/.werkator.yml`; credentials reach the sandbox like any build setting, through `werkdock.env`/`docker.env`, and a file such as an SSH key through the werkdock sandbox's persistent toolchain home, `.git/werkator/buildenv/home/` on the host, which the sandbox mounts as `/root`.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# .git/werkator/.werkator.yml — the host's part: when, for which branches, with what
|
||||||
|
builds:
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: frontend
|
||||||
|
branches: ["main"]
|
||||||
|
statusContext: werkator/deploy
|
||||||
|
werkdock:
|
||||||
|
env:
|
||||||
|
DEPLOY_TARGET: user@host:~/doms/example.org/htdocs-ssl
|
||||||
|
```
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# .werkator.yml — the repository's part: what
|
||||||
|
builds:
|
||||||
|
deploy:
|
||||||
|
cleanCommand: ""
|
||||||
|
buildCommand: scripts/deploy-prod.sh -y "$DEPLOY_TARGET"
|
||||||
|
```
|
||||||
|
|
||||||
|
A follow-up whose predecessor no definition has, and a cycle of follow-ups, refuse the start with a message naming the definition — a deployment that silently never runs is the failure the flat-key refusal exists to prevent.
|
||||||
|
A branch whose committed config drops or renames the predecessor only loses its follow-up, with a warning naming the branch.
|
||||||
|
A one-shot `werkator build` runs no follow-ups — its process ends with its build — and says which ones the server would have run.
|
||||||
|
|
||||||
Selector: `trigger.branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches.
|
Selector: `trigger.branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches.
|
||||||
A pattern prefixed with `!` excludes instead, and an exclusion always wins regardless of order — `["*", "!master"]` is every branch but master.
|
A pattern prefixed with `!` excludes instead, and an exclusion always wins regardless of order — `["*", "!master"]` is every branch but master.
|
||||||
That is how a branch gets a build of its own without being built by the default one as well.
|
That is how a branch gets a build of its own without being built by the default one as well.
|
||||||
@@ -409,6 +453,7 @@ Both parts combine as an intersection.
|
|||||||
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `werkdock` with all their keys.
|
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `werkdock` with all their keys.
|
||||||
A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults.
|
A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults.
|
||||||
`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
|
`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `werkdock.enabled`, `werkdock.rootfs`, and `werkdock.binary` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
|
||||||
|
So is the trigger of a [follow-up build](#follow-up-builds).
|
||||||
Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited.
|
Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited.
|
||||||
Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only.
|
Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only.
|
||||||
Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of.
|
Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of.
|
||||||
@@ -475,9 +520,7 @@ The section was called `bwrap` and its binary key `bwrap.werkdock` until v1.2.0;
|
|||||||
`werkdock.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
|
`werkdock.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
|
||||||
It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`.
|
It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`.
|
||||||
Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker.
|
Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker.
|
||||||
The archive is imported once per source as the werkdock image `werkator-buildenv-<hash>` into werkdock's store (`$WERKDOCK_HOME`, default `~/.werkdock`) — shared by every repository of this OS user; the hash derives from the source string, so a changed `rootfs` imports a fresh image and stale ones can be removed from the store.
|
The archive is loaded once per source as the werkdock image `werkator-buildenv-<hash>` into werkdock's store (`$WERKDOCK_HOME`, default `~/.werkdock`) — shared by every repository of this OS user; the hash derives from the source string, so a changed `rootfs` loads a fresh image and stale ones can be removed from the store.
|
||||||
Werkator uses `werkdock import ARCHIVE IMAGE` (docker import semantics) and falls back to the older `werkdock load -i ARCHIVE --name IMAGE` when the installed werkdock does not know the verb yet, so Werkator and werkdock can be updated in either order.
|
|
||||||
The image keeps its untagged name (it equals `werkator-buildenv-<hash>:latest` in werkdock's docker-style naming), so an existing store needs no re-import.
|
|
||||||
Per-repo Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`.
|
Per-repo Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`.
|
||||||
`werkdock.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in.
|
`werkdock.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in.
|
||||||
Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user.
|
Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user.
|
||||||
|
|||||||
+1
-2
@@ -123,7 +123,6 @@ Adding a repository is editing a registry entry — never a data migration, beca
|
|||||||
It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry.
|
It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry.
|
||||||
It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back.
|
It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back.
|
||||||
Cloning and initialising is mechanical; registering is a decision.
|
Cloning and initialising is mechanical; registering is a decision.
|
||||||
A private `https` origin authenticates with the shared `defaults.git.account`/`defaults.git.token` of `~/.werkator.yml` (the token travels via a one-shot `GIT_ASKPASS` on the host, never in a URL or process list); enter those once before cloning a private repository — without them only public origins clone.
|
|
||||||
|
|
||||||
4. **Restart** the service; startup recovery re-enqueues what was in flight:
|
4. **Restart** the service; startup recovery re-enqueues what was in flight:
|
||||||
|
|
||||||
@@ -342,7 +341,7 @@ tools/remote --env-file .env.mih34 port-forward start # browser tunne
|
|||||||
|
|
||||||
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
|
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
|
||||||
That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation).
|
That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation).
|
||||||
The rootfs archive is imported once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
|
The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
|
||||||
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
|
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
|
||||||
|
|
||||||
Updates are one command, refused while a build runs (`FORCE=1` overrides):
|
Updates are one command, refused while a build runs (`FORCE=1` overrides):
|
||||||
|
|||||||
@@ -1,90 +0,0 @@
|
|||||||
> **WARNING:** This document describes only the change applied in this PR.
|
|
||||||
> It may already be outdated once the next PR is merged.
|
|
||||||
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
|
||||||
|
|
||||||
## The Problem
|
|
||||||
|
|
||||||
`init --systemd` generates the host integration — the systemd unit's resource limits, the Apache `.htaccess` and the maintenance page — from the effective configuration.
|
|
||||||
It read that configuration through a helper with two independent defects, both of which fail silently.
|
|
||||||
|
|
||||||
**It swallowed every error.**
|
|
||||||
The load sat in `try { … } catch (_: Exception) { ServerConfig() }`.
|
|
||||||
Any configuration error at all — a missing required field, a malformed layer, a version floor violation — produced a default `ServerConfig` with a blank `publicBaseUrl`.
|
|
||||||
A repository with a broken `.werkator.yml` then looked exactly like one that simply has no public base URL configured:
|
|
||||||
the `.htaccess` and the maintenance page were skipped without a word.
|
|
||||||
This surfaced while verifying [PR#17](2026-09-03-PR%2317-maintenance-page.md) on mih09, where the missing files looked like an unconfigured `publicBaseUrl` and were in fact an unrelated validation error.
|
|
||||||
|
|
||||||
**It read from the wrong directory.**
|
|
||||||
The helper called `configLoader.load(Paths.get("."))` — the process's current directory — while everything else in the command works off the git top level resolved by `GitService.getTopLevel`.
|
|
||||||
`ConfigLoader.loadRaw` resolves the layers directly under the directory it is given and does not walk up to the repository root, so the two agree only when `init` happens to be invoked from the root itself.
|
|
||||||
From a subdirectory the command read another repository's configuration, or none.
|
|
||||||
That also broke `--apply`: the fragment is installed into the repository root deliberately before the systemd files are written, so that its port and limits reach the generated unit, and a current-directory read does not see it.
|
|
||||||
|
|
||||||
`init --systemd` runs during initial deployment setup, which is exactly when a silent wrong answer is most expensive.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
- The fallback itself is kept: a configuration that cannot be loaded is not fatal for `init`, the units are still generated with the defaults.
|
|
||||||
During the very first bootstrap there is legitimately nothing to load yet.
|
|
||||||
- No change to `ConfigLoader`, to the configuration schema, or to any other command.
|
|
||||||
- No sweep for catch-all exception handlers elsewhere in the code base;
|
|
||||||
the two other `catch` blocks in `InitCommand` already print an `Error:` and abort, so they were only checked, not changed.
|
|
||||||
|
|
||||||
## The Scenarios
|
|
||||||
|
|
||||||
### Feature: init reports what it read and where it read it from
|
|
||||||
|
|
||||||
#### Background
|
|
||||||
|
|
||||||
- The *repository root* is the git top level as resolved by `GitService.getTopLevel`, the directory holding `.werkator.yml`, `.git/werkator/.werkator.yml` and an applied fragment.
|
|
||||||
- The *current directory* is the process working directory, which is the repository root only when `init` is invoked there.
|
|
||||||
|
|
||||||
#### Scenario#22.01: A broken configuration is named, not defaulted over
|
|
||||||
|
|
||||||
So that a validation error during deployment setup is not mistaken for an unconfigured installation.
|
|
||||||
|
|
||||||
- **Given** a repository whose effective configuration cannot be loaded
|
|
||||||
- **When** `init --systemd` runs
|
|
||||||
- **Then** the exception message is printed as a warning
|
|
||||||
- **and** the unit files are still generated with the default settings
|
|
||||||
- **and** the warning appears exactly once, although three settings are read from the configuration
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- [InitCommandTest: `--systemd warns once when the effective configuration cannot be loaded`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
|
|
||||||
|
|
||||||
#### Scenario#22.02: The configuration is read from the repository root
|
|
||||||
|
|
||||||
So that the generated host integration reflects the repository being initialized, whatever directory `init` was invoked from.
|
|
||||||
|
|
||||||
- **Given** a repository whose root configuration sets `server.publicBaseUrl` and `server.port`
|
|
||||||
- **and** a current directory that is not that repository root
|
|
||||||
- **When** `init --systemd` runs
|
|
||||||
- **Then** the `.htaccess` and the maintenance page are generated
|
|
||||||
- **and** the `.htaccess` proxies to the port from the root configuration
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- [InitCommandTest: `--systemd reads the configuration from the repository root, not the current directory`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
|
|
||||||
|
|
||||||
## The Solution
|
|
||||||
|
|
||||||
The catch-all now prints the exception message before falling back:
|
|
||||||
|
|
||||||
```
|
|
||||||
Warning: the effective configuration could not be loaded (<message>)
|
|
||||||
continuing with default server settings — check the generated unit and host files
|
|
||||||
```
|
|
||||||
|
|
||||||
The configuration is read three times while the systemd files are written (`memoryMax`, `tasksMax`, `publicBaseUrl`), which would repeat the warning three times.
|
|
||||||
It is therefore loaded once per run and cached in the command, and the cache is reset at the top of `run()` so a reused instance — the command is a Spring singleton — re-reads.
|
|
||||||
|
|
||||||
The repository root is passed down into the two accessors instead of `Paths.get(".")`.
|
|
||||||
This matches every other caller of `ConfigLoader.load` in the code base, all of which pass an explicit working directory;
|
|
||||||
`InitCommand` was the only one relying on the process's current directory.
|
|
||||||
|
|
||||||
Both fixes are the same failure in two forms — the command answered from a configuration it never actually read — which is why they are in one PR.
|
|
||||||
|
|
||||||
## Additional Changes
|
|
||||||
|
|
||||||
- None.
|
|
||||||
@@ -0,0 +1,199 @@
|
|||||||
|
> **WARNING:** This document describes only the change applied in this PR.
|
||||||
|
> It may already be outdated once the next PR is merged.
|
||||||
|
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
||||||
|
|
||||||
|
## Related Links
|
||||||
|
|
||||||
|
- [ADR 0007 — build definitions](../adrs/0007-2026-08-31.build-definitions.md): the `builds` section and its `trigger` block this PR extends.
|
||||||
|
- [configuration.md — build definitions](../configuration.md#build-definitions): the reference this PR updates.
|
||||||
|
- Werkbaum's `scripts/deploy-prod.sh`: the first deployment meant to run this way.
|
||||||
|
|
||||||
|
## The Problem
|
||||||
|
|
||||||
|
Werkator builds and reports, but it cannot deploy.
|
||||||
|
Werkbaum's production deployment is still a script run by hand from a developer machine, after looking at the Werkator status.
|
||||||
|
The step "if the build is green, run this" is exactly what a CI system is for, and the hand-off between the two is where releases go wrong: the wrong commit gets deployed, or a red commit, or nothing.
|
||||||
|
|
||||||
|
Three ways to add deployments were considered:
|
||||||
|
|
||||||
|
- A `deployCommand` next to `buildCommand`, run after a green `buildCommand` inside the same run.
|
||||||
|
Simple, but it creates a second command path inside one build — one log, one status, one duration for two different things — and a failed deployment would turn a green build red.
|
||||||
|
- A separate `deploy` section with its own executor path, statuses, and cancellation.
|
||||||
|
A second execution path next to `buildCommand`, with everything the first one has to be built again.
|
||||||
|
- **A deployment is a build that follows a green build.** Chosen.
|
||||||
|
A build definition may declare that it runs whenever another definition of the same branch turns green.
|
||||||
|
Everything a build has comes for free: its own row in History with log and duration, its own Gitea check under its own `statusContext`, restart without rebuilding, cancel, artifacts, and the per-branch serialization.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- A native execution path for deployments.
|
||||||
|
A follow-up build runs where every build of its branch runs — in the werkdock sandbox or the Docker container — so a deployment tool such as the Docker CLI is provided the same way a compiler is.
|
||||||
|
- Waiting for *several* builds to be green.
|
||||||
|
`afterSuccessOf` names one build; an `all of` form is a follow-up PR if a repository ever needs it.
|
||||||
|
- A separate secrets mechanism.
|
||||||
|
Deployment credentials reach the sandbox the way any build setting does — the host's `werkdock.env`/`docker.env` for the definition, and files placed in the sandbox's persistent toolchain home — and the pinning below keeps them on the branches the host names.
|
||||||
|
- Handing the predecessor's artifacts to the follow-up.
|
||||||
|
The follow-up runs in the same worktree, so the predecessor's output is *usually* still there, but a deployment command must not rely on it (see Open Questions).
|
||||||
|
- Recovering a follow-up whose enqueueing was lost to a server restart between the two builds.
|
||||||
|
The operator restarts the predecessor, which triggers the follow-up again.
|
||||||
|
|
||||||
|
## The Scenarios
|
||||||
|
|
||||||
|
### Feature: a build that follows a green build
|
||||||
|
|
||||||
|
#### Background
|
||||||
|
|
||||||
|
- A *follow-up build* is a build definition whose `trigger` block carries `afterSuccessOf: <name>`, naming another definition of the same configuration — its *predecessor*.
|
||||||
|
- The follow-up runs on the predecessor's branch at the predecessor's commit, never at the branch's current origin head, so a deployment always ships the commit that was tested.
|
||||||
|
- Like every non-default build it records under its own pool `<branch>@<name>`, and it should carry its own `statusContext` so that Gitea shows the deployment as a check of its own.
|
||||||
|
- *Green* is every run of the predecessor that ends with `SUCCESS`, whatever started it: the push watcher, an `atTimes` slot, a UI restart, `werkator retry`, or the startup recovery.
|
||||||
|
- The key belongs to the `trigger` block: it says *when* the build runs, so it is never inherited from `builds.default`, and writing it flat is refused like every other trigger key.
|
||||||
|
|
||||||
|
#### Scenario#23.01: A green predecessor triggers the follow-up on the same commit
|
||||||
|
|
||||||
|
So that a deployment ships exactly the commit that was just tested, with its own log, duration, and Gitea check.
|
||||||
|
|
||||||
|
- **Given** a definition `deploy` with `trigger.afterSuccessOf: frontend` and `statusContext: werkator/deploy`
|
||||||
|
- **and** the build `frontend` of branch `main` at commit `c1` is running
|
||||||
|
- **When** that build finishes with `SUCCESS`
|
||||||
|
- **Then** a build `deploy` of branch `main` at commit `c1` is enqueued
|
||||||
|
- **and** it is recorded under the pool `main@deploy`
|
||||||
|
- **and** its Gitea status is posted under the context `werkator/deploy`
|
||||||
|
- **and** the origin head of `main` having moved on to `c2` meanwhile changes nothing about that
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [FollowUpTriggerTest — "a green predecessor enqueues the follow-up at the predecessor's commit"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
|
||||||
|
|
||||||
|
#### Scenario#23.02: Every green run triggers again, including a repeated run of the same commit
|
||||||
|
|
||||||
|
So that a deployment can be repeated by rerunning the build — and so that no run of a green build is silently *not* deployed, which would be more confusing than a redundant deployment.
|
||||||
|
|
||||||
|
- **Given** the build `frontend` of `main` at `c1` was green and `deploy` ran for it
|
||||||
|
- **When** `frontend` at `c1` is restarted from the UI, retried from the CLI, or rebuilt by an `atTimes` slot, and turns green again
|
||||||
|
- **Then** `deploy` is enqueued for `c1` again
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [FollowUpTriggerTest — "every green run of the predecessor triggers the follow-up again"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
|
||||||
|
|
||||||
|
#### Scenario#23.03: A run that is not green triggers nothing
|
||||||
|
|
||||||
|
So that nothing is ever deployed from a failed, cancelled, or interrupted build.
|
||||||
|
|
||||||
|
- **Given** the same definitions
|
||||||
|
- **When** the build `frontend` ends as `FAILED`, `CANCELLED`, or `INTERRUPTED`
|
||||||
|
- **or** a build other than `frontend` ends as `SUCCESS`
|
||||||
|
- **Then** no `deploy` build is enqueued
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [FollowUpTriggerTest — "only a SUCCESS of the named predecessor triggers"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
|
||||||
|
|
||||||
|
#### Scenario#23.04: The trigger of a follow-up is host-pinned
|
||||||
|
|
||||||
|
So that a branch can never deploy itself: a follow-up build is the host's way to hand real-world effects — targets, credentials — to a commit, and the host alone decides *when* and *for which branches* that happens.
|
||||||
|
|
||||||
|
- **Given** the host configuration defines `deploy` with `trigger: {afterSuccessOf: frontend, branches: [main]}`
|
||||||
|
- **When** a branch's committed `.werkator.yml` sets `builds.deploy.trigger.branches: ["*"]`
|
||||||
|
- **or** adds `afterSuccessOf` to any definition's trigger
|
||||||
|
- **Then** the host's trigger block of `deploy` is used unchanged, and the branch's `afterSuccessOf` is dropped with a warning naming the branch
|
||||||
|
- **and** a branch the host's selector does not name never runs `deploy`, however green its own builds are
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [ConfigLoaderTest — "a follow-up trigger is pinned to the host, a branch cannot add or widen one"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||||
|
|
||||||
|
#### Scenario#23.05: What the follow-up does comes with the repository
|
||||||
|
|
||||||
|
So that the deployment command is versioned with the code it deploys, like every other build command — while the host keeps the targets and credentials.
|
||||||
|
|
||||||
|
- **Given** the host defines `deploy` with its trigger and `werkdock.env: {DEPLOY_TARGET: …}`
|
||||||
|
- **and** the committed `.werkator.yml` of `main` defines `deploy` with `cleanCommand: ""` and `buildCommand: scripts/deploy-prod.sh -y "$DEPLOY_TARGET"`
|
||||||
|
- **When** `deploy` runs for `main`
|
||||||
|
- **Then** it runs the committed command with the host's environment, in the branch's sandbox, in the branch's worktree, after its predecessor and serialized with the branch's other builds
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [BuildExecutorTest — "a follow-up build runs in its branch's worktree after its predecessor"](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorTest.kt)
|
||||||
|
- [ConfigLoaderTest — "a branch supplies the command of a host-triggered follow-up"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||||
|
|
||||||
|
#### Scenario#23.06: A follow-up that could never fire is refused at start
|
||||||
|
|
||||||
|
So that a deployment that silently never runs cannot exist — the same reasoning that refuses a flat trigger key.
|
||||||
|
|
||||||
|
- **Given** a configuration whose `afterSuccessOf` names a definition that does not exist
|
||||||
|
- **or** whose follow-ups form a cycle (`a` after `b`, `b` after `a`)
|
||||||
|
- **or** which writes `afterSuccessOf` outside the `trigger` block
|
||||||
|
- **When** the configuration is loaded
|
||||||
|
- **Then** loading fails with a message naming the definition and the reason
|
||||||
|
- **and** a definition with only `afterSuccessOf` counts as triggered, so the "no build triggered" warning is not raised for it
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [ConfigLoaderTest — "afterSuccessOf must name an existing definition and must not form a cycle"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||||
|
- [ConfigLoaderTest — "afterSuccessOf written flat is refused like every trigger key"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||||
|
|
||||||
|
#### Scenario#23.07: Follow-ups fire in server mode only
|
||||||
|
|
||||||
|
So that the invariant "nothing is scheduled during CLI runs or tests" holds: a CLI `build` ends when its build ends, and a follow-up enqueued into a process that is about to exit would only ever be recorded as interrupted.
|
||||||
|
|
||||||
|
- **Given** `werkator build main` runs from the CLI and turns green
|
||||||
|
- **When** the command finishes
|
||||||
|
- **Then** no follow-up was enqueued, and the CLI says which follow-up the server would have run
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [FollowUpTriggerTest — "the trigger listens only while the watcher runs"](../../src/test/kotlin/de/hoennig/werkator/watcher/FollowUpTriggerTest.kt)
|
||||||
|
- [WatcherTest — "start arms the follow-up trigger before the recovery, stop disarms it"](../../src/test/kotlin/de/hoennig/werkator/watcher/WatcherTest.kt)
|
||||||
|
- [BuildCommandTest — "a green CLI build names the follow-up builds the server would run, and runs none"](../../src/test/kotlin/de/hoennig/werkator/commands/BuildCommandTest.kt)
|
||||||
|
|
||||||
|
## The Solution
|
||||||
|
|
||||||
|
`TriggerConfig` gets a fourth key, `afterSuccessOf: String` (empty: none).
|
||||||
|
It sits inside the `trigger` block on purpose, next to `onPush` and `atTimes`: it answers "when does this build run", so the structural rule of ADR 0007 makes it non-inheritable without touching any list, and `checkTriggerBlocks` refuses it written flat by adding it to `FLAT_TRIGGER_KEYS`.
|
||||||
|
`isTriggered` counts it, so a definition with nothing but a predecessor is not reported as never triggering.
|
||||||
|
A definition may itself be followed; `ConfigLoader` refuses an unknown predecessor and a cycle when it validates the merged configuration.
|
||||||
|
|
||||||
|
A new `FollowUpTrigger` in the `watcher` package listens to `BuildStatusChangedEvent`.
|
||||||
|
On a `SUCCESS` it resolves the definitions of the result's branch at the result's commit — the same `definitionsFor` the watcher uses for the branch layer — and enqueues, through `BuildExecutor.startBuild(repo, branch, commit, name)`, every definition whose `afterSuccessOf` names the finished build and whose selector selects the branch.
|
||||||
|
Passing the commit explicitly is what makes Scenario#23.01's last line true; the executor's duplicate guard folds a follow-up that is already queued for the same commit.
|
||||||
|
The listener is armed by the watcher's `start()` and disarmed by `stop()`, which is how it stays silent in CLI runs and tests (Scenario#23.07).
|
||||||
|
|
||||||
|
The pinning extends `stripPinned`: `afterSuccessOf` is removed from every trigger of the branch layer, and for every definition whose host trigger carries `afterSuccessOf` the branch layer's whole `trigger` block is dropped.
|
||||||
|
That is the smallest rule that makes Scenario#23.04 hold: a branch keeps the right to describe what its deployment does, and loses only the right to decide that — or where — it happens.
|
||||||
|
The pull-request gate is not consulted for a follow-up: the predecessor passed it already for the same commit, and the host's selector is the follow-up's own gate.
|
||||||
|
|
||||||
|
The follow-up runs like any other build of its branch — same worktree, same sandbox, serialized behind its predecessor.
|
||||||
|
It inherits `builds.default`, so a deployment that wants the predecessor's output has to set `cleanCommand: ""` itself.
|
||||||
|
|
||||||
|
Three places stay in sync with the new key, as the invariant demands: the data classes, the `init` templates (a commented `afterSuccessOf` line under the `trigger` block), and `docs/configuration.md`, which gets a "Follow-up builds" paragraph under build definitions and the new pinning in the branch-layer section.
|
||||||
|
`AGENTS.md` names the key in the trigger and pinning invariants.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- **Credentials inside the sandbox.**
|
||||||
|
Werkdock clears the environment and mounts the repository's persistent toolchain home as `/root`, so an SSH key placed under `.git/werkator/buildenv/home/.ssh/` on the host is `/root/.ssh/` inside the sandbox, and `werkdock.env` carries the target.
|
||||||
|
For Docker there is no equivalent mount today; a key passed through `docker.env` works but is visible in `docker inspect`.
|
||||||
|
Implemented: nothing new — the PR documents the werkdock path in `configuration.md` and leaves a Docker mount to a follow-up if vm4006 ever deploys.
|
||||||
|
- **Restarting the follow-up alone.**
|
||||||
|
Restart re-runs `deploy` without its predecessor, in a worktree that may have been cleaned by a later build of the branch.
|
||||||
|
Implemented: allowed, and the reference recommends a self-contained deployment command that rebuilds what it ships (Werkbaum's `deploy-prod.sh` does).
|
||||||
|
- **A predecessor that exists only on a branch.**
|
||||||
|
The host's `afterSuccessOf: frontend` refers to a name the branch layer may rename.
|
||||||
|
Implemented: refused at start for the primary configuration; for a branch whose layer lacks the name, a warning once per branch and commit, and no follow-up
|
||||||
|
(verified by [ConfigLoaderTest — "a branch whose layer lacks the predecessor loses only its follow-up"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)).
|
||||||
|
An instance fragment checked by `init --apply` is not checked for its predecessor at all: the build it names may well live in the project config it is merged with, and the merged configuration is checked on every load anyway.
|
||||||
|
|
||||||
|
## Additional Changes
|
||||||
|
|
||||||
|
- `BuildStatusChangedEvent` now carries the `RepoContext` the result belongs to: a `BuildResult` does not know its repository, and the listener has to enqueue into the right one.
|
||||||
|
- `ConfigLoader.loadWithBranchLayer` and `loadForWorktree` take an optional branch name, used only to name the branch in the pinning warnings; the watcher passes it.
|
||||||
|
- The follow-up check distinguishes three loads: the primary configuration refuses a missing predecessor, a branch layer warns, and an instance fragment (`init --apply`) skips the check — its predecessor may live in the project config it is merged with.
|
||||||
|
- The "no build triggered" warning now names `afterSuccessOf` as the third trigger.
|
||||||
|
|
||||||
|
## Follow-up PRs
|
||||||
|
|
||||||
|
- Werkbaum: commit the `deploy` definition to its `.werkator.yml`, add the host part on mih09, and retire the manual `deploy-prod.sh` invocation from the README.
|
||||||
|
- `afterSuccessOf` as a list (all green), if a repository ever needs a deployment gated on more than one build.
|
||||||
|
- A Docker bind mount for credential files, if a Docker host deploys.
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
> **WARNING:** This document describes only the change applied in this PR.
|
|
||||||
> It may already be outdated once the next PR is merged.
|
|
||||||
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
|
||||||
|
|
||||||
## Related Links
|
|
||||||
|
|
||||||
- ADR 0009 — multi-repo instance: the `defaults` block carries shared repository-level keys, read by `ConfigLoader`, never directly by consumers.
|
|
||||||
- `docs/deployment.md` — registry setup: `repo-add` clones, initialises, and prints the registry entry.
|
|
||||||
|
|
||||||
## The Problem
|
|
||||||
|
|
||||||
`tools/remote werkator repo-add <private-https-url>` fails on the host with `could not read Username`.
|
|
||||||
The clone runs anonymously, but the credentials exist only in the instance file's `defaults.git` block — which the script never consults.
|
|
||||||
Registering a private repository therefore needs a manual SSH session today.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
The script still does not write `~/.werkator.yml`: registering stays the operator's decision.
|
|
||||||
No SSH-URL support: the forge is reached over `https` from the host.
|
|
||||||
No new config keys: `defaults.git.account`/`defaults.git.token` already exist.
|
|
||||||
|
|
||||||
## The Scenarios
|
|
||||||
|
|
||||||
### Feature: authenticated clone for private origins
|
|
||||||
|
|
||||||
#### Background
|
|
||||||
|
|
||||||
- The shared credentials live in `~/.werkator.yml` under `defaults.git` (ADR 0009).
|
|
||||||
- Public origins and instances without shared credentials must keep cloning anonymously.
|
|
||||||
|
|
||||||
#### Scenario#000.01: Private https origin clones with shared credentials
|
|
||||||
|
|
||||||
- **Given** `defaults.git.account`/`defaults.git.token` in `~/.werkator.yml` on the host
|
|
||||||
- **When** `tools/remote werkator repo-add <private-https-url>` runs
|
|
||||||
- **Then** the clone authenticates with those credentials and succeeds.
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- Manual stub-`git` test: URL carries `account@`, askpass answers the token, `GIT_TERMINAL_PROMPT=0`.
|
|
||||||
|
|
||||||
#### Scenario#000.02: Public origin clones anonymously
|
|
||||||
|
|
||||||
- **Given** no shared credentials (or a public repository)
|
|
||||||
- **When** `repo-add` or `repo-init` runs
|
|
||||||
- **Then** the clone runs exactly as before, without authentication.
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- Local helper test against a `file://` origin with and without an instance file.
|
|
||||||
|
|
||||||
#### Scenario#000.03: Token never leaks locally
|
|
||||||
|
|
||||||
- **Given** an authenticated clone
|
|
||||||
- **When** the command runs from the workstation
|
|
||||||
- **Then** the token appears in neither the local process list nor a repository config.
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- Code inspection: the token is read on the host and passed via a one-shot `GIT_ASKPASS` script.
|
|
||||||
|
|
||||||
## The Solution
|
|
||||||
|
|
||||||
`tools/remote` gained a `clone_repo` helper used by both `repo-init` and `repo-add`.
|
|
||||||
For `https://` URLs it ships a small Python helper (base64-encoded, so no `$` is expanded locally) to the host.
|
|
||||||
The helper reads `defaults.git.account`/`defaults.git.token` from `~/.werkator.yml`, puts the account into the URL, and hands the token to git via a one-shot `0700` `GIT_ASKPASS` script deleted in `finally`.
|
|
||||||
Without credentials it falls back to the plain anonymous clone; non-`https` URLs clone unchanged.
|
|
||||||
`docs/deployment.md` documents that the shared credentials must exist before cloning a private repository.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
- None.
|
|
||||||
|
|
||||||
## Attachments
|
|
||||||
|
|
||||||
### Adding a Gitea repository — example
|
|
||||||
|
|
||||||
From the workstation, clone and initialise, then register:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
|
|
||||||
```
|
|
||||||
|
|
||||||
Add the printed entry to `~/.werkator.yml` under `repositories:`, then restart the service.
|
|
||||||
A public repository needs nothing else: the clone runs anonymously.
|
|
||||||
A private repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning (token: Gitea → Settings → Applications → Generate Token, scope `read:repository`):
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
defaults:
|
|
||||||
git:
|
|
||||||
account: <gitea-user>
|
|
||||||
token: <token>
|
|
||||||
```
|
|
||||||
@@ -1,102 +0,0 @@
|
|||||||
> **WARNING:** This document describes only the change applied in this PR.
|
|
||||||
> It may already be outdated once the next PR is merged.
|
|
||||||
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
|
||||||
|
|
||||||
## Related Links
|
|
||||||
|
|
||||||
- Werkdock `CHANGELOG.md` (repository `mi/werkdock`, Unreleased section) — the release notes this PR adapts to: `import` replaces `load -i --name` for rootfs archives, `load` is reserved for docker/OCI image archives, image names gained docker-style tags.
|
|
||||||
- ADR 0008 — the bwrap build runtime; `docs/configuration.md`, notes on `builds.<name>.werkdock`.
|
|
||||||
|
|
||||||
## The Problem
|
|
||||||
|
|
||||||
Werkdock moves its CLI closer to `docker run --rm`.
|
|
||||||
For the rootfs archives Werkator uses, the verb is now `werkdock import ARCHIVE IMAGE` (docker import semantics).
|
|
||||||
`werkdock load -i ARCHIVE --name IMAGE` still works but is a compatibility path with a note on stderr, and is announced to go away once Werkator has switched.
|
|
||||||
Werkator ships the werkdock binary with its deployment, but nothing forces the two to be updated together: an installation may run a new Werkator against an older werkdock for a while, or the other way round.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
- No switch of the existence check from `werkdock images` to `werkdock inspect`: `images` prints the bare name for untagged images on every werkdock version, the exact-line match keeps working, and `inspect` would tie Werkator to the new werkdock.
|
|
||||||
- No tagged image name (`werkator-buildenv:<hash>`): it would re-import every build environment on the webspace and orphan the old image, for no functional gain today.
|
|
||||||
- None of the new `run` flags (`--mount`, `--entrypoint`, `--network host`): Werkator's invocation needs none of them.
|
|
||||||
|
|
||||||
## The Scenarios
|
|
||||||
|
|
||||||
### Feature: rootfs archives are imported with werkdock's docker-shaped verb
|
|
||||||
|
|
||||||
#### Background
|
|
||||||
|
|
||||||
- The build environment is a rootfs archive named by `builds.<name>.werkdock.rootfs`.
|
|
||||||
- Werkator creates the werkdock image `werkator-buildenv-<hash>` from it once per source.
|
|
||||||
- Werkdock answers an unknown verb with `werkdock: unknown command "import"` and exit code 125, its code for its own errors.
|
|
||||||
|
|
||||||
#### Scenario#000.01: A missing image is imported with `werkdock import`
|
|
||||||
|
|
||||||
So that Werkator uses the verb werkdock names for rootfs archives, and the deprecated path can be removed on werkdock's side.
|
|
||||||
|
|
||||||
- **Given** `werkdock images` does not list the image
|
|
||||||
- **When** a build starts
|
|
||||||
- **Then** Werkator runs `werkdock import ARCHIVE werkator-buildenv-<hash>`
|
|
||||||
- **and** runs no `werkdock load`.
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- [WerkdockBuildRunnerTest "imports the image once when werkdock does not know it yet"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
|
|
||||||
- [WerkdockBuildRunnerTest "downloads a URL rootfs once before importing it"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
|
|
||||||
|
|
||||||
#### Scenario#000.02: An older werkdock without the verb still works
|
|
||||||
|
|
||||||
So that Werkator and werkdock can be updated in either order.
|
|
||||||
|
|
||||||
- **Given** the installed werkdock answers `import` with `unknown command` and exit 125
|
|
||||||
- **When** a build starts with a missing image
|
|
||||||
- **Then** Werkator falls back to `werkdock load -i ARCHIVE --name werkator-buildenv-<hash>`
|
|
||||||
- **and** logs the fallback.
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- [WerkdockBuildRunnerTest "falls back to load -i --name on a werkdock without the import verb"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
|
|
||||||
|
|
||||||
#### Scenario#000.03: A real import failure is not masked by the fallback
|
|
||||||
|
|
||||||
So that a broken archive fails the build with werkdock's message, as before.
|
|
||||||
|
|
||||||
- **Given** `werkdock import` fails for any other reason (any other exit code, or exit 125 without `unknown command`)
|
|
||||||
- **When** a build starts with a missing image
|
|
||||||
- **Then** the build fails with that command's output
|
|
||||||
- **and** no `werkdock load` runs.
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- [WerkdockBuildRunnerTest "propagates an import failure that is not a missing verb"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
|
|
||||||
|
|
||||||
#### Scenario#000.04: An existing image is neither imported nor loaded
|
|
||||||
|
|
||||||
- **Given** `werkdock images` lists the image
|
|
||||||
- **When** a build starts
|
|
||||||
- **Then** neither `import` nor `load` runs.
|
|
||||||
|
|
||||||
##### Verified by
|
|
||||||
|
|
||||||
- [WerkdockBuildRunnerTest "does not import an image werkdock already has"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
|
|
||||||
|
|
||||||
## The Solution
|
|
||||||
|
|
||||||
`WerkdockBuildRunner.ensureImage` calls the new `importImage`: `werkdock import ARCHIVE IMAGE` through the non-throwing `run`, then either returns, falls back to `load -i --name` on exactly the unknown-verb signature (exit 125 plus `unknown command` on stderr), or rethrows the import's result as a `GitCommandException` — the same exception and message the old code produced.
|
|
||||||
The fallback is keyed on werkdock's own error signature rather than on a version string, because werkdock's version output does not yet distinguish the two builds.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
- When to drop the fallback: once every installation runs a werkdock with `import`, the `load -i --name` branch and its test go, and werkdock can remove the compatibility path.
|
|
||||||
|
|
||||||
## Additional Changes
|
|
||||||
|
|
||||||
- `docs/configuration.md` and `docs/deployment.md` say "imported" where they said "loaded", and name the fallback and the reason the image name stays untagged.
|
|
||||||
|
|
||||||
## Prerequisite PRs
|
|
||||||
|
|
||||||
- The werkdock change that introduces `import` (repository `mi/werkdock`, CHANGELOG Unreleased); against an older werkdock the fallback path runs.
|
|
||||||
|
|
||||||
## Follow-up PRs
|
|
||||||
|
|
||||||
- Remove the fallback (see Open Questions).
|
|
||||||
@@ -119,7 +119,7 @@ class BuildExecutor(
|
|||||||
artifactKey = runningBuild.artifactKey,
|
artifactKey = runningBuild.artifactKey,
|
||||||
)
|
)
|
||||||
repo.results.append(pending)
|
repo.results.append(pending)
|
||||||
eventPublisher.publishEvent(BuildStatusChangedEvent(pending))
|
eventPublisher.publishEvent(BuildStatusChangedEvent(pending, repo))
|
||||||
val activeBuild = ActiveBuild(runningBuild, repo)
|
val activeBuild = ActiveBuild(runningBuild, repo)
|
||||||
builds[runningBuild.artifactKey] = activeBuild
|
builds[runningBuild.artifactKey] = activeBuild
|
||||||
publishGiteaStatus(activeBuild, BuildStatus.PENDING, duration = null)
|
publishGiteaStatus(activeBuild, BuildStatus.PENDING, duration = null)
|
||||||
@@ -384,7 +384,7 @@ class BuildExecutor(
|
|||||||
duration = duration,
|
duration = duration,
|
||||||
artifactKey = runningBuild.artifactKey,
|
artifactKey = runningBuild.artifactKey,
|
||||||
).also { build.repo.results.append(it) }
|
).also { build.repo.results.append(it) }
|
||||||
eventPublisher.publishEvent(BuildStatusChangedEvent(updated))
|
eventPublisher.publishEvent(BuildStatusChangedEvent(updated, build.repo))
|
||||||
publishGiteaStatus(build, status, duration)
|
publishGiteaStatus(build, status, duration)
|
||||||
return updated
|
return updated
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -35,7 +35,12 @@ data class RunningBuild(
|
|||||||
var runningSince: Instant? = null
|
var runningSince: Instant? = null
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Published via Spring's `ApplicationEventPublisher` on every persisted status transition. */
|
/**
|
||||||
|
* Published via Spring's `ApplicationEventPublisher` on every persisted status transition.
|
||||||
|
* Carries the repository because a [BuildResult] does not: a listener that reacts to the
|
||||||
|
* transition — the follow-up trigger — has to act on that repository.
|
||||||
|
*/
|
||||||
data class BuildStatusChangedEvent(
|
data class BuildStatusChangedEvent(
|
||||||
val result: BuildResult,
|
val result: BuildResult,
|
||||||
|
val repo: RepoContext,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package de.hoennig.werkator.build
|
|||||||
|
|
||||||
import de.hoennig.werkator.config.BranchConfig
|
import de.hoennig.werkator.config.BranchConfig
|
||||||
import de.hoennig.werkator.config.WerkdockConfig
|
import de.hoennig.werkator.config.WerkdockConfig
|
||||||
import de.hoennig.werkator.git.GitCommandException
|
|
||||||
import de.hoennig.werkator.git.GitCommandRunner
|
import de.hoennig.werkator.git.GitCommandRunner
|
||||||
import org.slf4j.LoggerFactory
|
import org.slf4j.LoggerFactory
|
||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
@@ -17,9 +16,9 @@ import java.security.MessageDigest
|
|||||||
* it shells out to the `werkdock` CLI (`werkdock.binary`, default via PATH) — the same
|
* it shells out to the `werkdock` CLI (`werkdock.binary`, default via PATH) — the same
|
||||||
* pattern as git and docker, CLI, no library.
|
* pattern as git and docker, CLI, no library.
|
||||||
*
|
*
|
||||||
* The rootfs archive becomes a werkdock *image*, imported once per source
|
* The rootfs archive becomes a werkdock *image*, loaded once per source
|
||||||
* (`werkator-buildenv-<hash>`, the hash over the source string, so a changed source
|
* (`werkator-buildenv-<hash>`, the hash over the source string, so a changed source
|
||||||
* imports a fresh image) into werkdock's own store (`$WERKDOCK_HOME`, default
|
* loads a fresh image) into werkdock's own store (`$WERKDOCK_HOME`, default
|
||||||
* `~/.werkdock`) — shared by every repository of this OS user, unlike the old
|
* `~/.werkdock`) — shared by every repository of this OS user, unlike the old
|
||||||
* per-repo unpack. Only the download cache for URL sources and the persistent
|
* per-repo unpack. Only the download cache for URL sources and the persistent
|
||||||
* toolchain home (bound to `/root` for Gradle/Go caches) stay under
|
* toolchain home (bound to `/root` for Gradle/Go caches) stay under
|
||||||
@@ -67,11 +66,9 @@ class WerkdockBuildRunner(
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Imports the rootfs archive into the werkdock image store once per source.
|
* Loads the rootfs archive into the werkdock image store once per source.
|
||||||
* `werkdock images` answers existence through the CLI, like `docker image
|
* `werkdock images` answers existence through the CLI, like `docker image
|
||||||
* inspect` does for the Docker runner; it prints the bare name for an untagged
|
* inspect` does for the Docker runner.
|
||||||
* image on every werkdock version, so the exact-line match holds across the
|
|
||||||
* werkdock upgrade that introduced tags.
|
|
||||||
*/
|
*/
|
||||||
private fun ensureImage(
|
private fun ensureImage(
|
||||||
werkdock: String,
|
werkdock: String,
|
||||||
@@ -87,33 +84,7 @@ class WerkdockBuildRunner(
|
|||||||
val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(sandbox.rootfs))
|
val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(sandbox.rootfs))
|
||||||
Files.createDirectories(envDir)
|
Files.createDirectories(envDir)
|
||||||
val archive = localArchive(sandbox.rootfs, envDir, repoDir, onAuxProcess)
|
val archive = localArchive(sandbox.rootfs, envDir, repoDir, onAuxProcess)
|
||||||
log.info("importing build environment {} as werkdock image {}", sandbox.rootfs, image)
|
log.info("loading build environment {} as werkdock image {}", sandbox.rootfs, image)
|
||||||
importImage(werkdock, archive, image, repoDir, onAuxProcess)
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* `werkdock import ARCHIVE IMAGE` (docker import semantics) creates the image from a
|
|
||||||
* rootfs archive. A werkdock that predates the verb answers `unknown command` with
|
|
||||||
* exit 125; then the older `load -i ARCHIVE --name IMAGE` does the same job, so an
|
|
||||||
* installation can update Werkator and werkdock in either order. Any other failure
|
|
||||||
* is the import's own and propagates as it did before.
|
|
||||||
*/
|
|
||||||
private fun importImage(
|
|
||||||
werkdock: String,
|
|
||||||
archive: String,
|
|
||||||
image: String,
|
|
||||||
repoDir: Path,
|
|
||||||
onAuxProcess: (Process) -> Unit,
|
|
||||||
) {
|
|
||||||
val importCommand = listOf(werkdock, "import", archive, image)
|
|
||||||
val imported = commandRunner.run(importCommand, repoDir, onProcess = onAuxProcess)
|
|
||||||
if (imported.isSuccess) {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if (imported.exitCode != WERKDOCK_CLI_ERROR || "unknown command" !in imported.stderr) {
|
|
||||||
throw GitCommandException(importCommand, imported)
|
|
||||||
}
|
|
||||||
log.info("this werkdock has no import verb yet, falling back to load -i --name")
|
|
||||||
commandRunner.runOrThrow(
|
commandRunner.runOrThrow(
|
||||||
listOf(werkdock, "load", "-i", archive, "--name", image),
|
listOf(werkdock, "load", "-i", archive, "--name", image),
|
||||||
repoDir,
|
repoDir,
|
||||||
@@ -238,8 +209,5 @@ class WerkdockBuildRunner(
|
|||||||
companion object {
|
companion object {
|
||||||
const val BUILDENV_DIR = ".git/werkator/buildenv"
|
const val BUILDENV_DIR = ".git/werkator/buildenv"
|
||||||
const val HOME_DIR = "home"
|
const val HOME_DIR = "home"
|
||||||
|
|
||||||
/** werkdock's exit code for its own errors (docker's 125), as opposed to the sandboxed command's. */
|
|
||||||
const val WERKDOCK_CLI_ERROR = 125
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,9 +1,11 @@
|
|||||||
package de.hoennig.werkator.commands
|
package de.hoennig.werkator.commands
|
||||||
|
|
||||||
import de.hoennig.werkator.build.BuildStatus
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
|
import de.hoennig.werkator.config.BuildDefinition
|
||||||
import de.hoennig.werkator.git.GitService
|
import de.hoennig.werkator.git.GitService
|
||||||
import de.hoennig.werkator.repo.RepoContext
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
import de.hoennig.werkator.repo.RepoRegistry
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
|
import de.hoennig.werkator.watcher.FollowUpTrigger
|
||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
import picocli.CommandLine.Command
|
import picocli.CommandLine.Command
|
||||||
import picocli.CommandLine.ExitCode
|
import picocli.CommandLine.ExitCode
|
||||||
@@ -27,6 +29,7 @@ class BuildCommand(
|
|||||||
private val gitService: GitService,
|
private val gitService: GitService,
|
||||||
private val consoleBuildRunner: ConsoleBuildRunner,
|
private val consoleBuildRunner: ConsoleBuildRunner,
|
||||||
private val registry: RepoRegistry,
|
private val registry: RepoRegistry,
|
||||||
|
private val followUpTrigger: FollowUpTrigger,
|
||||||
) : Callable<Int> {
|
) : Callable<Int> {
|
||||||
@Mixin
|
@Mixin
|
||||||
var repoOption = RepoOption()
|
var repoOption = RepoOption()
|
||||||
@@ -58,9 +61,33 @@ class BuildCommand(
|
|||||||
}
|
}
|
||||||
println("building branch $branch at commit ${commit.take(12)}")
|
println("building branch $branch at commit ${commit.take(12)}")
|
||||||
val status = consoleBuildRunner.buildAndStream(repo, branch, commit)
|
val status = consoleBuildRunner.buildAndStream(repo, branch, commit)
|
||||||
|
if (status == BuildStatus.SUCCESS) {
|
||||||
|
reportSkippedFollowUps(branch, commit)
|
||||||
|
}
|
||||||
return if (status == BuildStatus.SUCCESS) ExitCode.OK else ExitCode.SOFTWARE
|
return if (status == BuildStatus.SUCCESS) ExitCode.OK else ExitCode.SOFTWARE
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A one-shot build ends with its process, so it never runs the follow-up builds the
|
||||||
|
* server would enqueue after a green run (PR#23) — it says which ones instead of
|
||||||
|
* leaving the operator to wonder why nothing was deployed.
|
||||||
|
*/
|
||||||
|
private fun reportSkippedFollowUps(
|
||||||
|
branch: String,
|
||||||
|
commit: String,
|
||||||
|
) {
|
||||||
|
val followUps =
|
||||||
|
try {
|
||||||
|
followUpTrigger.followUpsOf(repo, branch, commit, BuildDefinition.DEFAULT)
|
||||||
|
} catch (e: Exception) {
|
||||||
|
System.err.println("warning: could not determine the follow-up builds (${e.message})")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (followUps.isNotEmpty()) {
|
||||||
|
println("note: the server would now run the follow-up build(s) ${followUps.joinToString(", ")}; a CLI build does not")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** A one-shot build should still work offline, from the last fetched origin state. */
|
/** A one-shot build should still work offline, from the last fetched origin state. */
|
||||||
private fun fetchBestEffort() {
|
private fun fetchBestEffort() {
|
||||||
try {
|
try {
|
||||||
|
|||||||
@@ -48,7 +48,6 @@ class InitCommand(
|
|||||||
internal var javaExecutableResolver: () -> Path = { Paths.get(System.getProperty("java.home"), "bin", "java") }
|
internal var javaExecutableResolver: () -> Path = { Paths.get(System.getProperty("java.home"), "bin", "java") }
|
||||||
|
|
||||||
override fun run() {
|
override fun run() {
|
||||||
cachedServerConfig = null
|
|
||||||
val normalizedWorkingDir = workingDir.toAbsolutePath().normalize()
|
val normalizedWorkingDir = workingDir.toAbsolutePath().normalize()
|
||||||
val root =
|
val root =
|
||||||
try {
|
try {
|
||||||
@@ -89,7 +88,7 @@ class InitCommand(
|
|||||||
if (url == null) return DetectedValues()
|
if (url == null) return DetectedValues()
|
||||||
|
|
||||||
if (url.startsWith("http")) {
|
if (url.startsWith("http")) {
|
||||||
val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/(.+?)(?:\.git)?$""")
|
val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/([^/.]+)(?:\.git)?""")
|
||||||
val match = regex.find(url)
|
val match = regex.find(url)
|
||||||
if (match != null) {
|
if (match != null) {
|
||||||
val (user, host, owner, repo) = match.destructured
|
val (user, host, owner, repo) = match.destructured
|
||||||
@@ -102,7 +101,7 @@ class InitCommand(
|
|||||||
}
|
}
|
||||||
} else if (url.contains("@") && url.contains(":")) {
|
} else if (url.contains("@") && url.contains(":")) {
|
||||||
// Assume SSH: git@host:owner/repo.git
|
// Assume SSH: git@host:owner/repo.git
|
||||||
val regex = Regex("""([^@]+)@([^:]+):([^/]+)/(.+?)(?:\.git)?$""")
|
val regex = Regex("""([^@]+)@([^:]+):([^/]+)/([^/.]+)(?:\.git)?""")
|
||||||
val match = regex.find(url)
|
val match = regex.find(url)
|
||||||
if (match != null) {
|
if (match != null) {
|
||||||
val (_, host, owner, repo) = match.destructured
|
val (_, host, owner, repo) = match.destructured
|
||||||
@@ -223,6 +222,7 @@ class InitCommand(
|
|||||||
# branches: ["*", "!master"] # names or globs; "!" excludes; default: all
|
# branches: ["*", "!master"] # names or globs; "!" excludes; default: all
|
||||||
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
|
# atTimes: ["01:00"] # daily UTC times HH:MM ("??:05" = every hour at :05)
|
||||||
# activeWithin: 24h # only branches with recent commits
|
# activeWithin: 24h # only branches with recent commits
|
||||||
|
# afterSuccessOf: test # run after every green run of that build, at its commit (pinned)
|
||||||
# run before each build
|
# run before each build
|
||||||
cleanCommand: rm -rf build
|
cleanCommand: rm -rf build
|
||||||
# shell command for each build
|
# shell command for each build
|
||||||
@@ -297,31 +297,14 @@ class InitCommand(
|
|||||||
* already loadable (re-running `init --systemd` on an installed instance); during
|
* already loadable (re-running `init --systemd` on an installed instance); during
|
||||||
* the very first bootstrap they stay unset and the defaults (no directives) apply.
|
* the very first bootstrap they stay unset and the defaults (no directives) apply.
|
||||||
*/
|
*/
|
||||||
private fun loadedSystemdConfig(root: Path): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig(root).systemd
|
private fun loadedSystemdConfig(): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig().systemd
|
||||||
|
|
||||||
/** Loaded once per run, so a broken configuration is reported once and not per caller. */
|
private fun loadedServerConfig(): de.hoennig.werkator.config.ServerConfig =
|
||||||
private var cachedServerConfig: de.hoennig.werkator.config.ServerConfig? = null
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Read from the repository root like every other file this command touches — the
|
|
||||||
* layers sit there, not in whatever directory the process happens to run in, and
|
|
||||||
* an applied fragment must reach the generated unit even when `init` is invoked
|
|
||||||
* from a subdirectory.
|
|
||||||
*
|
|
||||||
* A configuration error here is not fatal — the units are still generated with defaults —
|
|
||||||
* but it must not pass for "nothing configured": without the warning a broken `.werkator.yml`
|
|
||||||
* looks exactly like an unset `publicBaseUrl` and the host integration is skipped silently.
|
|
||||||
*/
|
|
||||||
private fun loadedServerConfig(root: Path): de.hoennig.werkator.config.ServerConfig =
|
|
||||||
cachedServerConfig ?: run {
|
|
||||||
try {
|
try {
|
||||||
configLoader.load(root).server
|
configLoader.load(Paths.get(".")).server
|
||||||
} catch (e: Exception) {
|
} catch (_: Exception) {
|
||||||
println("Warning: the effective configuration could not be loaded (${e.message})")
|
|
||||||
println(" continuing with default server settings — check the generated unit and host files")
|
|
||||||
de.hoennig.werkator.config
|
de.hoennig.werkator.config
|
||||||
.ServerConfig()
|
.ServerConfig()
|
||||||
}.also { cachedServerConfig = it }
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun createSystemdFiles(
|
private fun createSystemdFiles(
|
||||||
@@ -345,8 +328,8 @@ class InitCommand(
|
|||||||
javaExecutable = javaExecutableResolver(),
|
javaExecutable = javaExecutableResolver(),
|
||||||
jarPath = jarPath,
|
jarPath = jarPath,
|
||||||
envFile = envFile,
|
envFile = envFile,
|
||||||
memoryMax = loadedSystemdConfig(root).memoryMax,
|
memoryMax = loadedSystemdConfig().memoryMax,
|
||||||
tasksMax = loadedSystemdConfig(root).tasksMax,
|
tasksMax = loadedSystemdConfig().tasksMax,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}")
|
println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}")
|
||||||
@@ -369,7 +352,7 @@ class InitCommand(
|
|||||||
|
|
||||||
// generated host integration like the units: only meaningful behind a web
|
// generated host integration like the units: only meaningful behind a web
|
||||||
// frontend, so it needs a public base URL; unused elsewhere and harmless
|
// frontend, so it needs a public base URL; unused elsewhere and harmless
|
||||||
val server = loadedServerConfig(root)
|
val server = loadedServerConfig()
|
||||||
if (server.publicBaseUrl.isNotBlank()) {
|
if (server.publicBaseUrl.isNotBlank()) {
|
||||||
val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME)
|
val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME)
|
||||||
htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port))
|
htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port))
|
||||||
|
|||||||
@@ -89,8 +89,9 @@ data class BuildDefinition(
|
|||||||
* When a build runs and for which branches — the `trigger` block of a build definition,
|
* When a build runs and for which branches — the `trigger` block of a build definition,
|
||||||
* and the one part of it that is never inherited from `builds.default`.
|
* and the one part of it that is never inherited from `builds.default`.
|
||||||
*
|
*
|
||||||
* A definition with neither [onPush] nor [atTimes] never triggers automatically; that is
|
* A definition with neither [onPush] nor [atTimes] nor [afterSuccessOf] never triggers
|
||||||
* how `builds.default` is written when it is meant as a settings base only.
|
* automatically; that is how `builds.default` is written when it is meant as a settings
|
||||||
|
* base only.
|
||||||
*/
|
*/
|
||||||
data class TriggerConfig(
|
data class TriggerConfig(
|
||||||
/** Build every new commit of the selected branches. */
|
/** Build every new commit of the selected branches. */
|
||||||
@@ -112,7 +113,19 @@ data class TriggerConfig(
|
|||||||
* empty applies no age filter. Combines with [branches] as an intersection.
|
* empty applies no age filter. Combines with [branches] as an intersection.
|
||||||
*/
|
*/
|
||||||
val activeWithin: String = "",
|
val activeWithin: String = "",
|
||||||
|
/**
|
||||||
|
* Name of another definition of this configuration — the *predecessor*: this build
|
||||||
|
* runs on the predecessor's branch at the predecessor's commit whenever a run of it
|
||||||
|
* ends with `SUCCESS`, whatever started that run (PR#23). Empty means none. Sits in
|
||||||
|
* the trigger block because it says *when* this build runs, so it is never inherited;
|
||||||
|
* and it is host-pinned, because a follow-up build is the host's way to hand
|
||||||
|
* real-world effects — deployment targets, credentials — to a green commit.
|
||||||
|
*/
|
||||||
|
val afterSuccessOf: String = "",
|
||||||
) {
|
) {
|
||||||
|
/** True when this build follows another one, see [afterSuccessOf]. */
|
||||||
|
fun isFollowUp(): Boolean = afterSuccessOf.isNotBlank()
|
||||||
|
|
||||||
/** True when [branch] matches the [branches] patterns (or none are configured) and none excludes it. */
|
/** True when [branch] matches the [branches] patterns (or none are configured) and none excludes it. */
|
||||||
fun selectsByName(branch: String): Boolean {
|
fun selectsByName(branch: String): Boolean {
|
||||||
val (excluding, including) = branches.partition { it.startsWith(EXCLUDE_PREFIX) }
|
val (excluding, including) = branches.partition { it.startsWith(EXCLUDE_PREFIX) }
|
||||||
|
|||||||
@@ -90,7 +90,8 @@ class ConfigLoader(
|
|||||||
fun loadForWorktree(
|
fun loadForWorktree(
|
||||||
workingDir: Path,
|
workingDir: Path,
|
||||||
worktreeDir: Path,
|
worktreeDir: Path,
|
||||||
): WerkatorConfig = withBranchLayer(workingDir, loadFile(worktreeDir.resolve(ConfigFiles.firstExisting(worktreeDir)).toFile()))
|
branch: String? = null,
|
||||||
|
): WerkatorConfig = withBranchLayer(workingDir, loadFile(worktreeDir.resolve(ConfigFiles.firstExisting(worktreeDir)).toFile()), branch)
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The primary/`.git` config with the committed `.werkator.yml` of one branch
|
* The primary/`.git` config with the committed `.werkator.yml` of one branch
|
||||||
@@ -107,30 +108,53 @@ class ConfigLoader(
|
|||||||
* (`requirePullRequest`, which decides whether the branch is built at all).
|
* (`requirePullRequest`, which decides whether the branch is built at all).
|
||||||
* They are stripped from the branch layer before merging, so a branch can neither
|
* They are stripped from the branch layer before merging, so a branch can neither
|
||||||
* escape its container, nor bypass its own pull-request gate, nor raise the global
|
* escape its container, nor bypass its own pull-request gate, nor raise the global
|
||||||
* concurrency, nor reach the credentials.
|
* concurrency, nor reach the credentials. The trigger of a follow-up build is pinned
|
||||||
|
* the same way (PR#23): a branch may say what its deployment does, never that — or
|
||||||
|
* where — it happens. [branch] only names the branch in the warnings.
|
||||||
*/
|
*/
|
||||||
fun loadWithBranchLayer(
|
fun loadWithBranchLayer(
|
||||||
workingDir: Path,
|
workingDir: Path,
|
||||||
branchConfigYaml: String?,
|
branchConfigYaml: String?,
|
||||||
): WerkatorConfig = withBranchLayer(workingDir, parseYaml(branchConfigYaml))
|
branch: String? = null,
|
||||||
|
): WerkatorConfig = withBranchLayer(workingDir, parseYaml(branchConfigYaml), branch)
|
||||||
|
|
||||||
private fun withBranchLayer(
|
private fun withBranchLayer(
|
||||||
workingDir: Path,
|
workingDir: Path,
|
||||||
branchLayer: Map<String, Any?>,
|
branchLayer: Map<String, Any?>,
|
||||||
|
branch: String?,
|
||||||
): WerkatorConfig {
|
): WerkatorConfig {
|
||||||
// scoped to this branch: an incompatible branch config fails its own builds and
|
// scoped to this branch: an incompatible branch config fails its own builds and
|
||||||
// must never stop the server or hold up the branches that are fine
|
// must never stop the server or hold up the branches that are fine
|
||||||
checkVersion(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
|
checkVersion(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
|
||||||
checkTriggerBlocks(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
|
checkTriggerBlocks(branchLayer, "the committed .werkator.yml of this branch", BRANCH_HINT)
|
||||||
return toConfig(deepMerge(loadRaw(workingDir), stripPinned(branchLayer)))
|
val primary = loadRaw(workingDir)
|
||||||
|
return toConfig(deepMerge(primary, stripPinned(branchLayer, primary, branch)), MissingPredecessor.WARN)
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun toConfig(raw: Map<String, Any?>): WerkatorConfig {
|
/**
|
||||||
|
* What a follow-up whose predecessor no definition has means for this load, see
|
||||||
|
* [checkFollowUps].
|
||||||
|
*/
|
||||||
|
private enum class MissingPredecessor {
|
||||||
|
/** The primary configuration: a deployment that could never fire refuses the start. */
|
||||||
|
REFUSE,
|
||||||
|
|
||||||
|
/** A branch layer on top: the branch renamed or dropped the build the host's trigger names, and only loses its follow-up. */
|
||||||
|
WARN,
|
||||||
|
|
||||||
|
/** A fragment checked on its own: the predecessor may well live in the project config it is merged with later. */
|
||||||
|
SKIP,
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun toConfig(
|
||||||
|
raw: Map<String, Any?>,
|
||||||
|
missingPredecessor: MissingPredecessor = MissingPredecessor.REFUSE,
|
||||||
|
): WerkatorConfig {
|
||||||
val config =
|
val config =
|
||||||
if (raw.isEmpty()) {
|
if (raw.isEmpty()) {
|
||||||
WerkatorConfig()
|
WerkatorConfig()
|
||||||
} else {
|
} else {
|
||||||
yaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw)), WerkatorConfig::class.java)
|
yaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw), missingPredecessor), WerkatorConfig::class.java)
|
||||||
}
|
}
|
||||||
return defaultPublicBaseUrl(config)
|
return defaultPublicBaseUrl(config)
|
||||||
}
|
}
|
||||||
@@ -178,7 +202,11 @@ class ConfigLoader(
|
|||||||
* second as soon as the committed configuration carries them.
|
* second as soon as the committed configuration carries them.
|
||||||
*/
|
*/
|
||||||
@Suppress("UNCHECKED_CAST")
|
@Suppress("UNCHECKED_CAST")
|
||||||
private fun stripPinned(branchLayer: Map<String, Any?>): Map<String, Any?> {
|
private fun stripPinned(
|
||||||
|
branchLayer: Map<String, Any?>,
|
||||||
|
primary: Map<String, Any?>,
|
||||||
|
branch: String?,
|
||||||
|
): Map<String, Any?> {
|
||||||
if (branchLayer.isEmpty()) {
|
if (branchLayer.isEmpty()) {
|
||||||
return branchLayer
|
return branchLayer
|
||||||
}
|
}
|
||||||
@@ -188,9 +216,55 @@ class ConfigLoader(
|
|||||||
val entries = result[section] as? Map<String, Any?> ?: continue
|
val entries = result[section] as? Map<String, Any?> ?: continue
|
||||||
result[section] = entries.mapValues { (_, value) -> stripPinnedSettings(value) }
|
result[section] = entries.mapValues { (_, value) -> stripPinnedSettings(value) }
|
||||||
}
|
}
|
||||||
|
(result["builds"] as? Map<String, Any?>)?.let { builds ->
|
||||||
|
val hostBuilds = primary["builds"] as? Map<String, Any?> ?: emptyMap()
|
||||||
|
result["builds"] = builds.mapValues { (name, value) -> stripPinnedTrigger(name, value, hostBuilds[name], branch) }
|
||||||
|
}
|
||||||
return result
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The follow-up part of the pinning (PR#23): a branch's definition loses its
|
||||||
|
* `afterSuccessOf`, and where the host's definition of the same name is a follow-up,
|
||||||
|
* the branch's whole `trigger` block — otherwise a branch could widen the host's
|
||||||
|
* selector to include itself, and deploy itself with the host's credentials. Said
|
||||||
|
* out loud, because a trigger the branch wrote and does not see in effect is a
|
||||||
|
* question it would otherwise ask the log in vain.
|
||||||
|
*/
|
||||||
|
@Suppress("UNCHECKED_CAST")
|
||||||
|
private fun stripPinnedTrigger(
|
||||||
|
name: String,
|
||||||
|
value: Any?,
|
||||||
|
hostDefinition: Any?,
|
||||||
|
branch: String?,
|
||||||
|
): Any? {
|
||||||
|
val definition = value as? Map<String, Any?> ?: return value
|
||||||
|
val trigger = definition["trigger"] as? Map<String, Any?> ?: return value
|
||||||
|
val where = branch?.let { "branch '$it'" } ?: "this branch"
|
||||||
|
if (predecessorOf(hostDefinition) != null) {
|
||||||
|
log.warn(
|
||||||
|
"ignoring the trigger block of builds.{} in the committed {} of {}: the host defines that build as a follow-up, " +
|
||||||
|
"and when and where a follow-up runs is the host's decision alone",
|
||||||
|
name,
|
||||||
|
ConfigFiles.COMMITTED,
|
||||||
|
where,
|
||||||
|
)
|
||||||
|
return definition - "trigger"
|
||||||
|
}
|
||||||
|
if (predecessorOf(definition) == null) {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
log.warn(
|
||||||
|
"ignoring builds.{}.trigger.afterSuccessOf in the committed {} of {}: a branch cannot make a build follow another, " +
|
||||||
|
"only the host can",
|
||||||
|
name,
|
||||||
|
ConfigFiles.COMMITTED,
|
||||||
|
where,
|
||||||
|
)
|
||||||
|
val stripped = trigger - "afterSuccessOf"
|
||||||
|
return if (stripped.isEmpty()) definition - "trigger" else definition + ("trigger" to stripped)
|
||||||
|
}
|
||||||
|
|
||||||
@Suppress("UNCHECKED_CAST")
|
@Suppress("UNCHECKED_CAST")
|
||||||
private fun stripPinnedSettings(value: Any?): Any? {
|
private fun stripPinnedSettings(value: Any?): Any? {
|
||||||
val entry = value as? Map<String, Any?> ?: return value
|
val entry = value as? Map<String, Any?> ?: return value
|
||||||
@@ -225,12 +299,16 @@ class ConfigLoader(
|
|||||||
* an empty docker policy and run natively on the host, which is exactly the escape
|
* an empty docker policy and run natively on the host, which is exactly the escape
|
||||||
* the pinned keys exist to prevent.
|
* the pinned keys exist to prevent.
|
||||||
*/
|
*/
|
||||||
private fun resolveBuildSections(raw: Map<String, Any?>): Map<String, Any?> {
|
private fun resolveBuildSections(
|
||||||
|
raw: Map<String, Any?>,
|
||||||
|
missingPredecessor: MissingPredecessor,
|
||||||
|
): Map<String, Any?> {
|
||||||
@Suppress("UNCHECKED_CAST")
|
@Suppress("UNCHECKED_CAST")
|
||||||
val definitions = raw["builds"] as? Map<String, Any?> ?: emptyMap()
|
val definitions = raw["builds"] as? Map<String, Any?> ?: emptyMap()
|
||||||
if (definitions.isEmpty()) {
|
if (definitions.isEmpty()) {
|
||||||
return mergeBranchDefaults(raw)
|
return mergeBranchDefaults(raw)
|
||||||
}
|
}
|
||||||
|
checkFollowUps(definitions, missingPredecessor)
|
||||||
if (raw.containsKey("branches") && warnedSections.add(LEGACY_BRANCHES_WARNING)) {
|
if (raw.containsKey("branches") && warnedSections.add(LEGACY_BRANCHES_WARNING)) {
|
||||||
log.warn(
|
log.warn(
|
||||||
"ignoring the branches section: this configuration defines builds, and a build definition " +
|
"ignoring the branches section: this configuration defines builds, and a build definition " +
|
||||||
@@ -252,13 +330,59 @@ class ConfigLoader(
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
if (warnedSections.add(NO_TRIGGER_WARNING)) {
|
if (warnedSections.add(NO_TRIGGER_WARNING)) {
|
||||||
log.warn("no build defines onPush or atTimes; the watcher will never start a build on its own")
|
log.warn("no build defines onPush, atTimes, or afterSuccessOf; the watcher will never start a build on its own")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun isTriggered(definition: Any?): Boolean {
|
private fun isTriggered(definition: Any?): Boolean {
|
||||||
val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return false
|
val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return false
|
||||||
return trigger["onPush"] == true || (trigger["atTimes"] as? List<*>)?.isNotEmpty() == true
|
return trigger["onPush"] == true ||
|
||||||
|
(trigger["atTimes"] as? List<*>)?.isNotEmpty() == true ||
|
||||||
|
predecessorOf(definition) != null
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun predecessorOf(definition: Any?): String? {
|
||||||
|
val trigger = (definition as? Map<*, *>)?.get("trigger") as? Map<*, *> ?: return null
|
||||||
|
return (trigger["afterSuccessOf"] as? String)?.takeIf { it.isNotBlank() }
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A follow-up build that could never fire must not exist, for the same reason a flat
|
||||||
|
* trigger key is refused: a deployment that silently never runs is worse than a
|
||||||
|
* configuration that refuses to load (PR#23). Refused are a predecessor no definition
|
||||||
|
* has, and a cycle of follow-ups (a build following itself included) — a cycle is a
|
||||||
|
* configuration error whoever wrote it, while a missing predecessor depends on what
|
||||||
|
* is being loaded ([MissingPredecessor]).
|
||||||
|
*/
|
||||||
|
private fun checkFollowUps(
|
||||||
|
definitions: Map<String, Any?>,
|
||||||
|
missingPredecessor: MissingPredecessor,
|
||||||
|
) {
|
||||||
|
val predecessors = definitions.mapValues { (_, definition) -> predecessorOf(definition) }
|
||||||
|
val effective = definitions.keys + BuildDefinition.DEFAULT
|
||||||
|
for ((name, predecessor) in predecessors) {
|
||||||
|
if (predecessor == null || predecessor in effective) continue
|
||||||
|
val message = "builds.$name follows '$predecessor' (trigger.afterSuccessOf), but no build of that name is defined"
|
||||||
|
when (missingPredecessor) {
|
||||||
|
MissingPredecessor.REFUSE -> throw ConfigFormatException("$message. Name an existing build, or remove the follow-up.")
|
||||||
|
MissingPredecessor.WARN -> log.warn("$message on this branch; the follow-up will not run for it")
|
||||||
|
MissingPredecessor.SKIP -> {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (start in predecessors.keys) {
|
||||||
|
val path = mutableListOf(start)
|
||||||
|
var current = predecessors[start]
|
||||||
|
while (current != null && current !in path) {
|
||||||
|
path += current
|
||||||
|
current = predecessors[current]
|
||||||
|
}
|
||||||
|
if (current == start) {
|
||||||
|
throw ConfigFormatException(
|
||||||
|
"builds.$start follows itself through trigger.afterSuccessOf (${path.joinToString(" -> ")} -> $start); " +
|
||||||
|
"a follow-up build cannot wait for its own success.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -426,7 +550,10 @@ class ConfigLoader(
|
|||||||
checkVersion(raw, fragment.toString(), ROLLBACK_HINT)
|
checkVersion(raw, fragment.toString(), ROLLBACK_HINT)
|
||||||
checkTriggerBlocks(raw, fragment.toString(), ROLLBACK_HINT)
|
checkTriggerBlocks(raw, fragment.toString(), ROLLBACK_HINT)
|
||||||
try {
|
try {
|
||||||
strictYaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw)), WerkatorConfig::class.java)
|
strictYaml.convertValue(
|
||||||
|
resolveBuildSections(dropNonDefinitionBuilds(raw), MissingPredecessor.SKIP),
|
||||||
|
WerkatorConfig::class.java,
|
||||||
|
)
|
||||||
} catch (e: IllegalArgumentException) {
|
} catch (e: IllegalArgumentException) {
|
||||||
throw IllegalArgumentException(
|
throw IllegalArgumentException(
|
||||||
"instance fragment $fragment does not match the configuration schema: ${e.message}",
|
"instance fragment $fragment does not match the configuration schema: ${e.message}",
|
||||||
@@ -610,7 +737,7 @@ class ConfigLoader(
|
|||||||
private val TRIGGER_KEYS = setOf("trigger")
|
private val TRIGGER_KEYS = setOf("trigger")
|
||||||
|
|
||||||
/** The keys that moved into [TRIGGER_KEYS]; still writing them flat is refused, not ignored. */
|
/** The keys that moved into [TRIGGER_KEYS]; still writing them flat is refused, not ignored. */
|
||||||
private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin")
|
private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin", "afterSuccessOf")
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Top-level sections owned by the instance once a home config exists (ADR 0009):
|
* Top-level sections owned by the instance once a home config exists (ADR 0009):
|
||||||
|
|||||||
@@ -0,0 +1,134 @@
|
|||||||
|
package de.hoennig.werkator.watcher
|
||||||
|
|
||||||
|
import de.hoennig.werkator.build.BuildExecutor
|
||||||
|
import de.hoennig.werkator.build.BuildResult
|
||||||
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
|
import de.hoennig.werkator.build.BuildStatusChangedEvent
|
||||||
|
import de.hoennig.werkator.config.BuildDefinition
|
||||||
|
import de.hoennig.werkator.config.ConfigFiles
|
||||||
|
import de.hoennig.werkator.config.ConfigLoader
|
||||||
|
import de.hoennig.werkator.git.GitService
|
||||||
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import org.slf4j.LoggerFactory
|
||||||
|
import org.springframework.context.event.EventListener
|
||||||
|
import org.springframework.stereotype.Component
|
||||||
|
import java.time.Clock
|
||||||
|
import java.util.concurrent.atomic.AtomicBoolean
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs the follow-up builds (PR#23): whenever a build ends with `SUCCESS`, every
|
||||||
|
* definition of that branch whose `trigger.afterSuccessOf` names the finished build —
|
||||||
|
* and whose selector selects the branch — is enqueued on the same branch at the *same
|
||||||
|
* commit*, never at the branch's current origin head, so a deployment always ships the
|
||||||
|
* commit that was tested. Every green run counts, whatever started it, and a repeated
|
||||||
|
* green run of the same commit triggers again: a run of a green build that is silently
|
||||||
|
* not deployed would be more confusing than a redundant deployment.
|
||||||
|
*
|
||||||
|
* The definitions are resolved with the branch's committed config at the finished
|
||||||
|
* build's commit — the same layering the watcher applies — so the follow-up's command
|
||||||
|
* comes with the repository while its trigger stays the host's (pinned by
|
||||||
|
* [ConfigLoader]). The pull-request gate is not consulted: the predecessor passed it for
|
||||||
|
* this very commit, and the host's selector is the follow-up's own gate.
|
||||||
|
*
|
||||||
|
* Armed by [Watcher.start] and disarmed by [Watcher.stop], so a CLI `build` — whose
|
||||||
|
* process ends with its build — never enqueues a follow-up into a JVM that is about to
|
||||||
|
* exit; the CLI names the follow-ups the server would have run instead.
|
||||||
|
*/
|
||||||
|
@Component
|
||||||
|
class FollowUpTrigger(
|
||||||
|
private val gitService: GitService,
|
||||||
|
private val configLoader: ConfigLoader,
|
||||||
|
private val buildExecutor: BuildExecutor,
|
||||||
|
private val clock: Clock,
|
||||||
|
) {
|
||||||
|
private val log = LoggerFactory.getLogger(FollowUpTrigger::class.java)
|
||||||
|
|
||||||
|
private val armed = AtomicBoolean(false)
|
||||||
|
|
||||||
|
fun arm() {
|
||||||
|
armed.set(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun disarm() {
|
||||||
|
armed.set(false)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun isArmed(): Boolean = armed.get()
|
||||||
|
|
||||||
|
@EventListener
|
||||||
|
fun onBuildStatusChanged(event: BuildStatusChangedEvent) {
|
||||||
|
if (!armed.get() || event.result.status != BuildStatus.SUCCESS) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
val result = event.result
|
||||||
|
try {
|
||||||
|
for (name in followUpsOf(event.repo, result)) {
|
||||||
|
log.info(
|
||||||
|
"[{}] enqueueing follow-up build {} of branch {} at commit {}, after {}",
|
||||||
|
event.repo.name,
|
||||||
|
name,
|
||||||
|
result.branch,
|
||||||
|
result.commit,
|
||||||
|
result.build,
|
||||||
|
)
|
||||||
|
buildExecutor.startBuild(event.repo, result.branch, result.commit, name)
|
||||||
|
}
|
||||||
|
} catch (e: Exception) {
|
||||||
|
log.error("[{}] could not enqueue the follow-ups of {} at {}", event.repo.name, result.name, result.commit, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The names of the builds that follow a green [result], in the order of their definitions; nothing is enqueued. */
|
||||||
|
fun followUpsOf(
|
||||||
|
repo: RepoContext,
|
||||||
|
result: BuildResult,
|
||||||
|
): List<String> = followUpsOf(repo, result.branch, result.commit, result.build)
|
||||||
|
|
||||||
|
/** The names of the builds that follow a green run of [build] on [branch] at [commit]; nothing is enqueued. */
|
||||||
|
fun followUpsOf(
|
||||||
|
repo: RepoContext,
|
||||||
|
branch: String,
|
||||||
|
commit: String,
|
||||||
|
build: String,
|
||||||
|
): List<String> {
|
||||||
|
val workingDir = repo.workingDir
|
||||||
|
val definitions = definitionsAt(repo, branch, commit)
|
||||||
|
val headCommittedAt = lazy { gitService.originBranchCommitTimes(workingDir)[branch] }
|
||||||
|
return definitions
|
||||||
|
.filter { (_, definition) -> definition.trigger.afterSuccessOf == build }
|
||||||
|
.filter { (_, definition) -> definition.trigger.selects(branch, { headCommittedAt.value }, clock.instant()) }
|
||||||
|
.keys
|
||||||
|
.toList()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The branch's definitions at [commit] — not at its head, which may have moved on
|
||||||
|
* since the predecessor started. An unreadable branch config falls back to the
|
||||||
|
* primary definitions, like the watcher does.
|
||||||
|
*/
|
||||||
|
private fun definitionsAt(
|
||||||
|
repo: RepoContext,
|
||||||
|
branch: String,
|
||||||
|
commit: String,
|
||||||
|
): Map<String, BuildDefinition> {
|
||||||
|
val workingDir = repo.workingDir
|
||||||
|
return try {
|
||||||
|
configLoader
|
||||||
|
.loadWithBranchLayer(
|
||||||
|
workingDir,
|
||||||
|
ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) },
|
||||||
|
branch,
|
||||||
|
).effectiveBuildDefinitions()
|
||||||
|
} catch (e: Exception) {
|
||||||
|
log.warn(
|
||||||
|
"[{}] ignoring the committed {} of branch {} at {} for its follow-ups: {}",
|
||||||
|
repo.name,
|
||||||
|
ConfigFiles.COMMITTED,
|
||||||
|
branch,
|
||||||
|
commit,
|
||||||
|
e.message ?: e.javaClass.simpleName,
|
||||||
|
)
|
||||||
|
configLoader.load(workingDir).effectiveBuildDefinitions()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -41,6 +41,7 @@ class Watcher(
|
|||||||
private val buildExecutor: BuildExecutor,
|
private val buildExecutor: BuildExecutor,
|
||||||
private val configLoader: ConfigLoader,
|
private val configLoader: ConfigLoader,
|
||||||
private val clock: Clock,
|
private val clock: Clock,
|
||||||
|
private val followUpTrigger: FollowUpTrigger,
|
||||||
) {
|
) {
|
||||||
private val log = LoggerFactory.getLogger(Watcher::class.java)
|
private val log = LoggerFactory.getLogger(Watcher::class.java)
|
||||||
|
|
||||||
@@ -83,11 +84,14 @@ class Watcher(
|
|||||||
* Runs the startup recovery of every repository and schedules the poll loop with the
|
* Runs the startup recovery of every repository and schedules the poll loop with the
|
||||||
* fixed delay `watcher.pollInterval` — one loop, one delay: the instance's setting,
|
* fixed delay `watcher.pollInterval` — one loop, one delay: the instance's setting,
|
||||||
* which every repository's effective config carries; the first poll runs immediately.
|
* which every repository's effective config carries; the first poll runs immediately.
|
||||||
|
* Arms the [FollowUpTrigger] first, so the recovery's re-enqueued builds get their
|
||||||
|
* follow-ups too.
|
||||||
*/
|
*/
|
||||||
@Synchronized
|
@Synchronized
|
||||||
fun start(repos: List<RepoContext>) {
|
fun start(repos: List<RepoContext>) {
|
||||||
check(scheduler == null) { "watcher is already running" }
|
check(scheduler == null) { "watcher is already running" }
|
||||||
require(repos.isNotEmpty()) { "no repository to watch" }
|
require(repos.isNotEmpty()) { "no repository to watch" }
|
||||||
|
followUpTrigger.arm()
|
||||||
repos.forEach { recoverSafely(it) }
|
repos.forEach { recoverSafely(it) }
|
||||||
val interval = DurationParser.parse(configLoader.load(repos.first().workingDir).watcher.pollInterval)
|
val interval = DurationParser.parse(configLoader.load(repos.first().workingDir).watcher.pollInterval)
|
||||||
scheduler =
|
scheduler =
|
||||||
@@ -111,6 +115,7 @@ class Watcher(
|
|||||||
|
|
||||||
@Synchronized
|
@Synchronized
|
||||||
fun stop() {
|
fun stop() {
|
||||||
|
followUpTrigger.disarm()
|
||||||
scheduler?.shutdownNow()
|
scheduler?.shutdownNow()
|
||||||
scheduler = null
|
scheduler = null
|
||||||
state = state.copy(running = false)
|
state = state.copy(running = false)
|
||||||
@@ -324,6 +329,7 @@ class Watcher(
|
|||||||
.loadWithBranchLayer(
|
.loadWithBranchLayer(
|
||||||
workingDir,
|
workingDir,
|
||||||
ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) },
|
ConfigFiles.readCommitted { gitService.showFileAtCommit(commit, it, workingDir) },
|
||||||
|
branch,
|
||||||
).effectiveBuildDefinitions()
|
).effectiveBuildDefinitions()
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
log.warn(
|
log.warn(
|
||||||
|
|||||||
@@ -292,6 +292,42 @@ class BuildExecutorTest : FunSpec() {
|
|||||||
.build shouldBe "default"
|
.build shouldBe "default"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test("a follow-up build runs in its branch's worktree after its predecessor") {
|
||||||
|
val h =
|
||||||
|
Harness(
|
||||||
|
"""
|
||||||
|
executor:
|
||||||
|
maxConcurrent: 2
|
||||||
|
builds:
|
||||||
|
default:
|
||||||
|
trigger:
|
||||||
|
onPush: true
|
||||||
|
cleanCommand: ""
|
||||||
|
buildCommand: "sleep 1; echo built > output.txt"
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: default
|
||||||
|
buildCommand: "cat output.txt"
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
|
||||||
|
h.executor.startBuild(h.repo, "main", "c1", "default")
|
||||||
|
h.executor.startBuild(h.repo, "main", "c1", "deploy")
|
||||||
|
|
||||||
|
awaitStatus(h, "main@deploy", BuildStatus.SUCCESS)
|
||||||
|
awaitIdle(h)
|
||||||
|
// same branch, same commit: the same worktree, and the follow-up saw the predecessor's output
|
||||||
|
h.workspaceCalls shouldContainExactly listOf("main" to "c1", "main" to "c1")
|
||||||
|
val predecessor = h.repository.latestFor("main").shouldNotBeNull()
|
||||||
|
val followUp = h.repository.latestFor("main@deploy").shouldNotBeNull()
|
||||||
|
predecessor.status shouldBe BuildStatus.SUCCESS
|
||||||
|
followUp.runningSince
|
||||||
|
.shouldNotBeNull()
|
||||||
|
.isBefore(predecessor.runningSince.shouldNotBeNull())
|
||||||
|
.shouldBeFalse()
|
||||||
|
Files.readString(h.workingDir.resolve("output.txt")).trim() shouldBe "built"
|
||||||
|
}
|
||||||
|
|
||||||
test("a build whose definition was removed from the config falls back to the branch's settings") {
|
test("a build whose definition was removed from the config falls back to the branch's settings") {
|
||||||
val h = harness(buildCommand = "echo regular-\$branch")
|
val h = harness(buildCommand = "echo regular-\$branch")
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package de.hoennig.werkator.build
|
|||||||
|
|
||||||
import de.hoennig.werkator.config.BranchConfig
|
import de.hoennig.werkator.config.BranchConfig
|
||||||
import de.hoennig.werkator.config.WerkdockConfig
|
import de.hoennig.werkator.config.WerkdockConfig
|
||||||
import de.hoennig.werkator.git.GitCommandException
|
|
||||||
import de.hoennig.werkator.git.GitCommandResult
|
import de.hoennig.werkator.git.GitCommandResult
|
||||||
import de.hoennig.werkator.git.GitCommandRunner
|
import de.hoennig.werkator.git.GitCommandRunner
|
||||||
import io.kotest.assertions.throwables.shouldThrow
|
import io.kotest.assertions.throwables.shouldThrow
|
||||||
@@ -38,11 +37,7 @@ class WerkdockBuildRunnerTest : FunSpec() {
|
|||||||
|
|
||||||
private fun imageName(rootfs: String = "/srv/buildenv.tar.zst"): String = "werkator-buildenv-${rootfs.sha12()}"
|
private fun imageName(rootfs: String = "/srv/buildenv.tar.zst"): String = "werkator-buildenv-${rootfs.sha12()}"
|
||||||
|
|
||||||
private fun importCommand(): List<String> = listOf("werkdock", "import", "/srv/buildenv.tar.zst", imageName())
|
/** The image is already loaded: `werkdock images` lists it, so no load runs. */
|
||||||
|
|
||||||
private fun loadCommand(): List<String> = listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName())
|
|
||||||
|
|
||||||
/** The image is already there: `werkdock images` lists it, so no import runs. */
|
|
||||||
private fun givenImageLoaded(rootfs: String = "/srv/buildenv.tar.zst") {
|
private fun givenImageLoaded(rootfs: String = "/srv/buildenv.tar.zst") {
|
||||||
every { commandRunner.runOrThrow(listOf("werkdock", "images"), repoDir, any(), any()) } returns
|
every { commandRunner.runOrThrow(listOf("werkdock", "images"), repoDir, any(), any()) } returns
|
||||||
GitCommandResult(0, imageName(rootfs) + "\n", "")
|
GitCommandResult(0, imageName(rootfs) + "\n", "")
|
||||||
@@ -93,47 +88,34 @@ class WerkdockBuildRunnerTest : FunSpec() {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
test("imports the image once when werkdock does not know it yet") {
|
test("loads the image once when werkdock does not know it yet") {
|
||||||
givenImageMissing()
|
givenImageMissing()
|
||||||
every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns GitCommandResult(0, "", "")
|
every {
|
||||||
|
commandRunner.runOrThrow(
|
||||||
|
listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName()),
|
||||||
|
repoDir,
|
||||||
|
any(),
|
||||||
|
any(),
|
||||||
|
)
|
||||||
|
} returns GitCommandResult(0, "", "")
|
||||||
|
|
||||||
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
|
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
|
||||||
|
|
||||||
verify { commandRunner.run(importCommand(), repoDir, any(), any()) }
|
verify {
|
||||||
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
|
commandRunner.runOrThrow(
|
||||||
|
listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName()),
|
||||||
|
repoDir,
|
||||||
|
any(),
|
||||||
|
any(),
|
||||||
|
)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
test("falls back to load -i --name on a werkdock without the import verb") {
|
test("does not load an image werkdock already has") {
|
||||||
givenImageMissing()
|
|
||||||
every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns
|
|
||||||
GitCommandResult(125, "", "werkdock: unknown command \"import\"\n")
|
|
||||||
every { commandRunner.runOrThrow(loadCommand(), repoDir, any(), any()) } returns GitCommandResult(0, "", "")
|
|
||||||
|
|
||||||
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
|
|
||||||
|
|
||||||
verify { commandRunner.runOrThrow(loadCommand(), repoDir, any(), any()) }
|
|
||||||
}
|
|
||||||
|
|
||||||
test("propagates an import failure that is not a missing verb") {
|
|
||||||
givenImageMissing()
|
|
||||||
every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns
|
|
||||||
GitCommandResult(125, "", "werkdock: unpacking /srv/buildenv.tar.zst failed\n")
|
|
||||||
|
|
||||||
val exception =
|
|
||||||
shouldThrow<GitCommandException> {
|
|
||||||
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
|
|
||||||
}
|
|
||||||
|
|
||||||
exception.message shouldContain "unpacking"
|
|
||||||
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
|
|
||||||
}
|
|
||||||
|
|
||||||
test("does not import an image werkdock already has") {
|
|
||||||
givenImageLoaded()
|
givenImageLoaded()
|
||||||
|
|
||||||
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
|
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
|
||||||
|
|
||||||
verify(exactly = 0) { commandRunner.run(match { "import" in it }, any(), any(), any()) }
|
|
||||||
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
|
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -227,7 +209,7 @@ class WerkdockBuildRunnerTest : FunSpec() {
|
|||||||
exception.message shouldContain "werkdock.rootfs"
|
exception.message shouldContain "werkdock.rootfs"
|
||||||
}
|
}
|
||||||
|
|
||||||
test("downloads a URL rootfs once before importing it") {
|
test("downloads a URL rootfs once before loading it") {
|
||||||
val url = "https://example.test/buildenv.tar.zst"
|
val url = "https://example.test/buildenv.tar.zst"
|
||||||
val downloadTarget =
|
val downloadTarget =
|
||||||
repoDir
|
repoDir
|
||||||
@@ -237,7 +219,7 @@ class WerkdockBuildRunnerTest : FunSpec() {
|
|||||||
givenImageMissing()
|
givenImageMissing()
|
||||||
every { commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) } returns
|
every { commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) } returns
|
||||||
GitCommandResult(0, "", "")
|
GitCommandResult(0, "", "")
|
||||||
every { commandRunner.run(match { "import" in it }, any(), any(), any()) } returns
|
every { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) } returns
|
||||||
GitCommandResult(0, "", "")
|
GitCommandResult(0, "", "")
|
||||||
|
|
||||||
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig(rootfs = url))
|
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig(rootfs = url))
|
||||||
@@ -246,8 +228,8 @@ class WerkdockBuildRunnerTest : FunSpec() {
|
|||||||
commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any())
|
commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any())
|
||||||
}
|
}
|
||||||
verify {
|
verify {
|
||||||
commandRunner.run(
|
commandRunner.runOrThrow(
|
||||||
listOf("werkdock", "import", downloadTarget.toString(), "werkator-buildenv-${url.sha12()}"),
|
listOf("werkdock", "load", "-i", downloadTarget.toString(), "--name", "werkator-buildenv-${url.sha12()}"),
|
||||||
repoDir,
|
repoDir,
|
||||||
any(),
|
any(),
|
||||||
any(),
|
any(),
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import de.hoennig.werkator.build.BuildStatus
|
|||||||
import de.hoennig.werkator.git.GitService
|
import de.hoennig.werkator.git.GitService
|
||||||
import de.hoennig.werkator.repo.RepoContext
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
import de.hoennig.werkator.repo.RepoRegistry
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
|
import de.hoennig.werkator.watcher.FollowUpTrigger
|
||||||
import io.kotest.core.spec.style.FunSpec
|
import io.kotest.core.spec.style.FunSpec
|
||||||
import io.kotest.matchers.shouldBe
|
import io.kotest.matchers.shouldBe
|
||||||
import io.kotest.matchers.string.shouldContain
|
import io.kotest.matchers.string.shouldContain
|
||||||
@@ -22,16 +23,31 @@ class BuildCommandTest : FunSpec() {
|
|||||||
private val dir: Path = Paths.get(".")
|
private val dir: Path = Paths.get(".")
|
||||||
private val repo = RepoContext("test", dir, mockk(), mockk())
|
private val repo = RepoContext("test", dir, mockk(), mockk())
|
||||||
private val registry = mockk<RepoRegistry>().also { every { it.current() } returns repo }
|
private val registry = mockk<RepoRegistry>().also { every { it.current() } returns repo }
|
||||||
|
private val followUpTrigger = mockk<FollowUpTrigger>()
|
||||||
|
|
||||||
private fun command(fragment: String? = null) =
|
private fun command(fragment: String? = null) =
|
||||||
BuildCommand(gitService, consoleBuildRunner, registry).apply {
|
BuildCommand(gitService, consoleBuildRunner, registry, followUpTrigger).apply {
|
||||||
branchFragment = fragment
|
branchFragment = fragment
|
||||||
}
|
}
|
||||||
|
|
||||||
init {
|
init {
|
||||||
beforeEach {
|
beforeEach {
|
||||||
clearMocks(gitService, consoleBuildRunner)
|
clearMocks(gitService, consoleBuildRunner, followUpTrigger)
|
||||||
justRun { gitService.fetchOrigin(dir) }
|
justRun { gitService.fetchOrigin(dir) }
|
||||||
|
every { followUpTrigger.followUpsOf(any(), any(), any(), any()) } returns emptyList()
|
||||||
|
}
|
||||||
|
|
||||||
|
test("a green CLI build names the follow-up builds the server would run, and runs none") {
|
||||||
|
every { gitService.currentBranch(dir) } returns "main"
|
||||||
|
every { gitService.localHeadCommit("main", dir) } returns "local-head"
|
||||||
|
every { gitService.hasNewCommits("main", dir) } returns false
|
||||||
|
every { consoleBuildRunner.buildAndStream(repo, "main", "local-head") } returns BuildStatus.SUCCESS
|
||||||
|
every { followUpTrigger.followUpsOf(repo, "main", "local-head", "default") } returns listOf("deploy")
|
||||||
|
|
||||||
|
val console = captureConsole { command().call() }
|
||||||
|
|
||||||
|
console.stdout.shouldContain("follow-up build(s) deploy")
|
||||||
|
verify(exactly = 1) { consoleBuildRunner.buildAndStream(repo, "main", "local-head") }
|
||||||
}
|
}
|
||||||
|
|
||||||
test("builds the current branch at its local head when no branch is given") {
|
test("builds the current branch at its local head when no branch is given") {
|
||||||
|
|||||||
@@ -109,21 +109,6 @@ class InitCommandTest : FunSpec() {
|
|||||||
projectContent shouldContain "repo: my-repo"
|
projectContent shouldContain "repo: my-repo"
|
||||||
}
|
}
|
||||||
|
|
||||||
test("keeps dots in repository names") {
|
|
||||||
val tempDir = Files.createTempDirectory("werkator-init-test")
|
|
||||||
initCommand.workingDir = tempDir
|
|
||||||
|
|
||||||
every { gitService.getTopLevel(tempDir) } returns tempDir
|
|
||||||
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/mi/michael.hoennig.de.git"
|
|
||||||
|
|
||||||
initCommand.run()
|
|
||||||
|
|
||||||
val projectConfig = tempDir.resolve(".werkator.yml")
|
|
||||||
val projectContent = projectConfig.toFile().readText()
|
|
||||||
projectContent shouldContain "owner: mi"
|
|
||||||
projectContent shouldContain "repo: michael.hoennig.de"
|
|
||||||
}
|
|
||||||
|
|
||||||
test("does not overwrite existing files") {
|
test("does not overwrite existing files") {
|
||||||
val tempDir = Files.createTempDirectory("werkator-init-test")
|
val tempDir = Files.createTempDirectory("werkator-init-test")
|
||||||
initCommand.workingDir = tempDir
|
initCommand.workingDir = tempDir
|
||||||
@@ -267,50 +252,5 @@ class InitCommandTest : FunSpec() {
|
|||||||
// This should not throw IllegalArgumentException
|
// This should not throw IllegalArgumentException
|
||||||
initCommand.run()
|
initCommand.run()
|
||||||
}
|
}
|
||||||
|
|
||||||
test("--systemd reads the configuration from the repository root, not the current directory") {
|
|
||||||
val tempDir = Files.createTempDirectory("werkator-init-test")
|
|
||||||
// written before the run, so `init` keeps it instead of creating a template
|
|
||||||
tempDir.resolve(".werkator.yml").toFile().writeText(
|
|
||||||
"server:\n publicBaseUrl: \"https://werkator.example.org/\"\n port: 18099\n",
|
|
||||||
)
|
|
||||||
initCommand.workingDir = tempDir
|
|
||||||
initCommand.systemd = true
|
|
||||||
initCommand.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
|
|
||||||
initCommand.javaExecutableResolver = { Paths.get("/usr/bin/java") }
|
|
||||||
|
|
||||||
every { gitService.getTopLevel(tempDir) } returns tempDir
|
|
||||||
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
|
|
||||||
|
|
||||||
initCommand.run()
|
|
||||||
|
|
||||||
// the host integration is generated only when the root's config has a public base URL
|
|
||||||
val htaccess = tempDir.resolve(".git/werkator/${SystemdServiceFiles.HTACCESS_NAME}")
|
|
||||||
htaccess.toFile().shouldExist()
|
|
||||||
htaccess.toFile().readText() shouldContain "18099"
|
|
||||||
tempDir.resolve(".git/werkator/${SystemdServiceFiles.MAINTENANCE_PAGE_NAME}").toFile().shouldExist()
|
|
||||||
}
|
|
||||||
|
|
||||||
test("--systemd warns once when the effective configuration cannot be loaded") {
|
|
||||||
val tempDir = Files.createTempDirectory("werkator-init-test")
|
|
||||||
val brokenLoader = mockk<de.hoennig.werkator.config.ConfigLoader>()
|
|
||||||
every { brokenLoader.load(any()) } throws IllegalStateException("gitea.owner is required")
|
|
||||||
val command = InitCommand(gitService, brokenLoader)
|
|
||||||
command.workingDir = tempDir
|
|
||||||
command.systemd = true
|
|
||||||
command.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
|
|
||||||
command.javaExecutableResolver = { Paths.get("/usr/bin/java") }
|
|
||||||
|
|
||||||
every { gitService.getTopLevel(tempDir) } returns tempDir
|
|
||||||
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
|
|
||||||
|
|
||||||
val console = captureConsole { command.run() }
|
|
||||||
|
|
||||||
console.stdout shouldContain "gitea.owner is required"
|
|
||||||
// the three readers of the configuration must not repeat the warning
|
|
||||||
console.stdout.windowed("Warning:".length).count { it == "Warning:" } shouldBe 1
|
|
||||||
// the units are still written with the defaults
|
|
||||||
tempDir.resolve(".git/werkator/${SystemdServiceFiles.unitName(tempDir)}").toFile().shouldExist()
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -739,6 +739,190 @@ class ConfigLoaderTest : FunSpec() {
|
|||||||
.shouldBeTrue()
|
.shouldBeTrue()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test("afterSuccessOf must name an existing definition and must not form a cycle") {
|
||||||
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
|
val project = dir.resolve(".werkator.yml").toFile()
|
||||||
|
project.writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
default:
|
||||||
|
trigger:
|
||||||
|
onPush: true
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: default
|
||||||
|
branches: ["main"]
|
||||||
|
buildCommand: scripts/deploy.sh
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
// a follow-up with nothing but its predecessor is a triggered build, and it binds
|
||||||
|
val deploy = loader.load(dir).buildDefinitions.getValue("deploy")
|
||||||
|
deploy.trigger.afterSuccessOf shouldBe "default"
|
||||||
|
deploy.trigger.isFollowUp().shouldBeTrue()
|
||||||
|
deploy.trigger.onPush.shouldBeFalse()
|
||||||
|
|
||||||
|
project.writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: frontend
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.let {
|
||||||
|
it.shouldContain("builds.deploy")
|
||||||
|
it.shouldContain("frontend")
|
||||||
|
}
|
||||||
|
|
||||||
|
project.writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
a:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: b
|
||||||
|
b:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: a
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.shouldContain("a -> b -> a")
|
||||||
|
|
||||||
|
project.writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
a:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: a
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
shouldThrow<ConfigFormatException> { loader.load(dir) }.message.shouldContain("builds.a follows itself")
|
||||||
|
}
|
||||||
|
|
||||||
|
test("a branch whose layer lacks the predecessor loses only its follow-up") {
|
||||||
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
|
dir.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
frontend:
|
||||||
|
trigger:
|
||||||
|
onPush: true
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: frontend
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
// the branch renamed the predecessor: a warning, not a failed load — the branch's
|
||||||
|
// own builds must keep running, its follow-up simply never fires for it
|
||||||
|
val config =
|
||||||
|
loader.loadWithBranchLayer(
|
||||||
|
dir,
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
frontend: null
|
||||||
|
ui:
|
||||||
|
trigger:
|
||||||
|
onPush: true
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
config.buildDefinitions
|
||||||
|
.getValue("ui")
|
||||||
|
.trigger.onPush
|
||||||
|
.shouldBeTrue()
|
||||||
|
config.buildDefinitions
|
||||||
|
.getValue("deploy")
|
||||||
|
.trigger.afterSuccessOf shouldBe "frontend"
|
||||||
|
}
|
||||||
|
|
||||||
|
test("afterSuccessOf written flat is refused like every trigger key") {
|
||||||
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
|
dir.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
default:
|
||||||
|
trigger:
|
||||||
|
onPush: true
|
||||||
|
deploy:
|
||||||
|
afterSuccessOf: default
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
val thrown = shouldThrow<ConfigFormatException> { loader.load(dir) }
|
||||||
|
thrown.message.shouldContain("builds.deploy: afterSuccessOf")
|
||||||
|
thrown.message.shouldContain("trigger:")
|
||||||
|
}
|
||||||
|
|
||||||
|
test("a follow-up trigger is pinned to the host, a branch cannot add or widen one") {
|
||||||
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
|
dir.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
frontend:
|
||||||
|
trigger:
|
||||||
|
onPush: true
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: frontend
|
||||||
|
branches: ["main"]
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
val config =
|
||||||
|
loader.loadWithBranchLayer(
|
||||||
|
dir,
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: frontend
|
||||||
|
branches: ["*"]
|
||||||
|
nightly:
|
||||||
|
trigger:
|
||||||
|
atTimes: ["01:00"]
|
||||||
|
afterSuccessOf: frontend
|
||||||
|
""".trimIndent(),
|
||||||
|
"feature/x",
|
||||||
|
)
|
||||||
|
// the host's trigger block of the follow-up is used unchanged
|
||||||
|
val deploy = config.buildDefinitions.getValue("deploy").trigger
|
||||||
|
deploy.afterSuccessOf shouldBe "frontend"
|
||||||
|
deploy.branches shouldBe listOf("main")
|
||||||
|
deploy.selectsByName("feature/x").shouldBeFalse()
|
||||||
|
// and a branch cannot make any build of its own a follow-up
|
||||||
|
val nightly = config.buildDefinitions.getValue("nightly").trigger
|
||||||
|
nightly.afterSuccessOf shouldBe ""
|
||||||
|
nightly.atTimes shouldBe listOf("01:00")
|
||||||
|
}
|
||||||
|
|
||||||
|
test("a branch supplies the command of a host-triggered follow-up") {
|
||||||
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
|
Files.createDirectories(dir.resolve(".git/werkator"))
|
||||||
|
dir.resolve(".git/werkator/.werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
deploy:
|
||||||
|
trigger:
|
||||||
|
afterSuccessOf: default
|
||||||
|
branches: ["main"]
|
||||||
|
werkdock:
|
||||||
|
env:
|
||||||
|
DEPLOY_TARGET: host:/srv/www
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
val worktree = Files.createTempDirectory("werkator-test-worktree")
|
||||||
|
worktree.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
builds:
|
||||||
|
deploy:
|
||||||
|
cleanCommand: ""
|
||||||
|
buildCommand: scripts/deploy-prod.sh -y "${'$'}DEPLOY_TARGET"
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
|
||||||
|
val settings = loader.loadForWorktree(dir, worktree, "main").buildSettings("main", "deploy")
|
||||||
|
|
||||||
|
settings.buildCommand shouldBe "scripts/deploy-prod.sh -y \"${'$'}DEPLOY_TARGET\""
|
||||||
|
settings.cleanCommand shouldBe ""
|
||||||
|
settings.werkdock.env shouldBe mapOf("DEPLOY_TARGET" to "host:/srv/www")
|
||||||
|
}
|
||||||
|
|
||||||
test("builds.default is the base of every other build, but never its trigger") {
|
test("builds.default is the base of every other build, but never its trigger") {
|
||||||
val dir = Files.createTempDirectory("werkator-test")
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
dir.resolve(".werkator.yml").toFile().writeText(
|
dir.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
|||||||
@@ -113,7 +113,7 @@ class PermanentBranchRoutesTest : FunSpec() {
|
|||||||
every { registry.byName(any()) } returns null
|
every { registry.byName(any()) } returns null
|
||||||
every { registry.byName("test") } returns repo
|
every { registry.byName("test") } returns repo
|
||||||
every { configLoader.load(any()) } returns WerkatorConfig()
|
every { configLoader.load(any()) } returns WerkatorConfig()
|
||||||
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns WerkatorConfig()
|
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns WerkatorConfig()
|
||||||
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
||||||
every { controlTokens.token() } returns "test-token"
|
every { controlTokens.token() } returns "test-token"
|
||||||
every { branchListing.branches(any()) } returns emptyList()
|
every { branchListing.branches(any()) } returns emptyList()
|
||||||
|
|||||||
@@ -142,7 +142,7 @@ class UiControllerTest : FunSpec() {
|
|||||||
server = ServerConfig(impressumUrl = "https://example.org/imprint"),
|
server = ServerConfig(impressumUrl = "https://example.org/imprint"),
|
||||||
gitea = GiteaConfig(baseUrl = "https://git.example.org", owner = "acme", repo = "widget"),
|
gitea = GiteaConfig(baseUrl = "https://git.example.org", owner = "acme", repo = "widget"),
|
||||||
)
|
)
|
||||||
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns WerkatorConfig()
|
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns WerkatorConfig()
|
||||||
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
||||||
every { controlTokens.token() } returns "test-token"
|
every { controlTokens.token() } returns "test-token"
|
||||||
every { repository.latestGreenFor(any()) } returns null
|
every { repository.latestGreenFor(any()) } returns null
|
||||||
@@ -385,7 +385,7 @@ class UiControllerTest : FunSpec() {
|
|||||||
)
|
)
|
||||||
every { repository.history() } returns listOf(pitestResult)
|
every { repository.history() } returns listOf(pitestResult)
|
||||||
every { artifactStore.artifactDir("main-pitest-key") } returns null
|
every { artifactStore.artifactDir("main-pitest-key") } returns null
|
||||||
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns
|
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns
|
||||||
WerkatorConfig(
|
WerkatorConfig(
|
||||||
branches = mapOf("default" to BranchConfig(buildCommand = "./gradlew quick-check")),
|
branches = mapOf("default" to BranchConfig(buildCommand = "./gradlew quick-check")),
|
||||||
buildDefinitions = mapOf("pitest" to BuildDefinition(buildCommand = "./gradlew pitestFull")),
|
buildDefinitions = mapOf("pitest" to BuildDefinition(buildCommand = "./gradlew pitestFull")),
|
||||||
|
|||||||
@@ -0,0 +1,173 @@
|
|||||||
|
package de.hoennig.werkator.watcher
|
||||||
|
|
||||||
|
import de.hoennig.werkator.build.ArtifactKeys
|
||||||
|
import de.hoennig.werkator.build.BuildExecutor
|
||||||
|
import de.hoennig.werkator.build.BuildResult
|
||||||
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
|
import de.hoennig.werkator.build.BuildStatusChangedEvent
|
||||||
|
import de.hoennig.werkator.build.RunningBuild
|
||||||
|
import de.hoennig.werkator.config.BuildDefinition
|
||||||
|
import de.hoennig.werkator.config.ConfigLoader
|
||||||
|
import de.hoennig.werkator.config.TriggerConfig
|
||||||
|
import de.hoennig.werkator.config.WerkatorConfig
|
||||||
|
import de.hoennig.werkator.git.GitService
|
||||||
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import io.kotest.core.spec.style.FunSpec
|
||||||
|
import io.kotest.matchers.collections.shouldBeEmpty
|
||||||
|
import io.kotest.matchers.collections.shouldContainExactly
|
||||||
|
import io.mockk.every
|
||||||
|
import io.mockk.mockk
|
||||||
|
import java.nio.file.Files
|
||||||
|
import java.time.Clock
|
||||||
|
import java.time.Instant
|
||||||
|
import java.time.ZoneOffset
|
||||||
|
import java.util.concurrent.CopyOnWriteArrayList
|
||||||
|
|
||||||
|
class FollowUpTriggerTest : FunSpec() {
|
||||||
|
private val noon = Instant.parse("2026-09-04T12:00:00Z")
|
||||||
|
|
||||||
|
/** Which build was started on which branch at which commit. */
|
||||||
|
private data class Started(
|
||||||
|
val branch: String,
|
||||||
|
val commit: String,
|
||||||
|
val build: String,
|
||||||
|
)
|
||||||
|
|
||||||
|
private inner class Harness(
|
||||||
|
config: WerkatorConfig,
|
||||||
|
) {
|
||||||
|
val workingDir = Files.createTempDirectory("werkator-followup-test")
|
||||||
|
val gitService = mockk<GitService>()
|
||||||
|
val configLoader = mockk<ConfigLoader>()
|
||||||
|
val buildExecutor = mockk<BuildExecutor>()
|
||||||
|
val repo = RepoContext("test", workingDir, mockk(), mockk())
|
||||||
|
val started = CopyOnWriteArrayList<Started>()
|
||||||
|
val trigger = FollowUpTrigger(gitService, configLoader, buildExecutor, Clock.fixed(noon, ZoneOffset.UTC))
|
||||||
|
|
||||||
|
init {
|
||||||
|
every { configLoader.load(any()) } returns config
|
||||||
|
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns config
|
||||||
|
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
||||||
|
// the branch moved on since the predecessor started
|
||||||
|
every { gitService.originHeadCommit(any(), any()) } returns "c2"
|
||||||
|
every { gitService.originBranchCommitTimes(any()) } returns mapOf("main" to noon.minusSeconds(60))
|
||||||
|
every { buildExecutor.startBuild(any(), any(), any(), any()) } answers {
|
||||||
|
val branch = secondArg<String>()
|
||||||
|
val commit = thirdArg<String>()
|
||||||
|
val build = arg<String>(3)
|
||||||
|
started += Started(branch, commit, build)
|
||||||
|
val staging = Files.createTempDirectory("werkator-followup-staging")
|
||||||
|
RunningBuild(
|
||||||
|
repo = repo,
|
||||||
|
branch = branch,
|
||||||
|
build = build,
|
||||||
|
commit = commit,
|
||||||
|
artifactKey = ArtifactKeys.buildKey(BuildDefinition.poolName(branch, build), noon),
|
||||||
|
startedAt = noon,
|
||||||
|
stagingDir = staging,
|
||||||
|
liveLogFile = staging.resolve("build.log"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun finished(
|
||||||
|
build: String,
|
||||||
|
status: BuildStatus,
|
||||||
|
branch: String = "main",
|
||||||
|
commit: String = "c1",
|
||||||
|
) {
|
||||||
|
val result =
|
||||||
|
BuildResult(
|
||||||
|
branch = branch,
|
||||||
|
build = build,
|
||||||
|
commit = commit,
|
||||||
|
status = status,
|
||||||
|
startedAt = noon,
|
||||||
|
artifactKey = ArtifactKeys.buildKey(BuildDefinition.poolName(branch, build), noon),
|
||||||
|
)
|
||||||
|
trigger.onBuildStatusChanged(BuildStatusChangedEvent(result, repo))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun deployAfter(
|
||||||
|
predecessor: String,
|
||||||
|
branches: List<String> = emptyList(),
|
||||||
|
): WerkatorConfig =
|
||||||
|
WerkatorConfig(
|
||||||
|
buildDefinitions =
|
||||||
|
mapOf(
|
||||||
|
"frontend" to BuildDefinition(trigger = TriggerConfig(onPush = true)),
|
||||||
|
"backend" to BuildDefinition(trigger = TriggerConfig(onPush = true)),
|
||||||
|
"deploy" to
|
||||||
|
BuildDefinition(
|
||||||
|
trigger = TriggerConfig(afterSuccessOf = predecessor, branches = branches),
|
||||||
|
buildCommand = "scripts/deploy.sh",
|
||||||
|
),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
init {
|
||||||
|
test("a green predecessor enqueues the follow-up at the predecessor's commit") {
|
||||||
|
val h = Harness(deployAfter("frontend", branches = listOf("main")))
|
||||||
|
h.trigger.arm()
|
||||||
|
|
||||||
|
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
|
||||||
|
|
||||||
|
// c1, not the origin head c2 the branch has moved on to
|
||||||
|
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
|
||||||
|
}
|
||||||
|
|
||||||
|
test("every green run of the predecessor triggers the follow-up again") {
|
||||||
|
val h = Harness(deployAfter("frontend"))
|
||||||
|
h.trigger.arm()
|
||||||
|
|
||||||
|
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
|
||||||
|
h.finished("frontend", BuildStatus.SUCCESS, commit = "c1")
|
||||||
|
|
||||||
|
h.started shouldContainExactly
|
||||||
|
listOf(
|
||||||
|
Started("main", "c1", "deploy"),
|
||||||
|
Started("main", "c1", "deploy"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
test("only a SUCCESS of the named predecessor triggers") {
|
||||||
|
val h = Harness(deployAfter("frontend", branches = listOf("main")))
|
||||||
|
h.trigger.arm()
|
||||||
|
|
||||||
|
h.finished("frontend", BuildStatus.FAILED)
|
||||||
|
h.finished("frontend", BuildStatus.CANCELLED)
|
||||||
|
h.finished("frontend", BuildStatus.INTERRUPTED)
|
||||||
|
h.finished("frontend", BuildStatus.PENDING)
|
||||||
|
h.finished("frontend", BuildStatus.RUNNING)
|
||||||
|
h.finished("backend", BuildStatus.SUCCESS)
|
||||||
|
// a branch the host's selector does not name never runs the follow-up
|
||||||
|
h.finished("frontend", BuildStatus.SUCCESS, branch = "feature/x")
|
||||||
|
|
||||||
|
h.started.shouldBeEmpty()
|
||||||
|
}
|
||||||
|
|
||||||
|
test("the trigger listens only while the watcher runs") {
|
||||||
|
val h = Harness(deployAfter("frontend"))
|
||||||
|
|
||||||
|
h.finished("frontend", BuildStatus.SUCCESS)
|
||||||
|
h.started.shouldBeEmpty()
|
||||||
|
|
||||||
|
h.trigger.arm()
|
||||||
|
h.finished("frontend", BuildStatus.SUCCESS)
|
||||||
|
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
|
||||||
|
|
||||||
|
h.trigger.disarm()
|
||||||
|
h.finished("frontend", BuildStatus.SUCCESS)
|
||||||
|
h.started shouldContainExactly listOf(Started("main", "c1", "deploy"))
|
||||||
|
}
|
||||||
|
|
||||||
|
test("followUpsOf names the follow-ups without enqueueing anything") {
|
||||||
|
val h = Harness(deployAfter("default"))
|
||||||
|
|
||||||
|
h.trigger.followUpsOf(h.repo, "main", "c1", "default") shouldContainExactly listOf("deploy")
|
||||||
|
h.trigger.followUpsOf(h.repo, "main", "c1", "frontend").shouldBeEmpty()
|
||||||
|
h.started.shouldBeEmpty()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -63,6 +63,7 @@ class WatcherTest : FunSpec() {
|
|||||||
val artifactStore = mockk<ArtifactStore>()
|
val artifactStore = mockk<ArtifactStore>()
|
||||||
val startedBuilds = CopyOnWriteArrayList<Pair<String, String>>()
|
val startedBuilds = CopyOnWriteArrayList<Pair<String, String>>()
|
||||||
val configLoader = mockk<ConfigLoader>()
|
val configLoader = mockk<ConfigLoader>()
|
||||||
|
val followUpTrigger = mockk<FollowUpTrigger>(relaxed = true)
|
||||||
val repo = RepoContext("test", workingDir, repository, artifactStore)
|
val repo = RepoContext("test", workingDir, repository, artifactStore)
|
||||||
val watcher =
|
val watcher =
|
||||||
Watcher(
|
Watcher(
|
||||||
@@ -70,6 +71,7 @@ class WatcherTest : FunSpec() {
|
|||||||
buildExecutor = buildExecutor,
|
buildExecutor = buildExecutor,
|
||||||
configLoader = configLoader,
|
configLoader = configLoader,
|
||||||
clock = Clock.fixed(noon, ZoneOffset.UTC),
|
clock = Clock.fixed(noon, ZoneOffset.UTC),
|
||||||
|
followUpTrigger = followUpTrigger,
|
||||||
)
|
)
|
||||||
|
|
||||||
private var seedCounter = 0L
|
private var seedCounter = 0L
|
||||||
@@ -85,7 +87,7 @@ class WatcherTest : FunSpec() {
|
|||||||
every { gitService.originBranchCommitTimes(any()) } returns emptyMap()
|
every { gitService.originBranchCommitTimes(any()) } returns emptyMap()
|
||||||
every { gitService.originBranchHeads(any()) } returns emptyMap()
|
every { gitService.originBranchHeads(any()) } returns emptyMap()
|
||||||
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
||||||
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns config
|
every { configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns config
|
||||||
every { gitService.pullRequestHeads(any()) } returns emptySet()
|
every { gitService.pullRequestHeads(any()) } returns emptySet()
|
||||||
every { gitService.worktreePrune(any()) } returns Unit
|
every { gitService.worktreePrune(any()) } returns Unit
|
||||||
every { gitService.fastForwardLocalBranches(any()) } returns emptyList()
|
every { gitService.fastForwardLocalBranches(any()) } returns emptyList()
|
||||||
@@ -157,6 +159,19 @@ class WatcherTest : FunSpec() {
|
|||||||
)
|
)
|
||||||
|
|
||||||
init {
|
init {
|
||||||
|
test("start arms the follow-up trigger before the recovery, stop disarms it") {
|
||||||
|
val harness = Harness()
|
||||||
|
|
||||||
|
harness.watcher.start(listOf(harness.repo))
|
||||||
|
try {
|
||||||
|
verify(exactly = 1) { harness.followUpTrigger.arm() }
|
||||||
|
verify(exactly = 0) { harness.followUpTrigger.disarm() }
|
||||||
|
} finally {
|
||||||
|
harness.watcher.stop()
|
||||||
|
}
|
||||||
|
verify(exactly = 1) { harness.followUpTrigger.disarm() }
|
||||||
|
}
|
||||||
|
|
||||||
test("a fetch failure is exposed in the state and only retried next cycle") {
|
test("a fetch failure is exposed in the state and only retried next cycle") {
|
||||||
val harness = Harness()
|
val harness = Harness()
|
||||||
every { harness.gitService.fetchOrigin(any()) } throws RuntimeException("origin unreachable")
|
every { harness.gitService.fetchOrigin(any()) } throws RuntimeException("origin unreachable")
|
||||||
@@ -526,7 +541,7 @@ class WatcherTest : FunSpec() {
|
|||||||
every { harness.gitService.originBranchHeads(any()) } returns
|
every { harness.gitService.originBranchHeads(any()) } returns
|
||||||
mapOf("main" to "commit-main", "experiment" to "commit-exp")
|
mapOf("main" to "commit-main", "experiment" to "commit-exp")
|
||||||
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
|
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
|
||||||
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
|
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer
|
||||||
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
|
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
|
||||||
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
|
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
|
||||||
|
|
||||||
@@ -549,7 +564,7 @@ class WatcherTest : FunSpec() {
|
|||||||
every { harness.gitService.originBranches(any()) } returns listOf("experiment")
|
every { harness.gitService.originBranches(any()) } returns listOf("experiment")
|
||||||
every { harness.gitService.originBranchHeads(any()) } returns mapOf("experiment" to "commit-exp")
|
every { harness.gitService.originBranchHeads(any()) } returns mapOf("experiment" to "commit-exp")
|
||||||
every { harness.gitService.showFileAtCommit("commit-exp", ".gittally.yml", any()) } returns "branch-yaml"
|
every { harness.gitService.showFileAtCommit("commit-exp", ".gittally.yml", any()) } returns "branch-yaml"
|
||||||
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
|
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer
|
||||||
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
|
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
|
||||||
|
|
||||||
harness.watcher.poll(harness.repo)
|
harness.watcher.poll(harness.repo)
|
||||||
@@ -568,7 +583,7 @@ class WatcherTest : FunSpec() {
|
|||||||
every { harness.gitService.originBranchHeads(any()) } returns
|
every { harness.gitService.originBranchHeads(any()) } returns
|
||||||
mapOf("main" to "commit-main", "experiment" to "commit-exp")
|
mapOf("main" to "commit-main", "experiment" to "commit-exp")
|
||||||
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
|
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
|
||||||
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
|
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml", anyNullable()) } returns branchLayer
|
||||||
every { harness.gitService.originHeadCommit(any(), any()) } returns "commit-any"
|
every { harness.gitService.originHeadCommit(any(), any()) } returns "commit-any"
|
||||||
|
|
||||||
harness.watcher.poll(harness.repo)
|
harness.watcher.poll(harness.repo)
|
||||||
@@ -608,7 +623,7 @@ class WatcherTest : FunSpec() {
|
|||||||
buildDefinitions = mapOf("nightly" to BuildDefinition(trigger = TriggerConfig(atTimes = listOf("11:00")))),
|
buildDefinitions = mapOf("nightly" to BuildDefinition(trigger = TriggerConfig(atTimes = listOf("11:00")))),
|
||||||
)
|
)
|
||||||
every { harness.configLoader.load(any()) } returns edited
|
every { harness.configLoader.load(any()) } returns edited
|
||||||
every { harness.configLoader.loadWithBranchLayer(any(), anyNullable()) } returns edited
|
every { harness.configLoader.loadWithBranchLayer(any(), anyNullable(), anyNullable()) } returns edited
|
||||||
|
|
||||||
harness.watcher.poll(harness.repo)
|
harness.watcher.poll(harness.repo)
|
||||||
|
|
||||||
@@ -622,7 +637,7 @@ class WatcherTest : FunSpec() {
|
|||||||
every { harness.gitService.hasNewCommits("main", any()) } returns true
|
every { harness.gitService.hasNewCommits("main", any()) } returns true
|
||||||
every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-main")
|
every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-main")
|
||||||
every { harness.gitService.showFileAtCommit("commit-main", Watcher.CONFIG_FILE, any()) } returns "broken"
|
every { harness.gitService.showFileAtCommit("commit-main", Watcher.CONFIG_FILE, any()) } returns "broken"
|
||||||
every { harness.configLoader.loadWithBranchLayer(any(), "broken") } throws
|
every { harness.configLoader.loadWithBranchLayer(any(), "broken", anyNullable()) } throws
|
||||||
RuntimeException("mapping problem")
|
RuntimeException("mapping problem")
|
||||||
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
|
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
|
||||||
|
|
||||||
|
|||||||
+13
-61
@@ -332,65 +332,9 @@ instance_update() {
|
|||||||
echo "==> Instance updated."
|
echo "==> Instance updated."
|
||||||
}
|
}
|
||||||
|
|
||||||
# Clones one URL into one directory on the host.
|
# Sets up the WATCHED repository: an anonymous https clone (a private origin
|
||||||
# A private https origin authenticates with the shared `defaults.git.account` /
|
# gets its credentials via git.account/git.token in the machine config that
|
||||||
# `defaults.git.token` of `~/.werkator.yml` (ADR 0009). The whole authenticated
|
# `werkator init` creates), the werkator init with the instance fragment
|
||||||
# clone runs in one remote python script: the token is read from the instance
|
|
||||||
# file on the host and passed to git via a one-shot GIT_ASKPASS script, so it
|
|
||||||
# appears in neither the local process list nor a repository config.
|
|
||||||
# Public origins (or an instance without shared credentials) clone anonymously.
|
|
||||||
clone_repo() {
|
|
||||||
local url="$1" dest="$2"
|
|
||||||
if ssh "$HOST" "test -d '$dest/.git'"; then
|
|
||||||
echo " (already cloned, skipping)"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
case "$url" in
|
|
||||||
https://*)
|
|
||||||
# The python helper travels base64-encoded: the clone command itself
|
|
||||||
# stays a plain `ssh` line, so no `$` inside the script is ever
|
|
||||||
# expanded by the local shell, and the token never leaves the host.
|
|
||||||
local helper_b64
|
|
||||||
helper_b64="$(python3 -c 'import base64,sys; print(base64.b64encode(sys.stdin.read().encode()).decode())' <<'PYEOF_CLONE'
|
|
||||||
import os, stat, subprocess, sys, tempfile, urllib.parse
|
|
||||||
url, dest = sys.argv[1], sys.argv[2]
|
|
||||||
try:
|
|
||||||
import yaml
|
|
||||||
cfg = yaml.safe_load(open(os.path.expanduser("~/.werkator.yml"))) or {}
|
|
||||||
except (FileNotFoundError, ImportError):
|
|
||||||
cfg = {}
|
|
||||||
d = (cfg.get("defaults") or {}).get("git") or {}
|
|
||||||
account, token = d.get("account"), d.get("token")
|
|
||||||
env = dict(os.environ, GIT_TERMINAL_PROMPT="0")
|
|
||||||
ask = None
|
|
||||||
if account and token:
|
|
||||||
parts = urllib.parse.urlsplit(url)
|
|
||||||
host = parts.netloc.rsplit("@", 1)[-1]
|
|
||||||
url = urllib.parse.urlunsplit(parts._replace(netloc=account + "@" + host))
|
|
||||||
ask = tempfile.NamedTemporaryFile(mode="w", prefix="werkator-clone-askpass-",
|
|
||||||
suffix=".sh", delete=False)
|
|
||||||
ask.write("#!/bin/sh\nexec echo \"$WERKATOR_CLONE_TOKEN\"\n")
|
|
||||||
ask.close()
|
|
||||||
os.chmod(ask.name, stat.S_IRWXU)
|
|
||||||
env.update(GIT_ASKPASS=ask.name, WERKATOR_CLONE_TOKEN=token)
|
|
||||||
try:
|
|
||||||
subprocess.run(["git", "clone", url, dest], env=env, check=True)
|
|
||||||
finally:
|
|
||||||
if ask is not None:
|
|
||||||
os.unlink(ask.name)
|
|
||||||
PYEOF_CLONE
|
|
||||||
)"
|
|
||||||
ssh "$HOST" "echo '$helper_b64' | base64 -d | python3 - '$url' '$dest'"
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
ssh "$HOST" "git clone '$url' '$dest'"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
# Sets up the WATCHED repository: an https clone (a private origin
|
|
||||||
# authenticates with the shared `defaults.git.*` credentials of
|
|
||||||
# `~/.werkator.yml`; see `clone_repo`), the werkator init with the instance
|
|
||||||
# applied, and the rootfs archive for the sandbox builds. All configuration
|
# applied, and the rootfs archive for the sandbox builds. All configuration
|
||||||
# writing is init's — this script transports and invokes (step 23).
|
# writing is init's — this script transports and invokes (step 23).
|
||||||
repo_init() {
|
repo_init() {
|
||||||
@@ -400,7 +344,11 @@ repo_init() {
|
|||||||
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
||||||
|
|
||||||
echo "==> Cloning the watched repository"
|
echo "==> Cloning the watched repository"
|
||||||
clone_repo "$REPO_URL" "$REPO_DIR"
|
if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then
|
||||||
|
echo " (already cloned, skipping)"
|
||||||
|
else
|
||||||
|
ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
|
||||||
|
fi
|
||||||
|
|
||||||
if [ "$SANDBOX" = "docker" ]; then
|
if [ "$SANDBOX" = "docker" ]; then
|
||||||
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
|
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
|
||||||
@@ -453,7 +401,11 @@ repo_add() {
|
|||||||
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
||||||
|
|
||||||
echo "==> Cloning $url as '$name'"
|
echo "==> Cloning $url as '$name'"
|
||||||
clone_repo "$url" "$SIBLING_DIR/$name"
|
if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then
|
||||||
|
echo " (already cloned, skipping)"
|
||||||
|
else
|
||||||
|
ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
|
||||||
|
fi
|
||||||
|
|
||||||
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
|
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
|
||||||
# binary). Without it a watched repository builds on the bare host, where the
|
# binary). Without it a watched repository builds on the bare host, where the
|
||||||
|
|||||||
Reference in New Issue
Block a user