Author SHA1 Message Date
mhoennigandClaude Fable 5.1 89430952f1 docs(prs): name werkdock 0.2.0 as the prerequisite
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-06 13:39:44 +02:00
mhoennigandClaude Fable 5.1 ec07269cca docs(prs): refer to werkdock's change, not its branch
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 23:29:55 +02:00
mhoennigandClaude Fable 5.1 b8600bb556 werkdock: import the rootfs archive, fall back to load -i on an older werkdock
Werkdock's docker-compat branch names the verb for rootfs archives
'import' (docker import semantics) and keeps 'load -i --name' only as
a compatibility path. Werkator now runs 'werkdock import ARCHIVE IMAGE'
and falls back to 'load -i --name' on exactly the unknown-verb
signature (exit 125, 'unknown command' on stderr), so Werkator and
werkdock can be updated in either order. Any other import failure
propagates as before. The image name stays untagged: it equals
werkator-buildenv-<hash>:latest in werkdock's naming, and the existing
store needs no re-import. PR-doc under docs/prs/ with the PR#000
placeholder.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 22:49:46 +02:00
mhoennig 0f9f119687 init: keep dots in detected repository names 2026-09-05 15:15:12 +02:00
miandmhoennig 5c64391f5f Feature/remote authenticated clone (#24)
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #24
2026-09-05 14:06:58 +02:00
mhoennigandClaude Opus 5 cd6f915231 init reads its configuration from the repository root, and says so when it cannot (#22)
`init --systemd` generated the systemd unit, the .htaccess and the maintenance
page from a configuration it never actually read reliably: the load swallowed
every exception and fell back to a default `ServerConfig`, and it resolved the
layers from the process's current directory instead of the git top level every
other file of the command goes through.

A broken `.werkator.yml` was therefore indistinguishable from an unconfigured
`publicBaseUrl` — the host integration was skipped without a word — and running
`init` from a subdirectory read a foreign configuration or none, which also
defeated `--apply`'s promise that the fragment reaches the generated unit.

The fallback stays, but the exception message is printed, once per run, and the
repository root is passed down like every other caller of `ConfigLoader.load`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 19:38:41 +02:00
092183ca30 Clone the watched repository from Gitea by default (#21)
The repository moved to git.javagil.de, but repo-init still defaulted to
the GitHub mirror, which is no longer updated.
A freshly cloned instance therefore watched a stale main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #21
2026-09-04 13:33:08 +02:00
mhoennig 0761a274db Merge main into the deflake-maxconcurrent-test branch 2026-09-04 12:46:25 +02:00
mhoennigandClaude Opus 5 5c73c4cc21 docs(prs): the deflake-maxconcurrent-test PR-doc
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 08:53:00 +02:00
16543f038b the sandbox config section is werkdock, not bwrap (#19)
`bwrap` named the mechanism one layer below the tool that actually runs it: since
v1.0.0 Werkator does not invoke bwrap at all, it shells out to the werkdock CLI —
which made `bwrap.werkdock` a key naming its own executor.

The section is `werkdock` now and that key is `werkdock.binary`; BwrapConfig,
BwrapOverrides and BwrapBuildRunner follow the name. A file still writing `bwrap`
is read as before and warned about once per file, in `renameLegacySandbox` on the
raw map of every layer before merging — so nothing downstream knows two names, and
the old name is not a way around the pinning either. Renaming rather than refusing,
because the section lives in the machine configuration of every webspace instance,
which no repository tracks; the hard refusal belongs to the release that sets
ConfigVersions.FORMAT_BROKE_IN, where a file declaring no version can be caught
by name at all.

WERKATOR_SANDBOX in tools/remote follows, and still accepts `bwrap`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #19
2026-09-03 20:41:18 +02:00
3ccc901d1b tools/remote drives any host layout (#18)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #18
2026-09-03 20:37:58 +02:00
mhoennigandClaude Fable 5.1 11f6f9bd26 docs(rfcs): RFC 0001 proposes the Instrument Panel web UI redesign
Six explored directions, the chosen one (E) worked out in a light and a dark
teal palette, with a repository strip previewing every served repository,
the ten-repository and the phone case, and the backend it needs (/api/repos).
Renderings of all artboards live next to the RFC; AGENTS.md registers docs/rfcs/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 20:15:04 +02:00
mhoennigandClaude Opus 5 9221550a2a test(build): gate the maxConcurrent-1 test instead of racing a sleep
The build of branch-a slept one second while the test asserted, without
any synchronization, that branch-b was still PENDING.
Under CPU contention on the mih09 host the sleep could elapse first, so
branch-b was already RUNNING or SUCCESS when the assertion ran.
branch-a now blocks until the test creates a gate file, and the test
first waits for branch-a to be RUNNING; the PENDING assertion no longer
depends on timing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 19:32:22 +02:00
27 changed files with 777 additions and 61 deletions
+1
View File
@@ -68,6 +68,7 @@ Keep sentences short.
- `docs/deployment.md` — running Werkator as a systemd user service behind an existing reverse proxy (`init --systemd` generates the unit). - `docs/deployment.md` — running Werkator as a systemd user service behind an existing reverse proxy (`init --systemd` generates the unit).
- `docs/werkator-migrationsplan.md` — renaming a running installation from GitTally to Werkator: what the name fallback covers and what has to be moved by hand. - `docs/werkator-migrationsplan.md` — renaming a running installation from GitTally to Werkator: what the name fallback covers and what has to be moved by hand.
- `docs/plan/` — the step-by-step rewrite plan; `docs/plan/README.md` explains how to execute a step, `docs/plan/00-legacy-analysis.md` summarizes the legacy bash script. - `docs/plan/` — the step-by-step rewrite plan; `docs/plan/README.md` explains how to execute a step, `docs/plan/00-legacy-analysis.md` summarizes the legacy bash script.
- `docs/rfcs/` — requests for comments: proposals that are larger than one PR and not yet a decision (an accepted RFC becomes an ADR or a plan step).
- `docs/prs/` — one document per pull request; every PR needs one. IMPORTANT: Before opening or finishing a pull request, load the [pr-doc skill](.claude/skills/pr-doc/SKILL.md) and write the PR-doc. - `docs/prs/` — one document per pull request; every PR needs one. IMPORTANT: Before opening or finishing a pull request, load the [pr-doc skill](.claude/skills/pr-doc/SKILL.md) and write the PR-doc.
## Key Architectural Decisions ## Key Architectural Decisions
+22
View File
@@ -8,6 +8,28 @@ Lightweight, declarative and highly opinionated software build system (CI/CD).
- [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init` - [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init`
- [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy - [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy
## Adding a Gitea Repository
One instance serves several repositories (`docs/deployment.md`, ADR 0009).
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
It prints the registry entry: add it to `~/.werkator.yml` under `repositories:`, then restart the service.
The optional `[<name>]` overrides the directory basename: it becomes the route segment (`/repos/<name>/…`) and the UI switcher entry, so it must be unique.
Needed only when the clone directory name is wrong or collides — e.g. `michael.hoennig.de.git` checked out as `michael.hoennig.de`, or two forges serving a repo of the same name.
A **public** repository needs nothing else: the clone runs anonymously.
A **private** repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning:
```yaml
defaults:
git:
account: <gitea-user>
token: <token> # Gitea → Settings → Applications → Generate Token, scope read:repository
```
## Developer Setup ## Developer Setup
Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`: Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`:
+3 -1
View File
@@ -475,7 +475,9 @@ The section was called `bwrap` and its binary key `bwrap.werkdock` until v1.2.0;
`werkdock.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target. `werkdock.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`. It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`.
Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker. Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker.
The archive is loaded once per source as the werkdock image `werkator-buildenv-<hash>` into werkdock's store (`$WERKDOCK_HOME`, default `~/.werkdock`) — shared by every repository of this OS user; the hash derives from the source string, so a changed `rootfs` loads a fresh image and stale ones can be removed from the store. The archive is imported once per source as the werkdock image `werkator-buildenv-<hash>` into werkdock's store (`$WERKDOCK_HOME`, default `~/.werkdock`) — shared by every repository of this OS user; the hash derives from the source string, so a changed `rootfs` imports a fresh image and stale ones can be removed from the store.
Werkator uses `werkdock import ARCHIVE IMAGE` (docker import semantics) and falls back to the older `werkdock load -i ARCHIVE --name IMAGE` when the installed werkdock does not know the verb yet, so Werkator and werkdock can be updated in either order.
The image keeps its untagged name (it equals `werkator-buildenv-<hash>:latest` in werkdock's docker-style naming), so an existing store needs no re-import.
Per-repo Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`. Per-repo Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`.
`werkdock.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in. `werkdock.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in.
Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user. Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user.
+2 -1
View File
@@ -123,6 +123,7 @@ Adding a repository is editing a registry entry — never a data migration, beca
It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry. It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry.
It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back. It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back.
Cloning and initialising is mechanical; registering is a decision. Cloning and initialising is mechanical; registering is a decision.
A private `https` origin authenticates with the shared `defaults.git.account`/`defaults.git.token` of `~/.werkator.yml` (the token travels via a one-shot `GIT_ASKPASS` on the host, never in a URL or process list); enter those once before cloning a private repository — without them only public origins clone.
4. **Restart** the service; startup recovery re-enqueues what was in flight: 4. **Restart** the service; startup recovery re-enqueues what was in flight:
@@ -341,7 +342,7 @@ tools/remote --env-file .env.mih34 port-forward start # browser tunne
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`. Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation). That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation).
The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user. The rootfs archive is imported once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`. Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
Updates are one command, refused while a build runs (`FORCE=1` overrides): Updates are one command, refused while a build runs (`FORCE=1` overrides):
@@ -0,0 +1,42 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
The build for commit `7028ca8` on `main` failed on the mih09 production instance with
`BuildExecutorTest > with maxConcurrent 1 a second branch stays PENDING until the first finished`,
while a retry of the very same commit passed, and the test passes locally.
The test was timing-dependent.
It started a build for `branch-a` whose build command was `sleep 1`, immediately started a second build for `branch-b`,
and then asserted — without any synchronization at all — that `branch-b` was still `PENDING`.
That assertion only held as long as the test thread reached it within the one second `branch-a` slept.
On a shared host under CPU contention the executor can get through `branch-a` entirely (queued, running, slept, succeeded) first,
and `branch-b` is then already `RUNNING` or `SUCCESS` when the assertion runs.
The failure therefore says nothing about the executor; it is pure scheduling noise that costs a build and a retry every time it hits.
## Non-Goals
- No change to production code — `BuildExecutor` is not touched, its queueing behaviour is unchanged.
- No sweep of the other timing-sensitive tests in the suite; only the one that actually flaked is fixed.
## The Solution
`branch-a` no longer sleeps for a fixed time, it blocks until the test says so:
its build command is `until [ -f gate ]; do sleep 0.05; done`, and the build workspace is the test's working directory.
The test now
1. waits (via `eventually`) until `branch-a` is `RUNNING`, so the single executor slot is provably occupied,
2. asserts that `branch-b` is `PENDING` — which cannot race anything, because `branch-a` cannot finish before the gate file exists,
3. creates the gate file, and only then awaits both builds' `SUCCESS`.
The assertion on the event transitions (`branch-b` goes `RUNNING` only after `branch-a` reached `SUCCESS`) is unchanged.
A blocking gate was chosen over a mocked `BuildRunner` because it keeps the test on the real `ProcessBuildRunner`,
so it still covers the actual process handling rather than only the executor's bookkeeping.
Verified by running `BuildExecutorTest` five times on an idle machine and three more times with twice `nproc` busy-loops saturating the CPU,
which is the condition that produced the original failure.
- [BuildExecutorTest](../../src/test/kotlin/de/hoennig/werkator/build/BuildExecutorTest.kt)
@@ -0,0 +1,30 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
The repository's home is `https://git.javagil.de/mi/werkator.git` since the move to the own Gitea instance,
but `tools/remote` still defaulted `WERKATOR_REPO_URL` to the GitHub mirror.
The mih09 production instance was cloned from that mirror and consequently watched a `main` that nobody pushes to any more:
it kept reporting the last GitHub state as green while three merged pull requests sat unbuilt on the real `main`.
The failure that started this — a flaky test already fixed on Gitea's `main` — could not be re-verified live,
because the instance had no way to see the fix.
## Non-Goals
- The existing clone on mih09 is not touched by this change; its remote was repointed by hand (`git remote set-url`), and `repo-init` skips an existing clone.
- The generic placeholder URLs in `docs/deployment.md` stay as they are — they describe cloning *any* watched repository, not werkator's own.
- No decision about mirroring to GitHub; the mirror simply stops being the source an instance builds from.
## The Solution
`REPO_URL` in [tools/remote](../../tools/remote) defaults to the Gitea URL, and the usage comment says so.
The value stays overridable via `WERKATOR_REPO_URL` in the instance's env file,
so an installation that deliberately watches a different remote is unaffected.
Anonymous HTTPS works against Gitea exactly as it did against GitHub, so no deploy key or token is involved.
Note that pull requests opened via AGit-Flow create no branch in Gitea,
so an instance watching this remote sees `main` only — branch builds require pushing real branches.
@@ -0,0 +1,90 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
`init --systemd` generates the host integration — the systemd unit's resource limits, the Apache `.htaccess` and the maintenance page — from the effective configuration.
It read that configuration through a helper with two independent defects, both of which fail silently.
**It swallowed every error.**
The load sat in `try { … } catch (_: Exception) { ServerConfig() }`.
Any configuration error at all — a missing required field, a malformed layer, a version floor violation — produced a default `ServerConfig` with a blank `publicBaseUrl`.
A repository with a broken `.werkator.yml` then looked exactly like one that simply has no public base URL configured:
the `.htaccess` and the maintenance page were skipped without a word.
This surfaced while verifying [PR#17](2026-09-03-PR%2317-maintenance-page.md) on mih09, where the missing files looked like an unconfigured `publicBaseUrl` and were in fact an unrelated validation error.
**It read from the wrong directory.**
The helper called `configLoader.load(Paths.get("."))` — the process's current directory — while everything else in the command works off the git top level resolved by `GitService.getTopLevel`.
`ConfigLoader.loadRaw` resolves the layers directly under the directory it is given and does not walk up to the repository root, so the two agree only when `init` happens to be invoked from the root itself.
From a subdirectory the command read another repository's configuration, or none.
That also broke `--apply`: the fragment is installed into the repository root deliberately before the systemd files are written, so that its port and limits reach the generated unit, and a current-directory read does not see it.
`init --systemd` runs during initial deployment setup, which is exactly when a silent wrong answer is most expensive.
## Non-Goals
- The fallback itself is kept: a configuration that cannot be loaded is not fatal for `init`, the units are still generated with the defaults.
During the very first bootstrap there is legitimately nothing to load yet.
- No change to `ConfigLoader`, to the configuration schema, or to any other command.
- No sweep for catch-all exception handlers elsewhere in the code base;
the two other `catch` blocks in `InitCommand` already print an `Error:` and abort, so they were only checked, not changed.
## The Scenarios
### Feature: init reports what it read and where it read it from
#### Background
- The *repository root* is the git top level as resolved by `GitService.getTopLevel`, the directory holding `.werkator.yml`, `.git/werkator/.werkator.yml` and an applied fragment.
- The *current directory* is the process working directory, which is the repository root only when `init` is invoked there.
#### Scenario#22.01: A broken configuration is named, not defaulted over
So that a validation error during deployment setup is not mistaken for an unconfigured installation.
- **Given** a repository whose effective configuration cannot be loaded
- **When** `init --systemd` runs
- **Then** the exception message is printed as a warning
- **and** the unit files are still generated with the default settings
- **and** the warning appears exactly once, although three settings are read from the configuration
##### Verified by
- [InitCommandTest: `--systemd warns once when the effective configuration cannot be loaded`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
#### Scenario#22.02: The configuration is read from the repository root
So that the generated host integration reflects the repository being initialized, whatever directory `init` was invoked from.
- **Given** a repository whose root configuration sets `server.publicBaseUrl` and `server.port`
- **and** a current directory that is not that repository root
- **When** `init --systemd` runs
- **Then** the `.htaccess` and the maintenance page are generated
- **and** the `.htaccess` proxies to the port from the root configuration
##### Verified by
- [InitCommandTest: `--systemd reads the configuration from the repository root, not the current directory`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
## The Solution
The catch-all now prints the exception message before falling back:
```
Warning: the effective configuration could not be loaded (<message>)
continuing with default server settings — check the generated unit and host files
```
The configuration is read three times while the systemd files are written (`memoryMax`, `tasksMax`, `publicBaseUrl`), which would repeat the warning three times.
It is therefore loaded once per run and cached in the command, and the cache is reset at the top of `run()` so a reused instance — the command is a Spring singleton — re-reads.
The repository root is passed down into the two accessors instead of `Paths.get(".")`.
This matches every other caller of `ConfigLoader.load` in the code base, all of which pass an explicit working directory;
`InitCommand` was the only one relying on the process's current directory.
Both fixes are the same failure in two forms — the command answered from a configuration it never actually read — which is why they are in one PR.
## Additional Changes
- None.
@@ -0,0 +1,92 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- ADR 0009 — multi-repo instance: the `defaults` block carries shared repository-level keys, read by `ConfigLoader`, never directly by consumers.
- `docs/deployment.md` — registry setup: `repo-add` clones, initialises, and prints the registry entry.
## The Problem
`tools/remote werkator repo-add <private-https-url>` fails on the host with `could not read Username`.
The clone runs anonymously, but the credentials exist only in the instance file's `defaults.git` block — which the script never consults.
Registering a private repository therefore needs a manual SSH session today.
## Non-Goals
The script still does not write `~/.werkator.yml`: registering stays the operator's decision.
No SSH-URL support: the forge is reached over `https` from the host.
No new config keys: `defaults.git.account`/`defaults.git.token` already exist.
## The Scenarios
### Feature: authenticated clone for private origins
#### Background
- The shared credentials live in `~/.werkator.yml` under `defaults.git` (ADR 0009).
- Public origins and instances without shared credentials must keep cloning anonymously.
#### Scenario#000.01: Private https origin clones with shared credentials
- **Given** `defaults.git.account`/`defaults.git.token` in `~/.werkator.yml` on the host
- **When** `tools/remote werkator repo-add <private-https-url>` runs
- **Then** the clone authenticates with those credentials and succeeds.
##### Verified by
- Manual stub-`git` test: URL carries `account@`, askpass answers the token, `GIT_TERMINAL_PROMPT=0`.
#### Scenario#000.02: Public origin clones anonymously
- **Given** no shared credentials (or a public repository)
- **When** `repo-add` or `repo-init` runs
- **Then** the clone runs exactly as before, without authentication.
##### Verified by
- Local helper test against a `file://` origin with and without an instance file.
#### Scenario#000.03: Token never leaks locally
- **Given** an authenticated clone
- **When** the command runs from the workstation
- **Then** the token appears in neither the local process list nor a repository config.
##### Verified by
- Code inspection: the token is read on the host and passed via a one-shot `GIT_ASKPASS` script.
## The Solution
`tools/remote` gained a `clone_repo` helper used by both `repo-init` and `repo-add`.
For `https://` URLs it ships a small Python helper (base64-encoded, so no `$` is expanded locally) to the host.
The helper reads `defaults.git.account`/`defaults.git.token` from `~/.werkator.yml`, puts the account into the URL, and hands the token to git via a one-shot `0700` `GIT_ASKPASS` script deleted in `finally`.
Without credentials it falls back to the plain anonymous clone; non-`https` URLs clone unchanged.
`docs/deployment.md` documents that the shared credentials must exist before cloning a private repository.
## Open Questions
- None.
## Attachments
### Adding a Gitea repository — example
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
Add the printed entry to `~/.werkator.yml` under `repositories:`, then restart the service.
A public repository needs nothing else: the clone runs anonymously.
A private repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning (token: Gitea → Settings → Applications → Generate Token, scope `read:repository`):
```yaml
defaults:
git:
account: <gitea-user>
token: <token>
```
@@ -0,0 +1,102 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- Werkdock `CHANGELOG.md` 0.2.0 (repository `mi/werkdock`) — the release notes this PR adapts to: `import` replaces `load -i --name` for rootfs archives, `load` is reserved for docker/OCI image archives, image names gained docker-style tags.
- ADR 0008 — the bwrap build runtime; `docs/configuration.md`, notes on `builds.<name>.werkdock`.
## The Problem
Werkdock moves its CLI closer to `docker run --rm`.
For the rootfs archives Werkator uses, the verb is now `werkdock import ARCHIVE IMAGE` (docker import semantics).
`werkdock load -i ARCHIVE --name IMAGE` still works but is a compatibility path with a note on stderr, and is announced to go away once Werkator has switched.
Werkator ships the werkdock binary with its deployment, but nothing forces the two to be updated together: an installation may run a new Werkator against an older werkdock for a while, or the other way round.
## Non-Goals
- No switch of the existence check from `werkdock images` to `werkdock inspect`: `images` prints the bare name for untagged images on every werkdock version, the exact-line match keeps working, and `inspect` would tie Werkator to the new werkdock.
- No tagged image name (`werkator-buildenv:<hash>`): it would re-import every build environment on the webspace and orphan the old image, for no functional gain today.
- None of the new `run` flags (`--mount`, `--entrypoint`, `--network host`): Werkator's invocation needs none of them.
## The Scenarios
### Feature: rootfs archives are imported with werkdock's docker-shaped verb
#### Background
- The build environment is a rootfs archive named by `builds.<name>.werkdock.rootfs`.
- Werkator creates the werkdock image `werkator-buildenv-<hash>` from it once per source.
- Werkdock answers an unknown verb with `werkdock: unknown command "import"` and exit code 125, its code for its own errors.
#### Scenario#000.01: A missing image is imported with `werkdock import`
So that Werkator uses the verb werkdock names for rootfs archives, and the deprecated path can be removed on werkdock's side.
- **Given** `werkdock images` does not list the image
- **When** a build starts
- **Then** Werkator runs `werkdock import ARCHIVE werkator-buildenv-<hash>`
- **and** runs no `werkdock load`.
##### Verified by
- [WerkdockBuildRunnerTest "imports the image once when werkdock does not know it yet"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
- [WerkdockBuildRunnerTest "downloads a URL rootfs once before importing it"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
#### Scenario#000.02: An older werkdock without the verb still works
So that Werkator and werkdock can be updated in either order.
- **Given** the installed werkdock answers `import` with `unknown command` and exit 125
- **When** a build starts with a missing image
- **Then** Werkator falls back to `werkdock load -i ARCHIVE --name werkator-buildenv-<hash>`
- **and** logs the fallback.
##### Verified by
- [WerkdockBuildRunnerTest "falls back to load -i --name on a werkdock without the import verb"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
#### Scenario#000.03: A real import failure is not masked by the fallback
So that a broken archive fails the build with werkdock's message, as before.
- **Given** `werkdock import` fails for any other reason (any other exit code, or exit 125 without `unknown command`)
- **When** a build starts with a missing image
- **Then** the build fails with that command's output
- **and** no `werkdock load` runs.
##### Verified by
- [WerkdockBuildRunnerTest "propagates an import failure that is not a missing verb"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
#### Scenario#000.04: An existing image is neither imported nor loaded
- **Given** `werkdock images` lists the image
- **When** a build starts
- **Then** neither `import` nor `load` runs.
##### Verified by
- [WerkdockBuildRunnerTest "does not import an image werkdock already has"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
## The Solution
`WerkdockBuildRunner.ensureImage` calls the new `importImage`: `werkdock import ARCHIVE IMAGE` through the non-throwing `run`, then either returns, falls back to `load -i --name` on exactly the unknown-verb signature (exit 125 plus `unknown command` on stderr), or rethrows the import's result as a `GitCommandException` — the same exception and message the old code produced.
The fallback is keyed on werkdock's own error signature rather than on parsing `werkdock version`: the signature is what actually decides whether the verb exists, and it needs no version table in Werkator.
## Open Questions
- When to drop the fallback: once every installation runs a werkdock with `import`, the `load -i --name` branch and its test go, and werkdock can remove the compatibility path.
## Additional Changes
- `docs/configuration.md` and `docs/deployment.md` say "imported" where they said "loaded", and name the fallback and the reason the image name stays untagged.
## Prerequisite PRs
- Werkdock 0.2.0 (repository `mi/werkdock`), which introduces `import`; against an older werkdock the fallback path runs.
## Follow-up PRs
- Remove the fallback (see Open Questions).
+151
View File
@@ -0,0 +1,151 @@
# RFC 0001: Web UI Redesign — the Instrument Panel
**Status:**
- proposed: 2026-09-03
- accepted: -
- rejected: -
**Proposal:** The Werkator web UI adopts the **Instrument Panel** direction: a teal palette in a light and a dark mode, IBM Plex typography, a repository strip that previews the state of every served repository, a title hierarchy that names the view first and explains it second, and a tab bar at the foot that becomes the mobile navigation.
The architecture does not change: server-rendered Thymeleaf, one `werkator.css`, one hand-written `werkator.js`, JSON polling, no framework, no frontend build pipeline.
## Context and Problem Statement
The current UI is a functional port of the legacy generated pages: a table per view, pill badges, system font, blue links.
It is correct and calm, but it looks like every other CI page and gives no hint of the other repositories an instance serves (ADR 0009).
The brief for this RFC was "fancy, but serious and trustworthy", with two references from the same author for visual kinship:
- [werkbaum.javagil.de](https://werkbaum.javagil.de/) — light paper with a fine grid, IBM Plex, a petrol accent, panel labels in small caps.
- [javagil.de/vibe-engineering](https://javagil.de/vibe-engineering) — a dark instrument panel: ink and petrol, clay for warnings, monospaced spaced labels, a tab bar at the foot.
A hard constraint of this RFC is honesty towards the data.
The mockups show only what the API delivers today; nothing is invented to make a screen look richer.
### What the UI Has to Work With
Per build row (`BuildRowView`, `BuildResultDto`): status, branch name, commit (12-character abbreviation, full id for copying), started at (`yyyy-MM-dd HH:mm`), duration (`m:ss`; a pending build shows its wait time in italics), artifact key with the artifact, permalink and live-log links, and the actions restart and delete (history has no restart).
Statuses: `pending`, `running`, `success`, `failed`, `interrupted`, `cancelled`, plus `unknown` for a never-built branch and the client-side `finished` on a card whose build has left the current list.
Views: Latest (one build per name), Branches (every origin branch and its latest build), History (all stored builds), Current (running builds with their live log), System (seven metric rows with current/min/max/avg, warn from 80 %, critical from 90 %), the artifact page, and the release notes.
Live state: the indicator is `static`, `live` or `error`; the watcher banner reports `watcher stopped`, `origin unreachable` or `poll cycle failed`.
Multi-repo: the repository switcher is a server-rendered `<select>` of names; `WatcherState.repositories` already carries a per-repository watcher state that the UI does not show.
What does **not** exist, and therefore appears in no mockup: a commit subject line, a typical or expected duration, an ETA, a per-branch build history, test counts on a row, and any cross-repository status summary in the API.
## Considered Options
Six directions were sketched on a shared design canvas, two rounds of three, all with the same sample rows.
| Option | Idea | Why | Tradeoff |
|---|---|---|---|
| A · Quiet Console | Today's design refined: top bar, dot-plus-word statuses, hover actions | Smallest step, everything stays valid | Least distinctive |
| B · Mission Board | Health tiles, one card per branch with a history strip, running build with progress | Answers "is everything fine?" at a glance | Needs data the API does not have (history, typical duration) |
| C · Ledger | Warm paper, serif masthead, hairline rules, typographic status marks | The most "serious"; reads like a signed record | Leaves the system font, needs its own dark theme |
| D · Paper Rail | Werkbaum's paper and grid, a repository rail on the left with per-branch dots | Family resemblance to Werkbaum; other repositories visible | 250 px of table width lost; empty with one repository |
| **E · Instrument Panel** | Vibe-Engineering's dark panel, repositories as tiles, tab bar at the foot | Reads like a control room; failures in clay stay serious without alarm | Dark-only as drawn; needs a light palette |
| F · Fleet Overview | A new landing page with one panel per repository, ledger typography on paper | One page answers the question for the whole instance | Becomes a list beyond five repositories |
Round one (AC) still contained invented data; it is kept on the canvas for the visual ideas only.
**E was chosen**, and round three worked out what it lacked: the light mode, the ten-repository case, the title hierarchy, and the phone layout.
## The Design
### Palette
Both modes are CSS custom properties on `:root`, switched by `prefers-color-scheme` as today (`color-scheme: light dark`).
Failures use clay, not red, so they stay serious without shouting; the accent is teal in both modes.
| Token | Dark | Light | Used for |
|---|---|---|---|
| bg | `#061C1F` | `#EAF4F2` | page ground |
| panel | `#0A2A2E` | `#FFFFFF` | tables, cards, chips |
| panel-2 | `#0F3A3D` | `#D6ECE8` | the current repository, the active tab |
| line | `#17474B` | `#C9DFDB` | borders and rules |
| text | `#E4EEEC` | `#0B2B2E` | body text |
| text-2 | `#B4CBC8` | `#35595B` | timestamps |
| muted | `#7DA19E` | `#5E8583` | labels, footers |
| accent | `#5FD3C7` | `#0E8079` | links, success, running, the live indicator |
| accent-2 | `#1E9A93` | `#149A90` | underlines, the current repository's border |
| clay | `#E09070` | `#B0563B` | failed, error, delete, watcher warnings |
| clay-2 | `#C4664A` | `#C4664A` | the border of a failing repository chip |
| ghost | `#4A7370` | `#BFD4D1` | cancelled, interrupted, unknown |
Tinted rows: a running row gets 16 % (dark) or 10 % (light) of accent-2 as background, a failed row 12 % or 10 % of clay-2.
The reference's background grid was tried and dropped: it competes with the table, especially in light mode.
### Typography
IBM Plex Sans for text, IBM Plex Mono for commits, timestamps, durations and every label.
Labels are 10 px Mono, uppercase, letter-spaced 0.12 em, in `muted`; statuses are 11 px Mono uppercase in their status color, each preceded by an 8 px dot (outlined for pending, pulsing for running).
Fallback stacks: `"IBM Plex Sans", "Segoe UI", system-ui, sans-serif` and `"IBM Plex Mono", ui-monospace, Consolas, monospace`.
Whether Plex is bundled under `static/` or the fallback stack is accepted is an open question below.
### Anatomy of a Page (desktop)
1. **Header**, 52 px: logo, `Werkator` with the Gitea repository name in accent, a small label `updated HH:mm:ss`; right: the live indicator as an outlined chip with a pulsing dot, the reload button.
2. **Repository strip**: see below.
3. **Panel** with the view's title: the view name at 22 px semibold with a 2 px accent-2 underline, followed by a one-line label that explains it (`Latest``one build per branch, newest first`; `Branches``every origin branch and its latest build`; `History``all stored builds, newest first`; `System``instance metrics since first start`); on the right a Mono line with the row count, the last poll and the watcher state.
4. **Table**, columns as today (Status, Branch, Commit, Started, Duration, Artifacts, Actions), rows 9 px padding on a 1 px `line` rule; copy buttons as outlined 13 px icons; artifact links and actions as stroke icons (no emoji).
5. **Footer**: version and copyright left, the navigation as a Mono tab bar in the middle (Latest, Branches, History, System with icons; the active tab in panel-2 with an accent underline), Impressum and Privacy right.
### The Repository Strip
The `<select>` switcher is replaced by a strip below the header that shows every served repository with its state, so a failure elsewhere is visible without leaving the page.
- Up to about three repositories: **tiles** (220 px), each with `current` or `repo` label, the name, one dot per branch in the branch's latest status, a summary line (`6 builds · 1 failed · 1 running`), and the watcher warning in clay when that repository's watcher reports an error.
- More repositories: **chips** (30 px), each with one dot for the worst status in the repository, the name, an optional short finding (`1 failed`, `main`, `never built`), and a warning triangle when the watcher reports an error; the current repository has an accent-2 border on panel-2, a failing one a clay-2 border on the clay tint.
- Order is *failing first*: the current repository, then failing, running, then green; a summary line above (`10 served · 2 failing · 1 unreachable · 2 running`) and a sort control on the right.
- The strip **scrolls**: horizontally on the phone, and on the desktop it wraps to a second row up to about ten repositories and becomes a horizontally scrollable band beyond that, with the failing chips pinned at the front so they never scroll out of view.
- Beyond roughly twenty repositories the strip shows only the conspicuous chips (failing, running, unreachable) plus a search field for the rest.
- With a single served repository the strip is omitted, as the switcher is today.
### Phone (below 680 px)
The existing breakpoint behaviour is kept and restyled: rows become cards with the `data-label` captions, the live indicator collapses to a dot.
The header stacks `Werkator` over the repository name; the repository strip scrolls horizontally under its summary line; the panel title keeps its hierarchy; each card carries the status line with the branch, then commit, started and duration, then the artifact icons and the actions as 44 px targets.
The footer tab bar becomes a fixed bottom tab bar with icons — the same four entries as on the desktop.
No painted status bar or keyboard; the device provides those.
### What Is Deliberately Not in the Proposal
- The `DE` language button in the mockups is a leftover of the reference; the UI stays English-only.
- No commit subjects, typical durations, ETAs or history strips: they need data the server does not have, and each would be its own RFC with its own storage.
- No manual theme toggle; `prefers-color-scheme` decides, as today.
## Consequences
### Backend
- One new endpoint, `GET /api/repos`: for every served repository its name, its UI root (`/repos/<name>`), whether it is the current one, the latest status per build name (the Latest view's `latestPerName` reduced to counts, plus the worst status), and its `RepoWatcherState` (`lastFetchError`, `lastPollError`, `lastPollAt`).
With one served repository the endpoint returns a list of one and the strip stays hidden.
- `werkator.js` polls it on the table interval (10 s) and renders the strip; every fetch keeps the timeout and the explicit error badge.
- `UiFormats` and `werkator.js` keep producing identical formats; the palette and the title labels are template and CSS only.
### Rollout, One Concern per Pull Request
1. Palette, typography and the title hierarchy in `werkator.css` and the fragments — no data change, both modes.
2. Header and footer tab bar, including the phone tab bar.
3. `GET /api/repos` and the repository strip, replacing the `<select>`.
4. Card refinements on the phone and the System and artifact pages in the new vocabulary.
Each step leaves the UI usable, and the tests in `server` that assert on markup are adjusted with the step that changes it.
## Open Questions
- **Fonts:** bundle IBM Plex Sans and Mono under `static/fonts/` (about 100150 KB in WOFF2 for the four faces), or accept the fallback stack on hosts without the font; the reference sites load Plex from a CDN, which the deployment behind a strict reverse proxy may not want.
- **Current view:** it is reachable today only from a running row's live icon; the tab bar has room for it as a fifth entry with a count badge, or it stays a link from the row.
- **Instance pages:** `/system` and `/releases` are instance-level; in the tab bar they sit next to the per-repository views, which the strip makes visible enough, or they move to the footer's right side.
## Design Sources
The design canvas with all eleven artboards (rounds one to three, desktop and phone) is a private Claude artifact of the author; its renderings live next to this RFC under `0001-web-ui-instrument-panel/`.
The sample rows are real field shapes with invented values; the repositories other than `werkator` are invented.
The proposal:
- [E · dark, desktop](0001-web-ui-instrument-panel/e-dark-desktop.png) · [E · light, desktop](0001-web-ui-instrument-panel/e-light-desktop.png)
- [E · dark, ten repositories](0001-web-ui-instrument-panel/e-dark-10-repos.png) · [E · light, ten repositories](0001-web-ui-instrument-panel/e-light-10-repos.png)
- [E · dark, phone](0001-web-ui-instrument-panel/e-dark-phone.png) · [E · light, phone](0001-web-ui-instrument-panel/e-light-phone.png)
The alternatives, for the record:
- [A · Quiet Console](0001-web-ui-instrument-panel/a-quiet-console.png), [B · Mission Board](0001-web-ui-instrument-panel/b-mission-board.png), [C · Ledger](0001-web-ui-instrument-panel/c-ledger.png) — round one, still with invented data.
- [D · Paper Rail](0001-web-ui-instrument-panel/d-paper-rail.png), [F · Fleet Overview](0001-web-ui-instrument-panel/f-fleet-overview.png) — round two.
Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 171 KiB

@@ -2,6 +2,7 @@ package de.hoennig.werkator.build
import de.hoennig.werkator.config.BranchConfig import de.hoennig.werkator.config.BranchConfig
import de.hoennig.werkator.config.WerkdockConfig import de.hoennig.werkator.config.WerkdockConfig
import de.hoennig.werkator.git.GitCommandException
import de.hoennig.werkator.git.GitCommandRunner import de.hoennig.werkator.git.GitCommandRunner
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.stereotype.Component import org.springframework.stereotype.Component
@@ -16,9 +17,9 @@ import java.security.MessageDigest
* it shells out to the `werkdock` CLI (`werkdock.binary`, default via PATH) — the same * it shells out to the `werkdock` CLI (`werkdock.binary`, default via PATH) — the same
* pattern as git and docker, CLI, no library. * pattern as git and docker, CLI, no library.
* *
* The rootfs archive becomes a werkdock *image*, loaded once per source * The rootfs archive becomes a werkdock *image*, imported once per source
* (`werkator-buildenv-<hash>`, the hash over the source string, so a changed source * (`werkator-buildenv-<hash>`, the hash over the source string, so a changed source
* loads a fresh image) into werkdock's own store (`$WERKDOCK_HOME`, default * imports a fresh image) into werkdock's own store (`$WERKDOCK_HOME`, default
* `~/.werkdock`) — shared by every repository of this OS user, unlike the old * `~/.werkdock`) — shared by every repository of this OS user, unlike the old
* per-repo unpack. Only the download cache for URL sources and the persistent * per-repo unpack. Only the download cache for URL sources and the persistent
* toolchain home (bound to `/root` for Gradle/Go caches) stay under * toolchain home (bound to `/root` for Gradle/Go caches) stay under
@@ -66,9 +67,11 @@ class WerkdockBuildRunner(
} }
/** /**
* Loads the rootfs archive into the werkdock image store once per source. * Imports the rootfs archive into the werkdock image store once per source.
* `werkdock images` answers existence through the CLI, like `docker image * `werkdock images` answers existence through the CLI, like `docker image
* inspect` does for the Docker runner. * inspect` does for the Docker runner; it prints the bare name for an untagged
* image on every werkdock version, so the exact-line match holds across the
* werkdock upgrade that introduced tags.
*/ */
private fun ensureImage( private fun ensureImage(
werkdock: String, werkdock: String,
@@ -84,7 +87,33 @@ class WerkdockBuildRunner(
val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(sandbox.rootfs)) val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(sandbox.rootfs))
Files.createDirectories(envDir) Files.createDirectories(envDir)
val archive = localArchive(sandbox.rootfs, envDir, repoDir, onAuxProcess) val archive = localArchive(sandbox.rootfs, envDir, repoDir, onAuxProcess)
log.info("loading build environment {} as werkdock image {}", sandbox.rootfs, image) log.info("importing build environment {} as werkdock image {}", sandbox.rootfs, image)
importImage(werkdock, archive, image, repoDir, onAuxProcess)
}
/**
* `werkdock import ARCHIVE IMAGE` (docker import semantics) creates the image from a
* rootfs archive. A werkdock that predates the verb answers `unknown command` with
* exit 125; then the older `load -i ARCHIVE --name IMAGE` does the same job, so an
* installation can update Werkator and werkdock in either order. Any other failure
* is the import's own and propagates as it did before.
*/
private fun importImage(
werkdock: String,
archive: String,
image: String,
repoDir: Path,
onAuxProcess: (Process) -> Unit,
) {
val importCommand = listOf(werkdock, "import", archive, image)
val imported = commandRunner.run(importCommand, repoDir, onProcess = onAuxProcess)
if (imported.isSuccess) {
return
}
if (imported.exitCode != WERKDOCK_CLI_ERROR || "unknown command" !in imported.stderr) {
throw GitCommandException(importCommand, imported)
}
log.info("this werkdock has no import verb yet, falling back to load -i --name")
commandRunner.runOrThrow( commandRunner.runOrThrow(
listOf(werkdock, "load", "-i", archive, "--name", image), listOf(werkdock, "load", "-i", archive, "--name", image),
repoDir, repoDir,
@@ -209,5 +238,8 @@ class WerkdockBuildRunner(
companion object { companion object {
const val BUILDENV_DIR = ".git/werkator/buildenv" const val BUILDENV_DIR = ".git/werkator/buildenv"
const val HOME_DIR = "home" const val HOME_DIR = "home"
/** werkdock's exit code for its own errors (docker's 125), as opposed to the sandboxed command's. */
const val WERKDOCK_CLI_ERROR = 125
} }
} }
@@ -48,6 +48,7 @@ class InitCommand(
internal var javaExecutableResolver: () -> Path = { Paths.get(System.getProperty("java.home"), "bin", "java") } internal var javaExecutableResolver: () -> Path = { Paths.get(System.getProperty("java.home"), "bin", "java") }
override fun run() { override fun run() {
cachedServerConfig = null
val normalizedWorkingDir = workingDir.toAbsolutePath().normalize() val normalizedWorkingDir = workingDir.toAbsolutePath().normalize()
val root = val root =
try { try {
@@ -88,7 +89,7 @@ class InitCommand(
if (url == null) return DetectedValues() if (url == null) return DetectedValues()
if (url.startsWith("http")) { if (url.startsWith("http")) {
val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/([^/.]+)(?:\.git)?""") val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/(.+?)(?:\.git)?$""")
val match = regex.find(url) val match = regex.find(url)
if (match != null) { if (match != null) {
val (user, host, owner, repo) = match.destructured val (user, host, owner, repo) = match.destructured
@@ -101,7 +102,7 @@ class InitCommand(
} }
} else if (url.contains("@") && url.contains(":")) { } else if (url.contains("@") && url.contains(":")) {
// Assume SSH: git@host:owner/repo.git // Assume SSH: git@host:owner/repo.git
val regex = Regex("""([^@]+)@([^:]+):([^/]+)/([^/.]+)(?:\.git)?""") val regex = Regex("""([^@]+)@([^:]+):([^/]+)/(.+?)(?:\.git)?$""")
val match = regex.find(url) val match = regex.find(url)
if (match != null) { if (match != null) {
val (_, host, owner, repo) = match.destructured val (_, host, owner, repo) = match.destructured
@@ -296,14 +297,31 @@ class InitCommand(
* already loadable (re-running `init --systemd` on an installed instance); during * already loadable (re-running `init --systemd` on an installed instance); during
* the very first bootstrap they stay unset and the defaults (no directives) apply. * the very first bootstrap they stay unset and the defaults (no directives) apply.
*/ */
private fun loadedSystemdConfig(): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig().systemd private fun loadedSystemdConfig(root: Path): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig(root).systemd
private fun loadedServerConfig(): de.hoennig.werkator.config.ServerConfig = /** Loaded once per run, so a broken configuration is reported once and not per caller. */
private var cachedServerConfig: de.hoennig.werkator.config.ServerConfig? = null
/**
* Read from the repository root like every other file this command touches — the
* layers sit there, not in whatever directory the process happens to run in, and
* an applied fragment must reach the generated unit even when `init` is invoked
* from a subdirectory.
*
* A configuration error here is not fatal — the units are still generated with defaults —
* but it must not pass for "nothing configured": without the warning a broken `.werkator.yml`
* looks exactly like an unset `publicBaseUrl` and the host integration is skipped silently.
*/
private fun loadedServerConfig(root: Path): de.hoennig.werkator.config.ServerConfig =
cachedServerConfig ?: run {
try { try {
configLoader.load(Paths.get(".")).server configLoader.load(root).server
} catch (_: Exception) { } catch (e: Exception) {
println("Warning: the effective configuration could not be loaded (${e.message})")
println(" continuing with default server settings — check the generated unit and host files")
de.hoennig.werkator.config de.hoennig.werkator.config
.ServerConfig() .ServerConfig()
}.also { cachedServerConfig = it }
} }
private fun createSystemdFiles( private fun createSystemdFiles(
@@ -327,8 +345,8 @@ class InitCommand(
javaExecutable = javaExecutableResolver(), javaExecutable = javaExecutableResolver(),
jarPath = jarPath, jarPath = jarPath,
envFile = envFile, envFile = envFile,
memoryMax = loadedSystemdConfig().memoryMax, memoryMax = loadedSystemdConfig(root).memoryMax,
tasksMax = loadedSystemdConfig().tasksMax, tasksMax = loadedSystemdConfig(root).tasksMax,
), ),
) )
println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}") println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}")
@@ -351,7 +369,7 @@ class InitCommand(
// generated host integration like the units: only meaningful behind a web // generated host integration like the units: only meaningful behind a web
// frontend, so it needs a public base URL; unused elsewhere and harmless // frontend, so it needs a public base URL; unused elsewhere and harmless
val server = loadedServerConfig() val server = loadedServerConfig(root)
if (server.publicBaseUrl.isNotBlank()) { if (server.publicBaseUrl.isNotBlank()) {
val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME) val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME)
htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port)) htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port))
@@ -505,6 +505,8 @@ class BuildExecutorTest : FunSpec() {
} }
test("with maxConcurrent 1 a second branch stays PENDING until the first finished") { test("with maxConcurrent 1 a second branch stays PENDING until the first finished") {
// branch-a blocks on a gate file the test creates, so the PENDING assertion
// below cannot race the first build finishing on a loaded machine
val h = val h =
Harness( Harness(
""" """
@@ -512,7 +514,7 @@ class BuildExecutorTest : FunSpec() {
maxConcurrent: 1 maxConcurrent: 1
branches: branches:
branch-a: branch-a:
buildCommand: "sleep 1" buildCommand: "until [ -f gate ]; do sleep 0.05; done"
cleanCommand: "" cleanCommand: ""
branch-b: branch-b:
buildCommand: "echo ok" buildCommand: "echo ok"
@@ -523,8 +525,13 @@ class BuildExecutorTest : FunSpec() {
h.executor.startBuild(h.repo, "branch-a", "sha-a") h.executor.startBuild(h.repo, "branch-a", "sha-a")
h.executor.startBuild(h.repo, "branch-b", "sha-b") h.executor.startBuild(h.repo, "branch-b", "sha-b")
eventually(30.seconds) {
h.repository.latestFor("branch-a")?.status shouldBe BuildStatus.RUNNING
}
h.repository.latestFor("branch-b")?.status shouldBe BuildStatus.PENDING h.repository.latestFor("branch-b")?.status shouldBe BuildStatus.PENDING
Files.createFile(h.workingDir.resolve("gate"))
awaitStatus(h, "branch-b", BuildStatus.SUCCESS) awaitStatus(h, "branch-b", BuildStatus.SUCCESS)
awaitStatus(h, "branch-a", BuildStatus.SUCCESS) awaitStatus(h, "branch-a", BuildStatus.SUCCESS)
val transitions = h.events.map { it.result.branch to it.result.status } val transitions = h.events.map { it.result.branch to it.result.status }
@@ -2,6 +2,7 @@ package de.hoennig.werkator.build
import de.hoennig.werkator.config.BranchConfig import de.hoennig.werkator.config.BranchConfig
import de.hoennig.werkator.config.WerkdockConfig import de.hoennig.werkator.config.WerkdockConfig
import de.hoennig.werkator.git.GitCommandException
import de.hoennig.werkator.git.GitCommandResult import de.hoennig.werkator.git.GitCommandResult
import de.hoennig.werkator.git.GitCommandRunner import de.hoennig.werkator.git.GitCommandRunner
import io.kotest.assertions.throwables.shouldThrow import io.kotest.assertions.throwables.shouldThrow
@@ -37,7 +38,11 @@ class WerkdockBuildRunnerTest : FunSpec() {
private fun imageName(rootfs: String = "/srv/buildenv.tar.zst"): String = "werkator-buildenv-${rootfs.sha12()}" private fun imageName(rootfs: String = "/srv/buildenv.tar.zst"): String = "werkator-buildenv-${rootfs.sha12()}"
/** The image is already loaded: `werkdock images` lists it, so no load runs. */ private fun importCommand(): List<String> = listOf("werkdock", "import", "/srv/buildenv.tar.zst", imageName())
private fun loadCommand(): List<String> = listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName())
/** The image is already there: `werkdock images` lists it, so no import runs. */
private fun givenImageLoaded(rootfs: String = "/srv/buildenv.tar.zst") { private fun givenImageLoaded(rootfs: String = "/srv/buildenv.tar.zst") {
every { commandRunner.runOrThrow(listOf("werkdock", "images"), repoDir, any(), any()) } returns every { commandRunner.runOrThrow(listOf("werkdock", "images"), repoDir, any(), any()) } returns
GitCommandResult(0, imageName(rootfs) + "\n", "") GitCommandResult(0, imageName(rootfs) + "\n", "")
@@ -88,34 +93,47 @@ class WerkdockBuildRunnerTest : FunSpec() {
) )
} }
test("loads the image once when werkdock does not know it yet") { test("imports the image once when werkdock does not know it yet") {
givenImageMissing() givenImageMissing()
every { every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns GitCommandResult(0, "", "")
commandRunner.runOrThrow(
listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName()),
repoDir,
any(),
any(),
)
} returns GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig()) runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify { verify { commandRunner.run(importCommand(), repoDir, any(), any()) }
commandRunner.runOrThrow( verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName()),
repoDir,
any(),
any(),
)
}
} }
test("does not load an image werkdock already has") { test("falls back to load -i --name on a werkdock without the import verb") {
givenImageMissing()
every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns
GitCommandResult(125, "", "werkdock: unknown command \"import\"\n")
every { commandRunner.runOrThrow(loadCommand(), repoDir, any(), any()) } returns GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify { commandRunner.runOrThrow(loadCommand(), repoDir, any(), any()) }
}
test("propagates an import failure that is not a missing verb") {
givenImageMissing()
every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns
GitCommandResult(125, "", "werkdock: unpacking /srv/buildenv.tar.zst failed\n")
val exception =
shouldThrow<GitCommandException> {
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
}
exception.message shouldContain "unpacking"
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
}
test("does not import an image werkdock already has") {
givenImageLoaded() givenImageLoaded()
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig()) runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify(exactly = 0) { commandRunner.run(match { "import" in it }, any(), any(), any()) }
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) } verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
} }
@@ -209,7 +227,7 @@ class WerkdockBuildRunnerTest : FunSpec() {
exception.message shouldContain "werkdock.rootfs" exception.message shouldContain "werkdock.rootfs"
} }
test("downloads a URL rootfs once before loading it") { test("downloads a URL rootfs once before importing it") {
val url = "https://example.test/buildenv.tar.zst" val url = "https://example.test/buildenv.tar.zst"
val downloadTarget = val downloadTarget =
repoDir repoDir
@@ -219,7 +237,7 @@ class WerkdockBuildRunnerTest : FunSpec() {
givenImageMissing() givenImageMissing()
every { commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) } returns every { commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) } returns
GitCommandResult(0, "", "") GitCommandResult(0, "", "")
every { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) } returns every { commandRunner.run(match { "import" in it }, any(), any(), any()) } returns
GitCommandResult(0, "", "") GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig(rootfs = url)) runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig(rootfs = url))
@@ -228,8 +246,8 @@ class WerkdockBuildRunnerTest : FunSpec() {
commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any())
} }
verify { verify {
commandRunner.runOrThrow( commandRunner.run(
listOf("werkdock", "load", "-i", downloadTarget.toString(), "--name", "werkator-buildenv-${url.sha12()}"), listOf("werkdock", "import", downloadTarget.toString(), "werkator-buildenv-${url.sha12()}"),
repoDir, repoDir,
any(), any(),
any(), any(),
@@ -109,6 +109,21 @@ class InitCommandTest : FunSpec() {
projectContent shouldContain "repo: my-repo" projectContent shouldContain "repo: my-repo"
} }
test("keeps dots in repository names") {
val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/mi/michael.hoennig.de.git"
initCommand.run()
val projectConfig = tempDir.resolve(".werkator.yml")
val projectContent = projectConfig.toFile().readText()
projectContent shouldContain "owner: mi"
projectContent shouldContain "repo: michael.hoennig.de"
}
test("does not overwrite existing files") { test("does not overwrite existing files") {
val tempDir = Files.createTempDirectory("werkator-init-test") val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir initCommand.workingDir = tempDir
@@ -252,5 +267,50 @@ class InitCommandTest : FunSpec() {
// This should not throw IllegalArgumentException // This should not throw IllegalArgumentException
initCommand.run() initCommand.run()
} }
test("--systemd reads the configuration from the repository root, not the current directory") {
val tempDir = Files.createTempDirectory("werkator-init-test")
// written before the run, so `init` keeps it instead of creating a template
tempDir.resolve(".werkator.yml").toFile().writeText(
"server:\n publicBaseUrl: \"https://werkator.example.org/\"\n port: 18099\n",
)
initCommand.workingDir = tempDir
initCommand.systemd = true
initCommand.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
initCommand.javaExecutableResolver = { Paths.get("/usr/bin/java") }
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
initCommand.run()
// the host integration is generated only when the root's config has a public base URL
val htaccess = tempDir.resolve(".git/werkator/${SystemdServiceFiles.HTACCESS_NAME}")
htaccess.toFile().shouldExist()
htaccess.toFile().readText() shouldContain "18099"
tempDir.resolve(".git/werkator/${SystemdServiceFiles.MAINTENANCE_PAGE_NAME}").toFile().shouldExist()
}
test("--systemd warns once when the effective configuration cannot be loaded") {
val tempDir = Files.createTempDirectory("werkator-init-test")
val brokenLoader = mockk<de.hoennig.werkator.config.ConfigLoader>()
every { brokenLoader.load(any()) } throws IllegalStateException("gitea.owner is required")
val command = InitCommand(gitService, brokenLoader)
command.workingDir = tempDir
command.systemd = true
command.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
command.javaExecutableResolver = { Paths.get("/usr/bin/java") }
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
val console = captureConsole { command.run() }
console.stdout shouldContain "gitea.owner is required"
// the three readers of the configuration must not repeat the warning
console.stdout.windowed("Warning:".length).count { it == "Warning:" } shouldBe 1
// the units are still written with the defaults
tempDir.resolve(".git/werkator/${SystemdServiceFiles.unitName(tempDir)}").toFile().shouldExist()
}
} }
} }
+63 -15
View File
@@ -44,7 +44,7 @@
# Optional in the env file: # Optional in the env file:
# WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start) # WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start)
# WERKATOR_REPO_URL https clone URL of the watched repository # WERKATOR_REPO_URL https clone URL of the watched repository
# (default: https://github.com/mhoennig/werkator.git) # (default: https://git.javagil.de/mi/werkator.git)
# WERKATOR_REPO_DIR directory of the watched repository, absolute or relative to # WERKATOR_REPO_DIR directory of the watched repository, absolute or relative to
# WERKATOR_PATH (default: werkator); it also names the systemd # WERKATOR_PATH (default: werkator); it also names the systemd
# unit, exactly as `init --systemd` derives it # unit, exactly as `init --systemd` derives it
@@ -122,7 +122,7 @@ require_env WERKATOR_REMOTE WERKATOR_PATH
HOST="$WERKATOR_REMOTE" HOST="$WERKATOR_REMOTE"
TARGET_DIR="$WERKATOR_PATH" TARGET_DIR="$WERKATOR_PATH"
ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}" ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}"
REPO_URL="${WERKATOR_REPO_URL:-https://github.com/mhoennig/werkator.git}" REPO_URL="${WERKATOR_REPO_URL:-https://git.javagil.de/mi/werkator.git}"
# The host layout is three values, not one convention: an installation that grew # The host layout is three values, not one convention: an installation that grew
# before this script existed puts them elsewhere, and the defaults are exactly what # before this script existed puts them elsewhere, and the defaults are exactly what
@@ -332,9 +332,65 @@ instance_update() {
echo "==> Instance updated." echo "==> Instance updated."
} }
# Sets up the WATCHED repository: an anonymous https clone (a private origin # Clones one URL into one directory on the host.
# gets its credentials via git.account/git.token in the machine config that # A private https origin authenticates with the shared `defaults.git.account` /
# `werkator init` creates), the werkator init with the instance fragment # `defaults.git.token` of `~/.werkator.yml` (ADR 0009). The whole authenticated
# clone runs in one remote python script: the token is read from the instance
# file on the host and passed to git via a one-shot GIT_ASKPASS script, so it
# appears in neither the local process list nor a repository config.
# Public origins (or an instance without shared credentials) clone anonymously.
clone_repo() {
local url="$1" dest="$2"
if ssh "$HOST" "test -d '$dest/.git'"; then
echo " (already cloned, skipping)"
return 0
fi
case "$url" in
https://*)
# The python helper travels base64-encoded: the clone command itself
# stays a plain `ssh` line, so no `$` inside the script is ever
# expanded by the local shell, and the token never leaves the host.
local helper_b64
helper_b64="$(python3 -c 'import base64,sys; print(base64.b64encode(sys.stdin.read().encode()).decode())' <<'PYEOF_CLONE'
import os, stat, subprocess, sys, tempfile, urllib.parse
url, dest = sys.argv[1], sys.argv[2]
try:
import yaml
cfg = yaml.safe_load(open(os.path.expanduser("~/.werkator.yml"))) or {}
except (FileNotFoundError, ImportError):
cfg = {}
d = (cfg.get("defaults") or {}).get("git") or {}
account, token = d.get("account"), d.get("token")
env = dict(os.environ, GIT_TERMINAL_PROMPT="0")
ask = None
if account and token:
parts = urllib.parse.urlsplit(url)
host = parts.netloc.rsplit("@", 1)[-1]
url = urllib.parse.urlunsplit(parts._replace(netloc=account + "@" + host))
ask = tempfile.NamedTemporaryFile(mode="w", prefix="werkator-clone-askpass-",
suffix=".sh", delete=False)
ask.write("#!/bin/sh\nexec echo \"$WERKATOR_CLONE_TOKEN\"\n")
ask.close()
os.chmod(ask.name, stat.S_IRWXU)
env.update(GIT_ASKPASS=ask.name, WERKATOR_CLONE_TOKEN=token)
try:
subprocess.run(["git", "clone", url, dest], env=env, check=True)
finally:
if ask is not None:
os.unlink(ask.name)
PYEOF_CLONE
)"
ssh "$HOST" "echo '$helper_b64' | base64 -d | python3 - '$url' '$dest'"
;;
*)
ssh "$HOST" "git clone '$url' '$dest'"
;;
esac
}
# Sets up the WATCHED repository: an https clone (a private origin
# authenticates with the shared `defaults.git.*` credentials of
# `~/.werkator.yml`; see `clone_repo`), the werkator init with the instance
# applied, and the rootfs archive for the sandbox builds. All configuration # applied, and the rootfs archive for the sandbox builds. All configuration
# writing is init's — this script transports and invokes (step 23). # writing is init's — this script transports and invokes (step 23).
repo_init() { repo_init() {
@@ -344,11 +400,7 @@ repo_init() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning the watched repository" echo "==> Cloning the watched repository"
if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then clone_repo "$REPO_URL" "$REPO_DIR"
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
fi
if [ "$SANDBOX" = "docker" ]; then if [ "$SANDBOX" = "docker" ]; then
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile" echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
@@ -401,11 +453,7 @@ repo_add() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning $url as '$name'" echo "==> Cloning $url as '$name'"
if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then clone_repo "$url" "$SIBLING_DIR/$name"
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
fi
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock # The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
# binary). Without it a watched repository builds on the bare host, where the # binary). Without it a watched repository builds on the bare host, where the