Author SHA1 Message Date
4db294e1bc Follow-up builds (#23)
A build definition may declare afterSuccessOf, running whenever the named
build of the same branch turns green — the deployment path chosen over a
deployCommand or a separate deploy section.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #23
2026-09-05 20:02:32 +02:00
mhoennig 0f9f119687 init: keep dots in detected repository names 2026-09-05 15:15:12 +02:00
miandmhoennig 5c64391f5f Feature/remote authenticated clone (#24)
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #24
2026-09-05 14:06:58 +02:00
mhoennigandClaude Opus 5 cd6f915231 init reads its configuration from the repository root, and says so when it cannot (#22)
`init --systemd` generated the systemd unit, the .htaccess and the maintenance
page from a configuration it never actually read reliably: the load swallowed
every exception and fell back to a default `ServerConfig`, and it resolved the
layers from the process's current directory instead of the git top level every
other file of the command goes through.

A broken `.werkator.yml` was therefore indistinguishable from an unconfigured
`publicBaseUrl` — the host integration was skipped without a word — and running
`init` from a subdirectory read a foreign configuration or none, which also
defeated `--apply`'s promise that the fragment reaches the generated unit.

The fallback stays, but the exception message is printed, once per run, and the
repository root is passed down like every other caller of `ConfigLoader.load`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 19:38:41 +02:00
7 changed files with 359 additions and 28 deletions
+22
View File
@@ -8,6 +8,28 @@ Lightweight, declarative and highly opinionated software build system (CI/CD).
- [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init`
- [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy
## Adding a Gitea Repository
One instance serves several repositories (`docs/deployment.md`, ADR 0009).
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
It prints the registry entry: add it to `~/.werkator.yml` under `repositories:`, then restart the service.
The optional `[<name>]` overrides the directory basename: it becomes the route segment (`/repos/<name>/…`) and the UI switcher entry, so it must be unique.
Needed only when the clone directory name is wrong or collides — e.g. `michael.hoennig.de.git` checked out as `michael.hoennig.de`, or two forges serving a repo of the same name.
A **public** repository needs nothing else: the clone runs anonymously.
A **private** repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning:
```yaml
defaults:
git:
account: <gitea-user>
token: <token> # Gitea → Settings → Applications → Generate Token, scope read:repository
```
## Developer Setup
Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`:
+1
View File
@@ -123,6 +123,7 @@ Adding a repository is editing a registry entry — never a data migration, beca
It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry.
It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back.
Cloning and initialising is mechanical; registering is a decision.
A private `https` origin authenticates with the shared `defaults.git.account`/`defaults.git.token` of `~/.werkator.yml` (the token travels via a one-shot `GIT_ASKPASS` on the host, never in a URL or process list); enter those once before cloning a private repository — without them only public origins clone.
4. **Restart** the service; startup recovery re-enqueues what was in flight:
@@ -0,0 +1,90 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
`init --systemd` generates the host integration — the systemd unit's resource limits, the Apache `.htaccess` and the maintenance page — from the effective configuration.
It read that configuration through a helper with two independent defects, both of which fail silently.
**It swallowed every error.**
The load sat in `try { … } catch (_: Exception) { ServerConfig() }`.
Any configuration error at all — a missing required field, a malformed layer, a version floor violation — produced a default `ServerConfig` with a blank `publicBaseUrl`.
A repository with a broken `.werkator.yml` then looked exactly like one that simply has no public base URL configured:
the `.htaccess` and the maintenance page were skipped without a word.
This surfaced while verifying [PR#17](2026-09-03-PR%2317-maintenance-page.md) on mih09, where the missing files looked like an unconfigured `publicBaseUrl` and were in fact an unrelated validation error.
**It read from the wrong directory.**
The helper called `configLoader.load(Paths.get("."))` — the process's current directory — while everything else in the command works off the git top level resolved by `GitService.getTopLevel`.
`ConfigLoader.loadRaw` resolves the layers directly under the directory it is given and does not walk up to the repository root, so the two agree only when `init` happens to be invoked from the root itself.
From a subdirectory the command read another repository's configuration, or none.
That also broke `--apply`: the fragment is installed into the repository root deliberately before the systemd files are written, so that its port and limits reach the generated unit, and a current-directory read does not see it.
`init --systemd` runs during initial deployment setup, which is exactly when a silent wrong answer is most expensive.
## Non-Goals
- The fallback itself is kept: a configuration that cannot be loaded is not fatal for `init`, the units are still generated with the defaults.
During the very first bootstrap there is legitimately nothing to load yet.
- No change to `ConfigLoader`, to the configuration schema, or to any other command.
- No sweep for catch-all exception handlers elsewhere in the code base;
the two other `catch` blocks in `InitCommand` already print an `Error:` and abort, so they were only checked, not changed.
## The Scenarios
### Feature: init reports what it read and where it read it from
#### Background
- The *repository root* is the git top level as resolved by `GitService.getTopLevel`, the directory holding `.werkator.yml`, `.git/werkator/.werkator.yml` and an applied fragment.
- The *current directory* is the process working directory, which is the repository root only when `init` is invoked there.
#### Scenario#22.01: A broken configuration is named, not defaulted over
So that a validation error during deployment setup is not mistaken for an unconfigured installation.
- **Given** a repository whose effective configuration cannot be loaded
- **When** `init --systemd` runs
- **Then** the exception message is printed as a warning
- **and** the unit files are still generated with the default settings
- **and** the warning appears exactly once, although three settings are read from the configuration
##### Verified by
- [InitCommandTest: `--systemd warns once when the effective configuration cannot be loaded`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
#### Scenario#22.02: The configuration is read from the repository root
So that the generated host integration reflects the repository being initialized, whatever directory `init` was invoked from.
- **Given** a repository whose root configuration sets `server.publicBaseUrl` and `server.port`
- **and** a current directory that is not that repository root
- **When** `init --systemd` runs
- **Then** the `.htaccess` and the maintenance page are generated
- **and** the `.htaccess` proxies to the port from the root configuration
##### Verified by
- [InitCommandTest: `--systemd reads the configuration from the repository root, not the current directory`](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
## The Solution
The catch-all now prints the exception message before falling back:
```
Warning: the effective configuration could not be loaded (<message>)
continuing with default server settings — check the generated unit and host files
```
The configuration is read three times while the systemd files are written (`memoryMax`, `tasksMax`, `publicBaseUrl`), which would repeat the warning three times.
It is therefore loaded once per run and cached in the command, and the cache is reset at the top of `run()` so a reused instance — the command is a Spring singleton — re-reads.
The repository root is passed down into the two accessors instead of `Paths.get(".")`.
This matches every other caller of `ConfigLoader.load` in the code base, all of which pass an explicit working directory;
`InitCommand` was the only one relying on the process's current directory.
Both fixes are the same failure in two forms — the command answered from a configuration it never actually read — which is why they are in one PR.
## Additional Changes
- None.
@@ -0,0 +1,92 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- ADR 0009 — multi-repo instance: the `defaults` block carries shared repository-level keys, read by `ConfigLoader`, never directly by consumers.
- `docs/deployment.md` — registry setup: `repo-add` clones, initialises, and prints the registry entry.
## The Problem
`tools/remote werkator repo-add <private-https-url>` fails on the host with `could not read Username`.
The clone runs anonymously, but the credentials exist only in the instance file's `defaults.git` block — which the script never consults.
Registering a private repository therefore needs a manual SSH session today.
## Non-Goals
The script still does not write `~/.werkator.yml`: registering stays the operator's decision.
No SSH-URL support: the forge is reached over `https` from the host.
No new config keys: `defaults.git.account`/`defaults.git.token` already exist.
## The Scenarios
### Feature: authenticated clone for private origins
#### Background
- The shared credentials live in `~/.werkator.yml` under `defaults.git` (ADR 0009).
- Public origins and instances without shared credentials must keep cloning anonymously.
#### Scenario#000.01: Private https origin clones with shared credentials
- **Given** `defaults.git.account`/`defaults.git.token` in `~/.werkator.yml` on the host
- **When** `tools/remote werkator repo-add <private-https-url>` runs
- **Then** the clone authenticates with those credentials and succeeds.
##### Verified by
- Manual stub-`git` test: URL carries `account@`, askpass answers the token, `GIT_TERMINAL_PROMPT=0`.
#### Scenario#000.02: Public origin clones anonymously
- **Given** no shared credentials (or a public repository)
- **When** `repo-add` or `repo-init` runs
- **Then** the clone runs exactly as before, without authentication.
##### Verified by
- Local helper test against a `file://` origin with and without an instance file.
#### Scenario#000.03: Token never leaks locally
- **Given** an authenticated clone
- **When** the command runs from the workstation
- **Then** the token appears in neither the local process list nor a repository config.
##### Verified by
- Code inspection: the token is read on the host and passed via a one-shot `GIT_ASKPASS` script.
## The Solution
`tools/remote` gained a `clone_repo` helper used by both `repo-init` and `repo-add`.
For `https://` URLs it ships a small Python helper (base64-encoded, so no `$` is expanded locally) to the host.
The helper reads `defaults.git.account`/`defaults.git.token` from `~/.werkator.yml`, puts the account into the URL, and hands the token to git via a one-shot `0700` `GIT_ASKPASS` script deleted in `finally`.
Without credentials it falls back to the plain anonymous clone; non-`https` URLs clone unchanged.
`docs/deployment.md` documents that the shared credentials must exist before cloning a private repository.
## Open Questions
- None.
## Attachments
### Adding a Gitea repository — example
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
Add the printed entry to `~/.werkator.yml` under `repositories:`, then restart the service.
A public repository needs nothing else: the clone runs anonymously.
A private repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning (token: Gitea → Settings → Applications → Generate Token, scope `read:repository`):
```yaml
defaults:
git:
account: <gitea-user>
token: <token>
```
@@ -48,6 +48,7 @@ class InitCommand(
internal var javaExecutableResolver: () -> Path = { Paths.get(System.getProperty("java.home"), "bin", "java") }
override fun run() {
cachedServerConfig = null
val normalizedWorkingDir = workingDir.toAbsolutePath().normalize()
val root =
try {
@@ -88,7 +89,7 @@ class InitCommand(
if (url == null) return DetectedValues()
if (url.startsWith("http")) {
val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/([^/.]+)(?:\.git)?""")
val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/(.+?)(?:\.git)?$""")
val match = regex.find(url)
if (match != null) {
val (user, host, owner, repo) = match.destructured
@@ -101,7 +102,7 @@ class InitCommand(
}
} else if (url.contains("@") && url.contains(":")) {
// Assume SSH: git@host:owner/repo.git
val regex = Regex("""([^@]+)@([^:]+):([^/]+)/([^/.]+)(?:\.git)?""")
val regex = Regex("""([^@]+)@([^:]+):([^/]+)/(.+?)(?:\.git)?$""")
val match = regex.find(url)
if (match != null) {
val (_, host, owner, repo) = match.destructured
@@ -297,14 +298,31 @@ class InitCommand(
* already loadable (re-running `init --systemd` on an installed instance); during
* the very first bootstrap they stay unset and the defaults (no directives) apply.
*/
private fun loadedSystemdConfig(): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig().systemd
private fun loadedSystemdConfig(root: Path): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig(root).systemd
private fun loadedServerConfig(): de.hoennig.werkator.config.ServerConfig =
/** Loaded once per run, so a broken configuration is reported once and not per caller. */
private var cachedServerConfig: de.hoennig.werkator.config.ServerConfig? = null
/**
* Read from the repository root like every other file this command touches — the
* layers sit there, not in whatever directory the process happens to run in, and
* an applied fragment must reach the generated unit even when `init` is invoked
* from a subdirectory.
*
* A configuration error here is not fatal — the units are still generated with defaults —
* but it must not pass for "nothing configured": without the warning a broken `.werkator.yml`
* looks exactly like an unset `publicBaseUrl` and the host integration is skipped silently.
*/
private fun loadedServerConfig(root: Path): de.hoennig.werkator.config.ServerConfig =
cachedServerConfig ?: run {
try {
configLoader.load(Paths.get(".")).server
} catch (_: Exception) {
configLoader.load(root).server
} catch (e: Exception) {
println("Warning: the effective configuration could not be loaded (${e.message})")
println(" continuing with default server settings — check the generated unit and host files")
de.hoennig.werkator.config
.ServerConfig()
}.also { cachedServerConfig = it }
}
private fun createSystemdFiles(
@@ -328,8 +346,8 @@ class InitCommand(
javaExecutable = javaExecutableResolver(),
jarPath = jarPath,
envFile = envFile,
memoryMax = loadedSystemdConfig().memoryMax,
tasksMax = loadedSystemdConfig().tasksMax,
memoryMax = loadedSystemdConfig(root).memoryMax,
tasksMax = loadedSystemdConfig(root).tasksMax,
),
)
println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}")
@@ -352,7 +370,7 @@ class InitCommand(
// generated host integration like the units: only meaningful behind a web
// frontend, so it needs a public base URL; unused elsewhere and harmless
val server = loadedServerConfig()
val server = loadedServerConfig(root)
if (server.publicBaseUrl.isNotBlank()) {
val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME)
htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port))
@@ -109,6 +109,21 @@ class InitCommandTest : FunSpec() {
projectContent shouldContain "repo: my-repo"
}
test("keeps dots in repository names") {
val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/mi/michael.hoennig.de.git"
initCommand.run()
val projectConfig = tempDir.resolve(".werkator.yml")
val projectContent = projectConfig.toFile().readText()
projectContent shouldContain "owner: mi"
projectContent shouldContain "repo: michael.hoennig.de"
}
test("does not overwrite existing files") {
val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir
@@ -252,5 +267,50 @@ class InitCommandTest : FunSpec() {
// This should not throw IllegalArgumentException
initCommand.run()
}
test("--systemd reads the configuration from the repository root, not the current directory") {
val tempDir = Files.createTempDirectory("werkator-init-test")
// written before the run, so `init` keeps it instead of creating a template
tempDir.resolve(".werkator.yml").toFile().writeText(
"server:\n publicBaseUrl: \"https://werkator.example.org/\"\n port: 18099\n",
)
initCommand.workingDir = tempDir
initCommand.systemd = true
initCommand.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
initCommand.javaExecutableResolver = { Paths.get("/usr/bin/java") }
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
initCommand.run()
// the host integration is generated only when the root's config has a public base URL
val htaccess = tempDir.resolve(".git/werkator/${SystemdServiceFiles.HTACCESS_NAME}")
htaccess.toFile().shouldExist()
htaccess.toFile().readText() shouldContain "18099"
tempDir.resolve(".git/werkator/${SystemdServiceFiles.MAINTENANCE_PAGE_NAME}").toFile().shouldExist()
}
test("--systemd warns once when the effective configuration cannot be loaded") {
val tempDir = Files.createTempDirectory("werkator-init-test")
val brokenLoader = mockk<de.hoennig.werkator.config.ConfigLoader>()
every { brokenLoader.load(any()) } throws IllegalStateException("gitea.owner is required")
val command = InitCommand(gitService, brokenLoader)
command.workingDir = tempDir
command.systemd = true
command.jarPathResolver = { Paths.get("/home/ci/bin/werkator.jar") }
command.javaExecutableResolver = { Paths.get("/usr/bin/java") }
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
val console = captureConsole { command.run() }
console.stdout shouldContain "gitea.owner is required"
// the three readers of the configuration must not repeat the warning
console.stdout.windowed("Warning:".length).count { it == "Warning:" } shouldBe 1
// the units are still written with the defaults
tempDir.resolve(".git/werkator/${SystemdServiceFiles.unitName(tempDir)}").toFile().shouldExist()
}
}
}
+61 -13
View File
@@ -332,9 +332,65 @@ instance_update() {
echo "==> Instance updated."
}
# Sets up the WATCHED repository: an anonymous https clone (a private origin
# gets its credentials via git.account/git.token in the machine config that
# `werkator init` creates), the werkator init with the instance fragment
# Clones one URL into one directory on the host.
# A private https origin authenticates with the shared `defaults.git.account` /
# `defaults.git.token` of `~/.werkator.yml` (ADR 0009). The whole authenticated
# clone runs in one remote python script: the token is read from the instance
# file on the host and passed to git via a one-shot GIT_ASKPASS script, so it
# appears in neither the local process list nor a repository config.
# Public origins (or an instance without shared credentials) clone anonymously.
clone_repo() {
local url="$1" dest="$2"
if ssh "$HOST" "test -d '$dest/.git'"; then
echo " (already cloned, skipping)"
return 0
fi
case "$url" in
https://*)
# The python helper travels base64-encoded: the clone command itself
# stays a plain `ssh` line, so no `$` inside the script is ever
# expanded by the local shell, and the token never leaves the host.
local helper_b64
helper_b64="$(python3 -c 'import base64,sys; print(base64.b64encode(sys.stdin.read().encode()).decode())' <<'PYEOF_CLONE'
import os, stat, subprocess, sys, tempfile, urllib.parse
url, dest = sys.argv[1], sys.argv[2]
try:
import yaml
cfg = yaml.safe_load(open(os.path.expanduser("~/.werkator.yml"))) or {}
except (FileNotFoundError, ImportError):
cfg = {}
d = (cfg.get("defaults") or {}).get("git") or {}
account, token = d.get("account"), d.get("token")
env = dict(os.environ, GIT_TERMINAL_PROMPT="0")
ask = None
if account and token:
parts = urllib.parse.urlsplit(url)
host = parts.netloc.rsplit("@", 1)[-1]
url = urllib.parse.urlunsplit(parts._replace(netloc=account + "@" + host))
ask = tempfile.NamedTemporaryFile(mode="w", prefix="werkator-clone-askpass-",
suffix=".sh", delete=False)
ask.write("#!/bin/sh\nexec echo \"$WERKATOR_CLONE_TOKEN\"\n")
ask.close()
os.chmod(ask.name, stat.S_IRWXU)
env.update(GIT_ASKPASS=ask.name, WERKATOR_CLONE_TOKEN=token)
try:
subprocess.run(["git", "clone", url, dest], env=env, check=True)
finally:
if ask is not None:
os.unlink(ask.name)
PYEOF_CLONE
)"
ssh "$HOST" "echo '$helper_b64' | base64 -d | python3 - '$url' '$dest'"
;;
*)
ssh "$HOST" "git clone '$url' '$dest'"
;;
esac
}
# Sets up the WATCHED repository: an https clone (a private origin
# authenticates with the shared `defaults.git.*` credentials of
# `~/.werkator.yml`; see `clone_repo`), the werkator init with the instance
# applied, and the rootfs archive for the sandbox builds. All configuration
# writing is init's — this script transports and invokes (step 23).
repo_init() {
@@ -344,11 +400,7 @@ repo_init() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning the watched repository"
if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
fi
clone_repo "$REPO_URL" "$REPO_DIR"
if [ "$SANDBOX" = "docker" ]; then
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
@@ -401,11 +453,7 @@ repo_add() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning $url as '$name'"
if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
fi
clone_repo "$url" "$SIBLING_DIR/$name"
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
# binary). Without it a watched repository builds on the bare host, where the