Author SHA1 Message Date
mhoennigandClaude Fable 5.1 ec07269cca docs(prs): refer to werkdock's change, not its branch
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 23:29:55 +02:00
mhoennigandClaude Fable 5.1 b8600bb556 werkdock: import the rootfs archive, fall back to load -i on an older werkdock
Werkdock's docker-compat branch names the verb for rootfs archives
'import' (docker import semantics) and keeps 'load -i --name' only as
a compatibility path. Werkator now runs 'werkdock import ARCHIVE IMAGE'
and falls back to 'load -i --name' on exactly the unknown-verb
signature (exit 125, 'unknown command' on stderr), so Werkator and
werkdock can be updated in either order. Any other import failure
propagates as before. The image name stays untagged: it equals
werkator-buildenv-<hash>:latest in werkdock's naming, and the existing
store needs no re-import. PR-doc under docs/prs/ with the PR#000
placeholder.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 22:49:46 +02:00
mhoennig 0f9f119687 init: keep dots in detected repository names 2026-09-05 15:15:12 +02:00
miandmhoennig 5c64391f5f Feature/remote authenticated clone (#24)
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #24
2026-09-05 14:06:58 +02:00
mhoennigandClaude Opus 5 cd6f915231 init reads its configuration from the repository root, and says so when it cannot (#22)
`init --systemd` generated the systemd unit, the .htaccess and the maintenance
page from a configuration it never actually read reliably: the load swallowed
every exception and fell back to a default `ServerConfig`, and it resolved the
layers from the process's current directory instead of the git top level every
other file of the command goes through.

A broken `.werkator.yml` was therefore indistinguishable from an unconfigured
`publicBaseUrl` — the host integration was skipped without a word — and running
`init` from a subdirectory read a foreign configuration or none, which also
defeated `--apply`'s promise that the fragment reaches the generated unit.

The fallback stays, but the exception message is printed, once per run, and the
repository root is passed down like every other caller of `ConfigLoader.load`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 19:38:41 +02:00
10 changed files with 380 additions and 48 deletions
+22
View File
@@ -8,6 +8,28 @@ Lightweight, declarative and highly opinionated software build system (CI/CD).
- [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init` - [docs/bootstrapping.md](docs/bootstrapping.md) — initializing a repository with `init`
- [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy - [docs/deployment.md](docs/deployment.md) — running Werkator as a systemd service behind a reverse proxy
## Adding a Gitea Repository
One instance serves several repositories (`docs/deployment.md`, ADR 0009).
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
It prints the registry entry: add it to `~/.werkator.yml` under `repositories:`, then restart the service.
The optional `[<name>]` overrides the directory basename: it becomes the route segment (`/repos/<name>/…`) and the UI switcher entry, so it must be unique.
Needed only when the clone directory name is wrong or collides — e.g. `michael.hoennig.de.git` checked out as `michael.hoennig.de`, or two forges serving a repo of the same name.
A **public** repository needs nothing else: the clone runs anonymously.
A **private** repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning:
```yaml
defaults:
git:
account: <gitea-user>
token: <token> # Gitea → Settings → Applications → Generate Token, scope read:repository
```
## Developer Setup ## Developer Setup
Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`: Source `.envrc` to add `tools/` to your `PATH`, or install [direnv](#direnv) to have this done automatically on `cd`:
+3 -1
View File
@@ -475,7 +475,9 @@ The section was called `bwrap` and its binary key `bwrap.werkdock` until v1.2.0;
`werkdock.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target. `werkdock.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`. It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`.
Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker. Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker.
The archive is loaded once per source as the werkdock image `werkator-buildenv-<hash>` into werkdock's store (`$WERKDOCK_HOME`, default `~/.werkdock`) — shared by every repository of this OS user; the hash derives from the source string, so a changed `rootfs` loads a fresh image and stale ones can be removed from the store. The archive is imported once per source as the werkdock image `werkator-buildenv-<hash>` into werkdock's store (`$WERKDOCK_HOME`, default `~/.werkdock`) — shared by every repository of this OS user; the hash derives from the source string, so a changed `rootfs` imports a fresh image and stale ones can be removed from the store.
Werkator uses `werkdock import ARCHIVE IMAGE` (docker import semantics) and falls back to the older `werkdock load -i ARCHIVE --name IMAGE` when the installed werkdock does not know the verb yet, so Werkator and werkdock can be updated in either order.
The image keeps its untagged name (it equals `werkator-buildenv-<hash>:latest` in werkdock's docker-style naming), so an existing store needs no re-import.
Per-repo Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`. Per-repo Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`.
`werkdock.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in. `werkdock.env` adds environment variables inside the sandbox; the environment is otherwise cleared (docker semantics) — the server's environment does not leak in.
Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user. Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user.
+5 -4
View File
@@ -120,9 +120,10 @@ Adding a repository is editing a registry entry — never a data migration, beca
tools/remote --env-file .env.<instance> werkator repo-add https://github.com/<owner>/<repo>.git [<name>] tools/remote --env-file .env.<instance> werkator repo-add https://github.com/<owner>/<repo>.git [<name>]
``` ```
It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry. It clones the repository next to the ones already served, runs `init` in it, and **prints** the registry entry.
It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back. It does not write `~/.werkator.yml`: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back.
Cloning and initialising is mechanical; registering is a decision. Cloning and initialising is mechanical; registering is a decision.
A private `https` origin authenticates with the shared `defaults.git.account`/`defaults.git.token` of `~/.werkator.yml` (the token travels via a one-shot `GIT_ASKPASS` on the host, never in a URL or process list); enter those once before cloning a private repository — without them only public origins clone.
4. **Restart** the service; startup recovery re-enqueues what was in flight: 4. **Restart** the service; startup recovery re-enqueues what was in flight:
@@ -341,7 +342,7 @@ tools/remote --env-file .env.mih34 port-forward start # browser tunne
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`. Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation). That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation).
The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user. The rootfs archive is imported once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`. Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
Updates are one command, refused while a build runs (`FORCE=1` overrides): Updates are one command, refused while a build runs (`FORCE=1` overrides):
@@ -0,0 +1,92 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- ADR 0009 — multi-repo instance: the `defaults` block carries shared repository-level keys, read by `ConfigLoader`, never directly by consumers.
- `docs/deployment.md` — registry setup: `repo-add` clones, initialises, and prints the registry entry.
## The Problem
`tools/remote werkator repo-add <private-https-url>` fails on the host with `could not read Username`.
The clone runs anonymously, but the credentials exist only in the instance file's `defaults.git` block — which the script never consults.
Registering a private repository therefore needs a manual SSH session today.
## Non-Goals
The script still does not write `~/.werkator.yml`: registering stays the operator's decision.
No SSH-URL support: the forge is reached over `https` from the host.
No new config keys: `defaults.git.account`/`defaults.git.token` already exist.
## The Scenarios
### Feature: authenticated clone for private origins
#### Background
- The shared credentials live in `~/.werkator.yml` under `defaults.git` (ADR 0009).
- Public origins and instances without shared credentials must keep cloning anonymously.
#### Scenario#000.01: Private https origin clones with shared credentials
- **Given** `defaults.git.account`/`defaults.git.token` in `~/.werkator.yml` on the host
- **When** `tools/remote werkator repo-add <private-https-url>` runs
- **Then** the clone authenticates with those credentials and succeeds.
##### Verified by
- Manual stub-`git` test: URL carries `account@`, askpass answers the token, `GIT_TERMINAL_PROMPT=0`.
#### Scenario#000.02: Public origin clones anonymously
- **Given** no shared credentials (or a public repository)
- **When** `repo-add` or `repo-init` runs
- **Then** the clone runs exactly as before, without authentication.
##### Verified by
- Local helper test against a `file://` origin with and without an instance file.
#### Scenario#000.03: Token never leaks locally
- **Given** an authenticated clone
- **When** the command runs from the workstation
- **Then** the token appears in neither the local process list nor a repository config.
##### Verified by
- Code inspection: the token is read on the host and passed via a one-shot `GIT_ASKPASS` script.
## The Solution
`tools/remote` gained a `clone_repo` helper used by both `repo-init` and `repo-add`.
For `https://` URLs it ships a small Python helper (base64-encoded, so no `$` is expanded locally) to the host.
The helper reads `defaults.git.account`/`defaults.git.token` from `~/.werkator.yml`, puts the account into the URL, and hands the token to git via a one-shot `0700` `GIT_ASKPASS` script deleted in `finally`.
Without credentials it falls back to the plain anonymous clone; non-`https` URLs clone unchanged.
`docs/deployment.md` documents that the shared credentials must exist before cloning a private repository.
## Open Questions
- None.
## Attachments
### Adding a Gitea repository — example
From the workstation, clone and initialise, then register:
```bash
tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.org/<owner>/<repo>.git [<name>]
```
Add the printed entry to `~/.werkator.yml` under `repositories:`, then restart the service.
A public repository needs nothing else: the clone runs anonymously.
A private repository needs shared credentials once on the host, in `~/.werkator.yml` of the service user, before cloning (token: Gitea → Settings → Applications → Generate Token, scope `read:repository`):
```yaml
defaults:
git:
account: <gitea-user>
token: <token>
```
@@ -0,0 +1,102 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## Related Links
- Werkdock `CHANGELOG.md` (repository `mi/werkdock`, Unreleased section) — the release notes this PR adapts to: `import` replaces `load -i --name` for rootfs archives, `load` is reserved for docker/OCI image archives, image names gained docker-style tags.
- ADR 0008 — the bwrap build runtime; `docs/configuration.md`, notes on `builds.<name>.werkdock`.
## The Problem
Werkdock moves its CLI closer to `docker run --rm`.
For the rootfs archives Werkator uses, the verb is now `werkdock import ARCHIVE IMAGE` (docker import semantics).
`werkdock load -i ARCHIVE --name IMAGE` still works but is a compatibility path with a note on stderr, and is announced to go away once Werkator has switched.
Werkator ships the werkdock binary with its deployment, but nothing forces the two to be updated together: an installation may run a new Werkator against an older werkdock for a while, or the other way round.
## Non-Goals
- No switch of the existence check from `werkdock images` to `werkdock inspect`: `images` prints the bare name for untagged images on every werkdock version, the exact-line match keeps working, and `inspect` would tie Werkator to the new werkdock.
- No tagged image name (`werkator-buildenv:<hash>`): it would re-import every build environment on the webspace and orphan the old image, for no functional gain today.
- None of the new `run` flags (`--mount`, `--entrypoint`, `--network host`): Werkator's invocation needs none of them.
## The Scenarios
### Feature: rootfs archives are imported with werkdock's docker-shaped verb
#### Background
- The build environment is a rootfs archive named by `builds.<name>.werkdock.rootfs`.
- Werkator creates the werkdock image `werkator-buildenv-<hash>` from it once per source.
- Werkdock answers an unknown verb with `werkdock: unknown command "import"` and exit code 125, its code for its own errors.
#### Scenario#000.01: A missing image is imported with `werkdock import`
So that Werkator uses the verb werkdock names for rootfs archives, and the deprecated path can be removed on werkdock's side.
- **Given** `werkdock images` does not list the image
- **When** a build starts
- **Then** Werkator runs `werkdock import ARCHIVE werkator-buildenv-<hash>`
- **and** runs no `werkdock load`.
##### Verified by
- [WerkdockBuildRunnerTest "imports the image once when werkdock does not know it yet"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
- [WerkdockBuildRunnerTest "downloads a URL rootfs once before importing it"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
#### Scenario#000.02: An older werkdock without the verb still works
So that Werkator and werkdock can be updated in either order.
- **Given** the installed werkdock answers `import` with `unknown command` and exit 125
- **When** a build starts with a missing image
- **Then** Werkator falls back to `werkdock load -i ARCHIVE --name werkator-buildenv-<hash>`
- **and** logs the fallback.
##### Verified by
- [WerkdockBuildRunnerTest "falls back to load -i --name on a werkdock without the import verb"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
#### Scenario#000.03: A real import failure is not masked by the fallback
So that a broken archive fails the build with werkdock's message, as before.
- **Given** `werkdock import` fails for any other reason (any other exit code, or exit 125 without `unknown command`)
- **When** a build starts with a missing image
- **Then** the build fails with that command's output
- **and** no `werkdock load` runs.
##### Verified by
- [WerkdockBuildRunnerTest "propagates an import failure that is not a missing verb"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
#### Scenario#000.04: An existing image is neither imported nor loaded
- **Given** `werkdock images` lists the image
- **When** a build starts
- **Then** neither `import` nor `load` runs.
##### Verified by
- [WerkdockBuildRunnerTest "does not import an image werkdock already has"](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt)
## The Solution
`WerkdockBuildRunner.ensureImage` calls the new `importImage`: `werkdock import ARCHIVE IMAGE` through the non-throwing `run`, then either returns, falls back to `load -i --name` on exactly the unknown-verb signature (exit 125 plus `unknown command` on stderr), or rethrows the import's result as a `GitCommandException` — the same exception and message the old code produced.
The fallback is keyed on werkdock's own error signature rather than on a version string, because werkdock's version output does not yet distinguish the two builds.
## Open Questions
- When to drop the fallback: once every installation runs a werkdock with `import`, the `load -i --name` branch and its test go, and werkdock can remove the compatibility path.
## Additional Changes
- `docs/configuration.md` and `docs/deployment.md` say "imported" where they said "loaded", and name the fallback and the reason the image name stays untagged.
## Prerequisite PRs
- The werkdock change that introduces `import` (repository `mi/werkdock`, CHANGELOG Unreleased); against an older werkdock the fallback path runs.
## Follow-up PRs
- Remove the fallback (see Open Questions).
@@ -2,6 +2,7 @@ package de.hoennig.werkator.build
import de.hoennig.werkator.config.BranchConfig import de.hoennig.werkator.config.BranchConfig
import de.hoennig.werkator.config.WerkdockConfig import de.hoennig.werkator.config.WerkdockConfig
import de.hoennig.werkator.git.GitCommandException
import de.hoennig.werkator.git.GitCommandRunner import de.hoennig.werkator.git.GitCommandRunner
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.stereotype.Component import org.springframework.stereotype.Component
@@ -16,9 +17,9 @@ import java.security.MessageDigest
* it shells out to the `werkdock` CLI (`werkdock.binary`, default via PATH) — the same * it shells out to the `werkdock` CLI (`werkdock.binary`, default via PATH) — the same
* pattern as git and docker, CLI, no library. * pattern as git and docker, CLI, no library.
* *
* The rootfs archive becomes a werkdock *image*, loaded once per source * The rootfs archive becomes a werkdock *image*, imported once per source
* (`werkator-buildenv-<hash>`, the hash over the source string, so a changed source * (`werkator-buildenv-<hash>`, the hash over the source string, so a changed source
* loads a fresh image) into werkdock's own store (`$WERKDOCK_HOME`, default * imports a fresh image) into werkdock's own store (`$WERKDOCK_HOME`, default
* `~/.werkdock`) — shared by every repository of this OS user, unlike the old * `~/.werkdock`) — shared by every repository of this OS user, unlike the old
* per-repo unpack. Only the download cache for URL sources and the persistent * per-repo unpack. Only the download cache for URL sources and the persistent
* toolchain home (bound to `/root` for Gradle/Go caches) stay under * toolchain home (bound to `/root` for Gradle/Go caches) stay under
@@ -66,9 +67,11 @@ class WerkdockBuildRunner(
} }
/** /**
* Loads the rootfs archive into the werkdock image store once per source. * Imports the rootfs archive into the werkdock image store once per source.
* `werkdock images` answers existence through the CLI, like `docker image * `werkdock images` answers existence through the CLI, like `docker image
* inspect` does for the Docker runner. * inspect` does for the Docker runner; it prints the bare name for an untagged
* image on every werkdock version, so the exact-line match holds across the
* werkdock upgrade that introduced tags.
*/ */
private fun ensureImage( private fun ensureImage(
werkdock: String, werkdock: String,
@@ -84,7 +87,33 @@ class WerkdockBuildRunner(
val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(sandbox.rootfs)) val envDir = repoDir.resolve(BUILDENV_DIR).resolve(sourceKey(sandbox.rootfs))
Files.createDirectories(envDir) Files.createDirectories(envDir)
val archive = localArchive(sandbox.rootfs, envDir, repoDir, onAuxProcess) val archive = localArchive(sandbox.rootfs, envDir, repoDir, onAuxProcess)
log.info("loading build environment {} as werkdock image {}", sandbox.rootfs, image) log.info("importing build environment {} as werkdock image {}", sandbox.rootfs, image)
importImage(werkdock, archive, image, repoDir, onAuxProcess)
}
/**
* `werkdock import ARCHIVE IMAGE` (docker import semantics) creates the image from a
* rootfs archive. A werkdock that predates the verb answers `unknown command` with
* exit 125; then the older `load -i ARCHIVE --name IMAGE` does the same job, so an
* installation can update Werkator and werkdock in either order. Any other failure
* is the import's own and propagates as it did before.
*/
private fun importImage(
werkdock: String,
archive: String,
image: String,
repoDir: Path,
onAuxProcess: (Process) -> Unit,
) {
val importCommand = listOf(werkdock, "import", archive, image)
val imported = commandRunner.run(importCommand, repoDir, onProcess = onAuxProcess)
if (imported.isSuccess) {
return
}
if (imported.exitCode != WERKDOCK_CLI_ERROR || "unknown command" !in imported.stderr) {
throw GitCommandException(importCommand, imported)
}
log.info("this werkdock has no import verb yet, falling back to load -i --name")
commandRunner.runOrThrow( commandRunner.runOrThrow(
listOf(werkdock, "load", "-i", archive, "--name", image), listOf(werkdock, "load", "-i", archive, "--name", image),
repoDir, repoDir,
@@ -209,5 +238,8 @@ class WerkdockBuildRunner(
companion object { companion object {
const val BUILDENV_DIR = ".git/werkator/buildenv" const val BUILDENV_DIR = ".git/werkator/buildenv"
const val HOME_DIR = "home" const val HOME_DIR = "home"
/** werkdock's exit code for its own errors (docker's 125), as opposed to the sandboxed command's. */
const val WERKDOCK_CLI_ERROR = 125
} }
} }
@@ -89,7 +89,7 @@ class InitCommand(
if (url == null) return DetectedValues() if (url == null) return DetectedValues()
if (url.startsWith("http")) { if (url.startsWith("http")) {
val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/([^/.]+)(?:\.git)?""") val regex = Regex("""https?://(?:([^@]+)@)?([^/]+)/([^/]+)/(.+?)(?:\.git)?$""")
val match = regex.find(url) val match = regex.find(url)
if (match != null) { if (match != null) {
val (user, host, owner, repo) = match.destructured val (user, host, owner, repo) = match.destructured
@@ -102,7 +102,7 @@ class InitCommand(
} }
} else if (url.contains("@") && url.contains(":")) { } else if (url.contains("@") && url.contains(":")) {
// Assume SSH: git@host:owner/repo.git // Assume SSH: git@host:owner/repo.git
val regex = Regex("""([^@]+)@([^:]+):([^/]+)/([^/.]+)(?:\.git)?""") val regex = Regex("""([^@]+)@([^:]+):([^/]+)/(.+?)(?:\.git)?$""")
val match = regex.find(url) val match = regex.find(url)
if (match != null) { if (match != null) {
val (_, host, owner, repo) = match.destructured val (_, host, owner, repo) = match.destructured
@@ -2,6 +2,7 @@ package de.hoennig.werkator.build
import de.hoennig.werkator.config.BranchConfig import de.hoennig.werkator.config.BranchConfig
import de.hoennig.werkator.config.WerkdockConfig import de.hoennig.werkator.config.WerkdockConfig
import de.hoennig.werkator.git.GitCommandException
import de.hoennig.werkator.git.GitCommandResult import de.hoennig.werkator.git.GitCommandResult
import de.hoennig.werkator.git.GitCommandRunner import de.hoennig.werkator.git.GitCommandRunner
import io.kotest.assertions.throwables.shouldThrow import io.kotest.assertions.throwables.shouldThrow
@@ -37,7 +38,11 @@ class WerkdockBuildRunnerTest : FunSpec() {
private fun imageName(rootfs: String = "/srv/buildenv.tar.zst"): String = "werkator-buildenv-${rootfs.sha12()}" private fun imageName(rootfs: String = "/srv/buildenv.tar.zst"): String = "werkator-buildenv-${rootfs.sha12()}"
/** The image is already loaded: `werkdock images` lists it, so no load runs. */ private fun importCommand(): List<String> = listOf("werkdock", "import", "/srv/buildenv.tar.zst", imageName())
private fun loadCommand(): List<String> = listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName())
/** The image is already there: `werkdock images` lists it, so no import runs. */
private fun givenImageLoaded(rootfs: String = "/srv/buildenv.tar.zst") { private fun givenImageLoaded(rootfs: String = "/srv/buildenv.tar.zst") {
every { commandRunner.runOrThrow(listOf("werkdock", "images"), repoDir, any(), any()) } returns every { commandRunner.runOrThrow(listOf("werkdock", "images"), repoDir, any(), any()) } returns
GitCommandResult(0, imageName(rootfs) + "\n", "") GitCommandResult(0, imageName(rootfs) + "\n", "")
@@ -88,34 +93,47 @@ class WerkdockBuildRunnerTest : FunSpec() {
) )
} }
test("loads the image once when werkdock does not know it yet") { test("imports the image once when werkdock does not know it yet") {
givenImageMissing() givenImageMissing()
every { every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns GitCommandResult(0, "", "")
commandRunner.runOrThrow(
listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName()),
repoDir,
any(),
any(),
)
} returns GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig()) runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify { verify { commandRunner.run(importCommand(), repoDir, any(), any()) }
commandRunner.runOrThrow( verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
listOf("werkdock", "load", "-i", "/srv/buildenv.tar.zst", "--name", imageName()),
repoDir,
any(),
any(),
)
}
} }
test("does not load an image werkdock already has") { test("falls back to load -i --name on a werkdock without the import verb") {
givenImageMissing()
every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns
GitCommandResult(125, "", "werkdock: unknown command \"import\"\n")
every { commandRunner.runOrThrow(loadCommand(), repoDir, any(), any()) } returns GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify { commandRunner.runOrThrow(loadCommand(), repoDir, any(), any()) }
}
test("propagates an import failure that is not a missing verb") {
givenImageMissing()
every { commandRunner.run(importCommand(), repoDir, any(), any()) } returns
GitCommandResult(125, "", "werkdock: unpacking /srv/buildenv.tar.zst failed\n")
val exception =
shouldThrow<GitCommandException> {
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
}
exception.message shouldContain "unpacking"
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
}
test("does not import an image werkdock already has") {
givenImageLoaded() givenImageLoaded()
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig()) runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig())
verify(exactly = 0) { commandRunner.run(match { "import" in it }, any(), any(), any()) }
verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) } verify(exactly = 0) { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) }
} }
@@ -209,7 +227,7 @@ class WerkdockBuildRunnerTest : FunSpec() {
exception.message shouldContain "werkdock.rootfs" exception.message shouldContain "werkdock.rootfs"
} }
test("downloads a URL rootfs once before loading it") { test("downloads a URL rootfs once before importing it") {
val url = "https://example.test/buildenv.tar.zst" val url = "https://example.test/buildenv.tar.zst"
val downloadTarget = val downloadTarget =
repoDir repoDir
@@ -219,7 +237,7 @@ class WerkdockBuildRunnerTest : FunSpec() {
givenImageMissing() givenImageMissing()
every { commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) } returns every { commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) } returns
GitCommandResult(0, "", "") GitCommandResult(0, "", "")
every { commandRunner.runOrThrow(match { "load" in it }, any(), any(), any()) } returns every { commandRunner.run(match { "import" in it }, any(), any(), any()) } returns
GitCommandResult(0, "", "") GitCommandResult(0, "", "")
runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig(rootfs = url)) runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, werkdockBranchConfig(rootfs = url))
@@ -228,8 +246,8 @@ class WerkdockBuildRunnerTest : FunSpec() {
commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any())
} }
verify { verify {
commandRunner.runOrThrow( commandRunner.run(
listOf("werkdock", "load", "-i", downloadTarget.toString(), "--name", "werkator-buildenv-${url.sha12()}"), listOf("werkdock", "import", downloadTarget.toString(), "werkator-buildenv-${url.sha12()}"),
repoDir, repoDir,
any(), any(),
any(), any(),
@@ -109,6 +109,21 @@ class InitCommandTest : FunSpec() {
projectContent shouldContain "repo: my-repo" projectContent shouldContain "repo: my-repo"
} }
test("keeps dots in repository names") {
val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir
every { gitService.getTopLevel(tempDir) } returns tempDir
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/mi/michael.hoennig.de.git"
initCommand.run()
val projectConfig = tempDir.resolve(".werkator.yml")
val projectContent = projectConfig.toFile().readText()
projectContent shouldContain "owner: mi"
projectContent shouldContain "repo: michael.hoennig.de"
}
test("does not overwrite existing files") { test("does not overwrite existing files") {
val tempDir = Files.createTempDirectory("werkator-init-test") val tempDir = Files.createTempDirectory("werkator-init-test")
initCommand.workingDir = tempDir initCommand.workingDir = tempDir
+61 -13
View File
@@ -332,9 +332,65 @@ instance_update() {
echo "==> Instance updated." echo "==> Instance updated."
} }
# Sets up the WATCHED repository: an anonymous https clone (a private origin # Clones one URL into one directory on the host.
# gets its credentials via git.account/git.token in the machine config that # A private https origin authenticates with the shared `defaults.git.account` /
# `werkator init` creates), the werkator init with the instance fragment # `defaults.git.token` of `~/.werkator.yml` (ADR 0009). The whole authenticated
# clone runs in one remote python script: the token is read from the instance
# file on the host and passed to git via a one-shot GIT_ASKPASS script, so it
# appears in neither the local process list nor a repository config.
# Public origins (or an instance without shared credentials) clone anonymously.
clone_repo() {
local url="$1" dest="$2"
if ssh "$HOST" "test -d '$dest/.git'"; then
echo " (already cloned, skipping)"
return 0
fi
case "$url" in
https://*)
# The python helper travels base64-encoded: the clone command itself
# stays a plain `ssh` line, so no `$` inside the script is ever
# expanded by the local shell, and the token never leaves the host.
local helper_b64
helper_b64="$(python3 -c 'import base64,sys; print(base64.b64encode(sys.stdin.read().encode()).decode())' <<'PYEOF_CLONE'
import os, stat, subprocess, sys, tempfile, urllib.parse
url, dest = sys.argv[1], sys.argv[2]
try:
import yaml
cfg = yaml.safe_load(open(os.path.expanduser("~/.werkator.yml"))) or {}
except (FileNotFoundError, ImportError):
cfg = {}
d = (cfg.get("defaults") or {}).get("git") or {}
account, token = d.get("account"), d.get("token")
env = dict(os.environ, GIT_TERMINAL_PROMPT="0")
ask = None
if account and token:
parts = urllib.parse.urlsplit(url)
host = parts.netloc.rsplit("@", 1)[-1]
url = urllib.parse.urlunsplit(parts._replace(netloc=account + "@" + host))
ask = tempfile.NamedTemporaryFile(mode="w", prefix="werkator-clone-askpass-",
suffix=".sh", delete=False)
ask.write("#!/bin/sh\nexec echo \"$WERKATOR_CLONE_TOKEN\"\n")
ask.close()
os.chmod(ask.name, stat.S_IRWXU)
env.update(GIT_ASKPASS=ask.name, WERKATOR_CLONE_TOKEN=token)
try:
subprocess.run(["git", "clone", url, dest], env=env, check=True)
finally:
if ask is not None:
os.unlink(ask.name)
PYEOF_CLONE
)"
ssh "$HOST" "echo '$helper_b64' | base64 -d | python3 - '$url' '$dest'"
;;
*)
ssh "$HOST" "git clone '$url' '$dest'"
;;
esac
}
# Sets up the WATCHED repository: an https clone (a private origin
# authenticates with the shared `defaults.git.*` credentials of
# `~/.werkator.yml`; see `clone_repo`), the werkator init with the instance
# applied, and the rootfs archive for the sandbox builds. All configuration # applied, and the rootfs archive for the sandbox builds. All configuration
# writing is init's — this script transports and invokes (step 23). # writing is init's — this script transports and invokes (step 23).
repo_init() { repo_init() {
@@ -344,11 +400,7 @@ repo_init() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning the watched repository" echo "==> Cloning the watched repository"
if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then clone_repo "$REPO_URL" "$REPO_DIR"
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
fi
if [ "$SANDBOX" = "docker" ]; then if [ "$SANDBOX" = "docker" ]; then
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile" echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
@@ -401,11 +453,7 @@ repo_add() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning $url as '$name'" echo "==> Cloning $url as '$name'"
if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then clone_repo "$url" "$SIBLING_DIR/$name"
echo " (already cloned, skipping)"
else
ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
fi
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock # The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
# binary). Without it a watched repository builds on the bare host, where the # binary). Without it a watched repository builds on the bare host, where the