diff --git a/.gitignore b/.gitignore index a081a8f..3750b5c 100644 --- a/.gitignore +++ b/.gitignore @@ -34,3 +34,5 @@ replay_pid* # Other /.local/ +/.env + diff --git a/AGENTS.md b/AGENTS.md index d5849ba..254d830 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -39,9 +39,9 @@ All production code lives under `de.hoennig.werkator`, with sub-packages `comman - Builds run detached in worktrees under `.git/werkator/worktrees/`; the primary checkout is never used for builds; never assume a single running build. - When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`. - Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config. -- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker sandbox policy (`docker.enabled`, `docker.network`), and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container, change its network, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone. +- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and bubblewrap (`bwrap.enabled`, `bwrap.rootfs`) sandbox policies, and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone. - A build definition carries the complete description of its build, split in two: the `trigger` block (`onPush`, `atTimes`, `branches`, `activeWithin`) says when and for which branches it runs, everything else what it does. `builds.default` is the base every other definition inherits its settings — never its `trigger` — from. The split is structural so that a selector added to `TriggerConfig` later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A `!` prefix in `trigger.branches` excludes and always wins. -- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, and `docker.network`. +- The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are `requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `bwrap.enabled`, and `bwrap.rootfs`. Docker and bwrap are mutually exclusive per branch — enabling both is rejected at start. - `builds` or the legacy `branches`, never both: `branches` is read only while the merged config defines no build at all (`builds.maxConcurrent` is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then `ConfigVersions.FORMAT_BROKE_IN` gets set and a leftover `branches:` key must be rejected by name — the version check alone cannot catch a file that declares no version. - Web UI: server-rendered Thymeleaf plus one hand-written `static/werkator.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `werkator.js` must produce identical display formats. - Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK. diff --git a/docs/configuration.md b/docs/configuration.md index 6ad2150..7cb34b0 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -63,7 +63,7 @@ above, giving the precedence **branch > repo install > project**. It takes prece everything that describes how this branch is built: the whole `builds` section — its own definitions and its overrides of the definitions from the project config, with `buildCommand`, `cleanCommand`, `artifactDirs`, log file names, and -`docker.image`/`dockerfile`/`context`/`env` inside them. That is how a new configuration is tried out: change it on a branch, and +`docker.image`/`dockerfile`/`context`/`env` and `bwrap.env` inside them. That is how a new configuration is tried out: change it on a branch, and no other branch's builds are affected. The branch layer is used in both places where it matters: the watcher reads the committed @@ -90,7 +90,8 @@ single branch may decide it: - the repository-side settings: the whole `gitea`, `executor`, and `watcher` sections; - the trust gate: `requirePullRequest`, and the Gitea status context: `statusContext`; -- the container sandbox policy: `docker.enabled` and `docker.network` — host-pinned as +- the container sandbox policy: `docker.enabled`/`docker.network` and + `bwrap.enabled`/`bwrap.rootfs` — host-pinned as long as only the host's configuration sets them, master-pinned once the committed configuration does. @@ -138,6 +139,12 @@ server: bindAddress: 127.0.0.1 # optional Impressum (legal disclosure) link in the web UI footer; empty hides the link impressumUrl: "" + # Resource limits of the systemd user unit generated by `init --systemd`; empty = directive omitted. + # Needed where the service shares a memory slice, e.g. Hostsharing Managed Webspaces, where a + # runaway Gradle build would starve everything else in the package. + systemd: + memoryMax: "" # e.g. 1G — written as `MemoryMax=` into the unit + tasksMax: "" # e.g. 512 — written as `TasksMax=` into the unit # Opt-in managed nginx+certbot Docker container for HTTPS, for hosts without # a usable reverse proxy (ADR 0005; see notes below and deployment.md). nginx: @@ -342,9 +349,9 @@ That is how a branch gets a build of its own without being built by the default `activeWithin` (e.g. `24h`) additionally keeps only branches whose origin head commit is younger than the duration — useful to run a nightly deep check over all recently active branches. Both parts combine as an intersection. -Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` with all its keys. +Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `bwrap` with all their keys. A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults. -`requirePullRequest`, `statusContext`, `docker.enabled`, and `docker.network` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config. +`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `bwrap.enabled`, and `bwrap.rootfs` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config. Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited. Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only. Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of. @@ -400,6 +407,23 @@ Note that the rest of `.git` — including `.git/config` — is visible to build The Docker socket is mounted into the container and `DOCKER_HOST`/`TESTCONTAINERS_*` variables are set, so Testcontainers-based builds work inside the container. All Werkator containers carry `org.hoennig.werkator` labels; stale build containers of the repository are removed before the first Docker build after a restart. +### Notes on `builds..bwrap` + +With `bwrap.enabled`, Werkator shells out to the `bwrap` CLI (bubblewrap) instead of native execution. +This is the third runtime, for hosts without root and without a Docker daemon (e.g. Hostsharing managed webspaces); see `docs/plan/17-bwrap-build-runtime.md` and ADR 0007. +`bwrap` must be on the `PATH`. + +`bwrap.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target. +It is a local path or an `http(s)` URL; a URL is downloaded once. +Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker; verify the host's user-namespace capability first with `tools/werkator-build-prerequisites.sh`. +The archive is unpacked on demand (`tar --no-same-owner`) into `.git/werkator/buildenv//rootfs`, shared across all branch worktrees like the Docker Gradle cache volume; `` derives from a hash of the source, so a changed `rootfs` unpacks a fresh environment and stale ones can be pruned. +Per-branch Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`. +`bwrap.env` adds environment variables inside the sandbox. +Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user. + +`docker` and `bwrap` are mutually exclusive per branch: enabling both is rejected at start, not silently picked. +Git works inside the sandbox exactly as inside the Docker container: the primary `.git` is mounted read-only with `.git/werkator/` masked, so builds can run read-only git commands but never reach the machine config or the control token. + ## `.git/werkator/.werkator.yml` (not committed) ```yaml diff --git a/docs/plan/17-bwrap-build-runtime.md b/docs/plan/17-bwrap-build-runtime.md index 0cd22fe..42a2960 100644 --- a/docs/plan/17-bwrap-build-runtime.md +++ b/docs/plan/17-bwrap-build-runtime.md @@ -95,6 +95,17 @@ bwrap --unshare-user --unshare-pid --die-with-parent --uid 0 --gid 0 \ - No Docker inside the sandbox, so no Testcontainers-based tests; build commands must select a Docker-free test subset. For Werkator's own build this means `TestcontainersSmokeTest` must become conditional (`enabledIf` docker present) — that change is part of this step. +**Done (branch `bwrap-build-runtime`):** `TestcontainersSmokeTest` is now gated with `enabledIf docker available` — it is reported as skipped (never failed) when no Docker daemon is reachable, and runs as before when one is. +The two manual steps of the workflow have scripts in `tools/`: +`tools/build-bwrap-rootfs.sh` builds the rootfs archive (debootstrap-minbase Debian + JDK 21 + git + locales) on any Docker machine — the rootfs is *not* built on the target; `tools/werkator-build-prerequisites.sh` re-runs the precondition command line above on the target webspace and checks all three signals. +It also checks the disk/quota situation: free space via `df` plus group-quota headroom (`quota -g` limit minus usage) against the ~4 GiB build footprint (unpacked rootfs + Gradle cache + artifacts) — an undersized quota fails the check, since a build would otherwise be blocked mid-flight (experienced on h68: 1 GiB group quota). + +**Planned: a central `tools/remote` control script** (same pattern as the user's other repos; first argument is the repo, here `werkator`, then a command): + +- `tools/remote werkator check-prerequisites @` — runs `werkator-build-prerequisites.sh` on the target host (uploaded if missing). +- `tools/remote werkator install @` — full setup, idempotent: ensure SSH access (`ssh-copy-id`, first login asks for the password), run the prerequisites check, upload runtime bundle + rootfs archive to `~/.werkator/`, unpack, clone the repo, run `werkator init`, write the machine-local bwrap config. + The one manual dependency remains: the host's public SSH key must be added to GitHub once; the script prints the key and waits. + ## Web Access under a Domain (no Docker, no managed nginx) The managed nginx/TLS container from ADR 0005 is for container hosts without a reverse proxy. diff --git a/docs/prs/2026-08-31-PR#000-bwrap-build-runtime.md b/docs/prs/2026-08-31-PR#000-bwrap-build-runtime.md new file mode 100644 index 0000000..a13cd8e --- /dev/null +++ b/docs/prs/2026-08-31-PR#000-bwrap-build-runtime.md @@ -0,0 +1,135 @@ +> **WARNING:** This document describes only the change applied in this PR. +> It may already be outdated once the next PR is merged. +> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation. + +## The Problem + +Werkator runs its builds on the host, either natively or inside a Docker container. +A Hostsharing **Managed Webspace** has neither root nor a Docker daemon, so neither runtime works there — yet that is exactly where some users want to run a Werkator that builds Werkator itself. + +The only sandboxing primitive available there is `bwrap` (bubblewrap): unprivileged user namespaces with a uid-0 mapping and read-only root binds. +Step 17 (docs/plan/17-bwrap-build-runtime.md) defines a third build runtime behind the `BuildRunner` interface that is based on it. + +## Non-Goals + +- No change to the native and Docker runtimes; bwrap is added alongside them and selected per branch via config. +- No Docker inside the sandbox; Testcontainers-based tests cannot run there and are excluded by a branch's own build command. +- No overlayfs: the webspace's bubblewrap 0.8.0 predates `--overlay`, so a throwaway writable rootfs per build is out of scope. +- No web-access deployment; that half of step 17 (Apache `.htaccess` proxy, systemd user unit, Let's Encrypt) needs a real webspace and is written up separately. +- No per-repo build automation on the webspace; this PR makes it possible, and a follow-up runs it on a real host. + +## The Scenarios + +### Feature: bubblewrap as the third build runtime + +#### Background + +- A branch selects exactly one runtime: nothing (native), `docker.enabled`, or `bwrap.enabled`. +- `bwrap.enabled` and `bwrap.rootfs` are pinned (host-set), so a branch's committed config cannot switch its own sandbox off or substitute a foreign rootfs. +- `docker.enabled` and `bwrap.enabled` are mutually exclusive per branch; enabling both is rejected, not silently picked. + +#### Scenario#000.01: A bwrap build runs the command inside the sandbox as root + +So that the build is isolated from the host exactly as the native and Docker runtimes intend. + +- **Given** a branch with `bwrap.enabled` and a `bwrap.rootfs` archive +- **When** a build for that branch starts +- **Then** Werkator unpacks the rootfs on demand into `.git/werkator/buildenv//rootfs` + - **and** invokes `bwrap` with a uid-0 mapping, a read-only root bind of the rootfs, the workspace bound at its host path, and `--chdir` into it + - **and** the returned process is the attached `bwrap` process, so streaming and cancellation behave like native builds +- **and** the environment plus `bwrap.env` are passed via `--setenv` + +##### Verified by + +- [BwrapBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/BwrapBuildRunnerTest.kt) + +#### Scenario#000.02: Git metadata mounts keep secrets out of the sandbox + +So that builds can run read-only git commands but never reach the machine config or the control token. + +- **Given** a workspace that is a worktree of the repository +- **When** the sandbox is assembled +- **Then** the primary `.git` is bound read-only + - **and** `.git/werkator/` is masked by an empty tmpfs + - **and** the worktree admin directory is bound read-write + +##### Verified by + +- [BwrapBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/BwrapBuildRunnerTest.kt) + +#### Scenario#000.03: A branch cannot turn its sandbox off or swap its rootfs + +So that the pinned sandbox policy holds for builds a branch invents as well as for ones the host already knows. + +- **Given** a branch whose committed config sets `bwrap.enabled` or `bwrap.rootfs` +- **When** that config is resolved into a build +- **Then** the pinned keys are stripped from the worktree layer + - **and** enabling both `docker` and `bwrap` on a build is rejected, not picked silently + +##### Verified by + +- [ConfigLoaderTest](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt) + +#### Scenario#000.04: The dispatcher routes builds to the bwrap runtime + +So that a bwrap-explicit branch builds inside the sandbox rather than natively. + +- **Given** a branch with `bwrap.enabled` +- **When** its build is dispatched +- **Then** `BwrapBuildRunner` is selected + +##### Verified by + +- [DispatchingBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt) + +### Feature: Werkator builds itself without Docker + +#### Background + +- Werkator's own build runs the full test suite, which includes `TestcontainersSmokeTest`. +- On a Docker-less host (the webspace, and the bwrap sandbox that builds there), that test must not fail the self-build. + +#### Scenario#000.05: The Testcontainers smoke test is skipped, not failed, without Docker + +So that a Docker-less build of Werkator itself stays green. + +- **Given** no reachable Docker daemon +- **When** the test suite runs +- **Then** `TestcontainersSmokeTest` is reported as skipped + - **and** the build is not failed by it +- **and** with a Docker daemon present the test still runs and verifies a container + +##### Verified by + +- [TestcontainersSmokeTest](../../src/test/kotlin/de/hoennig/werkator/framework/TestcontainersSmokeTest.kt) + +## The Solution + +**A third `BuildRunner` by the same shell-out pattern as git and Docker.** +`BwrapBuildRunner` shells out to the `bwrap` CLI (no library), unpacks a prepared Debian rootfs on demand into `.git/werkator/buildenv//rootfs`, reuses the step-16 git-metadata mounts verbatim, binds a persistent `.git/werkator/buildenv/home` as `/root`, and returns the attached `bwrap` process so streaming and cancellation match the other runtimes. +`DispatchingBuildRunner` routes by `bwrap.enabled`. + +**Pinning and mutual exclusion hold at the choke point.** +`bwrap.enabled` and `bwrap.rootfs` join the pinned sandbox-policy set, stripped from the worktree layer so a branch cannot disable its sandbox or substitute a foreign rootfs. +`docker` and `bwrap` are mutually exclusive per build, rejected in `buildSettings` — the single point every build passes through — instead of picked silently. + +**Tooling makes the two manual steps reproducible.** +`tools/build-bwrap-rootfs.sh` builds the rootfs archive (debootstrap-minbase Debian + JDK 21 + git + locales) on any machine with Docker, since `debootstrap` is not available on the target. +`tools/werkator-build-prerequisites.sh` re-runs the exact precondition command line from the plan on the target webspace and checks all three signals. +`TestcontainersSmokeTest` is gated with `enabledIf docker available`, so a Docker-less self-build skips it. + +## Open Questions + +- Whether the rootfs archive built by `tools/build-bwrap-rootfs.sh` is complete for `./gradlew build` has not been exercised on a real webspace; the JDK 21, git and locales package set is a good baseline but project-specific tooling must be added. +- The `systemd` user unit for a webspace should gain `MemoryMax`/`TasksMax` (configurable per the plan); not implemented here. + +## Additional Changes + +- Config template (`InitCommand`), `docs/configuration.md` and `AGENTS.md` updated so all three config places stay in sync and the pinned set is documented. +- The plan file 17 records the precondition result and the new tooling. + +## Follow-up PRs + +- ADR 0007 recording the bubblewrap runtime decision (options: bwrap vs proot/fakechroot vs plain native). +- Web-access deployment on a real webspace and the `docs/deployment.md` third variant, written once verified there. +- `MemoryMax`/`TasksMax` on the webspace systemd unit. diff --git a/src/main/kotlin/de/hoennig/werkator/build/BuildRunner.kt b/src/main/kotlin/de/hoennig/werkator/build/BuildRunner.kt index d016b0d..9d1dcb8 100644 --- a/src/main/kotlin/de/hoennig/werkator/build/BuildRunner.kt +++ b/src/main/kotlin/de/hoennig/werkator/build/BuildRunner.kt @@ -44,13 +44,16 @@ class ProcessBuildRunner : BuildRunner { /** * Selects the runtime per branch: Docker when `branches..docker.enabled`, - * native shell execution otherwise (the unchanged default). + * bubblewrap when `branches..bwrap.enabled`, native shell execution otherwise + * (the unchanged default). Docker and bwrap are mutually exclusive per branch and are + * rejected together at config load, so the branch order here never has to "pick". */ @Primary @Component class DispatchingBuildRunner( private val processBuildRunner: ProcessBuildRunner, private val dockerBuildRunner: DockerBuildRunner, + private val bwrapBuildRunner: BwrapBuildRunner, ) : BuildRunner { override fun start( command: String, @@ -60,7 +63,12 @@ class DispatchingBuildRunner( branchConfig: BranchConfig, onAuxProcess: (Process) -> Unit, ): Process { - val runner = if (branchConfig.docker.enabled) dockerBuildRunner else processBuildRunner + val runner = + when { + branchConfig.docker.enabled -> dockerBuildRunner + branchConfig.bwrap.enabled -> bwrapBuildRunner + else -> processBuildRunner + } return runner.start(command, workingDir, environment, repoDir, branchConfig, onAuxProcess) } } diff --git a/src/main/kotlin/de/hoennig/werkator/build/BwrapBuildRunner.kt b/src/main/kotlin/de/hoennig/werkator/build/BwrapBuildRunner.kt new file mode 100644 index 0000000..f41cb69 --- /dev/null +++ b/src/main/kotlin/de/hoennig/werkator/build/BwrapBuildRunner.kt @@ -0,0 +1,266 @@ +package de.hoennig.werkator.build + +import de.hoennig.werkator.config.BranchConfig +import de.hoennig.werkator.config.BwrapConfig +import de.hoennig.werkator.git.GitCommandRunner +import org.slf4j.LoggerFactory +import org.springframework.stereotype.Component +import java.nio.file.Files +import java.nio.file.Path +import java.security.MessageDigest + +/** + * Runs build commands inside a bubblewrap user-namespace sandbox (Step 17 / ADR 0007), + * for hosts without root and without a Docker daemon (e.g. Hostsharing managed + * webspaces). Shells out to the `bwrap` CLI via the generic [GitCommandRunner] process + * wrapper — no library, consistent with git and docker. + * + * The prepared rootfs (a Debian-base archive built elsewhere, since `debootstrap` is not + * available on the target) is unpacked on demand into `.git/werkator/buildenv//rootfs`, + * shared across all branch worktrees like the Docker gradle cache volume; `` derives + * from a hash of the archive source, so a changed source unpacks a fresh rootfs and stale + * ones can be pruned. The returned [Process] is the attached `bwrap` process, so log + * streaming and cancellation work exactly like native builds (`--die-with-parent` plus + * `--unshare-pid` tear down the whole tree on cancel). Git works inside the sandbox with + * the same layered mounts as the Docker runner: the primary `.git` read-only with + * `.git/werkator/` masked, see [gitMetadataMounts]. + */ +@Component +class BwrapBuildRunner( + private val commandRunner: GitCommandRunner, +) : BuildRunner { + private val log = LoggerFactory.getLogger(BwrapBuildRunner::class.java) + + /** Replaceable process launcher so unit tests can capture the assembled `bwrap` argv. */ + internal var processStarter: (List, Path) -> Process = { command, dir -> + ProcessBuilder(command).directory(dir.toFile()).start() + } + + override fun start( + command: String, + workingDir: Path, + environment: Map, + repoDir: Path, + branchConfig: BranchConfig, + onAuxProcess: (Process) -> Unit, + ): Process { + val bwrap = branchConfig.bwrap + require(bwrap.rootfs.isNotBlank()) { "branches..bwrap.rootfs must be set when bwrap.enabled is true" } + val buildEnvRoot = buildEnvRoot(repoDir) + val envKey = envKey(bwrap.rootfs) + val rootfsDir = buildEnvRoot.resolve(envKey).resolve(ROOTFS_DIR) + ensureRootfs(bwrap, rootfsDir, repoDir, onAuxProcess) + val homeDir = buildEnvRoot.resolve(HOME_DIR) + Files.createDirectories(homeDir) + val args = + invocation(command, workingDir, environment, repoDir, bwrap, rootfsDir, homeDir) + ensureMountpoints(rootfsDir, args) + return processStarter(args, repoDir) + } + + /** + * bwrap creates mountpoint directories for bind destinations inside the sandbox — + * against the read-only rootfs bind that fails with "Can't mkdir parents ... + * Read-only file system" for every destination that does not exist in the rootfs + * (the workspace under the repo, for example). The rootfs directory itself is a + * plain host directory, so we pre-create the mountpoints there; bwrap then finds + * them and has nothing left to mkdir. + */ + private fun ensureMountpoints( + rootfsDir: Path, + args: List, + ) { + var i = 0 + while (i < args.size) { + val arg = args[i] + if (arg == "--bind" || arg == "--ro-bind") { + val dest = args[i + 2] + val mountpoint = rootfsDir.resolve(dest.substring(1)) + // Skip anything that already exists in the rootfs (e.g. /etc/resolv.conf + // is a file the rootfs ships); only missing dirs are created. + if (dest.startsWith("/") && !Files.exists(mountpoint)) { + Files.createDirectories(mountpoint) + } + i += 3 + } else if (arg == "--proc" || arg == "--dev" || arg == "--tmpfs") { + // The rootfs archive ships no /proc, /dev (excluded when packed), so + // these mountpoints must exist too. + val dest = args[i + 1] + if (dest.startsWith("/") && !Files.exists(rootfsDir.resolve(dest.substring(1)))) { + Files.createDirectories(rootfsDir.resolve(dest.substring(1))) + } + i += 2 + } else { + i += 1 + } + } + } + + /** + * Unpacks the configured archive into [rootfsDir] once per environment version + * (identified by [envKey]). Missing means "not yet unpacked"; the environment is a + * cache like the Docker image and the Gradle volume, and stale ones are pruned with + * the rest of `.git/werkator`. + */ + private fun ensureRootfs( + bwrap: BwrapConfig, + rootfsDir: Path, + repoDir: Path, + onAuxProcess: (Process) -> Unit, + ) { + if (Files.isDirectory(rootfsDir)) { + return + } + Files.createDirectories(rootfsDir) + val archive = localArchive(bwrap.rootfs, rootfsDir.parent, repoDir, onAuxProcess) + log.info("unpacking build environment {} into {}", bwrap.rootfs, rootfsDir) + commandRunner.runOrThrow( + listOf("tar", "--no-same-owner", "-xf", archive, "-C", rootfsDir.toString()), + repoDir, + onProcess = onAuxProcess, + ) + } + + /** + * Resolves [BwrapConfig.rootfs] to a local archive path: a bare or `file:` path is + * used as-is; an `http(s)` URL is downloaded once into the buildenv root. GNU tar + * auto-detects the compression from the archive magic, so a `.tar.gz` or `.tar.zst` + * needs no extra flag. + */ + private fun localArchive( + rootfs: String, + envDir: Path, + repoDir: Path, + onAuxProcess: (Process) -> Unit, + ): String { + if (!rootfs.startsWith("http://") && !rootfs.startsWith("https://")) { + return rootfs.removePrefix("file://") + } + val fileName = rootfs.substringAfterLast('/').ifBlank { "buildenv" } + val target = envDir.resolve(fileName) + if (!Files.exists(target)) { + log.info("downloading build environment {} from {}", fileName, rootfs) + commandRunner.runOrThrow( + listOf("curl", "-fsSL", "-o", target.toString(), rootfs), + repoDir, + onProcess = onAuxProcess, + ) + } + return target.toString() + } + + private fun invocation( + command: String, + workspace: Path, + environment: Map, + repoDir: Path, + bwrap: BwrapConfig, + rootfsDir: Path, + homeDir: Path, + ): List { + // bwrap creates mountpoints for bind destinations inside the sandbox; a + // relative workspace path would resolve there into the read-only rootfs + // ("Can't mkdir parents ...: Read-only file system"). Bind at absolute + // host paths instead — same contract as the Docker runner. Relative + // paths come from the CLI relative to the repo, so resolve them against + // repoDir, not against the process working directory. + val repoDirAbs = repoDir.toAbsolutePath().normalize() + val workspaceAbs = + if (workspace.isAbsolute) workspace.normalize() else repoDirAbs.resolve(workspace).normalize() + val homeDirAbs = + if (homeDir.isAbsolute) homeDir.normalize() else repoDirAbs.resolve(homeDir).normalize() + val args = + mutableListOf( + "bwrap", + "--unshare-user", + "--unshare-pid", + "--die-with-parent", + "--uid", + "0", + "--gid", + "0", + "--ro-bind", + rootfsDir.toString(), + "/", + ) + // Bind the repo read-write FIRST so bwrap can create the mountpoints of + // the later binds (workspace, worktree admin dir) inside it — creating + // them against the read-only rootfs fails with "Can't mkdir parents ... + // Read-only file system". The git metadata mounts below then layer the + // usual isolation on top: read-only .git, tmpfs mask over .git/werkator, + // read-write worktree admin dir. + args += listOf("--bind", "$repoDirAbs", "$repoDirAbs") + // Git metadata mounts BEFORE the workspace bind: the tmpfs mask over + // .git/werkator must not shadow the workspace, which lives under + // .git/werkator/worktrees — the later workspace bind shadows the mask + // at exactly its own path and nothing else. + args += gitMetadataMounts(workspaceAbs, repoDir) + args += listOf("--bind", "$workspaceAbs", "$workspaceAbs") + args += listOf("--bind", "$homeDirAbs", "/root") + args += listOf("--ro-bind", "/etc/resolv.conf", "/etc/resolv.conf") + args += listOf("--proc", "/proc", "--dev", "/dev", "--tmpfs", "/tmp") + args += listOf("--setenv", "HOME", "/root") + for ((key, value) in environment) { + args += listOf("--setenv", key, value) + } + for ((key, value) in bwrap.env) { + args += listOf("--setenv", key, value) + } + args += listOf("--chdir", "$workspaceAbs", "/bin/sh", "-c", command) + return args + } + + /** + * Makes git work inside the sandbox without exposing Werkator's secrets — the same + * three layered mounts as the Docker runner, expressed in `bwrap` flags (bwrap nests + * mounts by target path like Docker): the primary `.git` read-only, an empty tmpfs + * masking `.git/werkator/` (machine config with `git.token`, control token, build + * state), and this worktree's admin directory read-write so index-refreshing commands + * keep working. Object and ref writes stay blocked by the read-only `.git` mount. + * No mounts are added when the workspace is not a worktree of [repoDir]. + */ + private fun gitMetadataMounts( + workspace: Path, + repoDir: Path, + ): List { + val gitDir = repoDir.toAbsolutePath().normalize().resolve(".git") + val workspaceGitFile = workspace.resolve(".git") + if (!Files.isDirectory(gitDir) || !Files.isRegularFile(workspaceGitFile)) { + return emptyList() + } + val adminDir = + Files + .readString(workspaceGitFile) + .substringAfter("gitdir:", "") + .trim() + .takeIf { it.isNotEmpty() } + ?.let { workspace.resolve(it).normalize() } + ?: return emptyList() + if (!adminDir.startsWith(gitDir) || !Files.isDirectory(adminDir)) { + return emptyList() + } + val args = mutableListOf("--ro-bind", "$gitDir", "$gitDir") + val werkatorDir = gitDir.resolve("werkator") + if (Files.isDirectory(werkatorDir)) { + args += listOf("--tmpfs", "$werkatorDir") + } + args += listOf("--bind", "$adminDir", "$adminDir") + return args + } + + private fun buildEnvRoot(repoDir: Path): Path = repoDir.resolve(BUILDENV_DIR) + + /** A short hash of the archive source, so a changed source unpacks a fresh rootfs. */ + private fun envKey(rootfs: String): String = + MessageDigest + .getInstance("SHA-256") + .digest(rootfs.toByteArray()) + .joinToString("") { "%02x".format(it) } + .take(12) + + companion object { + const val BUILDENV_DIR = ".git/werkator/buildenv" + const val ROOTFS_DIR = "rootfs" + const val HOME_DIR = "home" + } +} diff --git a/src/main/kotlin/de/hoennig/werkator/commands/InitCommand.kt b/src/main/kotlin/de/hoennig/werkator/commands/InitCommand.kt index d4a1a3c..ac78d34 100644 --- a/src/main/kotlin/de/hoennig/werkator/commands/InitCommand.kt +++ b/src/main/kotlin/de/hoennig/werkator/commands/InitCommand.kt @@ -19,6 +19,7 @@ import java.nio.file.Paths ) class InitCommand( private val gitService: GitService, + private val configLoader: de.hoennig.werkator.config.ConfigLoader, /** The version written into the generated config as `werkator.version.since`. */ private val buildProperties: ObjectProvider? = null, ) : Runnable { @@ -157,6 +158,11 @@ class InitCommand( bindAddress: 127.0.0.1 # optional Impressum (legal disclosure) link in the web UI footer; empty hides the link impressumUrl: "" + # Resource limits of the systemd user unit (init --systemd); empty = directive omitted. + # Needed where the service shares a memory slice, e.g. Hostsharing Managed Webspaces. + systemd: + memoryMax: "" # e.g. 1G — a runaway Gradle build must not starve the package + tasksMax: "" # e.g. 512 # Opt-in managed nginx+certbot Docker container for HTTPS, for hosts without # a usable reverse proxy (see docs/deployment.md). Off by default. nginx: @@ -215,6 +221,12 @@ class InitCommand( context: "." # Docker build context used with dockerfile network: "" # Docker network mode for the build container; empty = Docker default (pinned) env: {} # additional environment variables set inside the build container + # bubblewrap user-namespace sandbox — for hosts without root and without a + # Docker daemon (e.g. Hostsharing managed webspaces). Mutually exclusive with docker. + bwrap: + enabled: false # run clean/build in a bwrap sandbox instead of natively (pinned) + rootfs: "" # prepared rootfs archive (path or URL); required when enabled (pinned) + env: {} # additional environment variables set inside the sandbox # Gitea check this build reports as; empty uses gitea.statusContext. # Two builds of one commit under the same context overwrite each other. statusContext: "" @@ -256,6 +268,19 @@ class InitCommand( println("created ${file.toFile().relativeTo(normalizedWorkingDir.toFile())}") } + /** + * Resource limits for the unit come from the effective configuration when one is + * already loadable (re-running `init --systemd` on an installed instance); during + * the very first bootstrap they stay unset and the defaults (no directives) apply. + */ + private fun loadedSystemdConfig(): de.hoennig.werkator.config.SystemdConfig = + try { + configLoader.load(Paths.get(".")).server.systemd + } catch (_: Exception) { + de.hoennig.werkator.config + .SystemdConfig() + } + private fun createSystemdFiles( root: Path, normalizedWorkingDir: Path, @@ -277,6 +302,8 @@ class InitCommand( javaExecutable = javaExecutableResolver(), jarPath = jarPath, envFile = envFile, + memoryMax = loadedSystemdConfig().memoryMax, + tasksMax = loadedSystemdConfig().tasksMax, ), ) println("created ${unitFile.toFile().relativeTo(normalizedWorkingDir.toFile())}") diff --git a/src/main/kotlin/de/hoennig/werkator/commands/SystemdServiceFiles.kt b/src/main/kotlin/de/hoennig/werkator/commands/SystemdServiceFiles.kt index 4dbcd12..566532c 100644 --- a/src/main/kotlin/de/hoennig/werkator/commands/SystemdServiceFiles.kt +++ b/src/main/kotlin/de/hoennig/werkator/commands/SystemdServiceFiles.kt @@ -22,24 +22,33 @@ object SystemdServiceFiles { javaExecutable: Path, jarPath: Path, envFile: Path, - ): String = - """ - [Unit] - Description=Werkator CI for ${repoRoot.fileName} - Wants=network-online.target - After=network-online.target docker.service + memoryMax: String = "", + tasksMax: String = "", + ): String { + val limits = + listOfNotNull( + "MemoryMax=$memoryMax".takeIf { memoryMax.isNotBlank() }, + "TasksMax=$tasksMax".takeIf { tasksMax.isNotBlank() }, + ).joinToString("\n") + val limitsLine = if (limits.isEmpty()) "" else "\n$limits" + return """ + [Unit] + Description=Werkator CI for ${repoRoot.fileName} + Wants=network-online.target + After=network-online.target docker.service - [Service] - Type=simple - WorkingDirectory=${systemdPath("$repoRoot")} - EnvironmentFile=-${systemdPath("$envFile")} - ExecStart=${systemdQuote("$javaExecutable")} ${'$'}JAVA_OPTS -jar ${systemdQuote("$jarPath")} server - Restart=always - RestartSec=30 + [Service] + Type=simple + WorkingDirectory=${systemdPath("$repoRoot")} + EnvironmentFile=-${systemdPath("$envFile")} + ExecStart=${systemdQuote("$javaExecutable")} ${'$'}JAVA_OPTS -jar ${systemdQuote("$jarPath")} server + Restart=always + RestartSec=30 - [Install] - WantedBy=default.target - """.trimIndent() + "\n" + [Install] + WantedBy=default.target + """.trimIndent().replace("\n\n[Install]", "$limitsLine\n\n[Install]") + "\n" + } /** * Nightly Docker cleanup like the legacy `docker-prune.service`, but without `--volumes`: diff --git a/src/main/kotlin/de/hoennig/werkator/config/BuildDefinition.kt b/src/main/kotlin/de/hoennig/werkator/config/BuildDefinition.kt index 7be9dcc..0c64dad 100644 --- a/src/main/kotlin/de/hoennig/werkator/config/BuildDefinition.kt +++ b/src/main/kotlin/de/hoennig/werkator/config/BuildDefinition.kt @@ -42,6 +42,8 @@ data class BuildDefinition( val statusContext: String? = null, /** Overrides of the docker settings; null inherits them. */ val docker: DockerOverrides? = null, + /** Overrides of the bwrap settings; null inherits them. */ + val bwrap: BwrapOverrides? = null, ) { /** The settings this build runs with: [branchConfig] with this definition applied; unset values fall through. */ fun applyTo(branchConfig: BranchConfig): BranchConfig = @@ -62,6 +64,12 @@ data class BuildDefinition( network = docker?.network ?: branchConfig.docker.network, env = docker?.env ?: branchConfig.docker.env, ), + bwrap = + branchConfig.bwrap.copy( + enabled = bwrap?.enabled ?: branchConfig.bwrap.enabled, + rootfs = bwrap?.rootfs ?: branchConfig.bwrap.rootfs, + env = bwrap?.env ?: branchConfig.bwrap.env, + ), ) companion object { @@ -159,3 +167,12 @@ data class DockerOverrides( val context: String? = null, val env: Map? = null, ) + +/** Nullable bubblewrap overrides of a [BuildDefinition]; null values inherit the branch's setting. */ +data class BwrapOverrides( + /** Run the build in the bwrap sandbox instead of natively. Pinned — a branch must not escape its sandbox. */ + val enabled: Boolean? = null, + /** Rootfs archive source. Pinned — a branch must not substitute a foreign rootfs. */ + val rootfs: String? = null, + val env: Map? = null, +) diff --git a/src/main/kotlin/de/hoennig/werkator/config/ConfigLoader.kt b/src/main/kotlin/de/hoennig/werkator/config/ConfigLoader.kt index f208927..ca59590 100644 --- a/src/main/kotlin/de/hoennig/werkator/config/ConfigLoader.kt +++ b/src/main/kotlin/de/hoennig/werkator/config/ConfigLoader.kt @@ -159,6 +159,11 @@ class ConfigLoader( val strippedDocker = docker.toMutableMap().apply { PINNED_DOCKER_KEYS.forEach { remove(it) } } if (strippedDocker.isEmpty()) result.remove("docker") else result["docker"] = strippedDocker } + val bwrap = entry["bwrap"] as? Map + if (bwrap != null) { + val strippedBwrap = bwrap.toMutableMap().apply { PINNED_BWRAP_KEYS.forEach { remove(it) } } + if (strippedBwrap.isEmpty()) result.remove("bwrap") else result["bwrap"] = strippedBwrap + } return result } @@ -398,6 +403,9 @@ class ConfigLoader( /** `docker` keys a branch must never override: the sandbox policy. */ private val PINNED_DOCKER_KEYS = setOf("enabled", "network") + /** `bwrap` keys a branch must never override: the sandbox policy (Step 17). */ + private val PINNED_BWRAP_KEYS = setOf("enabled", "rootfs") + /** * The one key of a build definition that says *when* and *for which branches* it * runs; never inherited from `builds.default`. A single key on purpose: a selector diff --git a/src/main/kotlin/de/hoennig/werkator/config/WerkatorConfig.kt b/src/main/kotlin/de/hoennig/werkator/config/WerkatorConfig.kt index 8555d01..38eb09c 100644 --- a/src/main/kotlin/de/hoennig/werkator/config/WerkatorConfig.kt +++ b/src/main/kotlin/de/hoennig/werkator/config/WerkatorConfig.kt @@ -37,7 +37,14 @@ data class WerkatorConfig( build: String, ): BranchConfig { val branchConfig = branches[branch] ?: branches["default"] ?: BranchConfig() - return effectiveBuildDefinitions()[build]?.applyTo(branchConfig) ?: branchConfig + val settings = effectiveBuildDefinitions()[build]?.applyTo(branchConfig) ?: branchConfig + if (settings.docker.enabled && settings.bwrap.enabled) { + throw IllegalArgumentException( + "builds.$build on '$branch' enables both docker and bwrap; a build runs in exactly one sandbox. " + + "Disable one of them.", + ) + } + return settings } } @@ -57,9 +64,24 @@ data class ServerConfig( val bindAddress: String = "127.0.0.1", /** Optional Impressum (legal disclosure) link shown in the web UI footer; empty hides the link. */ val impressumUrl: String = "", + /** + * Resource limits for the generated systemd user unit (`init --systemd`); empty + * means the directive is not written. Needed on platforms where the service runs + * inside a shared memory slice, e.g. Hostsharing Managed Webspaces, where a + * runaway Gradle build would starve everything else in the package. + */ + val systemd: SystemdConfig = SystemdConfig(), val nginx: NginxConfig = NginxConfig(), ) +/** Resource-limit directives of the systemd user unit (`server.systemd`, see [ServerConfig.systemd]). */ +data class SystemdConfig( + /** `MemoryMax=` of the unit, e.g. `1G`; empty omits the directive. */ + val memoryMax: String = "", + /** `TasksMax=` of the unit, e.g. `512`; empty omits the directive. */ + val tasksMax: String = "", +) + /** * Opt-in managed nginx+certbot Docker container serving Werkator over HTTPS, * for hosts without a usable reverse proxy (ADR 0005). Off by default; the @@ -157,6 +179,26 @@ data class BranchConfig( val statusContext: String = "", val autoBuild: AutoBuildConfig = AutoBuildConfig(), val docker: DockerConfig = DockerConfig(), + /** bubblewrap user-namespace sandbox; mutually exclusive with [docker]. */ + val bwrap: BwrapConfig = BwrapConfig(), +) + +/** + * bubblewrap build sandbox (Step 17): runs the build in an unprivileged user namespace + * with a prepared Debian root filesystem. For hosts without root and without a Docker + * daemon (e.g. Hostsharing managed webspaces); see `docs/plan/17-bwrap-build-runtime.md`. + */ +data class BwrapConfig( + /** Run the clean and build commands in a bwrap sandbox instead of natively. */ + val enabled: Boolean = false, + /** + * Path or URL of the prepared rootfs archive (e.g. `werkator-buildenv-trixie-java21.tar.zst`), + * unpacked on demand into `.git/werkator/buildenv//rootfs`; required when [enabled]. + * Pinned — a branch must not substitute a foreign rootfs via its committed config. + */ + val rootfs: String = "", + /** Additional environment variables set inside the sandbox. */ + val env: Map = emptyMap(), ) data class DockerConfig( diff --git a/src/test/kotlin/de/hoennig/werkator/build/BwrapBuildRunnerTest.kt b/src/test/kotlin/de/hoennig/werkator/build/BwrapBuildRunnerTest.kt new file mode 100644 index 0000000..65c8932 --- /dev/null +++ b/src/test/kotlin/de/hoennig/werkator/build/BwrapBuildRunnerTest.kt @@ -0,0 +1,240 @@ +package de.hoennig.werkator.build + +import de.hoennig.werkator.config.BranchConfig +import de.hoennig.werkator.config.BwrapConfig +import de.hoennig.werkator.git.GitCommandResult +import de.hoennig.werkator.git.GitCommandRunner +import io.kotest.assertions.throwables.shouldThrow +import io.kotest.core.spec.style.FunSpec +import io.kotest.matchers.shouldBe +import io.kotest.matchers.string.shouldContain +import io.mockk.clearMocks +import io.mockk.every +import io.mockk.mockk +import io.mockk.verify +import java.nio.file.Files +import java.nio.file.Path + +class BwrapBuildRunnerTest : FunSpec() { + private val commandRunner = mockk() + private lateinit var runner: BwrapBuildRunner + private lateinit var repoDir: Path + private lateinit var workspace: Path + private val captured = mutableListOf>() + + private fun bwrapBranchConfig( + rootfs: String = "/srv/buildenv.tar.zst", + env: Map = emptyMap(), + ): BranchConfig = + BranchConfig( + bwrap = + BwrapConfig( + enabled = true, + rootfs = rootfs, + env = env, + ), + ) + + private fun rootfsUnpacked(rootfs: String = "/srv/buildenv.tar.zst"): Path = + repoDir + .resolve(BwrapBuildRunner.BUILDENV_DIR) + .resolve(rootfs.sha12()) + .resolve(BwrapBuildRunner.ROOTFS_DIR) + + init { + beforeEach { + clearMocks(commandRunner) + captured.clear() + repoDir = Files.createTempDirectory("werkator-bwrap-runner") + workspace = repoDir.resolve("workspace") + runner = BwrapBuildRunner(commandRunner) + runner.processStarter = { command, _ -> + captured += command + ProcessBuilder("true").start() + } + } + + test("unpacks the rootfs on demand and assembles the exact bwrap command") { + every { + commandRunner.runOrThrow( + listOf("tar", "--no-same-owner", "-xf", "/srv/buildenv.tar.zst", "-C", rootfsUnpacked().toString()), + repoDir, + any(), + any(), + ) + } returns + GitCommandResult(0, "", "") + + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig()) + + val args = captured.single() + val rootfsDir = args[args.indexOf("--ro-bind") + 1] + args shouldBe + listOf( + "bwrap", + "--unshare-user", + "--unshare-pid", + "--die-with-parent", + "--uid", + "0", + "--gid", + "0", + "--ro-bind", + rootfsUnpacked().toString(), + "/", + "--bind", + repoDir.toString(), + repoDir.toString(), + "--bind", + workspace.toString(), + workspace.toString(), + "--bind", + repoDir.resolve(".git/werkator/buildenv/home").toString(), + "/root", + "--ro-bind", + "/etc/resolv.conf", + "/etc/resolv.conf", + "--proc", + "/proc", + "--dev", + "/dev", + "--tmpfs", + "/tmp", + "--setenv", + "HOME", + "/root", + "--setenv", + "branch", + "main", + "--chdir", + workspace.toString(), + "/bin/sh", + "-c", + "./gradlew test", + ) + Files.isDirectory(rootfsUnpacked()) shouldBe true + } + + test("binds a relative workspace path at its absolute location") { + // bwrap creates mountpoints for bind destinations inside the sandbox; + // a relative path would land in the read-only rootfs and fail with + // "Can't mkdir parents ...: Read-only file system" (seen on the webspace). + every { + commandRunner.runOrThrow( + listOf("tar", "--no-same-owner", "-xf", "/srv/buildenv.tar.zst", "-C", rootfsUnpacked().toString()), + repoDir, + any(), + any(), + ) + } returns + GitCommandResult(0, "", "") + + val relativeWorkspace = repoDir.relativize(workspace) + + runner.start("./gradlew test", relativeWorkspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig()) + + val args = captured.single() + val absolute = workspace.toAbsolutePath().normalize().toString() + val bindIdx = args.withIndex().filter { it.value == "--bind" }.map { it.index } + // first bind is the repo dir (mountpoint base), second is the workspace + args[bindIdx[1] + 1] shouldBe absolute + args[bindIdx[1] + 2] shouldBe absolute + args[args.indexOf("--chdir") + 1] shouldBe absolute + } + + test("does not re-unpack an already prepared rootfs") { + Files.createDirectories(rootfsUnpacked()) + + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig()) + + verify(exactly = 0) { commandRunner.runOrThrow(match { it.first() == "tar" }, any(), any(), any()) } + } + + test("adds bwrap env and passes the branch environment through") { + Files.createDirectories(rootfsUnpacked()) + + runner.start( + "./gradlew test", + workspace, + mapOf("branch" to "main"), + repoDir, + bwrapBranchConfig(env = mapOf("FOO" to "bar")), + ) + + val args = captured.single() + args[args.indexOf("branch") - 1] shouldBe "--setenv" + args[args.indexOf("branch") + 1] shouldBe "main" + args[args.indexOf("FOO") - 1] shouldBe "--setenv" + args[args.indexOf("FOO") + 1] shouldBe "bar" + } + + test("exposes git metadata read-only with the werkator dir masked for a worktree workspace") { + val gitDir = repoDir.resolve(".git") + val adminDir = gitDir.resolve("worktrees/workspace") + Files.createDirectories(adminDir) + Files.createDirectories(gitDir.resolve("werkator")) + Files.createDirectories(workspace) + Files.writeString(workspace.resolve(".git"), "gitdir: $adminDir\n") + Files.createDirectories(rootfsUnpacked()) + + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig()) + + val args = captured.single() + args[args.indexOf(gitDir.toString()) - 1] shouldBe "--ro-bind" + args[args.indexOf("$gitDir/werkator") - 1] shouldBe "--tmpfs" + args[args.indexOf(adminDir.toString()) - 1] shouldBe "--bind" + } + + test("mounts no git metadata when the workspace is not a worktree") { + Files.createDirectories(rootfsUnpacked()) + Files.createDirectories(workspace) + + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig()) + + val args = captured.single() + val gitDir = repoDir.resolve(".git") + // the sandbox's own /tmp tmpfs is always present; the point is that no tmpfs + // masks .git/werkator and no worktree admin dir is bound + args.none { it == "$gitDir/werkator" } shouldBe true + args.none { it.contains("worktrees/") } shouldBe true + } + + test("fails without a configured rootfs") { + val branchConfig = BranchConfig(bwrap = BwrapConfig(enabled = true)) + + val exception = + shouldThrow { + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, branchConfig) + } + + exception.message shouldContain "bwrap.rootfs" + } + + test("downloads a URL rootfs once before unpacking") { + val url = "https://example.test/buildenv.tar.zst" + val downloadTarget = + repoDir + .resolve(BwrapBuildRunner.BUILDENV_DIR) + .resolve(url.sha12()) + .resolve("buildenv.tar.zst") + every { commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) } returns + GitCommandResult(0, "", "") + every { commandRunner.runOrThrow(match { it.first() == "tar" }, any(), any(), any()) } returns + GitCommandResult(0, "", "") + + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, bwrapBranchConfig(rootfs = url)) + + verify { + commandRunner.runOrThrow(listOf("curl", "-fsSL", "-o", downloadTarget.toString(), url), repoDir, any(), any()) + } + verify { commandRunner.runOrThrow(match { it.first() == "tar" }, repoDir, any(), any()) } + } + } + + private fun String.sha12(): String = + java.security.MessageDigest + .getInstance("SHA-256") + .digest(toByteArray()) + .joinToString("") { "%02x".format(it) } + .take(12) +} diff --git a/src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt b/src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt index d39da45..906d451 100644 --- a/src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt +++ b/src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt @@ -1,6 +1,7 @@ package de.hoennig.werkator.build import de.hoennig.werkator.config.BranchConfig +import de.hoennig.werkator.config.BwrapConfig import de.hoennig.werkator.config.DockerConfig import io.kotest.core.spec.style.FunSpec import io.kotest.matchers.shouldBe @@ -14,12 +15,13 @@ import java.nio.file.Paths class DispatchingBuildRunnerTest : FunSpec() { private val processBuildRunner = mockk() private val dockerBuildRunner = mockk() - private val dispatcher = DispatchingBuildRunner(processBuildRunner, dockerBuildRunner) + private val bwrapBuildRunner = mockk() + private val dispatcher = DispatchingBuildRunner(processBuildRunner, dockerBuildRunner, bwrapBuildRunner) private val process = mockk() private val dir = Paths.get(".") init { - beforeEach { clearMocks(processBuildRunner, dockerBuildRunner) } + beforeEach { clearMocks(processBuildRunner, dockerBuildRunner, bwrapBuildRunner) } test("runs natively by default") { val branchConfig = BranchConfig() @@ -28,6 +30,7 @@ class DispatchingBuildRunnerTest : FunSpec() { dispatcher.start("cmd", dir, emptyMap(), dir, branchConfig) shouldBe process verify { dockerBuildRunner wasNot Called } + verify { bwrapBuildRunner wasNot Called } } test("runs in Docker when the branch enables it") { @@ -37,6 +40,17 @@ class DispatchingBuildRunnerTest : FunSpec() { dispatcher.start("cmd", dir, emptyMap(), dir, branchConfig) shouldBe process verify { processBuildRunner wasNot Called } + verify { bwrapBuildRunner wasNot Called } + } + + test("runs in bwrap when the branch enables it (and not Docker)") { + val branchConfig = BranchConfig(bwrap = BwrapConfig(enabled = true, rootfs = "/srv/buildenv.tar.zst")) + every { bwrapBuildRunner.start("cmd", dir, emptyMap(), dir, branchConfig) } returns process + + dispatcher.start("cmd", dir, emptyMap(), dir, branchConfig) shouldBe process + + verify { processBuildRunner wasNot Called } + verify { dockerBuildRunner wasNot Called } } } } diff --git a/src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt b/src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt index 57dd631..554d6d0 100644 --- a/src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt +++ b/src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt @@ -14,7 +14,12 @@ import java.nio.file.attribute.PosixFilePermissions class InitCommandTest : FunSpec() { private val gitService = mockk() - private val initCommand = InitCommand(gitService) + private val initCommand = + InitCommand( + gitService, + de.hoennig.werkator.config + .ConfigLoader(mockk(relaxed = true)), + ) init { test("creates config files with auto-detected values") { diff --git a/src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt b/src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt index 8b9c0e0..1417ac3 100644 --- a/src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt +++ b/src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt @@ -33,6 +33,26 @@ class SystemdServiceFilesTest : FunSpec() { content shouldContain "WantedBy=default.target" } + test("resource limits are written when configured and omitted when unset") { + fun unit( + memoryMax: String, + tasksMax: String, + ) = SystemdServiceFiles.unitFileContent( + repoRoot = Paths.get("/srv/repos/my-repo"), + javaExecutable = Paths.get("/usr/bin/java"), + jarPath = Paths.get("/srv/repos/my-repo/werkator.jar"), + envFile = Paths.get("/srv/repos/my-repo/werkator.env"), + memoryMax = memoryMax, + tasksMax = tasksMax, + ) + val with = unit(memoryMax = "1G", tasksMax = "512") + with shouldContain "MemoryMax=1G" + with shouldContain "TasksMax=512" + val without = unit(memoryMax = "", tasksMax = "") + without shouldNotContain "MemoryMax" + without shouldNotContain "TasksMax" + } + test("percent signs in paths are escaped for systemd") { val content = SystemdServiceFiles.unitFileContent( diff --git a/src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt b/src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt index cd34a4f..018f689 100644 --- a/src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt +++ b/src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt @@ -350,6 +350,39 @@ class ConfigLoaderTest : FunSpec() { settings.docker.image shouldBe "attacker-image" } + test("a branch cannot disable its bwrap sandbox or substitute a foreign rootfs through a build definition") { + val dir = Files.createTempDirectory("werkator-test") + dir.resolve(".werkator.yml").toFile().writeText( + """ + builds: + default: + bwrap: + enabled: true + rootfs: /host/rootfs.tar.zst + """.trimIndent(), + ) + val worktree = Files.createTempDirectory("werkator-test-worktree") + worktree.resolve(".werkator.yml").toFile().writeText( + """ + builds: + default: + bwrap: + enabled: false + rootfs: /attacker/rootfs.tar.zst + env: + FOO: from-branch + """.trimIndent(), + ) + + val settings = loader.loadForWorktree(dir, worktree).buildSettings("any-branch", "default") + + // pinned: the sandbox can neither be switched off nor pointed at a foreign rootfs + settings.bwrap.enabled shouldBe true + settings.bwrap.rootfs shouldBe "/host/rootfs.tar.zst" + // everything that describes the build itself stays the branch's own business + settings.bwrap.env shouldBe mapOf("FOO" to "from-branch") + } + test("a build the branch invents inherits the host's sandbox policy") { val dir = Files.createTempDirectory("werkator-test") dir.resolve(".werkator.yml").toFile().writeText( @@ -386,6 +419,31 @@ class ConfigLoaderTest : FunSpec() { settings.requirePullRequest shouldBe true } + test("enabling both docker and bwrap on a build is rejected, not picked silently") { + val dir = Files.createTempDirectory("werkator-test") + dir.resolve(".werkator.yml").toFile().writeText( + """ + builds: + default: + docker: + enabled: true + image: build-env + bwrap: + enabled: true + rootfs: /srv/rootfs.tar.zst + """.trimIndent(), + ) + + val config = loader.load(dir) + val exception = + shouldThrow { + config.buildSettings("any-branch", "default") + } + + exception.message shouldContain "both docker and bwrap" + exception.message shouldContain "builds.default" + } + test("an exclusion pattern takes a branch out of a build that would otherwise select it") { val dir = Files.createTempDirectory("werkator-test") dir.resolve(".werkator.yml").toFile().writeText( diff --git a/src/test/kotlin/de/hoennig/werkator/framework/TestcontainersSmokeTest.kt b/src/test/kotlin/de/hoennig/werkator/framework/TestcontainersSmokeTest.kt index fd8b3da..ece0d97 100644 --- a/src/test/kotlin/de/hoennig/werkator/framework/TestcontainersSmokeTest.kt +++ b/src/test/kotlin/de/hoennig/werkator/framework/TestcontainersSmokeTest.kt @@ -2,18 +2,35 @@ package de.hoennig.werkator.framework import io.kotest.core.spec.style.FunSpec import io.kotest.matchers.shouldBe +import org.testcontainers.DockerClientFactory import org.testcontainers.containers.GenericContainer import org.testcontainers.utility.DockerImageName +/** + * Probes that Testcontainers can actually start a container on this host. + * Skipped, not failed, when no Docker is present — that is what lets Werkator + * build itself on a Docker-less host (e.g. a Hostsharing webspace, where its own + * build runs in the bubblewrap sandbox); see `tools/werkator-build-prerequisites.sh`. + */ class TestcontainersSmokeTest : FunSpec({ - test("Testcontainers starts a container") { - val container = - GenericContainer(DockerImageName.parse("alpine:3")) - .withCommand("sh", "-c", "sleep 30") - container.start() - container.isRunning shouldBe true - container.stop() - } - }) + test("Testcontainers starts a container") + .config(enabledIf = { dockerAvailable() }) { + val container = + GenericContainer(DockerImageName.parse("alpine:3")) + .withCommand("sh", "-c", "sleep 30") + container.start() + container.isRunning shouldBe true + container.stop() + } + }) { + companion object { + private fun dockerAvailable(): Boolean = + try { + DockerClientFactory.instance().isDockerAvailable() + } catch (_: Throwable) { + false + } + } +} diff --git a/tools/build-bwrap-rootfs.sh b/tools/build-bwrap-rootfs.sh new file mode 100755 index 0000000..5b1a5fe --- /dev/null +++ b/tools/build-bwrap-rootfs.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +# +# Build the Werkator bwrap build environment (rootfs) archive. +# +# The bwrap build runtime (step 17 / ADR 0007) runs each build inside a +# bubblewrap user namespace, chrooted into a *prepared* Debian root filesystem. +# That rootfs is NOT built on the target (the webspace has no root and no +# debootstrap), it is built once on any machine that can — most comfortably a +# machine with Docker — and distributed as an archive, e.g. +# `werkator-buildenv-trixie-java21.tar.zst`. +# +# This script builds exactly that archive: a debootstrap-minbase Debian release +# plus the packages Werkator itself needs to run `./gradlew build` inside the +# sandbox (JDK 21, git, ca-certificates, locales, curl/unzip for the wrapper). +# The whole build runs inside a throwaway Docker container, so no root is +# needed on the machine running this script. +# +# Why it works this way (each quirk learned the hard way): +# - The whole job runs in ONE container whose stdin carries a base64-encoded +# script (no file is bind-mounted for the script — a bind-mounted script +# hit noexec/tmpfs trouble and vanished inside the container). +# - The rootfs is built inside the container's own writable layer, NOT on a +# bind-mounted host directory — debootstrap "Tried to extract package, but +# tar failed" when its target sat on some bind-mounted/special filesystems. +# - Only the final `tar --zstd` writes to stdout; every build step is +# redirected to stderr, so the archive coming out of `docker run` is pure. +# +# Usage: build-bwrap-rootfs.sh [--release trixie] [--mirror URL] [--out path] +# --release Debian release/architecture tail, default "trixie" +# --mirror apt mirror for debootstrap, default http://deb.debian.org/debian +# --out output archive path, default ./werkator-buildenv-.tar.zst +# + +set -euo pipefail + +die() { echo "ERROR: $*" >&2; exit 1; } +warn() { echo "WARNING: $*" >&2; } + +usage() { + echo "usage: build-bwrap-rootfs.sh [--release trixie] [--mirror URL] [--out path]" >&2 + exit 1 +} + +# --------------------------------------------------------------- arguments -- + +release="trixie" +out="" +while [ $# -gt 0 ]; do + case "$1" in + --release) release="${2:?missing value for --release}"; shift 2 ;; + --mirror) mirror="${2:?missing value for --mirror}"; shift 2 ;; + --out) out="${2:?missing value for --out}"; shift 2 ;; + -*) die "unknown option: $1" ;; + *) usage ;; + esac +done +mirror="${mirror:-http://deb.debian.org/debian}" +[ -n "$out" ] || out="$(pwd)/werkator-buildenv-${release}.tar.zst" + +command -v docker >/dev/null 2>&1 || die "docker is required to build the rootfs" + +# Rootfs content: Werkator's own build needs a JDK 21 toolchain (Gradle +# toolchain resolution), git, ca-certificates for HTTPS, locales for git, and +# curl/unzip/xz-utils/zstd for the Gradle wrapper and general build hygiene. +# Keep this list additive — project-specific tooling goes on top of this base. +PKGS="openjdk-21-jdk git ca-certificates locales procps file curl unzip xz-utils zstd" + +# The chroot step runs inside the freshly debootstrapped rootfs; passed into +# the container as base64 so no nested heredoc corrupts the piped script. +inner="$(printf '%s' '#!/bin/bash +set -euxo pipefail +mount -t proc none /proc +apt-get update -qq +apt-get install -y --no-install-recommends '"${PKGS}"' +apt-get clean +rm -f /etc/localtime +locale-gen en_US.UTF-8 de_DE.UTF-8 >/dev/null 2>&1 || true +update-locale LANG=en_US.UTF-8 >/dev/null 2>&1 || true +' | base64 -w0)" + +# The outer script runs inside the Debian container as root. Build noise goes +# to stderr (fd 1 is saved on fd 3 and restored only for the final tar), so +# docker stdout is exactly the archive. +outer="$(printf '%s' '#!/bin/bash +set -euo pipefail +export DEBIAN_FRONTEND=noninteractive +exec 3>&1 +exec 1>&2 +apt-get update -qq +apt-get install -y --no-install-recommends debootstrap zstd ca-certificates +mkdir -p /b/rootfs +debootstrap --variant=minbase --components=main,contrib --include=apt,ca-certificates '"${release}"' /b/rootfs '"${mirror}"' +mount --bind /proc /b/rootfs/proc +mount --bind /sys /b/rootfs/sys +mount --bind /dev /b/rootfs/dev +echo '"${inner}"' | base64 -d > /b/rootfs/inner.sh +chmod +x /b/rootfs/inner.sh +chroot /b/rootfs /bin/bash /inner.sh +umount /b/rootfs/proc; umount /b/rootfs/sys; umount /b/rootfs/dev +exec 1>&3 +tar --zstd --exclude=proc --exclude=sys --exclude=dev -C /b/rootfs -cf - . +' | base64 -w0)" + +echo "building ${release} rootfs (downloads packages, takes a while; log below)..." +echo "archive → $out" + +# Stream the base64-encoded outer script into the container over stdin; the +# archive lands on stdout (redirected to $out), the build log on stderr. +docker run --rm -i --privileged debian:"${release}-slim" \ + bash -c 'base64 -d | bash' \ + <<<"$outer" >"$out" + +echo +echo "OK: build environment written to $out" +echo " Configure it as branches..bwrap.rootfs (a bare path or a URL)" +echo " on the target Werkator instance to build in this environment." diff --git a/tools/remote b/tools/remote new file mode 100755 index 0000000..2fc45ea --- /dev/null +++ b/tools/remote @@ -0,0 +1,344 @@ +#!/usr/bin/env bash +# +# Central control script for remote Werkator operations (same pattern as the +# `remote` scripts in the other repos): the first argument is the repo selector, +# the second the command. All connection and deployment values come from the +# `.env` file in the repository root — never as command line parameters. +# +# Usage: +# tools/remote werkator check-prerequisites +# tools/remote werkator install +# tools/remote werkator build # WERKATOR_BRANCH to override, default main +# tools/remote werkator start +# tools/remote port-forward start # background tunnel to the Werkator UI +# tools/remote port-forward stop +# tools/remote werkator control-token +# +# Required in .env: +# WERKATOR_REMOTE user@host to operate on, e.g. mih34-werkator@mih34.hostsharing.net +# WERKATOR_PATH target directory on that host, e.g. /home/storage/mih34/users/werkator +# +# Required for `start`: +# WERKATOR_PORT the localhost port assigned by Hostsharing (eigener Serverdienst) +# WERKATOR_DOMAIN the domain served by the managed Apache, e.g. ci.example.de +# +# Required for `port-forward`: +# WERKATOR_LOCAL_PORT the local port the browser uses +# Optional in .env: +# WERKATOR_BRANCH branch for `build` (default: main) +# WERKATOR_MEMORY_MAX systemd MemoryMax for the unit, e.g. 1G (start) +# WERKATOR_TASKS_MAX systemd TasksMax for the unit, e.g. 512 (start) +# WERKATOR_ROOTFS rootfs archive path +# (default: /build/werkator-buildenv-trixie.tar.zst) +# +# Install layout on the host: +# $WERKATOR_PATH/werkator/ the repository clone +# $WERKATOR_PATH/.werkator/ runtime bundle + rootfs archive +# +# `install` performs, in order: +# 1. check-prerequisites (bwrap capability + disk/quota, aborts on FAIL) +# 2. ensure SSH access (ssh-copy-id on first use; asks for the password) +# 3. upload artifacts (runtime bundle, built locally if missing, + rootfs) +# 4. clone the repository (needs the host SSH key registered at GitHub once — +# the script prints the key and waits) +# 5. `werkator init` + machine-local bwrap configuration +# + +set -euo pipefail + +REPO="${1:-}" +COMMAND="${2:-}" + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +PREREQ_SCRIPT="$REPO_ROOT/tools/werkator-build-prerequisites.sh" +RUNTIME_BUNDLE="$REPO_ROOT/build/distributions/werkator-runtime-linux-x64.tar.gz" +PID_FILE="/tmp/werkator-port-forward-$(id -u).pid" +LOG_FILE="/tmp/werkator-port-forward-$(id -u).log" + +usage() { + sed -n '3,32p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' + exit 2 +} + +require_env() { + local missing=0 + for name in "$@"; do + if [ -z "${!name:-}" ]; then + echo "ERROR: $name is not set — define it in $REPO_ROOT/.env" >&2 + missing=1 + fi + done + [ "$missing" -eq 0 ] || exit 1 +} + +[ -n "$REPO" ] && [ -n "$COMMAND" ] || usage + +# Load the connection and deployment values; explicit environment wins, the +# .env in the repository root fills the rest. +set -a +[ -f "$REPO_ROOT/.env" ] && source "$REPO_ROOT/.env" +set +a + +require_env WERKATOR_REMOTE WERKATOR_PATH +HOST="$WERKATOR_REMOTE" +TARGET_DIR="$WERKATOR_PATH" +ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie.tar.zst}" + +ssh_present() { + ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST" true 2>/dev/null +} + +ensure_ssh() { + if ssh_present; then + echo "==> SSH access to $HOST: ok" + else + echo "==> No key-based SSH access yet; running ssh-copy-id (password prompt expected)" + ssh-copy-id "$HOST" + ssh_present || { echo "ERROR: SSH access still not working after ssh-copy-id" >&2; exit 1; } + fi +} + +# Run the prerequisites script remotely by piping it over stdin; TARGET_DIR and +# ROOTFS_ARCHIVE are passed as arguments to `bash -s --`. +check_prerequisites() { + echo "==> Checking prerequisites on $HOST (target dir: $TARGET_DIR)" + local rootfs_remote="$TARGET_DIR/.werkator/$(basename "$ROOTFS")" + if ! ssh "$HOST" "WERKATOR_SSH_TARGET='$HOST' bash -s -- '$TARGET_DIR' '$rootfs_remote'" < "$PREREQ_SCRIPT"; then + echo "ERROR: prerequisites failed on $HOST — install aborted" >&2 + exit 1 + fi +} + +ensure_local_artifacts() { + if [ ! -f "$RUNTIME_BUNDLE" ]; then + echo "==> Runtime bundle not found; building it locally (./gradlew runtimeBundle)" + (cd "$REPO_ROOT" && ./gradlew runtimeBundle --console=plain -q) + fi + [ -f "$RUNTIME_BUNDLE" ] || { echo "ERROR: runtime bundle missing: $RUNTIME_BUNDLE" >&2; exit 1; } + [ -f "$ROOTFS" ] || { + echo "ERROR: rootfs archive missing: $ROOTFS" >&2 + echo " build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS" >&2 + exit 1 + } +} + +ensure_github_access() { + # `ssh -T git@github.com` exits 1 even on success ("does not provide shell + # access") — neutralize remotely, then match on the greeting text. + if ssh "$HOST" 'ssh -o BatchMode=yes -o ConnectTimeout=10 -T git@github.com 2>&1 || true' | grep -q "successfully authenticated"; then + echo "==> GitHub SSH access from $HOST: ok" + return 0 + fi + echo + echo "==> The host cannot reach GitHub via SSH yet." + echo " Add THIS public key to GitHub (Settings > SSH and GPG keys > New SSH key):" + ssh "$HOST" 'cat ~/.ssh/id_*.pub 2>/dev/null' || { + echo "ERROR: no public key on the host; create one with ssh-keygen -t ed25519" >&2 + exit 1 + } + read -r -p " Press Enter once the key is registered at GitHub... " + ssh "$HOST" 'ssh -o BatchMode=yes -T git@github.com 2>&1 || true' | grep -q "successfully authenticated" || { + echo "ERROR: GitHub authentication from $HOST still failing" >&2 + exit 1 + } + echo "==> GitHub SSH access from $HOST: ok" +} + +install() { + ensure_ssh + check_prerequisites + ensure_local_artifacts + + echo "==> Uploading runtime bundle and rootfs archive" + ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator'" + scp -q "$RUNTIME_BUNDLE" "$HOST:$TARGET_DIR/.werkator/" + scp -q "$ROOTFS" "$HOST:$TARGET_DIR/.werkator/" + + echo "==> Unpacking runtime bundle" + ssh "$HOST" "tar xzf '$TARGET_DIR/.werkator/$(basename "$RUNTIME_BUNDLE")' -C '$TARGET_DIR/.werkator'" + ssh "$HOST" "'$TARGET_DIR/.werkator/werkator/bin/werkator' --version" + + ensure_github_access + + echo "==> Cloning the repository" + if ssh "$HOST" "test -d '$TARGET_DIR/werkator/.git'"; then + echo " (already cloned, skipping)" + else + ssh "$HOST" "git clone git@github.com:mhoennig/werkator.git '$TARGET_DIR/werkator'" + fi + + echo "==> Running werkator init" + ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$TARGET_DIR/.werkator/werkator/bin/werkator' init" + + echo "==> Writing machine-local bwrap configuration" + ssh "$HOST" "grep -q '^ bwrap:' '$TARGET_DIR/werkator/.git/werkator/.werkator.yml' 2>/dev/null" || ssh "$HOST" "cat >> '$TARGET_DIR/werkator/.git/werkator/.werkator.yml' <<'CFG' + +# Build in the bubblewrap sandbox instead of natively (Step 17 / ADR 0007). +# Both keys are pinned: read from this machine config even if a branch sets +# its own values in a committed .werkator.yml. +builds: + default: + bwrap: + enabled: true + rootfs: $TARGET_DIR/.werkator/$(basename "$ROOTFS") +CFG" + + echo "==> Verifying the effective configuration" + ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$TARGET_DIR/.werkator/werkator/bin/werkator' config:print 2>/dev/null | grep -A3 'bwrap:' | head -4" + + echo + echo "==> Install complete." + echo " Repo: $TARGET_DIR/werkator" + echo " Runtime: $TARGET_DIR/.werkator/werkator/bin/werkator" + echo " Next: tools/remote werkator build" +} + +build() { + ensure_ssh + local branch="${WERKATOR_BRANCH:-main}" + echo "==> Running one initial build of branch '$branch' on $HOST (in the bwrap sandbox)" + ssh -t "$HOST" "cd '$TARGET_DIR/werkator' && '$TARGET_DIR/.werkator/werkator/bin/werkator' build '$branch'" +} + +# Start the server as a systemd user unit behind the managed Apache. +# WERKATOR_MEMORY_MAX / WERKATOR_TASKS_MAX (optional) are written into the +# machine config so `init --systemd` bakes them into the unit. +start() { + ensure_ssh + require_env WERKATOR_PORT WERKATOR_DOMAIN + local machine="$TARGET_DIR/werkator/.git/werkator/.werkator.yml" + local unit="werkator-$(basename "$TARGET_DIR/werkator").service" + local htaccess="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/.htaccess" + + echo "==> Writing server settings to the machine config" + if ssh "$HOST" "grep -q '^server:' '$machine' 2>/dev/null"; then + # re-run: update port and publicBaseUrl in place (systemd limits stay as written) + ssh "$HOST" "sed -i 's/^ port: .*/ port: $WERKATOR_PORT/; s|^ publicBaseUrl: .*| publicBaseUrl: \"https://$WERKATOR_DOMAIN/\"|' '$machine'" + else + ssh "$HOST" "cat >> '$machine' <<'CFG' + +# Web access: the managed Apache terminates TLS and proxies to the localhost +# port assigned by Hostsharing (eigener Serverdienst); TLS is the domain's +# Let's Encrypt certificate, so Werkator itself stays on 127.0.0.1. +server: + port: $WERKATOR_PORT + bindAddress: 127.0.0.1 + publicBaseUrl: \"https://$WERKATOR_DOMAIN/\" + nginx: + enabled: false + systemd: + memoryMax: \"${WERKATOR_MEMORY_MAX:-}\" + tasksMax: \"${WERKATOR_TASKS_MAX:-}\" +CFG" + fi + + echo "==> Writing the Apache reverse proxy to $htaccess" + ssh "$HOST" "mkdir -p '$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www' && cat > '$htaccess' <<'HT' +DirectoryIndex disabled +RewriteEngine On +RewriteBase / +RewriteRule .* http://127.0.0.1:$WERKATOR_PORT%{REQUEST_URI} [proxy] +HT" + + echo "==> Generating the systemd user unit (init --systemd)" + ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$TARGET_DIR/.werkator/werkator/bin/werkator' init --systemd" + + echo "==> Linking the units into ~/.config/systemd/user and enabling the service" + ssh "$HOST" "mkdir -p ~/.config/systemd/user && \ + ln -sf '$TARGET_DIR/werkator/.git/werkator/$unit' ~/.config/systemd/user/ && \ + ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.service' ~/.config/systemd/user/ && \ + ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.timer' ~/.config/systemd/user/ && \ + systemctl --user daemon-reload && systemctl --user restart '$unit' && systemctl --user status '$unit' --no-pager -l | head -12" + + echo + echo "==> Server started. Verify: https://$WERKATOR_DOMAIN/" + echo " Logs: ssh $HOST -- systemctl --user status '$unit'" +} + +# Background SSH tunnel to the Werkator server, so the browser reaches the UI +# at http://localhost: without keeping a terminal busy. +# `start` runs ssh -N -L detached with a pid file; `stop` kills it. +port_forward() { + require_env WERKATOR_LOCAL_PORT + local remote_port + remote_port="$(ssh "$HOST" "awk '/^server:/{f=1;next} f && /^ port:/{print \$2; exit}' '$TARGET_DIR/werkator/.git/werkator/.werkator.yml'")" + [ -n "$remote_port" ] || { echo "ERROR: no server.port in the machine config — run 'tools/remote werkator start' first" >&2; exit 1; } + + case "$COMMAND" in + start) + if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + echo "==> Port-forward already running (pid $(cat "$PID_FILE")) — http://localhost:$WERKATOR_LOCAL_PORT" + exit 0 + fi + nohup ssh -N -L "$WERKATOR_LOCAL_PORT:127.0.0.1:$remote_port" "$HOST" \ + >"$LOG_FILE" 2>&1 & + echo $! > "$PID_FILE" + sleep 1 + if kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + echo "==> Forwarding http://localhost:$WERKATOR_LOCAL_PORT -> $HOST:127.0.0.1:$remote_port (pid $(cat "$PID_FILE"))" + else + echo "ERROR: port-forward failed to start — see $LOG_FILE" >&2 + rm -f "$PID_FILE" + exit 1 + fi + ;; + stop) + if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + kill "$(cat "$PID_FILE")" + rm -f "$PID_FILE" + echo "==> Port-forward stopped" + else + rm -f "$PID_FILE" + echo "==> Port-forward is not running" + fi + ;; + *) + echo "ERROR: unknown port-forward command: $COMMAND (use start or stop)" >&2 + exit 2 + ;; + esac +} + +# Print the control token guarding the mutating build endpoints. If the server +# has not created it yet (it does so on first use), generate one in place — the +# server reads the file lazily, so a pre-created token is equivalent. +control_token() { + ensure_ssh + local token_file="$TARGET_DIR/werkator/.git/werkator/control-token" + ssh "$HOST" "if [ -f '$token_file' ]; then cat '$token_file'; else \ + umask 077 && head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \\n' > '$token_file' && cat '$token_file'; fi" +} + +case "$REPO" in + port-forward) + port_forward + ;; + werkator) + case "$COMMAND" in + check-prerequisites) + ensure_ssh + check_prerequisites + ;; + install) + install + ;; + build) + build + ;; + start) + start + ;; + control-token) + control_token + ;; + *) + echo "ERROR: unknown command: $COMMAND" >&2 + usage + ;; + esac + ;; + *) + echo "ERROR: unknown repo selector: $REPO" >&2 + usage + ;; +esac diff --git a/tools/werkator-build-prerequisites.sh b/tools/werkator-build-prerequisites.sh new file mode 100755 index 0000000..8534448 --- /dev/null +++ b/tools/werkator-build-prerequisites.sh @@ -0,0 +1,164 @@ +#!/usr/bin/env bash +# +# Verify the bwrap (bubblewrap) build precondition on a target host before +# running Werkator's bwrap build runtime there (step 17 / ADR 0007). +# +# The whole "build Werkator inside bubblewrap on a Managed Webspace" approach +# hinges on one hard precondition: unprivileged user namespaces with a uid-0 +# mapping and read-only root binds must work. This script runs the exact +# command line recorded in docs/plan/17-bwrap-build-runtime.md, checks the +# expected signals, and additionally verifies the disk/quota situation: +# a bwrap build unpacks the rootfs (a zstd archive expands to several GiB) +# plus a Gradle distribution and per-branch caches, so the host needs both +# raw free space and enough group-quota headroom. +# +# Run this ON the target host (the webspace), no root needed. +# +# Optional: the rootfs archive to size the disk/quota check against, e.g. +# werkator-build-prerequisites.sh /path/to/werkator-buildenv-trixie.tar.zst +# When omitted, the check runs against a conservative default footprint. +# +# Usage: werkator-build-prerequisites.sh [TARGET_DIR] [ROOTFS_ARCHIVE] +# +# TARGET_DIR is the directory the build workspace will live in (default: $HOME). +# The check verifies it sits on the home filesystem and has enough free space. +# ROOTFS_ARCHIVE, when given, is the rootfs archive that will be used there. +# +# Output is one PASS/FAIL line per check plus a final RESULT line, e.g.: +# PASS: bwrap version: bubblewrap 0.8.0 +# PASS: build runs as root inside the namespace (uid 0) +# PASS: uid_map maps root back to the unprivileged user (uid 120957) +# PASS: read-only root bind is enforced +# PASS: at least 5 GiB free space on the build working filesystem +# FAIL: group quota headroom below the 5 GiB build footprint ... +# RESULT: FAIL (4/5) — Werkator bubblewrap builds are not usable on this host. +# + +set -euo pipefail + +target_dir_arg="${1:-}" +rootfs_arg="${2:-}" + +die() { echo "ERROR: $*" >&2; exit 1; } + +# Disk footprint a bwrap build needs headroom for, in 1K blocks: unpacked +# rootfs (zstd expands roughly 3-4x), Gradle distribution + per-branch cache, +# build output and artifacts. ~5 GiB. +MIN_FREE_BLOCKS=$((5 * 1024 * 1024)) + +# Reference filesystem: the one the invoking user's home directory lives on. +# Builds (repo clone, buildenv, caches) must run there — other mounts, such as +# a slow mass-storage volume, are rejected. +HOME_FS="$(df -Pk "$HOME" 2>/dev/null | awk 'NR==2 {print $1}')" + +pass=0 +fail=0 +result() { # result PASS|FAIL "message" + echo "$1: $2" + if [ "$1" = "PASS" ]; then pass=$((pass+1)); else fail=$((fail+1)); fi +} + +command -v bwrap >/dev/null 2>&1 || die "bwrap is not installed on this host" + +output="$(bwrap --unshare-user --unshare-pid --die-with-parent --uid 0 --gid 0 \ + --ro-bind / / --dev /dev --proc /proc --tmpfs /tmp \ + sh -c 'id -u && cat /proc/self/uid_map && (touch /usr/ro-test 2>&1 || true)' 2>&1)" || + die "bwrap invocation failed (no user namespace support?): $output" + +# Signal 0: bwrap itself is usable (version as a visible marker). +result PASS "bwrap version: $(bwrap --version 2>&1)" + +# Signal 1: runs as root (uid 0) inside the namespace. +first="$(printf '%s\n' "$output" | sed -n '1p')" +if [ "$first" = "0" ]; then + result PASS "build runs as root inside the namespace (uid 0)" +else + result FAIL "expected uid 0 inside the namespace, got: $first" +fi + +# Signal 2: uid_map maps root to the invoking unprivileged user. +uid_line="$(printf '%s\n' "$output" | sed -n '2p')" +self_uid="$(id -u)" +if printf '%s\n' "$uid_line" | grep -E "^[[:space:]]*0[[:space:]]+${self_uid}[[:space:]]+1" >/dev/null; then + result PASS "uid_map maps root back to the unprivileged user (uid $self_uid)" +else + result FAIL "expected uid_map '0 $self_uid 1', got: $uid_line" +fi + +# Signal 3: the read-only root bind is enforced (a write to /usr fails). +if printf '%s\n' "$output" | grep -qi "read-only file system"; then + result PASS "read-only root bind is enforced" +else + result FAIL "the read-only root bind did not reject a write to /usr" +fi + +# --- Disk / quota checks ------------------------------------------------ + +target_dir="${target_dir_arg:-$HOME}" +target_dir="$(realpath -m "$target_dir")" +min_gib=$((MIN_FREE_BLOCKS / 1024 / 1024)) + +if [ -n "$rootfs_arg" ] && [ ! -f "$rootfs_arg" ]; then + echo "WARNING: rootfs archive not found: $rootfs_arg (continuing without it)" +fi + +target_fs="$(df -Pk "$target_dir" 2>/dev/null | awk 'NR==2 {print $1}')" +df_output="$(df -Pk "$target_dir" 2>/dev/null | awk 'NR==2 {print int($4) " " $6}')" +if [ -n "$df_output" ]; then + avail_k="${df_output%% *}" + mount="${df_output##* }" + if [ -n "$HOME_FS" ] && [ "$target_fs" != "$HOME_FS" ]; then + # An explicitly chosen foreign filesystem is allowed (e.g. for testing) + # but flagged: builds there will be slow. + echo "WARNING: target dir is on $target_fs (mounted at $mount), not the home filesystem ($HOME_FS) — builds will run on slower storage" + fi + if [ "${avail_k:-0}" -lt "$MIN_FREE_BLOCKS" ]; then + result FAIL "less than ${min_gib} GiB free space on the build working filesystem ($mount)" + else + result PASS "at least ${min_gib} GiB free space on the build working filesystem ($mount, device $target_fs)" + fi +else + echo "WARNING: could not measure free space on $target_dir — only the quota check below applies" +fi + +if quota_output="$(quota -g 2>/dev/null)" && [ -n "$quota_output" ]; then + quota_ok=1 + quota_seen=0 + detail="" + while read -r fs blocks quota_limit; do + quota_seen=1 + # Only the quota of the target filesystem counts — other volumes may + # legitimately be full or unquota'd without affecting the build. + if [ -n "$target_fs" ] && [ "$(basename "$fs")" != "$(basename "$target_fs")" ] && [ "$fs" != "$target_fs" ]; then + continue + fi + headroom=$((quota_limit - blocks)) + if [ "$headroom" -lt "$MIN_FREE_BLOCKS" ]; then + quota_ok=0 + detail+=" $(basename "$fs"): $(awk -v b="$headroom" 'BEGIN{printf "%.1f", b/1024/1024}') GiB free of quota;" + fi + done < <(printf '%s\n' "$quota_output" | awk ' + NF==1 && $1 ~ /^\// { pending_fs=$1; next } + $1 ~ /^\// && $2 ~ /^[0-9]+$/ { print $1, $2, $4; pending_fs=""; next } + $1 ~ /^[0-9]+[*]?/ && pending_fs != "" { gsub(/\*/, "", $1); print pending_fs, $1, $3; pending_fs="" }') + if [ "$quota_seen" -eq 0 ]; then + echo "WARNING: quota tooling present but no group quota lines could be parsed — only free space was checked" + elif [ "$quota_ok" -eq 1 ]; then + result PASS "group quota headroom covers the ${min_gib} GiB build footprint" + else + result FAIL "group quota headroom below the ${min_gib} GiB build footprint (rootfs + Gradle cache); raise the quota before building.$detail" + fi +else + echo "WARNING: no readable group quota tooling on this host — only free space was checked" +fi + +total=$((pass + fail)) +echo +if [ "$fail" -eq 0 ]; then + echo "RESULT: PASS ($pass/$total) — Werkator bubblewrap builds are usable on this host." + echo "Next: install the Werkator instance with: tools/remote werkator install ${WERKATOR_SSH_TARGET:-@} '$target_dir'" + exit 0 +else + echo "RESULT: FAIL ($pass/$total) — Werkator bubblewrap builds are not usable on this host." + exit 1 +fi