mi/werkator
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Bwrap build runtime und Installation in Hostsharing Managed Webspace (#4)

Browse Source 
* Add the bubblewrap build runtime (step 17, ADR 0007)

BwrapBuildRunner: third runtime behind BuildRunner for hosts without root
and without Docker (e.g. Hostsharing managed webspaces). Shells out to the
bwrap CLI, unpacks a prepared rootfs on demand into
.git/werkator/buildenv/<envKey>/rootfs, reuses the Docker runner's git
metadata mounts, and returns the attached bwrap process for streaming and
cancellation.

Config: bwrap.enabled/rootfs/env on BranchConfig and BwrapOverrides on
BuildDefinition; enabled/rootfs are pinned like the docker sandbox policy.
Docker and bwrap are mutually exclusive per build, rejected in
buildSettings instead of picked silently. DispatchingBuildRunner routes
bwrap; InitCommand template, docs/configuration.md and AGENTS.md in sync.

* bwrap rollout tooling: remote script, prerequisites disk/quota check, absolute workspace binds

- tools/remote: central remote control script with check-prerequisites, install and build commands
- tools/werkator-build-prerequisites.sh: compact PASS/FAIL output, target-dir parameter, free-space and
  group-quota headroom checks against the ~5 GiB build footprint, home-filesystem reference
- BwrapBuildRunner: bind workspace and home at absolute paths resolved against repoDir — a relative
  path made bwrap create mountpoints inside the read-only rootfs (seen on the webspace); regression test
- TestcontainersSmokeTest: gated with enabledIf docker available (skip, never fail, without a daemon)
- docs: configuration reference, step-17 plan notes, PR-doc

* bwrap: bind the repo read-write before the workspace so mountpoints are creatable

bwrap creates mountpoints for bind destinations inside the sandbox; with only a
read-only rootfs bound at /, creating them for the workspace under .git/werkator/
worktrees failed with 'Read-only file system' (seen on the webspace). Binding the
repo dir read-write first provides the base; the git metadata mounts then layer
the usual isolation on top (read-only .git, tmpfs mask over .git/werkator,
read-write worktree admin dir).

* bwrap: pre-create bind mountpoints inside the unpacked rootfs

bwrap mkdirs mountpoints for bind destinations against the sandbox view; with the
rootfs ro-bound at / every destination missing from the rootfs (the repo dir under
/home/storage/... on the webspace) fails with 'Read-only file system'. The rootfs
directory is a plain host dir, so create the mountpoints there before launching
bwrap; it then finds them and has nothing left to create.

* bwrap: skip existing rootfs files when pre-creating bind mountpoints

/etc/resolv.conf is a file the rootfs already ships; createDirectories threw on it.
Only missing directories are created now.

* bwrap: pre-create proc/dev/tmpfs mountpoints in the rootfs too

The rootfs archive ships no /proc or /dev (excluded when packed), so bwrap failed
mkdir'ing their mountpoints against the read-only root.

* bwrap: bind the workspace after the git metadata mounts

The tmpfs mask over .git/werkator shadowed the earlier workspace bind, because the
worktree lives under .git/werkator/worktrees — chdir then failed with ENOENT. The
workspace bind now comes last and shadows the mask at exactly its own path.

* systemd resource limits and webspace start command (step 17, web access)

- server.systemd.memoryMax/tasksMax (empty = directive omitted): on platforms where
  the service runs in a shared memory slice (Hostsharing Managed Webspaces) a
  runaway Gradle build must not starve the whole package; init --systemd reads the
  effective config and bakes the values into the generated unit
- tools/remote werkator start: writes server settings (assigned port, loopback
  bind, publicBaseUrl, nginx off) plus the Apache reverse-proxy .htaccess into
  ~/doms/<domain>/subs/www, runs init --systemd and enables the user unit
- docs/configuration.md documents the new keys

* tools/remote: env-based configuration and background port-forward

All connection and deployment values come from .env in the repository root
(WERKATOR_REMOTE, WERKATOR_PATH, WERKATOR_PORT, WERKATOR_DOMAIN,
WERKATOR_LOCAL_PORT, optional WERKATOR_BRANCH/MEMORY_MAX/TASKS_MAX/ROOTFS);
missing values fail with a pointing error instead of positional parameters.

- port-forward is now 'tools/remote port-forward start|stop' with a detached
  ssh tunnel, pid file under /tmp, and idempotent start
- start restarts the systemd unit after updating the machine config
- control-token generates the token in place when the server has not yet
- the rootfs archive default moves to build/ (already gitignored)
This commit is contained in:
Michael Hönnig
2026-08-31 19:56:04 +02:00
committed by GitHub
parent 516765a717
commit 71f1fc62c6
21 changed files with 1564 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/configuration.md
+28 -4
View File
@@ -63,7 +63,7 @@ above, giving the precedence **branch > repo install > project**. It takes prece
everything that describes how this branch is built: the whole `builds` section — its own
definitions and its overrides of the definitions from the project config, with
`buildCommand`, `cleanCommand`, `artifactDirs`, log file names, and
`docker.image`/`dockerfile`/`context`/`env` inside them. That is how a new configuration is tried out: change it on a branch, and
`docker.image`/`dockerfile`/`context`/`env` and `bwrap.env` inside them. That is how a new configuration is tried out: change it on a branch, and
no other branch's builds are affected.
The branch layer is used in both places where it matters: the watcher reads the committed
@@ -90,7 +90,8 @@ single branch may decide it:
- the repository-side settings: the whole `gitea`, `executor`, and `watcher` sections;
- the trust gate: `requirePullRequest`, and the Gitea status context: `statusContext`;
- the container sandbox policy: `docker.enabled` and `docker.network` — host-pinned as
- the container sandbox policy: `docker.enabled`/`docker.network` and
`bwrap.enabled`/`bwrap.rootfs` — host-pinned as
long as only the host's configuration sets them, master-pinned once the committed
configuration does.
@@ -138,6 +139,12 @@ server:
bindAddress: 127.0.0.1
# optional Impressum (legal disclosure) link in the web UI footer; empty hides the link
impressumUrl: ""
# Resource limits of the systemd user unit generated by `init --systemd`; empty = directive omitted.
# Needed where the service shares a memory slice, e.g. Hostsharing Managed Webspaces, where a
# runaway Gradle build would starve everything else in the package.
systemd:
memoryMax: "" # e.g. 1G — written as `MemoryMax=` into the unit
tasksMax: "" # e.g. 512 — written as `TasksMax=` into the unit
# Opt-in managed nginx+certbot Docker container for HTTPS, for hosts without
# a usable reverse proxy (ADR 0005; see notes below and deployment.md).
nginx:
@@ -342,9 +349,9 @@ That is how a branch gets a build of its own without being built by the default
`activeWithin` (e.g. `24h`) additionally keeps only branches whose origin head commit is younger than the duration — useful to run a nightly deep check over all recently active branches.
Both parts combine as an intersection.
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` with all its keys.
Settings: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, `requirePullRequest`, `statusContext`, and `docker` and `bwrap` with all their keys.
A definition carries the complete description of its build; unset keys fall back to `builds.default` and then to Werkator's own defaults.
`requirePullRequest`, `statusContext`, `docker.enabled`, and `docker.network` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
`requirePullRequest`, `statusContext`, `docker.enabled`, `docker.network`, `bwrap.enabled`, and `bwrap.rootfs` are pinned (master-pinned, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci)): they are read from the repo install/project config even when a branch sets them in its own committed config.
Inheritance from `builds.default` covers the settings only — the `trigger` block says when and where *this* build runs and is never inherited.
Definitions are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only.
Because the inheritance is applied after all layers are merged, a build a branch invents still inherits the host's `builds.default` — its sandbox policy included, which is what keeps the pinning effective for a build the host has never heard of.
@@ -400,6 +407,23 @@ Note that the rest of `.git` — including `.git/config` — is visible to build
The Docker socket is mounted into the container and `DOCKER_HOST`/`TESTCONTAINERS_*` variables are set, so Testcontainers-based builds work inside the container.
All Werkator containers carry `org.hoennig.werkator` labels; stale build containers of the repository are removed before the first Docker build after a restart.
### Notes on `builds.<name>.bwrap`
With `bwrap.enabled`, Werkator shells out to the `bwrap` CLI (bubblewrap) instead of native execution.
This is the third runtime, for hosts without root and without a Docker daemon (e.g. Hostsharing managed webspaces); see `docs/plan/17-bwrap-build-runtime.md` and ADR 0007.
`bwrap` must be on the `PATH`.
`bwrap.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
It is a local path or an `http(s)` URL; a URL is downloaded once.
Build the archive with `tools/build-bwrap-rootfs.sh` on any machine with Docker; verify the host's user-namespace capability first with `tools/werkator-build-prerequisites.sh`.
The archive is unpacked on demand (`tar --no-same-owner`) into `.git/werkator/buildenv/<envKey>/rootfs`, shared across all branch worktrees like the Docker Gradle cache volume; `<envKey>` derives from a hash of the source, so a changed `rootfs` unpacks a fresh environment and stale ones can be pruned.
Per-branch Gradle caches persist in `.git/werkator/buildenv/home`, bound as `/root`.
`bwrap.env` adds environment variables inside the sandbox.
Files created inside the sandbox are owned by the host user, because uid 0 maps back to the unprivileged webspace user.
`docker` and `bwrap` are mutually exclusive per branch: enabling both is rejected at start, not silently picked.
Git works inside the sandbox exactly as inside the Docker container: the primary `.git` is mounted read-only with `.git/werkator/` masked, so builds can run read-only git commands but never reach the machine config or the control token.
## `.git/werkator/.werkator.yml` (not committed)
```yaml
docs/plan/17-bwrap-build-runtime.md
+11
View File
@@ -95,6 +95,17 @@ bwrap --unshare-user --unshare-pid --die-with-parent --uid 0 --gid 0 \
- No Docker inside the sandbox, so no Testcontainers-based tests; build commands must select a Docker-free test subset.
For Werkator's own build this means `TestcontainersSmokeTest` must become conditional (`enabledIf` docker present) — that change is part of this step.
**Done (branch `bwrap-build-runtime`):** `TestcontainersSmokeTest` is now gated with `enabledIf docker available` — it is reported as skipped (never failed) when no Docker daemon is reachable, and runs as before when one is.
The two manual steps of the workflow have scripts in `tools/`:
`tools/build-bwrap-rootfs.sh` builds the rootfs archive (debootstrap-minbase Debian + JDK 21 + git + locales) on any Docker machine — the rootfs is *not* built on the target; `tools/werkator-build-prerequisites.sh` re-runs the precondition command line above on the target webspace and checks all three signals.
It also checks the disk/quota situation: free space via `df` plus group-quota headroom (`quota -g` limit minus usage) against the ~4 GiB build footprint (unpacked rootfs + Gradle cache + artifacts) — an undersized quota fails the check, since a build would otherwise be blocked mid-flight (experienced on h68: 1 GiB group quota).
**Planned: a central `tools/remote` control script** (same pattern as the user's other repos; first argument is the repo, here `werkator`, then a command):
- `tools/remote werkator check-prerequisites <user>@<host>` — runs `werkator-build-prerequisites.sh` on the target host (uploaded if missing).
- `tools/remote werkator install <user>@<host>` — full setup, idempotent: ensure SSH access (`ssh-copy-id`, first login asks for the password), run the prerequisites check, upload runtime bundle + rootfs archive to `~/.werkator/`, unpack, clone the repo, run `werkator init`, write the machine-local bwrap config.
The one manual dependency remains: the host's public SSH key must be added to GitHub once; the script prints the key and waits.
## Web Access under a Domain (no Docker, no managed nginx)
The managed nginx/TLS container from ADR 0005 is for container hosts without a reverse proxy.
docs/prs/2026-08-31-PR#000-bwrap-build-runtime.md
+135
View File
@@ -0,0 +1,135 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
Werkator runs its builds on the host, either natively or inside a Docker container.
A Hostsharing **Managed Webspace** has neither root nor a Docker daemon, so neither runtime works there — yet that is exactly where some users want to run a Werkator that builds Werkator itself.
The only sandboxing primitive available there is `bwrap` (bubblewrap): unprivileged user namespaces with a uid-0 mapping and read-only root binds.
Step 17 (docs/plan/17-bwrap-build-runtime.md) defines a third build runtime behind the `BuildRunner` interface that is based on it.
## Non-Goals
- No change to the native and Docker runtimes; bwrap is added alongside them and selected per branch via config.
- No Docker inside the sandbox; Testcontainers-based tests cannot run there and are excluded by a branch's own build command.
- No overlayfs: the webspace's bubblewrap 0.8.0 predates `--overlay`, so a throwaway writable rootfs per build is out of scope.
- No web-access deployment; that half of step 17 (Apache `.htaccess` proxy, systemd user unit, Let's Encrypt) needs a real webspace and is written up separately.
- No per-repo build automation on the webspace; this PR makes it possible, and a follow-up runs it on a real host.
## The Scenarios
### Feature: bubblewrap as the third build runtime
#### Background
- A branch selects exactly one runtime: nothing (native), `docker.enabled`, or `bwrap.enabled`.
- `bwrap.enabled` and `bwrap.rootfs` are pinned (host-set), so a branch's committed config cannot switch its own sandbox off or substitute a foreign rootfs.
- `docker.enabled` and `bwrap.enabled` are mutually exclusive per branch; enabling both is rejected, not silently picked.
#### Scenario#000.01: A bwrap build runs the command inside the sandbox as root
So that the build is isolated from the host exactly as the native and Docker runtimes intend.
- **Given** a branch with `bwrap.enabled` and a `bwrap.rootfs` archive
- **When** a build for that branch starts
- **Then** Werkator unpacks the rootfs on demand into `.git/werkator/buildenv/<envKey>/rootfs`
- **and** invokes `bwrap` with a uid-0 mapping, a read-only root bind of the rootfs, the workspace bound at its host path, and `--chdir` into it
- **and** the returned process is the attached `bwrap` process, so streaming and cancellation behave like native builds
- **and** the environment plus `bwrap.env` are passed via `--setenv`
##### Verified by
- [BwrapBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/BwrapBuildRunnerTest.kt)
#### Scenario#000.02: Git metadata mounts keep secrets out of the sandbox
So that builds can run read-only git commands but never reach the machine config or the control token.
- **Given** a workspace that is a worktree of the repository
- **When** the sandbox is assembled
- **Then** the primary `.git` is bound read-only
- **and** `.git/werkator/` is masked by an empty tmpfs
- **and** the worktree admin directory is bound read-write
##### Verified by
- [BwrapBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/BwrapBuildRunnerTest.kt)
#### Scenario#000.03: A branch cannot turn its sandbox off or swap its rootfs
So that the pinned sandbox policy holds for builds a branch invents as well as for ones the host already knows.
- **Given** a branch whose committed config sets `bwrap.enabled` or `bwrap.rootfs`
- **When** that config is resolved into a build
- **Then** the pinned keys are stripped from the worktree layer
- **and** enabling both `docker` and `bwrap` on a build is rejected, not picked silently
##### Verified by
- [ConfigLoaderTest](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#000.04: The dispatcher routes builds to the bwrap runtime
So that a bwrap-explicit branch builds inside the sandbox rather than natively.
- **Given** a branch with `bwrap.enabled`
- **When** its build is dispatched
- **Then** `BwrapBuildRunner` is selected
##### Verified by
- [DispatchingBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt)
### Feature: Werkator builds itself without Docker
#### Background
- Werkator's own build runs the full test suite, which includes `TestcontainersSmokeTest`.
- On a Docker-less host (the webspace, and the bwrap sandbox that builds there), that test must not fail the self-build.
#### Scenario#000.05: The Testcontainers smoke test is skipped, not failed, without Docker
So that a Docker-less build of Werkator itself stays green.
- **Given** no reachable Docker daemon
- **When** the test suite runs
- **Then** `TestcontainersSmokeTest` is reported as skipped
- **and** the build is not failed by it
- **and** with a Docker daemon present the test still runs and verifies a container
##### Verified by
- [TestcontainersSmokeTest](../../src/test/kotlin/de/hoennig/werkator/framework/TestcontainersSmokeTest.kt)
## The Solution
**A third `BuildRunner` by the same shell-out pattern as git and Docker.**
`BwrapBuildRunner` shells out to the `bwrap` CLI (no library), unpacks a prepared Debian rootfs on demand into `.git/werkator/buildenv/<envKey>/rootfs`, reuses the step-16 git-metadata mounts verbatim, binds a persistent `.git/werkator/buildenv/home` as `/root`, and returns the attached `bwrap` process so streaming and cancellation match the other runtimes.
`DispatchingBuildRunner` routes by `bwrap.enabled`.
**Pinning and mutual exclusion hold at the choke point.**
`bwrap.enabled` and `bwrap.rootfs` join the pinned sandbox-policy set, stripped from the worktree layer so a branch cannot disable its sandbox or substitute a foreign rootfs.
`docker` and `bwrap` are mutually exclusive per build, rejected in `buildSettings` — the single point every build passes through — instead of picked silently.
**Tooling makes the two manual steps reproducible.**
`tools/build-bwrap-rootfs.sh` builds the rootfs archive (debootstrap-minbase Debian + JDK 21 + git + locales) on any machine with Docker, since `debootstrap` is not available on the target.
`tools/werkator-build-prerequisites.sh` re-runs the exact precondition command line from the plan on the target webspace and checks all three signals.
`TestcontainersSmokeTest` is gated with `enabledIf docker available`, so a Docker-less self-build skips it.
## Open Questions
- Whether the rootfs archive built by `tools/build-bwrap-rootfs.sh` is complete for `./gradlew build` has not been exercised on a real webspace; the JDK 21, git and locales package set is a good baseline but project-specific tooling must be added.
- The `systemd` user unit for a webspace should gain `MemoryMax`/`TasksMax` (configurable per the plan); not implemented here.
## Additional Changes
- Config template (`InitCommand`), `docs/configuration.md` and `AGENTS.md` updated so all three config places stay in sync and the pinned set is documented.
- The plan file 17 records the precondition result and the new tooling.
## Follow-up PRs
- ADR 0007 recording the bubblewrap runtime decision (options: bwrap vs proot/fakechroot vs plain native).
- Web-access deployment on a real webspace and the `docs/deployment.md` third variant, written once verified there.
- `MemoryMax`/`TasksMax` on the webspace systemd unit.