mi/werkator
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
3e1085fa52b8173f211c0ee1d31aa0a8beca6ad3
werkator/docs/prs/2026-08-31-PR#000-bwrap-build-runtime.md
T
Michael Hönnig 71f1fc62c6 Bwrap build runtime und Installation in Hostsharing Managed Webspace (#4) 
* Add the bubblewrap build runtime (step 17, ADR 0007)

BwrapBuildRunner: third runtime behind BuildRunner for hosts without root
and without Docker (e.g. Hostsharing managed webspaces). Shells out to the
bwrap CLI, unpacks a prepared rootfs on demand into
.git/werkator/buildenv/<envKey>/rootfs, reuses the Docker runner's git
metadata mounts, and returns the attached bwrap process for streaming and
cancellation.

Config: bwrap.enabled/rootfs/env on BranchConfig and BwrapOverrides on
BuildDefinition; enabled/rootfs are pinned like the docker sandbox policy.
Docker and bwrap are mutually exclusive per build, rejected in
buildSettings instead of picked silently. DispatchingBuildRunner routes
bwrap; InitCommand template, docs/configuration.md and AGENTS.md in sync.

* bwrap rollout tooling: remote script, prerequisites disk/quota check, absolute workspace binds

- tools/remote: central remote control script with check-prerequisites, install and build commands
- tools/werkator-build-prerequisites.sh: compact PASS/FAIL output, target-dir parameter, free-space and
  group-quota headroom checks against the ~5 GiB build footprint, home-filesystem reference
- BwrapBuildRunner: bind workspace and home at absolute paths resolved against repoDir — a relative
  path made bwrap create mountpoints inside the read-only rootfs (seen on the webspace); regression test
- TestcontainersSmokeTest: gated with enabledIf docker available (skip, never fail, without a daemon)
- docs: configuration reference, step-17 plan notes, PR-doc

* bwrap: bind the repo read-write before the workspace so mountpoints are creatable

bwrap creates mountpoints for bind destinations inside the sandbox; with only a
read-only rootfs bound at /, creating them for the workspace under .git/werkator/
worktrees failed with 'Read-only file system' (seen on the webspace). Binding the
repo dir read-write first provides the base; the git metadata mounts then layer
the usual isolation on top (read-only .git, tmpfs mask over .git/werkator,
read-write worktree admin dir).

* bwrap: pre-create bind mountpoints inside the unpacked rootfs

bwrap mkdirs mountpoints for bind destinations against the sandbox view; with the
rootfs ro-bound at / every destination missing from the rootfs (the repo dir under
/home/storage/... on the webspace) fails with 'Read-only file system'. The rootfs
directory is a plain host dir, so create the mountpoints there before launching
bwrap; it then finds them and has nothing left to create.

* bwrap: skip existing rootfs files when pre-creating bind mountpoints

/etc/resolv.conf is a file the rootfs already ships; createDirectories threw on it.
Only missing directories are created now.

* bwrap: pre-create proc/dev/tmpfs mountpoints in the rootfs too

The rootfs archive ships no /proc or /dev (excluded when packed), so bwrap failed
mkdir'ing their mountpoints against the read-only root.

* bwrap: bind the workspace after the git metadata mounts

The tmpfs mask over .git/werkator shadowed the earlier workspace bind, because the
worktree lives under .git/werkator/worktrees — chdir then failed with ENOENT. The
workspace bind now comes last and shadows the mask at exactly its own path.

* systemd resource limits and webspace start command (step 17, web access)

- server.systemd.memoryMax/tasksMax (empty = directive omitted): on platforms where
  the service runs in a shared memory slice (Hostsharing Managed Webspaces) a
  runaway Gradle build must not starve the whole package; init --systemd reads the
  effective config and bakes the values into the generated unit
- tools/remote werkator start: writes server settings (assigned port, loopback
  bind, publicBaseUrl, nginx off) plus the Apache reverse-proxy .htaccess into
  ~/doms/<domain>/subs/www, runs init --systemd and enables the user unit
- docs/configuration.md documents the new keys

* tools/remote: env-based configuration and background port-forward

All connection and deployment values come from .env in the repository root
(WERKATOR_REMOTE, WERKATOR_PATH, WERKATOR_PORT, WERKATOR_DOMAIN,
WERKATOR_LOCAL_PORT, optional WERKATOR_BRANCH/MEMORY_MAX/TASKS_MAX/ROOTFS);
missing values fail with a pointing error instead of positional parameters.

- port-forward is now 'tools/remote port-forward start|stop' with a detached
  ssh tunnel, pid file under /tmp, and idempotent start
- start restarts the systemd unit after updating the machine config
- control-token generates the token in place when the server has not yet
- the rootfs archive default moves to build/ (already gitignored)
2026-08-31 19:56:04 +02:00

7.3 KiB
Raw Blame History

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.

The Problem

Werkator runs its builds on the host, either natively or inside a Docker container. A Hostsharing Managed Webspace has neither root nor a Docker daemon, so neither runtime works there — yet that is exactly where some users want to run a Werkator that builds Werkator itself.

The only sandboxing primitive available there is bwrap (bubblewrap): unprivileged user namespaces with a uid-0 mapping and read-only root binds. Step 17 (docs/plan/17-bwrap-build-runtime.md) defines a third build runtime behind the BuildRunner interface that is based on it.

Non-Goals

  • No change to the native and Docker runtimes; bwrap is added alongside them and selected per branch via config.
  • No Docker inside the sandbox; Testcontainers-based tests cannot run there and are excluded by a branch's own build command.
  • No overlayfs: the webspace's bubblewrap 0.8.0 predates --overlay, so a throwaway writable rootfs per build is out of scope.
  • No web-access deployment; that half of step 17 (Apache .htaccess proxy, systemd user unit, Let's Encrypt) needs a real webspace and is written up separately.
  • No per-repo build automation on the webspace; this PR makes it possible, and a follow-up runs it on a real host.

The Scenarios

Feature: bubblewrap as the third build runtime

Background

  • A branch selects exactly one runtime: nothing (native), docker.enabled, or bwrap.enabled.
  • bwrap.enabled and bwrap.rootfs are pinned (host-set), so a branch's committed config cannot switch its own sandbox off or substitute a foreign rootfs.
  • docker.enabled and bwrap.enabled are mutually exclusive per branch; enabling both is rejected, not silently picked.

Scenario#000.01: A bwrap build runs the command inside the sandbox as root

So that the build is isolated from the host exactly as the native and Docker runtimes intend.

  • Given a branch with bwrap.enabled and a bwrap.rootfs archive
  • When a build for that branch starts
  • Then Werkator unpacks the rootfs on demand into .git/werkator/buildenv/<envKey>/rootfs
    • and invokes bwrap with a uid-0 mapping, a read-only root bind of the rootfs, the workspace bound at its host path, and --chdir into it
    • and the returned process is the attached bwrap process, so streaming and cancellation behave like native builds
  • and the environment plus bwrap.env are passed via --setenv
Verified by

Scenario#000.02: Git metadata mounts keep secrets out of the sandbox

So that builds can run read-only git commands but never reach the machine config or the control token.

  • Given a workspace that is a worktree of the repository
  • When the sandbox is assembled
  • Then the primary .git is bound read-only
    • and .git/werkator/ is masked by an empty tmpfs
    • and the worktree admin directory is bound read-write
Verified by

Scenario#000.03: A branch cannot turn its sandbox off or swap its rootfs

So that the pinned sandbox policy holds for builds a branch invents as well as for ones the host already knows.

  • Given a branch whose committed config sets bwrap.enabled or bwrap.rootfs
  • When that config is resolved into a build
  • Then the pinned keys are stripped from the worktree layer
    • and enabling both docker and bwrap on a build is rejected, not picked silently
Verified by

Scenario#000.04: The dispatcher routes builds to the bwrap runtime

So that a bwrap-explicit branch builds inside the sandbox rather than natively.

  • Given a branch with bwrap.enabled
  • When its build is dispatched
  • Then BwrapBuildRunner is selected
Verified by

Feature: Werkator builds itself without Docker

Background

  • Werkator's own build runs the full test suite, which includes TestcontainersSmokeTest.
  • On a Docker-less host (the webspace, and the bwrap sandbox that builds there), that test must not fail the self-build.

Scenario#000.05: The Testcontainers smoke test is skipped, not failed, without Docker

So that a Docker-less build of Werkator itself stays green.

  • Given no reachable Docker daemon
  • When the test suite runs
  • Then TestcontainersSmokeTest is reported as skipped
    • and the build is not failed by it
  • and with a Docker daemon present the test still runs and verifies a container
Verified by

The Solution

A third BuildRunner by the same shell-out pattern as git and Docker. BwrapBuildRunner shells out to the bwrap CLI (no library), unpacks a prepared Debian rootfs on demand into .git/werkator/buildenv/<envKey>/rootfs, reuses the step-16 git-metadata mounts verbatim, binds a persistent .git/werkator/buildenv/home as /root, and returns the attached bwrap process so streaming and cancellation match the other runtimes. DispatchingBuildRunner routes by bwrap.enabled.

Pinning and mutual exclusion hold at the choke point. bwrap.enabled and bwrap.rootfs join the pinned sandbox-policy set, stripped from the worktree layer so a branch cannot disable its sandbox or substitute a foreign rootfs. docker and bwrap are mutually exclusive per build, rejected in buildSettings — the single point every build passes through — instead of picked silently.

Tooling makes the two manual steps reproducible. tools/build-bwrap-rootfs.sh builds the rootfs archive (debootstrap-minbase Debian + JDK 21 + git + locales) on any machine with Docker, since debootstrap is not available on the target. tools/werkator-build-prerequisites.sh re-runs the exact precondition command line from the plan on the target webspace and checks all three signals. TestcontainersSmokeTest is gated with enabledIf docker available, so a Docker-less self-build skips it.

Open Questions

  • Whether the rootfs archive built by tools/build-bwrap-rootfs.sh is complete for ./gradlew build has not been exercised on a real webspace; the JDK 21, git and locales package set is a good baseline but project-specific tooling must be added.
  • The systemd user unit for a webspace should gain MemoryMax/TasksMax (configurable per the plan); not implemented here.

Additional Changes

  • Config template (InitCommand), docs/configuration.md and AGENTS.md updated so all three config places stay in sync and the pinned set is documented.
  • The plan file 17 records the precondition result and the new tooling.

Follow-up PRs

  • ADR 0007 recording the bubblewrap runtime decision (options: bwrap vs proot/fakechroot vs plain native).
  • Web-access deployment on a real webspace and the docs/deployment.md third variant, written once verified there.
  • MemoryMax/TasksMax on the webspace systemd unit.
Reference in New Issue View Git Blame Copy Permalink