Bwrap build runtime und Installation in Hostsharing Managed Webspace (#4)
* Add the bubblewrap build runtime (step 17, ADR 0007) BwrapBuildRunner: third runtime behind BuildRunner for hosts without root and without Docker (e.g. Hostsharing managed webspaces). Shells out to the bwrap CLI, unpacks a prepared rootfs on demand into .git/werkator/buildenv/<envKey>/rootfs, reuses the Docker runner's git metadata mounts, and returns the attached bwrap process for streaming and cancellation. Config: bwrap.enabled/rootfs/env on BranchConfig and BwrapOverrides on BuildDefinition; enabled/rootfs are pinned like the docker sandbox policy. Docker and bwrap are mutually exclusive per build, rejected in buildSettings instead of picked silently. DispatchingBuildRunner routes bwrap; InitCommand template, docs/configuration.md and AGENTS.md in sync. * bwrap rollout tooling: remote script, prerequisites disk/quota check, absolute workspace binds - tools/remote: central remote control script with check-prerequisites, install and build commands - tools/werkator-build-prerequisites.sh: compact PASS/FAIL output, target-dir parameter, free-space and group-quota headroom checks against the ~5 GiB build footprint, home-filesystem reference - BwrapBuildRunner: bind workspace and home at absolute paths resolved against repoDir — a relative path made bwrap create mountpoints inside the read-only rootfs (seen on the webspace); regression test - TestcontainersSmokeTest: gated with enabledIf docker available (skip, never fail, without a daemon) - docs: configuration reference, step-17 plan notes, PR-doc * bwrap: bind the repo read-write before the workspace so mountpoints are creatable bwrap creates mountpoints for bind destinations inside the sandbox; with only a read-only rootfs bound at /, creating them for the workspace under .git/werkator/ worktrees failed with 'Read-only file system' (seen on the webspace). Binding the repo dir read-write first provides the base; the git metadata mounts then layer the usual isolation on top (read-only .git, tmpfs mask over .git/werkator, read-write worktree admin dir). * bwrap: pre-create bind mountpoints inside the unpacked rootfs bwrap mkdirs mountpoints for bind destinations against the sandbox view; with the rootfs ro-bound at / every destination missing from the rootfs (the repo dir under /home/storage/... on the webspace) fails with 'Read-only file system'. The rootfs directory is a plain host dir, so create the mountpoints there before launching bwrap; it then finds them and has nothing left to create. * bwrap: skip existing rootfs files when pre-creating bind mountpoints /etc/resolv.conf is a file the rootfs already ships; createDirectories threw on it. Only missing directories are created now. * bwrap: pre-create proc/dev/tmpfs mountpoints in the rootfs too The rootfs archive ships no /proc or /dev (excluded when packed), so bwrap failed mkdir'ing their mountpoints against the read-only root. * bwrap: bind the workspace after the git metadata mounts The tmpfs mask over .git/werkator shadowed the earlier workspace bind, because the worktree lives under .git/werkator/worktrees — chdir then failed with ENOENT. The workspace bind now comes last and shadows the mask at exactly its own path. * systemd resource limits and webspace start command (step 17, web access) - server.systemd.memoryMax/tasksMax (empty = directive omitted): on platforms where the service runs in a shared memory slice (Hostsharing Managed Webspaces) a runaway Gradle build must not starve the whole package; init --systemd reads the effective config and bakes the values into the generated unit - tools/remote werkator start: writes server settings (assigned port, loopback bind, publicBaseUrl, nginx off) plus the Apache reverse-proxy .htaccess into ~/doms/<domain>/subs/www, runs init --systemd and enables the user unit - docs/configuration.md documents the new keys * tools/remote: env-based configuration and background port-forward All connection and deployment values come from .env in the repository root (WERKATOR_REMOTE, WERKATOR_PATH, WERKATOR_PORT, WERKATOR_DOMAIN, WERKATOR_LOCAL_PORT, optional WERKATOR_BRANCH/MEMORY_MAX/TASKS_MAX/ROOTFS); missing values fail with a pointing error instead of positional parameters. - port-forward is now 'tools/remote port-forward start|stop' with a detached ssh tunnel, pid file under /tmp, and idempotent start - start restarts the systemd unit after updating the machine config - control-token generates the token in place when the server has not yet - the rootfs archive default moves to build/ (already gitignored)
This commit is contained in:
Michael Hönnig
Executable
+116
@@ -0,0 +1,116 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Build the Werkator bwrap build environment (rootfs) archive.
|
||||
#
|
||||
# The bwrap build runtime (step 17 / ADR 0007) runs each build inside a
|
||||
# bubblewrap user namespace, chrooted into a *prepared* Debian root filesystem.
|
||||
# That rootfs is NOT built on the target (the webspace has no root and no
|
||||
# debootstrap), it is built once on any machine that can — most comfortably a
|
||||
# machine with Docker — and distributed as an archive, e.g.
|
||||
# `werkator-buildenv-trixie-java21.tar.zst`.
|
||||
#
|
||||
# This script builds exactly that archive: a debootstrap-minbase Debian release
|
||||
# plus the packages Werkator itself needs to run `./gradlew build` inside the
|
||||
# sandbox (JDK 21, git, ca-certificates, locales, curl/unzip for the wrapper).
|
||||
# The whole build runs inside a throwaway Docker container, so no root is
|
||||
# needed on the machine running this script.
|
||||
#
|
||||
# Why it works this way (each quirk learned the hard way):
|
||||
# - The whole job runs in ONE container whose stdin carries a base64-encoded
|
||||
# script (no file is bind-mounted for the script — a bind-mounted script
|
||||
# hit noexec/tmpfs trouble and vanished inside the container).
|
||||
# - The rootfs is built inside the container's own writable layer, NOT on a
|
||||
# bind-mounted host directory — debootstrap "Tried to extract package, but
|
||||
# tar failed" when its target sat on some bind-mounted/special filesystems.
|
||||
# - Only the final `tar --zstd` writes to stdout; every build step is
|
||||
# redirected to stderr, so the archive coming out of `docker run` is pure.
|
||||
#
|
||||
# Usage: build-bwrap-rootfs.sh [--release trixie] [--mirror URL] [--out path]
|
||||
# --release Debian release/architecture tail, default "trixie"
|
||||
# --mirror apt mirror for debootstrap, default http://deb.debian.org/debian
|
||||
# --out output archive path, default ./werkator-buildenv-<release>.tar.zst
|
||||
#
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||
warn() { echo "WARNING: $*" >&2; }
|
||||
|
||||
usage() {
|
||||
echo "usage: build-bwrap-rootfs.sh [--release trixie] [--mirror URL] [--out path]" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --------------------------------------------------------------- arguments --
|
||||
|
||||
release="trixie"
|
||||
out=""
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--release) release="${2:?missing value for --release}"; shift 2 ;;
|
||||
--mirror) mirror="${2:?missing value for --mirror}"; shift 2 ;;
|
||||
--out) out="${2:?missing value for --out}"; shift 2 ;;
|
||||
-*) die "unknown option: $1" ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
mirror="${mirror:-http://deb.debian.org/debian}"
|
||||
[ -n "$out" ] || out="$(pwd)/werkator-buildenv-${release}.tar.zst"
|
||||
|
||||
command -v docker >/dev/null 2>&1 || die "docker is required to build the rootfs"
|
||||
|
||||
# Rootfs content: Werkator's own build needs a JDK 21 toolchain (Gradle
|
||||
# toolchain resolution), git, ca-certificates for HTTPS, locales for git, and
|
||||
# curl/unzip/xz-utils/zstd for the Gradle wrapper and general build hygiene.
|
||||
# Keep this list additive — project-specific tooling goes on top of this base.
|
||||
PKGS="openjdk-21-jdk git ca-certificates locales procps file curl unzip xz-utils zstd"
|
||||
|
||||
# The chroot step runs inside the freshly debootstrapped rootfs; passed into
|
||||
# the container as base64 so no nested heredoc corrupts the piped script.
|
||||
inner="$(printf '%s' '#!/bin/bash
|
||||
set -euxo pipefail
|
||||
mount -t proc none /proc
|
||||
apt-get update -qq
|
||||
apt-get install -y --no-install-recommends '"${PKGS}"'
|
||||
apt-get clean
|
||||
rm -f /etc/localtime
|
||||
locale-gen en_US.UTF-8 de_DE.UTF-8 >/dev/null 2>&1 || true
|
||||
update-locale LANG=en_US.UTF-8 >/dev/null 2>&1 || true
|
||||
' | base64 -w0)"
|
||||
|
||||
# The outer script runs inside the Debian container as root. Build noise goes
|
||||
# to stderr (fd 1 is saved on fd 3 and restored only for the final tar), so
|
||||
# docker stdout is exactly the archive.
|
||||
outer="$(printf '%s' '#!/bin/bash
|
||||
set -euo pipefail
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
exec 3>&1
|
||||
exec 1>&2
|
||||
apt-get update -qq
|
||||
apt-get install -y --no-install-recommends debootstrap zstd ca-certificates
|
||||
mkdir -p /b/rootfs
|
||||
debootstrap --variant=minbase --components=main,contrib --include=apt,ca-certificates '"${release}"' /b/rootfs '"${mirror}"'
|
||||
mount --bind /proc /b/rootfs/proc
|
||||
mount --bind /sys /b/rootfs/sys
|
||||
mount --bind /dev /b/rootfs/dev
|
||||
echo '"${inner}"' | base64 -d > /b/rootfs/inner.sh
|
||||
chmod +x /b/rootfs/inner.sh
|
||||
chroot /b/rootfs /bin/bash /inner.sh
|
||||
umount /b/rootfs/proc; umount /b/rootfs/sys; umount /b/rootfs/dev
|
||||
exec 1>&3
|
||||
tar --zstd --exclude=proc --exclude=sys --exclude=dev -C /b/rootfs -cf - .
|
||||
' | base64 -w0)"
|
||||
|
||||
echo "building ${release} rootfs (downloads packages, takes a while; log below)..."
|
||||
echo "archive → $out"
|
||||
|
||||
# Stream the base64-encoded outer script into the container over stdin; the
|
||||
# archive lands on stdout (redirected to $out), the build log on stderr.
|
||||
docker run --rm -i --privileged debian:"${release}-slim" \
|
||||
bash -c 'base64 -d | bash' \
|
||||
<<<"$outer" >"$out"
|
||||
|
||||
echo
|
||||
echo "OK: build environment written to $out"
|
||||
echo " Configure it as branches.<name>.bwrap.rootfs (a bare path or a URL)"
|
||||
echo " on the target Werkator instance to build in this environment."
|
||||
Reference in New Issue
Block a user