Files
werkator/docs/plan
Michael Hönnig 71f1fc62c6 Bwrap build runtime und Installation in Hostsharing Managed Webspace (#4)
* Add the bubblewrap build runtime (step 17, ADR 0007)

BwrapBuildRunner: third runtime behind BuildRunner for hosts without root
and without Docker (e.g. Hostsharing managed webspaces). Shells out to the
bwrap CLI, unpacks a prepared rootfs on demand into
.git/werkator/buildenv/<envKey>/rootfs, reuses the Docker runner's git
metadata mounts, and returns the attached bwrap process for streaming and
cancellation.

Config: bwrap.enabled/rootfs/env on BranchConfig and BwrapOverrides on
BuildDefinition; enabled/rootfs are pinned like the docker sandbox policy.
Docker and bwrap are mutually exclusive per build, rejected in
buildSettings instead of picked silently. DispatchingBuildRunner routes
bwrap; InitCommand template, docs/configuration.md and AGENTS.md in sync.

* bwrap rollout tooling: remote script, prerequisites disk/quota check, absolute workspace binds

- tools/remote: central remote control script with check-prerequisites, install and build commands
- tools/werkator-build-prerequisites.sh: compact PASS/FAIL output, target-dir parameter, free-space and
  group-quota headroom checks against the ~5 GiB build footprint, home-filesystem reference
- BwrapBuildRunner: bind workspace and home at absolute paths resolved against repoDir — a relative
  path made bwrap create mountpoints inside the read-only rootfs (seen on the webspace); regression test
- TestcontainersSmokeTest: gated with enabledIf docker available (skip, never fail, without a daemon)
- docs: configuration reference, step-17 plan notes, PR-doc

* bwrap: bind the repo read-write before the workspace so mountpoints are creatable

bwrap creates mountpoints for bind destinations inside the sandbox; with only a
read-only rootfs bound at /, creating them for the workspace under .git/werkator/
worktrees failed with 'Read-only file system' (seen on the webspace). Binding the
repo dir read-write first provides the base; the git metadata mounts then layer
the usual isolation on top (read-only .git, tmpfs mask over .git/werkator,
read-write worktree admin dir).

* bwrap: pre-create bind mountpoints inside the unpacked rootfs

bwrap mkdirs mountpoints for bind destinations against the sandbox view; with the
rootfs ro-bound at / every destination missing from the rootfs (the repo dir under
/home/storage/... on the webspace) fails with 'Read-only file system'. The rootfs
directory is a plain host dir, so create the mountpoints there before launching
bwrap; it then finds them and has nothing left to create.

* bwrap: skip existing rootfs files when pre-creating bind mountpoints

/etc/resolv.conf is a file the rootfs already ships; createDirectories threw on it.
Only missing directories are created now.

* bwrap: pre-create proc/dev/tmpfs mountpoints in the rootfs too

The rootfs archive ships no /proc or /dev (excluded when packed), so bwrap failed
mkdir'ing their mountpoints against the read-only root.

* bwrap: bind the workspace after the git metadata mounts

The tmpfs mask over .git/werkator shadowed the earlier workspace bind, because the
worktree lives under .git/werkator/worktrees — chdir then failed with ENOENT. The
workspace bind now comes last and shadows the mask at exactly its own path.

* systemd resource limits and webspace start command (step 17, web access)

- server.systemd.memoryMax/tasksMax (empty = directive omitted): on platforms where
  the service runs in a shared memory slice (Hostsharing Managed Webspaces) a
  runaway Gradle build must not starve the whole package; init --systemd reads the
  effective config and bakes the values into the generated unit
- tools/remote werkator start: writes server settings (assigned port, loopback
  bind, publicBaseUrl, nginx off) plus the Apache reverse-proxy .htaccess into
  ~/doms/<domain>/subs/www, runs init --systemd and enables the user unit
- docs/configuration.md documents the new keys

* tools/remote: env-based configuration and background port-forward

All connection and deployment values come from .env in the repository root
(WERKATOR_REMOTE, WERKATOR_PATH, WERKATOR_PORT, WERKATOR_DOMAIN,
WERKATOR_LOCAL_PORT, optional WERKATOR_BRANCH/MEMORY_MAX/TASKS_MAX/ROOTFS);
missing values fail with a pointing error instead of positional parameters.

- port-forward is now 'tools/remote port-forward start|stop' with a detached
  ssh tunnel, pid file under /tmp, and idempotent start
- start restarts the systemd unit after updating the machine config
- control-token generates the token in place when the server has not yet
- the rootfs archive default moves to build/ (already gitignored)
2026-08-31 19:56:04 +02:00
..

Werkator Rewrite Plan

This directory contains the step-by-step plan for rewriting the legacy bash script as the Kotlin/Spring application in this repository. Each step file is self-contained and sized for one focused Claude Code session.

How to Execute a Step

Start a fresh Claude Code session and prompt, for example: "Execute docs/plan/01-build-state-domain.md". The executing session should:

  1. Read this file, 00-legacy-analysis.md, and the step file.
  2. Read the referenced parts of the legacy bash script only if the step file says so. It was removed from the tree with the rename to Werkator; its last state is git show 7f55068^:legacy/gitTally, and 00-legacy-analysis.md condenses it.
  3. Implement with tests, following CLAUDE.md conventions.
  4. Run ./gradlew ktlintFormat and then ./gradlew build until green.
  5. Update the step's checkbox below and note deviations inside the step file.

Guiding Principles

  • The legacy script defines intended behavior, but it is buggy — treat it as a reference, not a spec.
  • Fix the two known legacy defects by design, not by patching:
    • Build status must be observable while a build runs (event-driven status transitions, async build execution).
    • The web UI must never get stuck loading (JSON status endpoints with explicit error states instead of regex-rewritten HTML).
  • Do not port orphaned or half-implemented legacy config options (see 00-legacy-analysis.md).
  • Every step leaves the build green and the application runnable.
  • Config keys added by a step must be updated in three places: WerkatorConfig, the InitCommand templates, and docs/configuration.md.

Proposed Architecture Decisions

These are proposals baked into the steps. Revisit them in an ADR if a step uncovers problems.

  • Build results are persisted as a JSON file under .git/werkator/, behind a BuildResultRepository interface (no database, but replaceable).
  • Builds run concurrently up to builds.maxConcurrent (default 1), but never more than one build per branch at a time. Each branch builds in its own reusable git worktree under .git/werkator/worktrees/, checked out detached at the requested commit — never in the primary checkout. A later step must decide (possibly per config) whether a new commit on a branch cancels that branch's running build or waits for it; for now new builds queue behind the running one.
  • Artifacts stay on the filesystem, served by the Spring server.
  • The web UI is server-rendered HTML plus small JavaScript polling JSON endpoints (no SPA framework).
  • The watcher is a Spring-managed scheduled component, decoupled from the build executor via the result repository and events.
  • nginx/Let's Encrypt container management is NOT ported; deployment behind an existing reverse proxy is documented instead. Revised after step 12 (ADR 0005): an opt-in managed nginx/TLS container is required for Hostsharing container hosts — see step 13.

Steps

Foundation:

  • 01-build-state-domain.md — build result domain model and persistent repository
  • 02-git-gateway.md — full git access layer (fetch, branches, commits, checkout)
  • 03-gitea-client.md — Gitea API client for commit statuses

Core engine:

  • 04-build-executor.md — async build execution with logs, cancellation, status transitions
  • 05-artifact-store.md — artifact persistence, naming, retention
  • 06-watcher.md — branch watching, scheduling, auto-builds

Server and UI:

  • 07-server-mode.mdserver subcommand, REST/JSON endpoints, artifact serving
  • 08-web-ui.md — HTML views with robust live updates
  • 09-system-metrics.md — system resource monitoring page

Completion:

  • 10-cli-commands.md — CLI build/status commands
  • 11-docker-build-runtime.md — optional Docker build execution
  • 12-deployment.md — systemd service, migration from legacy, docs

Added after the initial plan (ADR 0005):

  • 13-nginx-tls.md — opt-in managed nginx/TLS container for hosts without a reverse proxy

Added after the 2026-08-10 overhead measurements on vm2176:

  • 14-build-phase-timing-and-overhead.md — per-build phase timing, overhead budget warning, ownership/metrics fixes — deferred until after step 15; revisit relevance on vm4006

Added for the vm2176 → vm4006 migration (2026-08-10):

  • 15-runtime-bundle-distribution.md — self-contained runtime bundle (jlink JRE + jar) for hosts without a Java runtime
  • 16-git-in-docker-builds.md — read-only git metadata inside Docker build containers, with .git/werkator/ masked

Added after v0.9.19 replaced the per-branch settings with build definitions (2026-08-29):

  • 18-remove-branches-section.md — delete the legacy branches section and its autoBuild schedule; run around 2026-09-05, after the precondition check in the step file

Added after a silent 57-minute fetch outage on vm4006 (2026-08-30):

  • 19-watcher-health-in-ui.md — show an unreachable origin in the web UI instead of only in the journal

Added for running Werkator on Hostsharing Managed Webspaces (2026-08-10):

  • 17-bwrap-build-runtime.md — Werkator on a Managed Webspace: bubblewrap user-namespace build sandbox with a prepared rootfs (precondition check first — see the step file), plus web access under a domain via the platform's Apache proxy and Let's Encrypt

Added for surfacing build time as a trend (2026-08-31):

  • 20-build-duration-tracking.md — a per-name duration trend over the existing history, derived on read in the History view: series, window average/min/max, and a visible marker when the latest build is slower than its window average (grouped by the history's own name, so branch builds and named jobs stay separate — complements Step 14, which owns phase timing)

Steps 0103 are independent of each other. Steps 0406 depend on 0103. Steps 0709 depend on 0406. Steps 11 and 12 are optional/deferrable; 10 only needs 0406. Step 13 depends on 07, 11, and 12. Step 15 depends on 12 and 13 and revises the containerized-runtime sketch in docs/bootstrapping.md (ADR 0006 is written as part of the step; GraalVM native image was evaluated and rejected there). Step 17 depends on 11, 15, and 16, and starts with a hard precondition check on the target webspace (ADR 0007 is written as part of the step). Step 18 depends on nothing in code but on the watched repository having migrated — its precondition check is a hard gate, not a formality. Step 19 depends on nothing; WatcherState and /api/watcher already carry everything it needs to render. Step 20 depends on nothing; the duration is already recorded, and the trend is derived read-only from repository.history().