Merge pull request #9 from mhoennig/init-apply-config
Step 23 sessions A-C: init --apply, applied instance fragment, tools/remote wraps
This commit is contained in:
mhoennig
@@ -19,12 +19,13 @@ WerkatorApplication ← @SpringBootApplication
|
||||
CliRunner ← CommandLineRunner + ExitCodeGenerator
|
||||
WerkatorCommand ← root @Command, delegates to subcommands
|
||||
commands/
|
||||
InitCommand ← "init [--systemd]"
|
||||
InitCommand ← "init [--systemd] [--apply FILE]"
|
||||
ServerCommand ← "server"
|
||||
StatusCommand ← "status [--history]"
|
||||
BuildCommand ← "build [<branch>]"
|
||||
RetryCommand ← "retry"
|
||||
ConfigPrintCommand ← "config:print [--full]"
|
||||
ControlTokenCommand ← "control-token"
|
||||
```
|
||||
|
||||
`status`, `build`, and `retry` implement `Callable<Int>` for their exit codes (0 success, 1 build failure, 2 usage/config errors).
|
||||
@@ -40,14 +41,15 @@ Two independent staleness signals, never merged: the `live-indicator` badge says
|
||||
|
||||
## Configuration System
|
||||
|
||||
Werkator is configured by two YAML files, deep-merged by `ConfigLoader` (later wins):
|
||||
Werkator is configured by three YAML files, deep-merged by `ConfigLoader` (later wins):
|
||||
|
||||
1. `.werkator.yml` at the repo root — committed, shared team settings.
|
||||
2. `.git/werkator/.werkator.yml` — not committed; machine-specific overrides and secrets (`git.account`, `git.token`).
|
||||
2. `.git/werkator/.werkator.applied.yml` — not committed; the instance fragment installed verbatim by `init --apply` (step 23), validated strictly (unknown keys refused) and replaced wholesale on re-apply.
|
||||
3. `.git/werkator/.werkator.yml` — not committed; machine-specific overrides and secrets (`git.account`, `git.token`); hand-edited, always wins.
|
||||
|
||||
Every lookup falls back to the pre-rename name (`ConfigFiles`): `.gittally.yml`, and `.git/gittally/.gittally.yml` for the machine layer. Current name first, and where both exist the old one is ignored rather than merged — a missing config is not an error, so an un-renamed installation would otherwise start on defaults without a single failure.
|
||||
|
||||
On top of those comes the **branch layer**: the `.werkator.yml` committed on a branch, applied by `loadWithBranchLayer` (the watcher passes the content read via `git show`, `loadForWorktree` the file in the build worktree). A branch describes its own CI and wins over both layers — the whole `builds` section — so a configuration can be tried out on a branch without touching other branches' builds. `stripPinned` removes what is not a description of this branch's build: `git`, `server`, `gitea`, `executor`, `watcher`, and — inside every `builds` definition as well as every legacy `branches` entry — `requirePullRequest`, `statusContext`, and `docker.enabled`/`docker.network`.
|
||||
On top of those comes the **branch layer**: the `.werkator.yml` committed on a branch, applied by `loadWithBranchLayer` (the watcher passes the content read via `git show`, `loadForWorktree` the file in the build worktree). A branch describes its own CI and wins over both layers — the whole `builds` section — so a configuration can be tried out on a branch without touching other branches' builds. `stripPinned` removes what is not a description of this branch's build: `git`, `server`, `gitea`, `executor`, `watcher`, and — inside every `builds` definition as well as every legacy `branches` entry — `requirePullRequest`, `statusContext`, `docker.enabled`/`docker.network`, and `bwrap.enabled`/`bwrap.rootfs`/`bwrap.werkdock`.
|
||||
|
||||
Each file is version-checked before merging (`werkator.version.since`/`below`, `ConfigVersions.verdict`), so the message can name the file to fix: `since` is hard in both directions — too old a Werkator, or a file written before `ConfigVersions.FORMAT_BROKE_IN` and read after it — while `below` only warns. There is no format version (`apiVersion`) on purpose: only one configuration generation is supported, and the declared version exists to make the incompatibility nameable.
|
||||
|
||||
|
||||
@@ -35,4 +35,5 @@ replay_pid*
|
||||
# Other
|
||||
/.local/
|
||||
/.env
|
||||
/.env.*
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ Good:
|
||||
|
||||
Bad:
|
||||
|
||||
- The rootfs archive is project infrastructure that must be built and uploaded (~1.5 GB unpacked, disk/quota checked by `tools/werkator-build-prerequisites.sh`).
|
||||
- The rootfs archive is project infrastructure that must be built and uploaded (~1.5 GB unpacked, disk/quota checked by `werkdock doctor`, which ported the original prerequisites script).
|
||||
- Filesystem-only isolation: network and process view are the host's — acceptable here, and pinned so a branch cannot widen it, but weaker than Docker.
|
||||
- Mountpoint pre-creation and mount ordering are subtle (hardened on the real webspace; see the fix messages preserved in commit `71f1fc6`).
|
||||
|
||||
|
||||
@@ -86,6 +86,12 @@ gitea:
|
||||
|
||||
Then, you have to configure *Werkator* by amending this config file according to [configuration.md](configuration.md).
|
||||
|
||||
### 5. Optionally Install an Instance Fragment (`--apply`)
|
||||
|
||||
`init --apply FILE` installs a YAML fragment in the configuration schema as the applied instance layer — see [configuration.md](configuration.md#the-applied-instance-fragment-init---apply).
|
||||
Deployment tooling hands its parameters over this way instead of patching config files; the fragment is validated strictly and replaced wholesale on re-apply.
|
||||
It runs before `--systemd`, so an applied `server.port` reaches the generated unit and the Apache `.htaccess` (written beside the units when a `publicBaseUrl` is configured).
|
||||
|
||||
## Output
|
||||
|
||||
`init` prints one line per action taken:
|
||||
|
||||
@@ -7,10 +7,17 @@ Werkator is configured via YAML files. Settings are merged from several sources
|
||||
| Layer | Path | Committed to Git | Purpose |
|
||||
|--------------------------|----------------------------|------------------|----------------------------------------------|
|
||||
| Project config | `.werkator.yml` | Yes | Shared team settings |
|
||||
| Applied instance fragment | `.git/werkator/.werkator.applied.yml` | No | Instance parameters installed by `init --apply` |
|
||||
| Repo installation config | `.git/werkator/.werkator.yml` | No | Machine- or user-specific overrides, secrets |
|
||||
| Branch config | `.werkator.yml` committed on a branch | Yes | That branch's build settings and build definitions |
|
||||
|
||||
The repo install config (`.git/werkator/.werkator.yml`) wins on any key present in both files. Typically used to set `git.token` and `git.account` without committing them.
|
||||
The repo install config (`.git/werkator/.werkator.yml`) wins on any key present in several files; the applied fragment wins over the project config. Typically the repo install config sets `git.token` and `git.account` without committing them.
|
||||
|
||||
### The applied instance fragment (`init --apply`)
|
||||
|
||||
`werkator init --apply FILE` installs a YAML fragment in this very schema as its own layer (step 23) — the file a deployment wrapper hands over instead of patching configs.
|
||||
The fragment is validated strictly before installing: an unknown key is refused loudly, never ignored, because a typo would otherwise install a value that silently does nothing.
|
||||
It is then copied verbatim (comments included) to `.git/werkator/.werkator.applied.yml`; re-applying replaces the file, so nothing accumulates or duplicates, and the hand-edited repo install config — which always wins — is never rewritten.
|
||||
|
||||
### Which Werkator a file is written for
|
||||
|
||||
@@ -414,7 +421,7 @@ All Werkator containers carry `org.hoennig.werkator` labels; stale build contain
|
||||
With `bwrap.enabled`, Werkator runs the build in a bubblewrap sandbox instead of native execution.
|
||||
This is the third runtime, for hosts without root and without a Docker daemon (e.g. Hostsharing managed webspaces); see `docs/plan/17-bwrap-build-runtime.md` and ADR 0008.
|
||||
Since step 21 session C the sandbox is executed by the `werkdock` CLI (`bwrap.werkdock`, default: resolved via `PATH`) — Werkator no longer invokes `bwrap` itself; `bwrap` must be installed for werkdock.
|
||||
`werkdock doctor` checks the host's capability, superseding `tools/werkator-build-prerequisites.sh`.
|
||||
`werkdock doctor` checks the host's capability (it replaced the retired `tools/werkator-build-prerequisites.sh` in step 23).
|
||||
|
||||
`bwrap.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
|
||||
It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`.
|
||||
|
||||
+9
-7
@@ -268,14 +268,16 @@ The third deployment variant (plan step 21, verified live on a real webspace): n
|
||||
Werkator runs as a systemd *user* service on the assigned localhost port ("eigener Serverdienst"), the platform's managed Apache terminates TLS and proxies via `.htaccess`, and builds run in the bubblewrap sandbox executed by the [werkdock](../werkdock/README.md) CLI (ADR 0008, step 21 session C).
|
||||
|
||||
Werkator is never built on the webspace: the runtime bundle and the werkdock binary are built locally and uploaded (ADR 0006).
|
||||
All steps are driven by `tools/remote`, configured through the `.env` file in the repository root; commands name their role — `instance-*` manages the installed Werkator, `repo-*` the repository it watches.
|
||||
All steps are driven by `tools/remote`; commands name their role — `instance-*` manages the installed Werkator, `repo-*` the repository it watches.
|
||||
Each instance is a pair of files (step 23): a transport env file selected with `--env-file` (default `.env`), and a YAML fragment in the configuration schema, named by its `WERKATOR_INIT_CONFIG` key and installed remotely via `werkator init --apply` — e.g. `.env.mih34` + `.env.mih34.yml`, both gitignored.
|
||||
The fragment carries the Werkator configuration (`server.port`, `publicBaseUrl`, systemd limits, `builds.default.bwrap.*`); the env file only says where and how to reach the host.
|
||||
|
||||
```bash
|
||||
tools/remote werkator check-prerequisites # bwrap capability, disk and quota headroom
|
||||
tools/remote werkator instance-install # upload + unpack the runtime bundle and werkdock
|
||||
tools/remote werkator repo-init # clone the watched repo, init, rootfs archive, bwrap config
|
||||
tools/remote werkator instance-start # server config, Apache proxy, systemd user unit
|
||||
tools/remote port-forward start # browser tunnel while no public domain is set up
|
||||
tools/remote --env-file .env.mih34 werkator check-prerequisites # uploads werkdock, runs its doctor
|
||||
tools/remote --env-file .env.mih34 werkator instance-install # upload + unpack the runtime bundle and werkdock
|
||||
tools/remote --env-file .env.mih34 werkator repo-init # clone the watched repo, rootfs archive, init --apply
|
||||
tools/remote --env-file .env.mih34 werkator instance-start # init --apply --systemd, place .htaccess, enable the unit
|
||||
tools/remote --env-file .env.mih34 port-forward start # browser tunnel while no public domain is set up
|
||||
```
|
||||
|
||||
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
|
||||
@@ -285,7 +287,7 @@ Fill `git.account`/`git.token` in the machine config when the origin is private,
|
||||
Updates are one command, refused while a build runs (`FORCE=1` overrides):
|
||||
|
||||
```bash
|
||||
tools/remote werkator instance-update
|
||||
tools/remote --env-file .env.mih34 werkator instance-update
|
||||
```
|
||||
|
||||
The previous runtime stays as `.werkator/werkator.prev` for one deployment as the rollback asset.
|
||||
|
||||
@@ -15,7 +15,7 @@ Werkator becomes the executing app wherever possible; `tools/remote` shrinks to
|
||||
|
||||
Parameters travel as **files**, not as many CLI options — and each side gets the format that is native to it (refined 2026-09-01):
|
||||
|
||||
- The **wrapper** keeps a small, bash-sourceable env file with the transport values only: `tools/remote --env .env.mih34 werkator repo-init` selects the target (default: `.env`), so several instances (`.env.mih34`, `.env.vm4006`, later a Werkbaum instance) are files, not edits.
|
||||
- The **wrapper** keeps a small, bash-sourceable env file with the transport values only: `tools/remote --env-file .env.mih34 werkator repo-init` selects the target (default: `.env`; named `--env-file` like docker's flag for the same thing, since `--env` means a single variable there), so several instances (`.env.mih34`, `.env.vm4006`, later a Werkbaum instance) are files, not edits.
|
||||
- **Werkator** takes a **YAML fragment in its own config schema**: `werkator init --apply mih34.yml` deep-merges the fragment into the machine config, idempotently — creating sections that are missing, updating the given values, never duplicating.
|
||||
No mapping table exists: the fragment says `server: {port: …}` and `builds: {default: {bwrap: …}}` directly, is validated by the existing schema binding, and is documented by the existing `docs/configuration.md`.
|
||||
- The wrapper uploads the fragment alongside the artifacts and calls `werkator init --apply …` remotely — the heredocs and `sed` calls in `tools/remote` disappear.
|
||||
@@ -31,25 +31,28 @@ The removed legacy env-to-YAML conversion stays removed — there is no conversi
|
||||
|
||||
## The Sessions
|
||||
|
||||
### A — Werkator side
|
||||
### A — Werkator side (implemented 2026-09-01 on branch `init-apply-config`)
|
||||
|
||||
- `init --apply FILE`: deep-merge the YAML fragment into the machine config — reusing the loader's merge, creating missing sections, updating given values, never duplicating (the duplication class dies here); a fragment that fails the schema binding or carries unknown keys is refused loudly.
|
||||
- `init --systemd` keeps generating the units; decide in the session whether the Apache `.htaccess` becomes part of the host-integration output when the applied config carries `server.port` and a public domain (proposal: yes, under `init --systemd`, since it is generated host integration exactly like the units).
|
||||
- New subcommand `werkator control-token`: print the token, creating it exactly like `ControlTokenService` does — the bash duplication in the wrapper dies.
|
||||
- Tests per the writing-tests conventions; `docs/bootstrapping.md` documents `--apply` (the fragment keys need no new reference — they are ordinary `docs/configuration.md` keys).
|
||||
|
||||
### B — Wrapper side
|
||||
### B — Wrapper side (implemented 2026-09-01 on branch `init-apply-config`)
|
||||
|
||||
- `tools/remote --env FILE` (default `.env`); the init fragment named by `WERKATOR_INIT_CONFIG` is uploaded, and the remote init runs with `--apply`.
|
||||
- `tools/remote --env-file FILE` (default `.env`); the init fragment named by `WERKATOR_INIT_CONFIG` is uploaded, and the remote init runs with `--apply`.
|
||||
- `repo-init` and `instance-start` lose their heredoc/`sed` config writing; `control-token` delegates to the new subcommand.
|
||||
- `check-prerequisites` uploads the werkdock binary first and runs `werkdock doctor`; `tools/werkator-build-prerequisites.sh` retires (its werkdock port is the survivor).
|
||||
|
||||
### C — Live verification and docs
|
||||
### C — Live verification and docs (done 2026-09-01)
|
||||
|
||||
- Run the full wrapper flow against mih34 (`instance-update`, `repo-init`, `instance-start` as no-op re-runs); `docs/deployment.md`'s webspace section switches to the `--env` invocations.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Session A: `werkator init --apply …` merges and re-merges a fragment idempotently; `werkator control-token` exists; full suite green.
|
||||
- Session B: `tools/remote` contains no YAML heredocs and no `sed` into the machine config; the prerequisites bash script is gone.
|
||||
- Session C: the mih34 re-runs change nothing on a configured host and the deployment docs show only `--env`-style calls.
|
||||
- Session A: done 2026-09-01 — `werkator init --apply …` installs and replaces a fragment idempotently; `werkator control-token` exists; full suite green.
|
||||
Deviation from the sketch above: the fragment is NOT merged into the machine config — it is installed verbatim as its own layer (`.git/werkator/.werkator.applied.yml`, above project, below machine config), because an in-place merge would re-serialize the machine config, destroying its comments and rewriting the file that holds the secrets; a verbatim copy also makes re-apply a plain file replacement.
|
||||
The `.htaccess` decision fell as proposed: generated beside the units by `init --systemd` whenever a `publicBaseUrl` is configured; the wrapper copies it into the domain docroot.
|
||||
- Session B: done 2026-09-01 — `tools/remote` contains no YAML heredocs and no `sed` into the machine config (the port lookups for idle check and port-forward read the *effective* config via `config:print`, so a port living in the applied fragment is found too); the prerequisites bash script is gone.
|
||||
- Session C: done 2026-09-01 — verified live on mih34 with the `.env.mih34` + `.env.mih34.yml` pair: instance-update, doctor-based check-prerequisites (PASS 6/6), repo-init applying the fragment, instance-start placing the generated `.htaccess` and restarting the unit, control-token via the CLI; `docs/deployment.md` shows only `--env-file`-style calls.
|
||||
Known niggle: validating a fragment that carries a `builds.default` without triggers logs the loader's "no build defines onPush" warning, although a fragment is judged out of context — cosmetic, fix when it annoys.
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
> **WARNING:** This document describes only the change applied in this PR.
|
||||
> It may already be outdated once the next PR is merged.
|
||||
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
||||
|
||||
## The Problem
|
||||
|
||||
`tools/remote` wrote the machine config by appending heredoc blocks and patching values with `sed`, duplicating configuration knowledge (YAML shape, indentation, key names) that `WerkatorConfig` already owns — an indentation mismatch in one append guard produced nine duplicate `bwrap` blocks on mih34 before it was found (step 21 session D).
|
||||
The script also re-implemented control-token generation in bash and still piped a prerequisites bash script whose generic half now exists as `werkdock doctor`.
|
||||
PR #8 laid out the fix; this PR implements it.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Multi-repository support for one Werkator instance (step 22, PR #10).
|
||||
- A mapping table between env keys and config keys — deliberately absent, see The Solution.
|
||||
|
||||
## The Scenarios
|
||||
|
||||
### Feature: `werkator init --apply` installs an instance config fragment
|
||||
|
||||
#### Background
|
||||
|
||||
- The applied fragment is a fourth config layer: project `.werkator.yml` → applied fragment → repo-install machine config (secrets, always wins) → branch layer.
|
||||
- It is strictly validated (unknown keys rejected) and installed verbatim, never merged in place — an in-place merge would re-serialize and destroy the machine config's comments and secrets.
|
||||
|
||||
#### Scenario#9.01: A valid fragment becomes its own layer, above the project config and below the machine config
|
||||
|
||||
So that an instance-specific setting (e.g. `server.port`) takes effect without touching the committed project config or the machine config's secrets.
|
||||
|
||||
- **Given** a project `.werkator.yml` and a machine config with a secret
|
||||
- **When** `init --apply FILE` installs a fragment that also sets a key the machine config sets
|
||||
- **Then** the effective config shows the fragment's value where the machine config is silent, and the machine config's value where both set the same key.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [the applied instance fragment layers above the project config and below the machine config](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
- [--apply installs the fragment as the applied layer and the effective config sees it](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
|
||||
|
||||
#### Scenario#9.02: Re-applying a fragment replaces it, never duplicates it
|
||||
|
||||
So that repeated `instance-update` runs stay idempotent — the duplication class PR #8 was written against dies here structurally.
|
||||
|
||||
- **Given** a fragment already installed as the applied layer
|
||||
- **When** `init --apply` runs again with a changed fragment
|
||||
- **Then** the applied layer file is atomically replaced, not appended to.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [applyInstanceFragment installs a valid fragment verbatim, and re-applying replaces it](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
|
||||
#### Scenario#9.03: An invalid fragment is refused loudly and installs nothing
|
||||
|
||||
So that a typo in a fragment never becomes a silent no-op or a half-applied layer.
|
||||
|
||||
- **Given** a fragment with an unknown key, or a missing/empty file
|
||||
- **When** `init --apply FILE` runs
|
||||
- **Then** it fails loudly and the applied layer is left exactly as it was before the attempt.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [applyInstanceFragment refuses an unknown key loudly instead of installing a silent no-op](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
- [applyInstanceFragment refuses a missing or empty fragment](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
- [--apply with an invalid fragment installs nothing](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
|
||||
|
||||
#### Scenario#9.04: `init --systemd` generates the Apache reverse-proxy file alongside the units
|
||||
|
||||
So that host integration for a public domain is generated exactly like the systemd units, not hand-written.
|
||||
|
||||
- **Given** an effective config with `server.publicBaseUrl` set
|
||||
- **When** `init --systemd` runs
|
||||
- **Then** `werkator.htaccess` is generated proxying to the configured localhost port, for the wrapper to place in the domain docroot.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [the htaccess proxies everything to the configured localhost port](../../src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt)
|
||||
|
||||
#### Scenario#9.05: `werkator control-token` prints the same token the server would create
|
||||
|
||||
So that the wrapper's bash re-implementation of token generation is no longer needed.
|
||||
|
||||
- **Given** a repository with or without an existing control token
|
||||
- **When** `werkator control-token` runs
|
||||
- **Then** it creates the token exactly like `ControlTokenService` would and prints the same value on a re-run
|
||||
- **and** it fails with exit code 2 outside a repository.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [creates the token like the server would and prints the same one on a re-run](../../src/test/kotlin/de/hoennig/werkator/commands/ControlTokenCommandTest.kt)
|
||||
- [fails with exit code 2 outside a repository](../../src/test/kotlin/de/hoennig/werkator/commands/ControlTokenCommandTest.kt)
|
||||
|
||||
### Feature: `tools/remote` becomes a thin wrapper
|
||||
|
||||
#### Scenario#9.06: The wrapper selects its instance by file, not by many options
|
||||
|
||||
So that several instances (mih34, vm4006, a future Werkbaum instance) are files, not edits to one script.
|
||||
|
||||
- **Given** a transport env file (default `.env`) naming a config fragment via `WERKATOR_INIT_CONFIG`
|
||||
- **When** `tools/remote --env-file .env.mih34 werkator repo-init` runs
|
||||
- **Then** the fragment is uploaded and `werkator init --apply` is invoked remotely with it — no heredoc or `sed` writes the machine config.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- live run on mih34 with `.env.mih34` + `.env.mih34.yml`: `instance-update`, `repo-init`, `instance-start` as idempotent re-runs (shell script; no automated test harness for `tools/remote`)
|
||||
|
||||
#### Scenario#9.07: `check-prerequisites` delegates to `werkdock doctor`
|
||||
|
||||
So that the generic host-readiness checks are not duplicated between a bash script and Werkdock's own porting of it.
|
||||
|
||||
- **Given** a target host
|
||||
- **When** `tools/remote werkator check-prerequisites` runs
|
||||
- **Then** it uploads the `werkdock` binary and runs `werkdock doctor`, and `tools/werkator-build-prerequisites.sh` is deleted.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- live run on mih34: doctor-based check-prerequisites, PASS 6/6
|
||||
|
||||
## The Solution
|
||||
|
||||
Session A (Werkator): `ConfigLoader` reads a new `ConfigFiles.APPLIED` path (`.git/werkator/.werkator.applied.yml`) as a layer between project and repo-install config; `applyInstanceFragment(workingDir, fragment)` validates the fragment against a strict Jackson mapper (`FAIL_ON_UNKNOWN_PROPERTIES=true`, fragment validation only — the regular mapper stays lenient for forward-compatibility) and then copies it verbatim via an atomic move, never merging in place.
|
||||
`InitCommand` gained `--apply FILE`, applied before `--systemd` handling so generated units/htaccess see the fragment; `SystemdServiceFiles.htaccessContent(port)` is the new generated file, written when `server.publicBaseUrl` is non-blank.
|
||||
`ControlTokenCommand` is a new subcommand delegating to the existing `ControlTokenService`.
|
||||
Session B (wrapper): `tools/remote --env-file FILE` (default `.env`) replaced positional/flag-heavy invocation; `repo-init`/`instance-start` lost their heredoc/`sed` config writing in favor of uploading the named fragment and calling `werkator init --apply`; port lookups (`require_idle`, `port_forward`) now parse `werkator config:print` output instead of grepping the machine config file directly, so a port living in the applied fragment is found too; `tools/werkator-build-prerequisites.sh` is deleted.
|
||||
Session C: verified live end-to-end on mih34 with a `.env.mih34` + `.env.mih34.yml` pair.
|
||||
|
||||
Deviation from the PR #8 plan: the fragment is not deep-merged into the machine config as first sketched — it is installed as its own verbatim layer, because an in-place merge would re-serialize the machine config and destroy its comments and secrets; a verbatim copy also makes re-apply a plain file replacement instead of a merge algorithm.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Applying a fragment that carries `builds.default` without triggers logs the loader's "no build defines onPush" warning, even though a fragment is judged out of context — cosmetic, fix when it annoys.
|
||||
|
||||
## Additional Changes
|
||||
|
||||
- `docs/configuration.md`: the layer table now shows four layers; a new subsection documents the applied instance fragment.
|
||||
- `docs/bootstrapping.md`: a new section documents `--apply`.
|
||||
- `docs/deployment.md`: the webspace section shows only `--env-file`-style invocations.
|
||||
- `.gitignore`: `/.env.*` added for per-instance env files.
|
||||
|
||||
## Prerequisite PRs
|
||||
|
||||
- PR #7 (webspace install path) — the role-separated `tools/remote` this PR simplifies.
|
||||
- PR #8 (step 23 plan) — the decision this PR implements.
|
||||
|
||||
## Follow-up PRs
|
||||
|
||||
- PR #10: multi-repository support for one Werkator instance (step 22).
|
||||
@@ -2,6 +2,7 @@ package de.hoennig.werkator
|
||||
|
||||
import de.hoennig.werkator.commands.BuildCommand
|
||||
import de.hoennig.werkator.commands.ConfigPrintCommand
|
||||
import de.hoennig.werkator.commands.ControlTokenCommand
|
||||
import de.hoennig.werkator.commands.InitCommand
|
||||
import de.hoennig.werkator.commands.RetryCommand
|
||||
import de.hoennig.werkator.commands.ServerCommand
|
||||
@@ -22,6 +23,7 @@ import picocli.CommandLine.Command
|
||||
BuildCommand::class,
|
||||
RetryCommand::class,
|
||||
ConfigPrintCommand::class,
|
||||
ControlTokenCommand::class,
|
||||
],
|
||||
mixinStandardHelpOptions = true,
|
||||
versionProvider = BuildPropertiesVersionProvider::class,
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
package de.hoennig.werkator.commands
|
||||
|
||||
import de.hoennig.werkator.git.GitService
|
||||
import de.hoennig.werkator.server.ControlTokenService
|
||||
import org.springframework.stereotype.Component
|
||||
import picocli.CommandLine.Command
|
||||
import java.nio.file.Path
|
||||
import java.nio.file.Paths
|
||||
import java.util.concurrent.Callable
|
||||
|
||||
/**
|
||||
* Prints the control token guarding the mutating build endpoints, creating it
|
||||
* exactly like the server does ([ControlTokenService] owns the format) — so no
|
||||
* wrapper script ever needs its own token generator (step 23).
|
||||
*/
|
||||
@Component
|
||||
@Command(
|
||||
name = "control-token",
|
||||
description = ["Print the control token for the mutating build endpoints, creating it if missing"],
|
||||
mixinStandardHelpOptions = true,
|
||||
)
|
||||
class ControlTokenCommand(
|
||||
private val gitService: GitService,
|
||||
) : Callable<Int> {
|
||||
var workingDir: Path = Paths.get(".")
|
||||
|
||||
override fun call(): Int {
|
||||
val root =
|
||||
try {
|
||||
gitService.getTopLevel(workingDir.toAbsolutePath().normalize())
|
||||
} catch (e: Exception) {
|
||||
println("Error: ${e.message}")
|
||||
return 2
|
||||
}
|
||||
println(ControlTokenService(root.resolve(".git/werkator/control-token")).token())
|
||||
return 0
|
||||
}
|
||||
}
|
||||
@@ -31,6 +31,16 @@ class InitCommand(
|
||||
)
|
||||
var systemd: Boolean = false
|
||||
|
||||
@Option(
|
||||
names = ["--apply"],
|
||||
description = [
|
||||
"install a config-schema YAML fragment as the applied instance layer " +
|
||||
"(validated strictly; re-applying replaces the previous fragment)",
|
||||
],
|
||||
paramLabel = "FILE",
|
||||
)
|
||||
var apply: Path? = null
|
||||
|
||||
/** Replaceable for tests: the jar this JVM was started from, or null when not run via `java -jar`. */
|
||||
internal var jarPathResolver: () -> Path? = { runningJarPath() }
|
||||
|
||||
@@ -52,6 +62,17 @@ class InitCommand(
|
||||
|
||||
createRepoInstallConfig(root, detected, normalizedWorkingDir)
|
||||
createProjectConfig(root, detected, normalizedWorkingDir)
|
||||
// before the systemd files, which read the effective configuration —
|
||||
// an applied fragment's port and limits must reach the generated unit
|
||||
apply?.let { fragment ->
|
||||
try {
|
||||
val target = configLoader.applyInstanceFragment(root, fragment)
|
||||
println("applied $fragment as ${target.toFile().relativeTo(normalizedWorkingDir.toFile())}")
|
||||
} catch (e: Exception) {
|
||||
println("Error: ${e.message}")
|
||||
return
|
||||
}
|
||||
}
|
||||
if (systemd) {
|
||||
createSystemdFiles(root, normalizedWorkingDir)
|
||||
}
|
||||
@@ -274,12 +295,14 @@ class InitCommand(
|
||||
* already loadable (re-running `init --systemd` on an installed instance); during
|
||||
* the very first bootstrap they stay unset and the defaults (no directives) apply.
|
||||
*/
|
||||
private fun loadedSystemdConfig(): de.hoennig.werkator.config.SystemdConfig =
|
||||
private fun loadedSystemdConfig(): de.hoennig.werkator.config.SystemdConfig = loadedServerConfig().systemd
|
||||
|
||||
private fun loadedServerConfig(): de.hoennig.werkator.config.ServerConfig =
|
||||
try {
|
||||
configLoader.load(Paths.get(".")).server.systemd
|
||||
configLoader.load(Paths.get(".")).server
|
||||
} catch (_: Exception) {
|
||||
de.hoennig.werkator.config
|
||||
.SystemdConfig()
|
||||
.ServerConfig()
|
||||
}
|
||||
|
||||
private fun createSystemdFiles(
|
||||
@@ -325,6 +348,19 @@ class InitCommand(
|
||||
pruneTimerFile.toFile().writeText(SystemdServiceFiles.pruneTimerContent())
|
||||
println("created ${pruneTimerFile.toFile().relativeTo(normalizedWorkingDir.toFile())}")
|
||||
|
||||
// generated host integration like the units: only meaningful behind a web
|
||||
// frontend, so it needs a public base URL; unused elsewhere and harmless
|
||||
val server = loadedServerConfig()
|
||||
if (server.publicBaseUrl.isNotBlank()) {
|
||||
val htaccessFile = werkatorDir.resolve(SystemdServiceFiles.HTACCESS_NAME)
|
||||
htaccessFile.toFile().writeText(SystemdServiceFiles.htaccessContent(server.port))
|
||||
println(
|
||||
"created ${htaccessFile.toFile().relativeTo(
|
||||
normalizedWorkingDir.toFile(),
|
||||
)} (Apache reverse proxy; copy it into the domain docroot on a managed webspace)",
|
||||
)
|
||||
}
|
||||
|
||||
println("install and start the service and the nightly Docker cleanup with:")
|
||||
println(" ln -sf $unitFile ~/.config/systemd/user/$unitName")
|
||||
println(" ln -sf $pruneServiceFile ~/.config/systemd/user/${SystemdServiceFiles.PRUNE_SERVICE_NAME}")
|
||||
|
||||
@@ -11,6 +11,8 @@ object SystemdServiceFiles {
|
||||
const val ENV_FILE_NAME = "werkator.env"
|
||||
|
||||
/** Host-global unit names of the nightly Docker cleanup — shared by all Werkator repositories on the host. */
|
||||
const val HTACCESS_NAME = "werkator.htaccess"
|
||||
|
||||
const val PRUNE_SERVICE_NAME = "werkator-docker-prune.service"
|
||||
const val PRUNE_TIMER_NAME = "werkator-docker-prune.timer"
|
||||
|
||||
@@ -89,6 +91,20 @@ object SystemdServiceFiles {
|
||||
#JAVA_OPTS=-Xmx256m
|
||||
""".trimIndent() + "\n"
|
||||
|
||||
/**
|
||||
* Apache reverse proxy for a Hostsharing managed webspace: the platform's
|
||||
* Apache terminates TLS for the domain and forwards everything to the
|
||||
* localhost port of the "eigener Serverdienst". Generated host integration
|
||||
* like the units — the wrapper copies it into the domain's docroot.
|
||||
*/
|
||||
fun htaccessContent(port: Int): String =
|
||||
"""
|
||||
DirectoryIndex disabled
|
||||
RewriteEngine On
|
||||
RewriteBase /
|
||||
RewriteRule .* http://127.0.0.1:$port%{REQUEST_URI} [proxy]
|
||||
""".trimIndent() + "\n"
|
||||
|
||||
private fun sanitize(name: String): String = name.replace(Regex("[^A-Za-z0-9_.-]"), "-")
|
||||
|
||||
/** Escape `%` specifiers in systemd unit values (legacy `systemd_path`). */
|
||||
|
||||
@@ -19,6 +19,15 @@ object ConfigFiles {
|
||||
/** The machine-specific configuration inside `.git`; secrets live here. */
|
||||
const val REPO_INSTALL = ".git/werkator/$COMMITTED"
|
||||
|
||||
/**
|
||||
* The applied instance fragment (`init --apply`, step 23): a config-schema YAML
|
||||
* fragment installed verbatim as its own layer — above the committed project
|
||||
* config, below the hand-edited machine config. Kept separate so applying never
|
||||
* rewrites the machine config (its comments and secrets stay untouched) and
|
||||
* re-applying is a plain file replacement, never a merge that can duplicate.
|
||||
*/
|
||||
const val APPLIED = ".git/werkator/.werkator.applied.yml"
|
||||
|
||||
/** The name the committed configuration had before the rename. */
|
||||
const val LEGACY_COMMITTED = ".gittally.yml"
|
||||
|
||||
|
||||
@@ -11,8 +11,10 @@ import org.springframework.beans.factory.ObjectProvider
|
||||
import org.springframework.boot.info.BuildProperties
|
||||
import org.springframework.stereotype.Service
|
||||
import java.io.File
|
||||
import java.nio.file.Files
|
||||
import java.nio.file.Path
|
||||
import java.nio.file.Paths
|
||||
import java.nio.file.StandardCopyOption
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
|
||||
@Service
|
||||
@@ -28,6 +30,17 @@ class ConfigLoader(
|
||||
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
|
||||
.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
|
||||
|
||||
/**
|
||||
* For validating instance fragments ([applyInstanceFragment]) only: unknown keys
|
||||
* fail there instead of being ignored — the regular layers stay lenient so an old
|
||||
* Werkator can read a newer file.
|
||||
*/
|
||||
private val strictYaml =
|
||||
ObjectMapper(YAMLFactory().disable(YAMLGenerator.Feature.WRITE_DOC_START_MARKER))
|
||||
.registerKotlinModule()
|
||||
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true)
|
||||
.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
|
||||
|
||||
/** Keys already reported by [dropNonDefinitionBuilds]; the config is loaded on every poll cycle. */
|
||||
private val warnedBuildKeys = ConcurrentHashMap.newKeySet<String>()
|
||||
|
||||
@@ -285,13 +298,50 @@ class ConfigLoader(
|
||||
val repoInstallName = ConfigFiles.firstExisting(workingDir, ConfigFiles.repoInstall)
|
||||
val projectName = ConfigFiles.firstExisting(workingDir)
|
||||
val repoInstall = loadFile(workingDir.resolve(repoInstallName).toFile())
|
||||
val applied = loadFile(workingDir.resolve(ConfigFiles.APPLIED).toFile())
|
||||
val project = loadFile(workingDir.resolve(projectName).toFile())
|
||||
// per file, so the message names the file to fix — the merged map has no provenance
|
||||
checkVersion(project, projectName, ROLLBACK_HINT)
|
||||
checkVersion(applied, ConfigFiles.APPLIED, ROLLBACK_HINT)
|
||||
checkVersion(repoInstall, repoInstallName, ROLLBACK_HINT)
|
||||
checkTriggerBlocks(project, projectName, ROLLBACK_HINT)
|
||||
checkTriggerBlocks(applied, ConfigFiles.APPLIED, ROLLBACK_HINT)
|
||||
checkTriggerBlocks(repoInstall, repoInstallName, ROLLBACK_HINT)
|
||||
return deepMerge(project, repoInstall)
|
||||
// the applied instance fragment sits above the committed project config and
|
||||
// below the hand-edited machine config, which always has the last word
|
||||
return deepMerge(deepMerge(project, applied), repoInstall)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validates and installs an instance fragment (`init --apply`, step 23): the file
|
||||
* must be non-empty, pass the version and trigger checks, and bind *strictly*
|
||||
* against the schema — an unknown key is refused loudly, never ignored, because a
|
||||
* typo in a fragment would otherwise install a value that silently does nothing.
|
||||
* The fragment is then copied verbatim (comments included) to [ConfigFiles.APPLIED];
|
||||
* re-applying replaces the file, so nothing can accumulate or duplicate.
|
||||
*/
|
||||
fun applyInstanceFragment(
|
||||
workingDir: Path,
|
||||
fragment: Path,
|
||||
): Path {
|
||||
val raw = loadFile(fragment.toFile())
|
||||
require(raw.isNotEmpty()) { "instance fragment $fragment is missing, empty, or not a YAML mapping" }
|
||||
checkVersion(raw, fragment.toString(), ROLLBACK_HINT)
|
||||
checkTriggerBlocks(raw, fragment.toString(), ROLLBACK_HINT)
|
||||
try {
|
||||
strictYaml.convertValue(resolveBuildSections(dropNonDefinitionBuilds(raw)), WerkatorConfig::class.java)
|
||||
} catch (e: IllegalArgumentException) {
|
||||
throw IllegalArgumentException(
|
||||
"instance fragment $fragment does not match the configuration schema: ${e.message}",
|
||||
e,
|
||||
)
|
||||
}
|
||||
val target = workingDir.resolve(ConfigFiles.APPLIED)
|
||||
Files.createDirectories(target.parent)
|
||||
val temp = Files.createTempFile(target.parent, ".werkator.applied", ".tmp")
|
||||
Files.copy(fragment, temp, StandardCopyOption.REPLACE_EXISTING)
|
||||
Files.move(temp, target, StandardCopyOption.REPLACE_EXISTING, StandardCopyOption.ATOMIC_MOVE)
|
||||
return target
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
package de.hoennig.werkator.commands
|
||||
|
||||
import de.hoennig.werkator.git.GitService
|
||||
import io.kotest.core.spec.style.FunSpec
|
||||
import io.kotest.matchers.shouldBe
|
||||
import io.kotest.matchers.string.shouldMatch
|
||||
import io.mockk.every
|
||||
import io.mockk.mockk
|
||||
import java.nio.file.Files
|
||||
|
||||
class ControlTokenCommandTest : FunSpec() {
|
||||
private val gitService = mockk<GitService>()
|
||||
private val command = ControlTokenCommand(gitService)
|
||||
|
||||
init {
|
||||
test("creates the token like the server would and prints the same one on a re-run") {
|
||||
val tempDir = Files.createTempDirectory("werkator-token-test")
|
||||
command.workingDir = tempDir
|
||||
every { gitService.getTopLevel(any()) } returns tempDir
|
||||
|
||||
command.call() shouldBe 0
|
||||
|
||||
val tokenFile = tempDir.resolve(".git/werkator/control-token")
|
||||
val token = tokenFile.toFile().readText().trim()
|
||||
token shouldMatch Regex("[0-9a-f]{48}")
|
||||
|
||||
command.call() shouldBe 0
|
||||
tokenFile.toFile().readText().trim() shouldBe token
|
||||
}
|
||||
|
||||
test("fails with exit code 2 outside a repository") {
|
||||
every { gitService.getTopLevel(any()) } throws IllegalStateException("not a git repository")
|
||||
|
||||
command.call() shouldBe 2
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -17,8 +17,10 @@ class InitCommandTest : FunSpec() {
|
||||
private val initCommand =
|
||||
InitCommand(
|
||||
gitService,
|
||||
// default (null) BuildProperties provider: a relaxed ObjectProvider mock
|
||||
// returns a raw Object under type erasure and breaks the version check
|
||||
de.hoennig.werkator.config
|
||||
.ConfigLoader(mockk(relaxed = true)),
|
||||
.ConfigLoader(),
|
||||
)
|
||||
|
||||
init {
|
||||
@@ -122,6 +124,47 @@ class InitCommandTest : FunSpec() {
|
||||
projectConfig.toFile().readText() shouldBe "existing: content"
|
||||
}
|
||||
|
||||
test("--apply installs the fragment as the applied layer and the effective config sees it") {
|
||||
val tempDir = Files.createTempDirectory("werkator-init-test")
|
||||
initCommand.workingDir = tempDir
|
||||
val fragment = tempDir.resolve("mih34.yml")
|
||||
fragment.toFile().writeText("server:\n port: 18088\n")
|
||||
initCommand.apply = fragment
|
||||
|
||||
every { gitService.getTopLevel(tempDir) } returns tempDir
|
||||
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
|
||||
|
||||
initCommand.run()
|
||||
|
||||
tempDir
|
||||
.resolve(de.hoennig.werkator.config.ConfigFiles.APPLIED)
|
||||
.toFile()
|
||||
.shouldExist()
|
||||
de.hoennig.werkator.config
|
||||
.ConfigLoader()
|
||||
.load(tempDir)
|
||||
.server.port shouldBe 18088
|
||||
initCommand.apply = null
|
||||
}
|
||||
|
||||
test("--apply with an invalid fragment installs nothing") {
|
||||
val tempDir = Files.createTempDirectory("werkator-init-test")
|
||||
initCommand.workingDir = tempDir
|
||||
val fragment = tempDir.resolve("typo.yml")
|
||||
fragment.toFile().writeText("server:\n prot: 18088\n")
|
||||
initCommand.apply = fragment
|
||||
|
||||
every { gitService.getTopLevel(tempDir) } returns tempDir
|
||||
every { gitService.getOriginUrl(tempDir) } returns "https://git.example.org/my-org/my-repo.git"
|
||||
|
||||
initCommand.run()
|
||||
|
||||
Files
|
||||
.exists(tempDir.resolve(de.hoennig.werkator.config.ConfigFiles.APPLIED))
|
||||
.shouldBeFalse()
|
||||
initCommand.apply = null
|
||||
}
|
||||
|
||||
test("--systemd generates unit and environment file with install instructions") {
|
||||
val tempDir = Files.createTempDirectory("werkator-init-test")
|
||||
initCommand.workingDir = tempDir
|
||||
|
||||
@@ -91,5 +91,11 @@ class SystemdServiceFilesTest : FunSpec() {
|
||||
content shouldContain "#JAVA_OPTS="
|
||||
content shouldContain ".werkator.yml"
|
||||
}
|
||||
test("the htaccess proxies everything to the configured localhost port") {
|
||||
val content = SystemdServiceFiles.htaccessContent(18088)
|
||||
|
||||
content shouldContain "DirectoryIndex disabled"
|
||||
content shouldContain "RewriteRule .* http://127.0.0.1:18088%{REQUEST_URI} [proxy]"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -110,6 +110,63 @@ class ConfigLoaderTest : FunSpec() {
|
||||
"./gradlew fromBranch"
|
||||
}
|
||||
|
||||
test("the applied instance fragment layers above the project config and below the machine config") {
|
||||
val dir = Files.createTempDirectory("werkator-test")
|
||||
dir.resolve(".werkator.yml").toFile().writeText("server:\n port: 1000\n publicBaseUrl: \"https://project/\"\n")
|
||||
Files.createDirectories(dir.resolve(".git/werkator"))
|
||||
dir.resolve(ConfigFiles.APPLIED).toFile().writeText("server:\n port: 2000\n bindAddress: 0.0.0.0\n")
|
||||
dir
|
||||
.resolve(".git/werkator/.werkator.yml")
|
||||
.toFile()
|
||||
.writeText("server:\n port: 3000\n")
|
||||
|
||||
val server = loader.load(dir).server
|
||||
|
||||
// machine wins over applied wins over project; untouched keys fall through
|
||||
server.port shouldBe 3000
|
||||
server.bindAddress shouldBe "0.0.0.0"
|
||||
server.publicBaseUrl shouldBe "https://project/"
|
||||
}
|
||||
|
||||
test("applyInstanceFragment installs a valid fragment verbatim, and re-applying replaces it") {
|
||||
val dir = Files.createTempDirectory("werkator-test")
|
||||
val fragment = dir.resolve("mih34.yml")
|
||||
fragment.toFile().writeText("# instance mih34\nserver:\n port: 18088\n")
|
||||
|
||||
val target = loader.applyInstanceFragment(dir, fragment)
|
||||
|
||||
target shouldBe dir.resolve(ConfigFiles.APPLIED)
|
||||
// verbatim copy: the comment survives
|
||||
target.toFile().readText() shouldContain "# instance mih34"
|
||||
loader.load(dir).server.port shouldBe 18088
|
||||
|
||||
fragment.toFile().writeText("server:\n port: 19099\n")
|
||||
loader.applyInstanceFragment(dir, fragment)
|
||||
loader.load(dir).server.port shouldBe 19099
|
||||
}
|
||||
|
||||
test("applyInstanceFragment refuses an unknown key loudly instead of installing a silent no-op") {
|
||||
val dir = Files.createTempDirectory("werkator-test")
|
||||
val fragment = dir.resolve("typo.yml")
|
||||
fragment.toFile().writeText("server:\n prot: 18088\n")
|
||||
|
||||
val exception =
|
||||
shouldThrow<IllegalArgumentException> {
|
||||
loader.applyInstanceFragment(dir, fragment)
|
||||
}
|
||||
|
||||
exception.message shouldContain "typo.yml"
|
||||
Files.exists(dir.resolve(ConfigFiles.APPLIED)).shouldBeFalse()
|
||||
}
|
||||
|
||||
test("applyInstanceFragment refuses a missing or empty fragment") {
|
||||
val dir = Files.createTempDirectory("werkator-test")
|
||||
|
||||
shouldThrow<IllegalArgumentException> {
|
||||
loader.applyInstanceFragment(dir, dir.resolve("absent.yml"))
|
||||
}
|
||||
}
|
||||
|
||||
test("reads executor.maxConcurrent and defaults it to 1") {
|
||||
val dir = Files.createTempDirectory("werkator-test")
|
||||
loader.load(dir).executor.maxConcurrent shouldBe 1
|
||||
|
||||
+95
-104
@@ -13,33 +13,38 @@
|
||||
# repository are the running instance's job (or `bin/werkator build` on the
|
||||
# host — the werkator CLI, not this script).
|
||||
#
|
||||
# Usage:
|
||||
# tools/remote werkator check-prerequisites bwrap capability + disk/quota on the host
|
||||
# tools/remote werkator instance-install first-time: upload + unpack bundle and werkdock
|
||||
# tools/remote werkator instance-update redeploy bundle + werkdock, restart the service
|
||||
# tools/remote werkator instance-start server config, Apache proxy, systemd unit
|
||||
# tools/remote werkator repo-init clone the watched repo, init, rootfs, bwrap config
|
||||
# tools/remote werkator control-token
|
||||
# tools/remote port-forward start background tunnel to the Werkator UI
|
||||
# tools/remote port-forward stop
|
||||
# Usage (step 23: `--env-file` selects the target instance, default `.env`):
|
||||
# tools/remote [--env-file FILE] werkator check-prerequisites werkdock doctor on the host
|
||||
# tools/remote [--env-file FILE] werkator instance-install first-time: upload + unpack bundle and werkdock
|
||||
# tools/remote [--env-file FILE] werkator instance-update redeploy bundle + werkdock, restart the service
|
||||
# tools/remote [--env-file FILE] werkator instance-start apply fragment, Apache proxy, systemd unit
|
||||
# tools/remote [--env-file FILE] werkator repo-init clone the watched repo, init --apply, rootfs
|
||||
# tools/remote [--env-file FILE] werkator control-token
|
||||
# tools/remote [--env-file FILE] port-forward start background tunnel to the Werkator UI
|
||||
# tools/remote [--env-file FILE] port-forward stop
|
||||
#
|
||||
# Required in .env:
|
||||
# The env file carries TRANSPORT values only; everything that is Werkator
|
||||
# configuration travels as a YAML fragment in the config schema, named by
|
||||
# WERKATOR_INIT_CONFIG and installed remotely via `werkator init --apply`
|
||||
# (docs/plan/23-init-owns-the-files.md). Pair the files per instance, e.g.
|
||||
# `.env.mih34` + `.env.mih34.yml` (both gitignored).
|
||||
#
|
||||
# Required in the env file:
|
||||
# WERKATOR_REMOTE user@host to operate on, e.g. mih34-werkator@mih34.hostsharing.net
|
||||
# WERKATOR_PATH target directory on that host, e.g. /home/storage/mih34/users/werkator
|
||||
#
|
||||
# Required for `instance-start`:
|
||||
# WERKATOR_PORT the localhost port assigned by Hostsharing (eigener Serverdienst)
|
||||
# WERKATOR_DOMAIN the domain served by the managed Apache, e.g. ci.example.de
|
||||
# WERKATOR_DOMAIN the domain served by the managed Apache (docroot location for the
|
||||
# generated .htaccess); the port lives in the fragment (server.port)
|
||||
#
|
||||
# Required for `port-forward`:
|
||||
# WERKATOR_LOCAL_PORT the local port the browser uses
|
||||
# Optional in .env:
|
||||
# WERKATOR_REPO_URL https clone URL of the watched repository
|
||||
# (default: https://github.com/mhoennig/werkator.git)
|
||||
# WERKATOR_MEMORY_MAX systemd MemoryMax for the unit, e.g. 1G (instance-start)
|
||||
# WERKATOR_TASKS_MAX systemd TasksMax for the unit, e.g. 512 (instance-start)
|
||||
# WERKATOR_ROOTFS rootfs archive path for repo-init
|
||||
# (default: <repo>/build/werkator-buildenv-trixie-java-go-node.tar.zst)
|
||||
# WERKATOR_LOCAL_PORT the local port the browser uses
|
||||
# Optional in the env file:
|
||||
# WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start)
|
||||
# WERKATOR_REPO_URL https clone URL of the watched repository
|
||||
# (default: https://github.com/mhoennig/werkator.git)
|
||||
# WERKATOR_ROOTFS rootfs archive path for repo-init
|
||||
# (default: <repo>/build/werkator-buildenv-trixie-java-go-node.tar.zst)
|
||||
#
|
||||
# Install layout on the host:
|
||||
# $WERKATOR_PATH/werkator/ the watched repository (clone)
|
||||
@@ -52,11 +57,16 @@ set -euo pipefail
|
||||
|
||||
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||
|
||||
ENV_FILE=""
|
||||
if [ "${1:-}" = "--env-file" ]; then
|
||||
ENV_FILE="${2:?missing value for --env-file}"
|
||||
shift 2
|
||||
fi
|
||||
|
||||
REPO="${1:-}"
|
||||
COMMAND="${2:-}"
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
PREREQ_SCRIPT="$REPO_ROOT/tools/werkator-build-prerequisites.sh"
|
||||
RUNTIME_BUNDLE="$REPO_ROOT/build/distributions/werkator-runtime-linux-x64.tar.gz"
|
||||
WERKDOCK_BINARY="$REPO_ROOT/werkdock/dist/werkdock"
|
||||
PID_FILE="/tmp/werkator-port-forward-$(id -u).pid"
|
||||
@@ -71,7 +81,7 @@ require_env() {
|
||||
local missing=0
|
||||
for name in "$@"; do
|
||||
if [ -z "${!name:-}" ]; then
|
||||
echo "ERROR: $name is not set — define it in $REPO_ROOT/.env" >&2
|
||||
echo "ERROR: $name is not set — define it in $ENV_FILE" >&2
|
||||
missing=1
|
||||
fi
|
||||
done
|
||||
@@ -80,10 +90,11 @@ require_env() {
|
||||
|
||||
[ -n "$REPO" ] && [ -n "$COMMAND" ] || usage
|
||||
|
||||
# Load the connection and deployment values; explicit environment wins, the
|
||||
# .env in the repository root fills the rest.
|
||||
# Load the transport values; explicit environment wins, the selected env file
|
||||
# (default: the .env in the repository root) fills the rest.
|
||||
ENV_FILE="${ENV_FILE:-$REPO_ROOT/.env}"
|
||||
set -a
|
||||
[ -f "$REPO_ROOT/.env" ] && source "$REPO_ROOT/.env"
|
||||
[ -f "$ENV_FILE" ] && source "$ENV_FILE"
|
||||
set +a
|
||||
|
||||
require_env WERKATOR_REMOTE WERKATOR_PATH
|
||||
@@ -109,16 +120,37 @@ ensure_ssh() {
|
||||
fi
|
||||
}
|
||||
|
||||
# Run the prerequisites script remotely by piping it over stdin; TARGET_DIR and
|
||||
# ROOTFS_ARCHIVE are passed as arguments to `bash -s --`.
|
||||
# Werkdock owns the host checks (`werkdock doctor` ports the old prerequisites
|
||||
# script); the binary is uploaded first, so the check works pre-install.
|
||||
check_prerequisites() {
|
||||
echo "==> Checking prerequisites on $HOST (target dir: $TARGET_DIR)"
|
||||
local rootfs_remote="$TARGET_DIR/.werkator/$(basename "$ROOTFS")"
|
||||
if ! ssh "$HOST" "WERKATOR_SSH_TARGET='$HOST' bash -s -- '$TARGET_DIR' '$rootfs_remote'" < "$PREREQ_SCRIPT"; then
|
||||
die "prerequisites failed on $HOST — install aborted"
|
||||
ensure_werkdock_binary
|
||||
echo "==> Uploading werkdock and running its doctor on $HOST (target dir: $TARGET_DIR)"
|
||||
ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator/bin'"
|
||||
scp -q "$WERKDOCK_BINARY" "$HOST:$TARGET_DIR/.werkator/bin/werkdock.new"
|
||||
ssh "$HOST" "mv '$TARGET_DIR/.werkator/bin/werkdock.new' '$TARGET_DIR/.werkator/bin/werkdock' && chmod 755 '$TARGET_DIR/.werkator/bin/werkdock'"
|
||||
if ! ssh "$HOST" "'$TARGET_DIR/.werkator/bin/werkdock' doctor '$TARGET_DIR'"; then
|
||||
die "werkdock doctor failed on $HOST — install aborted"
|
||||
fi
|
||||
}
|
||||
|
||||
ensure_werkdock_binary() {
|
||||
if [ ! -f "$WERKDOCK_BINARY" ]; then
|
||||
echo "==> werkdock binary not found; building it locally (go build)"
|
||||
(cd "$REPO_ROOT/werkdock" && CGO_ENABLED=0 go build -o dist/werkdock .)
|
||||
fi
|
||||
[ -f "$WERKDOCK_BINARY" ] || die "werkdock binary missing: $WERKDOCK_BINARY"
|
||||
}
|
||||
|
||||
# Uploads the init fragment named by WERKATOR_INIT_CONFIG and echoes its remote
|
||||
# path; empty when no fragment is configured.
|
||||
upload_fragment() {
|
||||
[ -n "${WERKATOR_INIT_CONFIG:-}" ] || { echo ""; return 0; }
|
||||
[ -f "$WERKATOR_INIT_CONFIG" ] || die "init fragment missing: $WERKATOR_INIT_CONFIG"
|
||||
local remote="$TARGET_DIR/.werkator/$(basename "$WERKATOR_INIT_CONFIG")"
|
||||
scp -q "$WERKATOR_INIT_CONFIG" "$HOST:$remote"
|
||||
echo "$remote"
|
||||
}
|
||||
|
||||
# The instance artifacts are built locally (ADR 0006): the runtime bundle via
|
||||
# Gradle, the werkdock binary via the Go toolchain. Both are rebuilt when
|
||||
# missing, never on the target.
|
||||
@@ -128,11 +160,7 @@ ensure_instance_artifacts() {
|
||||
(cd "$REPO_ROOT" && ./gradlew runtimeBundle --console=plain -q)
|
||||
fi
|
||||
[ -f "$RUNTIME_BUNDLE" ] || die "runtime bundle missing: $RUNTIME_BUNDLE"
|
||||
if [ ! -f "$WERKDOCK_BINARY" ]; then
|
||||
echo "==> werkdock binary not found; building it locally (go build)"
|
||||
(cd "$REPO_ROOT/werkdock" && CGO_ENABLED=0 go build -o dist/werkdock .)
|
||||
fi
|
||||
[ -f "$WERKDOCK_BINARY" ] || die "werkdock binary missing: $WERKDOCK_BINARY"
|
||||
ensure_werkdock_binary
|
||||
}
|
||||
|
||||
# Uploads and unpacks the instance artifacts. The previous runtime stays as
|
||||
@@ -168,7 +196,7 @@ instance_install() {
|
||||
# Refuse to swap the runtime under a running build; FORCE=1 overrides.
|
||||
require_idle() {
|
||||
local port
|
||||
port="$(ssh "$HOST" "awk '/^server:/{f=1;next} f && /^ port:/{print \$2; exit}' '$MACHINE_CONFIG' 2>/dev/null" || true)"
|
||||
port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' 2>/dev/null && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"' || true)"
|
||||
[ -n "$port" ] || return 0
|
||||
local current
|
||||
current="$(ssh "$HOST" "curl -s --max-time 5 http://127.0.0.1:$port/api/builds/current" || true)"
|
||||
@@ -202,8 +230,9 @@ instance_update() {
|
||||
|
||||
# Sets up the WATCHED repository: an anonymous https clone (a private origin
|
||||
# gets its credentials via git.account/git.token in the machine config that
|
||||
# `werkator init` creates), the werkator init, the rootfs archive for the
|
||||
# sandbox builds, and the machine-local bwrap configuration.
|
||||
# `werkator init` creates), the werkator init with the instance fragment
|
||||
# applied, and the rootfs archive for the sandbox builds. All configuration
|
||||
# writing is init's — this script transports and invokes (step 23).
|
||||
repo_init() {
|
||||
ensure_ssh
|
||||
[ -f "$ROOTFS" ] || die "rootfs archive missing: $ROOTFS — build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS"
|
||||
@@ -216,9 +245,6 @@ repo_init() {
|
||||
ssh "$HOST" "git clone '$REPO_URL' '$TARGET_DIR/werkator'"
|
||||
fi
|
||||
|
||||
echo "==> Running werkator init"
|
||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init"
|
||||
|
||||
echo "==> Uploading the rootfs archive (skipped when unchanged)"
|
||||
local rootfs_remote="$TARGET_DIR/.werkator/$(basename "$ROOTFS")"
|
||||
local local_sha remote_sha
|
||||
@@ -232,25 +258,10 @@ repo_init() {
|
||||
[ "$local_sha" = "$remote_sha" ] || die "rootfs upload checksum mismatch"
|
||||
fi
|
||||
|
||||
echo "==> Writing the machine-local bwrap configuration"
|
||||
# NOTE: the guard must match the block's real indentation — a mismatch here
|
||||
# once appended the block on every run.
|
||||
if ssh "$HOST" "grep -q '^ bwrap:' '$MACHINE_CONFIG' 2>/dev/null"; then
|
||||
echo " (bwrap block present, skipping)"
|
||||
else
|
||||
ssh "$HOST" "cat >> '$MACHINE_CONFIG' <<'CFG'
|
||||
|
||||
# Build in the bubblewrap sandbox instead of natively (ADR 0008), executed by
|
||||
# the werkdock CLI (step 21 session C). All three keys are pinned: read from
|
||||
# this machine config even if a branch sets its own values.
|
||||
builds:
|
||||
default:
|
||||
bwrap:
|
||||
enabled: true
|
||||
rootfs: $rootfs_remote
|
||||
werkdock: $TARGET_DIR/.werkator/bin/werkdock
|
||||
CFG"
|
||||
fi
|
||||
echo "==> Running werkator init${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
|
||||
local fragment_remote
|
||||
fragment_remote="$(upload_fragment)"
|
||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
|
||||
|
||||
echo "==> Verifying the effective configuration"
|
||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null | grep -A4 'bwrap:' | head -5"
|
||||
@@ -262,47 +273,28 @@ CFG"
|
||||
echo " then tools/remote werkator instance-start"
|
||||
}
|
||||
|
||||
# Start the server as a systemd user unit behind the managed Apache.
|
||||
# WERKATOR_MEMORY_MAX / WERKATOR_TASKS_MAX (optional) are written into the
|
||||
# machine config so `init --systemd` bakes them into the unit.
|
||||
# Start the server as a systemd user unit behind the managed Apache. All
|
||||
# configuration comes from the instance fragment (server.port, publicBaseUrl,
|
||||
# systemd limits); init generates the units AND the .htaccess — this script
|
||||
# only places and activates them (step 23).
|
||||
instance_start() {
|
||||
ensure_ssh
|
||||
require_env WERKATOR_PORT WERKATOR_DOMAIN
|
||||
require_env WERKATOR_DOMAIN
|
||||
|
||||
echo "==> Applying the instance fragment and generating the host integration (init --systemd)"
|
||||
local fragment_remote
|
||||
fragment_remote="$(upload_fragment)"
|
||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'} --systemd"
|
||||
|
||||
local htaccess_src="$TARGET_DIR/werkator/.git/werkator/werkator.htaccess"
|
||||
local htaccess="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/.htaccess"
|
||||
|
||||
echo "==> Writing server settings to the machine config"
|
||||
if ssh "$HOST" "grep -q '^server:' '$MACHINE_CONFIG' 2>/dev/null"; then
|
||||
# re-run: update port and publicBaseUrl in place (systemd limits stay as written)
|
||||
ssh "$HOST" "sed -i 's/^ port: .*/ port: $WERKATOR_PORT/; s|^ publicBaseUrl: .*| publicBaseUrl: \"https://$WERKATOR_DOMAIN/\"|' '$MACHINE_CONFIG'"
|
||||
if ssh "$HOST" "test -f '$htaccess_src'"; then
|
||||
echo "==> Placing the generated Apache reverse proxy at $htaccess"
|
||||
ssh "$HOST" "mkdir -p '$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www' && cp '$htaccess_src' '$htaccess'"
|
||||
else
|
||||
ssh "$HOST" "cat >> '$MACHINE_CONFIG' <<'CFG'
|
||||
|
||||
# Web access: the managed Apache terminates TLS and proxies to the localhost
|
||||
# port assigned by Hostsharing (eigener Serverdienst); TLS is the domain's
|
||||
# Let's Encrypt certificate, so Werkator itself stays on 127.0.0.1.
|
||||
server:
|
||||
port: $WERKATOR_PORT
|
||||
bindAddress: 127.0.0.1
|
||||
publicBaseUrl: \"https://$WERKATOR_DOMAIN/\"
|
||||
nginx:
|
||||
enabled: false
|
||||
systemd:
|
||||
memoryMax: \"${WERKATOR_MEMORY_MAX:-}\"
|
||||
tasksMax: \"${WERKATOR_TASKS_MAX:-}\"
|
||||
CFG"
|
||||
echo "==> No werkator.htaccess generated (no server.publicBaseUrl configured) — skipping the Apache proxy"
|
||||
fi
|
||||
|
||||
echo "==> Writing the Apache reverse proxy to $htaccess"
|
||||
ssh "$HOST" "mkdir -p '$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www' && cat > '$htaccess' <<'HT'
|
||||
DirectoryIndex disabled
|
||||
RewriteEngine On
|
||||
RewriteBase /
|
||||
RewriteRule .* http://127.0.0.1:$WERKATOR_PORT%{REQUEST_URI} [proxy]
|
||||
HT"
|
||||
|
||||
echo "==> Generating the systemd user unit (init --systemd)"
|
||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init --systemd"
|
||||
|
||||
echo "==> Linking the units into ~/.config/systemd/user and enabling the service"
|
||||
ssh "$HOST" "mkdir -p ~/.config/systemd/user && \
|
||||
ln -sf '$TARGET_DIR/werkator/.git/werkator/$UNIT' ~/.config/systemd/user/ && \
|
||||
@@ -322,9 +314,11 @@ HT"
|
||||
# `start` runs ssh -N -L detached with a pid file; `stop` kills it.
|
||||
port_forward() {
|
||||
require_env WERKATOR_LOCAL_PORT
|
||||
# the effective port, wherever it is configured (machine config or applied
|
||||
# fragment) — config:print is the single answer, not this script's parser
|
||||
local remote_port
|
||||
remote_port="$(ssh "$HOST" "awk '/^server:/{f=1;next} f && /^ port:/{print \$2; exit}' '$MACHINE_CONFIG'")"
|
||||
[ -n "$remote_port" ] || die "no server.port in the machine config — run 'tools/remote werkator instance-start' first"
|
||||
remote_port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"')"
|
||||
[ -n "$remote_port" ] || die "no server.port configured — run 'tools/remote werkator instance-start' first"
|
||||
|
||||
case "$COMMAND" in
|
||||
start)
|
||||
@@ -361,14 +355,11 @@ port_forward() {
|
||||
esac
|
||||
}
|
||||
|
||||
# Print the control token guarding the mutating build endpoints. If the server
|
||||
# has not created it yet (it does so on first use), generate one in place — the
|
||||
# server reads the file lazily, so a pre-created token is equivalent.
|
||||
# Print the control token guarding the mutating build endpoints — the werkator
|
||||
# CLI owns creation and format (step 23), this script only invokes it.
|
||||
control_token() {
|
||||
ensure_ssh
|
||||
local token_file="$TARGET_DIR/werkator/.git/werkator/control-token"
|
||||
ssh "$HOST" "if [ -f '$token_file' ]; then cat '$token_file'; else \
|
||||
umask 077 && head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \\n' > '$token_file' && cat '$token_file'; fi"
|
||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' control-token"
|
||||
}
|
||||
|
||||
case "$REPO" in
|
||||
|
||||
@@ -1,164 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Verify the bwrap (bubblewrap) build precondition on a target host before
|
||||
# running Werkator's bwrap build runtime there (step 17 / ADR 0007).
|
||||
#
|
||||
# The whole "build Werkator inside bubblewrap on a Managed Webspace" approach
|
||||
# hinges on one hard precondition: unprivileged user namespaces with a uid-0
|
||||
# mapping and read-only root binds must work. This script runs the exact
|
||||
# command line recorded in docs/plan/17-bwrap-build-runtime.md, checks the
|
||||
# expected signals, and additionally verifies the disk/quota situation:
|
||||
# a bwrap build unpacks the rootfs (a zstd archive expands to several GiB)
|
||||
# plus a Gradle distribution and per-branch caches, so the host needs both
|
||||
# raw free space and enough group-quota headroom.
|
||||
#
|
||||
# Run this ON the target host (the webspace), no root needed.
|
||||
#
|
||||
# Optional: the rootfs archive to size the disk/quota check against, e.g.
|
||||
# werkator-build-prerequisites.sh /path/to/werkator-buildenv-trixie.tar.zst
|
||||
# When omitted, the check runs against a conservative default footprint.
|
||||
#
|
||||
# Usage: werkator-build-prerequisites.sh [TARGET_DIR] [ROOTFS_ARCHIVE]
|
||||
#
|
||||
# TARGET_DIR is the directory the build workspace will live in (default: $HOME).
|
||||
# The check verifies it sits on the home filesystem and has enough free space.
|
||||
# ROOTFS_ARCHIVE, when given, is the rootfs archive that will be used there.
|
||||
#
|
||||
# Output is one PASS/FAIL line per check plus a final RESULT line, e.g.:
|
||||
# PASS: bwrap version: bubblewrap 0.8.0
|
||||
# PASS: build runs as root inside the namespace (uid 0)
|
||||
# PASS: uid_map maps root back to the unprivileged user (uid 120957)
|
||||
# PASS: read-only root bind is enforced
|
||||
# PASS: at least 5 GiB free space on the build working filesystem
|
||||
# FAIL: group quota headroom below the 5 GiB build footprint ...
|
||||
# RESULT: FAIL (4/5) — Werkator bubblewrap builds are not usable on this host.
|
||||
#
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
target_dir_arg="${1:-}"
|
||||
rootfs_arg="${2:-}"
|
||||
|
||||
die() { echo "ERROR: $*" >&2; exit 1; }
|
||||
|
||||
# Disk footprint a bwrap build needs headroom for, in 1K blocks: unpacked
|
||||
# rootfs (zstd expands roughly 3-4x), Gradle distribution + per-branch cache,
|
||||
# build output and artifacts. ~5 GiB.
|
||||
MIN_FREE_BLOCKS=$((5 * 1024 * 1024))
|
||||
|
||||
# Reference filesystem: the one the invoking user's home directory lives on.
|
||||
# Builds (repo clone, buildenv, caches) must run there — other mounts, such as
|
||||
# a slow mass-storage volume, are rejected.
|
||||
HOME_FS="$(df -Pk "$HOME" 2>/dev/null | awk 'NR==2 {print $1}')"
|
||||
|
||||
pass=0
|
||||
fail=0
|
||||
result() { # result PASS|FAIL "message"
|
||||
echo "$1: $2"
|
||||
if [ "$1" = "PASS" ]; then pass=$((pass+1)); else fail=$((fail+1)); fi
|
||||
}
|
||||
|
||||
command -v bwrap >/dev/null 2>&1 || die "bwrap is not installed on this host"
|
||||
|
||||
output="$(bwrap --unshare-user --unshare-pid --die-with-parent --uid 0 --gid 0 \
|
||||
--ro-bind / / --dev /dev --proc /proc --tmpfs /tmp \
|
||||
sh -c 'id -u && cat /proc/self/uid_map && (touch /usr/ro-test 2>&1 || true)' 2>&1)" ||
|
||||
die "bwrap invocation failed (no user namespace support?): $output"
|
||||
|
||||
# Signal 0: bwrap itself is usable (version as a visible marker).
|
||||
result PASS "bwrap version: $(bwrap --version 2>&1)"
|
||||
|
||||
# Signal 1: runs as root (uid 0) inside the namespace.
|
||||
first="$(printf '%s\n' "$output" | sed -n '1p')"
|
||||
if [ "$first" = "0" ]; then
|
||||
result PASS "build runs as root inside the namespace (uid 0)"
|
||||
else
|
||||
result FAIL "expected uid 0 inside the namespace, got: $first"
|
||||
fi
|
||||
|
||||
# Signal 2: uid_map maps root to the invoking unprivileged user.
|
||||
uid_line="$(printf '%s\n' "$output" | sed -n '2p')"
|
||||
self_uid="$(id -u)"
|
||||
if printf '%s\n' "$uid_line" | grep -E "^[[:space:]]*0[[:space:]]+${self_uid}[[:space:]]+1" >/dev/null; then
|
||||
result PASS "uid_map maps root back to the unprivileged user (uid $self_uid)"
|
||||
else
|
||||
result FAIL "expected uid_map '0 $self_uid 1', got: $uid_line"
|
||||
fi
|
||||
|
||||
# Signal 3: the read-only root bind is enforced (a write to /usr fails).
|
||||
if printf '%s\n' "$output" | grep -qi "read-only file system"; then
|
||||
result PASS "read-only root bind is enforced"
|
||||
else
|
||||
result FAIL "the read-only root bind did not reject a write to /usr"
|
||||
fi
|
||||
|
||||
# --- Disk / quota checks ------------------------------------------------
|
||||
|
||||
target_dir="${target_dir_arg:-$HOME}"
|
||||
target_dir="$(realpath -m "$target_dir")"
|
||||
min_gib=$((MIN_FREE_BLOCKS / 1024 / 1024))
|
||||
|
||||
if [ -n "$rootfs_arg" ] && [ ! -f "$rootfs_arg" ]; then
|
||||
echo "WARNING: rootfs archive not found: $rootfs_arg (continuing without it)"
|
||||
fi
|
||||
|
||||
target_fs="$(df -Pk "$target_dir" 2>/dev/null | awk 'NR==2 {print $1}')"
|
||||
df_output="$(df -Pk "$target_dir" 2>/dev/null | awk 'NR==2 {print int($4) " " $6}')"
|
||||
if [ -n "$df_output" ]; then
|
||||
avail_k="${df_output%% *}"
|
||||
mount="${df_output##* }"
|
||||
if [ -n "$HOME_FS" ] && [ "$target_fs" != "$HOME_FS" ]; then
|
||||
# An explicitly chosen foreign filesystem is allowed (e.g. for testing)
|
||||
# but flagged: builds there will be slow.
|
||||
echo "WARNING: target dir is on $target_fs (mounted at $mount), not the home filesystem ($HOME_FS) — builds will run on slower storage"
|
||||
fi
|
||||
if [ "${avail_k:-0}" -lt "$MIN_FREE_BLOCKS" ]; then
|
||||
result FAIL "less than ${min_gib} GiB free space on the build working filesystem ($mount)"
|
||||
else
|
||||
result PASS "at least ${min_gib} GiB free space on the build working filesystem ($mount, device $target_fs)"
|
||||
fi
|
||||
else
|
||||
echo "WARNING: could not measure free space on $target_dir — only the quota check below applies"
|
||||
fi
|
||||
|
||||
if quota_output="$(quota -g 2>/dev/null)" && [ -n "$quota_output" ]; then
|
||||
quota_ok=1
|
||||
quota_seen=0
|
||||
detail=""
|
||||
while read -r fs blocks quota_limit; do
|
||||
quota_seen=1
|
||||
# Only the quota of the target filesystem counts — other volumes may
|
||||
# legitimately be full or unquota'd without affecting the build.
|
||||
if [ -n "$target_fs" ] && [ "$(basename "$fs")" != "$(basename "$target_fs")" ] && [ "$fs" != "$target_fs" ]; then
|
||||
continue
|
||||
fi
|
||||
headroom=$((quota_limit - blocks))
|
||||
if [ "$headroom" -lt "$MIN_FREE_BLOCKS" ]; then
|
||||
quota_ok=0
|
||||
detail+=" $(basename "$fs"): $(awk -v b="$headroom" 'BEGIN{printf "%.1f", b/1024/1024}') GiB free of quota;"
|
||||
fi
|
||||
done < <(printf '%s\n' "$quota_output" | awk '
|
||||
NF==1 && $1 ~ /^\// { pending_fs=$1; next }
|
||||
$1 ~ /^\// && $2 ~ /^[0-9]+$/ { print $1, $2, $4; pending_fs=""; next }
|
||||
$1 ~ /^[0-9]+[*]?/ && pending_fs != "" { gsub(/\*/, "", $1); print pending_fs, $1, $3; pending_fs="" }')
|
||||
if [ "$quota_seen" -eq 0 ]; then
|
||||
echo "WARNING: quota tooling present but no group quota lines could be parsed — only free space was checked"
|
||||
elif [ "$quota_ok" -eq 1 ]; then
|
||||
result PASS "group quota headroom covers the ${min_gib} GiB build footprint"
|
||||
else
|
||||
result FAIL "group quota headroom below the ${min_gib} GiB build footprint (rootfs + Gradle cache); raise the quota before building.$detail"
|
||||
fi
|
||||
else
|
||||
echo "WARNING: no readable group quota tooling on this host — only free space was checked"
|
||||
fi
|
||||
|
||||
total=$((pass + fail))
|
||||
echo
|
||||
if [ "$fail" -eq 0 ]; then
|
||||
echo "RESULT: PASS ($pass/$total) — Werkator bubblewrap builds are usable on this host."
|
||||
echo "Next: install the Werkator instance with: tools/remote werkator install ${WERKATOR_SSH_TARGET:-<user>@<host>} '$target_dir'"
|
||||
exit 0
|
||||
else
|
||||
echo "RESULT: FAIL ($pass/$total) — Werkator bubblewrap builds are not usable on this host."
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user