Merge pull request #9 from mhoennig/init-apply-config
Step 23 sessions A-C: init --apply, applied instance fragment, tools/remote wraps
This commit is contained in:
mhoennig
@@ -42,7 +42,7 @@ Good:
|
||||
|
||||
Bad:
|
||||
|
||||
- The rootfs archive is project infrastructure that must be built and uploaded (~1.5 GB unpacked, disk/quota checked by `tools/werkator-build-prerequisites.sh`).
|
||||
- The rootfs archive is project infrastructure that must be built and uploaded (~1.5 GB unpacked, disk/quota checked by `werkdock doctor`, which ported the original prerequisites script).
|
||||
- Filesystem-only isolation: network and process view are the host's — acceptable here, and pinned so a branch cannot widen it, but weaker than Docker.
|
||||
- Mountpoint pre-creation and mount ordering are subtle (hardened on the real webspace; see the fix messages preserved in commit `71f1fc6`).
|
||||
|
||||
|
||||
@@ -86,6 +86,12 @@ gitea:
|
||||
|
||||
Then, you have to configure *Werkator* by amending this config file according to [configuration.md](configuration.md).
|
||||
|
||||
### 5. Optionally Install an Instance Fragment (`--apply`)
|
||||
|
||||
`init --apply FILE` installs a YAML fragment in the configuration schema as the applied instance layer — see [configuration.md](configuration.md#the-applied-instance-fragment-init---apply).
|
||||
Deployment tooling hands its parameters over this way instead of patching config files; the fragment is validated strictly and replaced wholesale on re-apply.
|
||||
It runs before `--systemd`, so an applied `server.port` reaches the generated unit and the Apache `.htaccess` (written beside the units when a `publicBaseUrl` is configured).
|
||||
|
||||
## Output
|
||||
|
||||
`init` prints one line per action taken:
|
||||
|
||||
@@ -7,10 +7,17 @@ Werkator is configured via YAML files. Settings are merged from several sources
|
||||
| Layer | Path | Committed to Git | Purpose |
|
||||
|--------------------------|----------------------------|------------------|----------------------------------------------|
|
||||
| Project config | `.werkator.yml` | Yes | Shared team settings |
|
||||
| Applied instance fragment | `.git/werkator/.werkator.applied.yml` | No | Instance parameters installed by `init --apply` |
|
||||
| Repo installation config | `.git/werkator/.werkator.yml` | No | Machine- or user-specific overrides, secrets |
|
||||
| Branch config | `.werkator.yml` committed on a branch | Yes | That branch's build settings and build definitions |
|
||||
|
||||
The repo install config (`.git/werkator/.werkator.yml`) wins on any key present in both files. Typically used to set `git.token` and `git.account` without committing them.
|
||||
The repo install config (`.git/werkator/.werkator.yml`) wins on any key present in several files; the applied fragment wins over the project config. Typically the repo install config sets `git.token` and `git.account` without committing them.
|
||||
|
||||
### The applied instance fragment (`init --apply`)
|
||||
|
||||
`werkator init --apply FILE` installs a YAML fragment in this very schema as its own layer (step 23) — the file a deployment wrapper hands over instead of patching configs.
|
||||
The fragment is validated strictly before installing: an unknown key is refused loudly, never ignored, because a typo would otherwise install a value that silently does nothing.
|
||||
It is then copied verbatim (comments included) to `.git/werkator/.werkator.applied.yml`; re-applying replaces the file, so nothing accumulates or duplicates, and the hand-edited repo install config — which always wins — is never rewritten.
|
||||
|
||||
### Which Werkator a file is written for
|
||||
|
||||
@@ -414,7 +421,7 @@ All Werkator containers carry `org.hoennig.werkator` labels; stale build contain
|
||||
With `bwrap.enabled`, Werkator runs the build in a bubblewrap sandbox instead of native execution.
|
||||
This is the third runtime, for hosts without root and without a Docker daemon (e.g. Hostsharing managed webspaces); see `docs/plan/17-bwrap-build-runtime.md` and ADR 0008.
|
||||
Since step 21 session C the sandbox is executed by the `werkdock` CLI (`bwrap.werkdock`, default: resolved via `PATH`) — Werkator no longer invokes `bwrap` itself; `bwrap` must be installed for werkdock.
|
||||
`werkdock doctor` checks the host's capability, superseding `tools/werkator-build-prerequisites.sh`.
|
||||
`werkdock doctor` checks the host's capability (it replaced the retired `tools/werkator-build-prerequisites.sh` in step 23).
|
||||
|
||||
`bwrap.rootfs` names the prepared root filesystem archive — a Debian-base rootfs with the build tools (JDK, git, locales, project-specific tooling) built elsewhere, since `debootstrap` is unavailable on the target.
|
||||
It is a local path or an `http(s)` URL; a URL is downloaded once into `.git/werkator/buildenv/`.
|
||||
|
||||
+9
-7
@@ -268,14 +268,16 @@ The third deployment variant (plan step 21, verified live on a real webspace): n
|
||||
Werkator runs as a systemd *user* service on the assigned localhost port ("eigener Serverdienst"), the platform's managed Apache terminates TLS and proxies via `.htaccess`, and builds run in the bubblewrap sandbox executed by the [werkdock](../werkdock/README.md) CLI (ADR 0008, step 21 session C).
|
||||
|
||||
Werkator is never built on the webspace: the runtime bundle and the werkdock binary are built locally and uploaded (ADR 0006).
|
||||
All steps are driven by `tools/remote`, configured through the `.env` file in the repository root; commands name their role — `instance-*` manages the installed Werkator, `repo-*` the repository it watches.
|
||||
All steps are driven by `tools/remote`; commands name their role — `instance-*` manages the installed Werkator, `repo-*` the repository it watches.
|
||||
Each instance is a pair of files (step 23): a transport env file selected with `--env-file` (default `.env`), and a YAML fragment in the configuration schema, named by its `WERKATOR_INIT_CONFIG` key and installed remotely via `werkator init --apply` — e.g. `.env.mih34` + `.env.mih34.yml`, both gitignored.
|
||||
The fragment carries the Werkator configuration (`server.port`, `publicBaseUrl`, systemd limits, `builds.default.bwrap.*`); the env file only says where and how to reach the host.
|
||||
|
||||
```bash
|
||||
tools/remote werkator check-prerequisites # bwrap capability, disk and quota headroom
|
||||
tools/remote werkator instance-install # upload + unpack the runtime bundle and werkdock
|
||||
tools/remote werkator repo-init # clone the watched repo, init, rootfs archive, bwrap config
|
||||
tools/remote werkator instance-start # server config, Apache proxy, systemd user unit
|
||||
tools/remote port-forward start # browser tunnel while no public domain is set up
|
||||
tools/remote --env-file .env.mih34 werkator check-prerequisites # uploads werkdock, runs its doctor
|
||||
tools/remote --env-file .env.mih34 werkator instance-install # upload + unpack the runtime bundle and werkdock
|
||||
tools/remote --env-file .env.mih34 werkator repo-init # clone the watched repo, rootfs archive, init --apply
|
||||
tools/remote --env-file .env.mih34 werkator instance-start # init --apply --systemd, place .htaccess, enable the unit
|
||||
tools/remote --env-file .env.mih34 port-forward start # browser tunnel while no public domain is set up
|
||||
```
|
||||
|
||||
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
|
||||
@@ -285,7 +287,7 @@ Fill `git.account`/`git.token` in the machine config when the origin is private,
|
||||
Updates are one command, refused while a build runs (`FORCE=1` overrides):
|
||||
|
||||
```bash
|
||||
tools/remote werkator instance-update
|
||||
tools/remote --env-file .env.mih34 werkator instance-update
|
||||
```
|
||||
|
||||
The previous runtime stays as `.werkator/werkator.prev` for one deployment as the rollback asset.
|
||||
|
||||
@@ -15,7 +15,7 @@ Werkator becomes the executing app wherever possible; `tools/remote` shrinks to
|
||||
|
||||
Parameters travel as **files**, not as many CLI options — and each side gets the format that is native to it (refined 2026-09-01):
|
||||
|
||||
- The **wrapper** keeps a small, bash-sourceable env file with the transport values only: `tools/remote --env .env.mih34 werkator repo-init` selects the target (default: `.env`), so several instances (`.env.mih34`, `.env.vm4006`, later a Werkbaum instance) are files, not edits.
|
||||
- The **wrapper** keeps a small, bash-sourceable env file with the transport values only: `tools/remote --env-file .env.mih34 werkator repo-init` selects the target (default: `.env`; named `--env-file` like docker's flag for the same thing, since `--env` means a single variable there), so several instances (`.env.mih34`, `.env.vm4006`, later a Werkbaum instance) are files, not edits.
|
||||
- **Werkator** takes a **YAML fragment in its own config schema**: `werkator init --apply mih34.yml` deep-merges the fragment into the machine config, idempotently — creating sections that are missing, updating the given values, never duplicating.
|
||||
No mapping table exists: the fragment says `server: {port: …}` and `builds: {default: {bwrap: …}}` directly, is validated by the existing schema binding, and is documented by the existing `docs/configuration.md`.
|
||||
- The wrapper uploads the fragment alongside the artifacts and calls `werkator init --apply …` remotely — the heredocs and `sed` calls in `tools/remote` disappear.
|
||||
@@ -31,25 +31,28 @@ The removed legacy env-to-YAML conversion stays removed — there is no conversi
|
||||
|
||||
## The Sessions
|
||||
|
||||
### A — Werkator side
|
||||
### A — Werkator side (implemented 2026-09-01 on branch `init-apply-config`)
|
||||
|
||||
- `init --apply FILE`: deep-merge the YAML fragment into the machine config — reusing the loader's merge, creating missing sections, updating given values, never duplicating (the duplication class dies here); a fragment that fails the schema binding or carries unknown keys is refused loudly.
|
||||
- `init --systemd` keeps generating the units; decide in the session whether the Apache `.htaccess` becomes part of the host-integration output when the applied config carries `server.port` and a public domain (proposal: yes, under `init --systemd`, since it is generated host integration exactly like the units).
|
||||
- New subcommand `werkator control-token`: print the token, creating it exactly like `ControlTokenService` does — the bash duplication in the wrapper dies.
|
||||
- Tests per the writing-tests conventions; `docs/bootstrapping.md` documents `--apply` (the fragment keys need no new reference — they are ordinary `docs/configuration.md` keys).
|
||||
|
||||
### B — Wrapper side
|
||||
### B — Wrapper side (implemented 2026-09-01 on branch `init-apply-config`)
|
||||
|
||||
- `tools/remote --env FILE` (default `.env`); the init fragment named by `WERKATOR_INIT_CONFIG` is uploaded, and the remote init runs with `--apply`.
|
||||
- `tools/remote --env-file FILE` (default `.env`); the init fragment named by `WERKATOR_INIT_CONFIG` is uploaded, and the remote init runs with `--apply`.
|
||||
- `repo-init` and `instance-start` lose their heredoc/`sed` config writing; `control-token` delegates to the new subcommand.
|
||||
- `check-prerequisites` uploads the werkdock binary first and runs `werkdock doctor`; `tools/werkator-build-prerequisites.sh` retires (its werkdock port is the survivor).
|
||||
|
||||
### C — Live verification and docs
|
||||
### C — Live verification and docs (done 2026-09-01)
|
||||
|
||||
- Run the full wrapper flow against mih34 (`instance-update`, `repo-init`, `instance-start` as no-op re-runs); `docs/deployment.md`'s webspace section switches to the `--env` invocations.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Session A: `werkator init --apply …` merges and re-merges a fragment idempotently; `werkator control-token` exists; full suite green.
|
||||
- Session B: `tools/remote` contains no YAML heredocs and no `sed` into the machine config; the prerequisites bash script is gone.
|
||||
- Session C: the mih34 re-runs change nothing on a configured host and the deployment docs show only `--env`-style calls.
|
||||
- Session A: done 2026-09-01 — `werkator init --apply …` installs and replaces a fragment idempotently; `werkator control-token` exists; full suite green.
|
||||
Deviation from the sketch above: the fragment is NOT merged into the machine config — it is installed verbatim as its own layer (`.git/werkator/.werkator.applied.yml`, above project, below machine config), because an in-place merge would re-serialize the machine config, destroying its comments and rewriting the file that holds the secrets; a verbatim copy also makes re-apply a plain file replacement.
|
||||
The `.htaccess` decision fell as proposed: generated beside the units by `init --systemd` whenever a `publicBaseUrl` is configured; the wrapper copies it into the domain docroot.
|
||||
- Session B: done 2026-09-01 — `tools/remote` contains no YAML heredocs and no `sed` into the machine config (the port lookups for idle check and port-forward read the *effective* config via `config:print`, so a port living in the applied fragment is found too); the prerequisites bash script is gone.
|
||||
- Session C: done 2026-09-01 — verified live on mih34 with the `.env.mih34` + `.env.mih34.yml` pair: instance-update, doctor-based check-prerequisites (PASS 6/6), repo-init applying the fragment, instance-start placing the generated `.htaccess` and restarting the unit, control-token via the CLI; `docs/deployment.md` shows only `--env-file`-style calls.
|
||||
Known niggle: validating a fragment that carries a `builds.default` without triggers logs the loader's "no build defines onPush" warning, although a fragment is judged out of context — cosmetic, fix when it annoys.
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
> **WARNING:** This document describes only the change applied in this PR.
|
||||
> It may already be outdated once the next PR is merged.
|
||||
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
||||
|
||||
## The Problem
|
||||
|
||||
`tools/remote` wrote the machine config by appending heredoc blocks and patching values with `sed`, duplicating configuration knowledge (YAML shape, indentation, key names) that `WerkatorConfig` already owns — an indentation mismatch in one append guard produced nine duplicate `bwrap` blocks on mih34 before it was found (step 21 session D).
|
||||
The script also re-implemented control-token generation in bash and still piped a prerequisites bash script whose generic half now exists as `werkdock doctor`.
|
||||
PR #8 laid out the fix; this PR implements it.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Multi-repository support for one Werkator instance (step 22, PR #10).
|
||||
- A mapping table between env keys and config keys — deliberately absent, see The Solution.
|
||||
|
||||
## The Scenarios
|
||||
|
||||
### Feature: `werkator init --apply` installs an instance config fragment
|
||||
|
||||
#### Background
|
||||
|
||||
- The applied fragment is a fourth config layer: project `.werkator.yml` → applied fragment → repo-install machine config (secrets, always wins) → branch layer.
|
||||
- It is strictly validated (unknown keys rejected) and installed verbatim, never merged in place — an in-place merge would re-serialize and destroy the machine config's comments and secrets.
|
||||
|
||||
#### Scenario#9.01: A valid fragment becomes its own layer, above the project config and below the machine config
|
||||
|
||||
So that an instance-specific setting (e.g. `server.port`) takes effect without touching the committed project config or the machine config's secrets.
|
||||
|
||||
- **Given** a project `.werkator.yml` and a machine config with a secret
|
||||
- **When** `init --apply FILE` installs a fragment that also sets a key the machine config sets
|
||||
- **Then** the effective config shows the fragment's value where the machine config is silent, and the machine config's value where both set the same key.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [the applied instance fragment layers above the project config and below the machine config](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
- [--apply installs the fragment as the applied layer and the effective config sees it](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
|
||||
|
||||
#### Scenario#9.02: Re-applying a fragment replaces it, never duplicates it
|
||||
|
||||
So that repeated `instance-update` runs stay idempotent — the duplication class PR #8 was written against dies here structurally.
|
||||
|
||||
- **Given** a fragment already installed as the applied layer
|
||||
- **When** `init --apply` runs again with a changed fragment
|
||||
- **Then** the applied layer file is atomically replaced, not appended to.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [applyInstanceFragment installs a valid fragment verbatim, and re-applying replaces it](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
|
||||
#### Scenario#9.03: An invalid fragment is refused loudly and installs nothing
|
||||
|
||||
So that a typo in a fragment never becomes a silent no-op or a half-applied layer.
|
||||
|
||||
- **Given** a fragment with an unknown key, or a missing/empty file
|
||||
- **When** `init --apply FILE` runs
|
||||
- **Then** it fails loudly and the applied layer is left exactly as it was before the attempt.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [applyInstanceFragment refuses an unknown key loudly instead of installing a silent no-op](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
- [applyInstanceFragment refuses a missing or empty fragment](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
- [--apply with an invalid fragment installs nothing](../../src/test/kotlin/de/hoennig/werkator/commands/InitCommandTest.kt)
|
||||
|
||||
#### Scenario#9.04: `init --systemd` generates the Apache reverse-proxy file alongside the units
|
||||
|
||||
So that host integration for a public domain is generated exactly like the systemd units, not hand-written.
|
||||
|
||||
- **Given** an effective config with `server.publicBaseUrl` set
|
||||
- **When** `init --systemd` runs
|
||||
- **Then** `werkator.htaccess` is generated proxying to the configured localhost port, for the wrapper to place in the domain docroot.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [the htaccess proxies everything to the configured localhost port](../../src/test/kotlin/de/hoennig/werkator/commands/SystemdServiceFilesTest.kt)
|
||||
|
||||
#### Scenario#9.05: `werkator control-token` prints the same token the server would create
|
||||
|
||||
So that the wrapper's bash re-implementation of token generation is no longer needed.
|
||||
|
||||
- **Given** a repository with or without an existing control token
|
||||
- **When** `werkator control-token` runs
|
||||
- **Then** it creates the token exactly like `ControlTokenService` would and prints the same value on a re-run
|
||||
- **and** it fails with exit code 2 outside a repository.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [creates the token like the server would and prints the same one on a re-run](../../src/test/kotlin/de/hoennig/werkator/commands/ControlTokenCommandTest.kt)
|
||||
- [fails with exit code 2 outside a repository](../../src/test/kotlin/de/hoennig/werkator/commands/ControlTokenCommandTest.kt)
|
||||
|
||||
### Feature: `tools/remote` becomes a thin wrapper
|
||||
|
||||
#### Scenario#9.06: The wrapper selects its instance by file, not by many options
|
||||
|
||||
So that several instances (mih34, vm4006, a future Werkbaum instance) are files, not edits to one script.
|
||||
|
||||
- **Given** a transport env file (default `.env`) naming a config fragment via `WERKATOR_INIT_CONFIG`
|
||||
- **When** `tools/remote --env-file .env.mih34 werkator repo-init` runs
|
||||
- **Then** the fragment is uploaded and `werkator init --apply` is invoked remotely with it — no heredoc or `sed` writes the machine config.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- live run on mih34 with `.env.mih34` + `.env.mih34.yml`: `instance-update`, `repo-init`, `instance-start` as idempotent re-runs (shell script; no automated test harness for `tools/remote`)
|
||||
|
||||
#### Scenario#9.07: `check-prerequisites` delegates to `werkdock doctor`
|
||||
|
||||
So that the generic host-readiness checks are not duplicated between a bash script and Werkdock's own porting of it.
|
||||
|
||||
- **Given** a target host
|
||||
- **When** `tools/remote werkator check-prerequisites` runs
|
||||
- **Then** it uploads the `werkdock` binary and runs `werkdock doctor`, and `tools/werkator-build-prerequisites.sh` is deleted.
|
||||
|
||||
##### Verified by
|
||||
|
||||
- live run on mih34: doctor-based check-prerequisites, PASS 6/6
|
||||
|
||||
## The Solution
|
||||
|
||||
Session A (Werkator): `ConfigLoader` reads a new `ConfigFiles.APPLIED` path (`.git/werkator/.werkator.applied.yml`) as a layer between project and repo-install config; `applyInstanceFragment(workingDir, fragment)` validates the fragment against a strict Jackson mapper (`FAIL_ON_UNKNOWN_PROPERTIES=true`, fragment validation only — the regular mapper stays lenient for forward-compatibility) and then copies it verbatim via an atomic move, never merging in place.
|
||||
`InitCommand` gained `--apply FILE`, applied before `--systemd` handling so generated units/htaccess see the fragment; `SystemdServiceFiles.htaccessContent(port)` is the new generated file, written when `server.publicBaseUrl` is non-blank.
|
||||
`ControlTokenCommand` is a new subcommand delegating to the existing `ControlTokenService`.
|
||||
Session B (wrapper): `tools/remote --env-file FILE` (default `.env`) replaced positional/flag-heavy invocation; `repo-init`/`instance-start` lost their heredoc/`sed` config writing in favor of uploading the named fragment and calling `werkator init --apply`; port lookups (`require_idle`, `port_forward`) now parse `werkator config:print` output instead of grepping the machine config file directly, so a port living in the applied fragment is found too; `tools/werkator-build-prerequisites.sh` is deleted.
|
||||
Session C: verified live end-to-end on mih34 with a `.env.mih34` + `.env.mih34.yml` pair.
|
||||
|
||||
Deviation from the PR #8 plan: the fragment is not deep-merged into the machine config as first sketched — it is installed as its own verbatim layer, because an in-place merge would re-serialize the machine config and destroy its comments and secrets; a verbatim copy also makes re-apply a plain file replacement instead of a merge algorithm.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Applying a fragment that carries `builds.default` without triggers logs the loader's "no build defines onPush" warning, even though a fragment is judged out of context — cosmetic, fix when it annoys.
|
||||
|
||||
## Additional Changes
|
||||
|
||||
- `docs/configuration.md`: the layer table now shows four layers; a new subsection documents the applied instance fragment.
|
||||
- `docs/bootstrapping.md`: a new section documents `--apply`.
|
||||
- `docs/deployment.md`: the webspace section shows only `--env-file`-style invocations.
|
||||
- `.gitignore`: `/.env.*` added for per-instance env files.
|
||||
|
||||
## Prerequisite PRs
|
||||
|
||||
- PR #7 (webspace install path) — the role-separated `tools/remote` this PR simplifies.
|
||||
- PR #8 (step 23 plan) — the decision this PR implements.
|
||||
|
||||
## Follow-up PRs
|
||||
|
||||
- PR #10: multi-repository support for one Werkator instance (step 22).
|
||||
Reference in New Issue
Block a user