From f8e8188fc618e44939bf0534e1d508542ebae2cd Mon Sep 17 00:00:00 2001 From: mhoennig Date: Tue, 1 Sep 2026 18:26:39 +0200 Subject: [PATCH 1/3] Plan step 23: init owns the files, tools/remote wraps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Decided 2026-09-01: Werkator becomes the executing app, the remote script a thin wrapper (build, transport, remote execution, service switching). Parameters travel as env files instead of many options — 'remote --env .env.mih34 werkator ...' selects the target, and 'werkator --env ... init' reads the same file and writes real values into the files it owns, idempotently; the wrapper's YAML heredocs and sed patches (the duplicate-block class) die. control-token becomes a werkator subcommand, check-prerequisites delegates to werkdock doctor. Explicitly not the removed legacy env-to-YAML conversion: the env file is setup-time input to init, never runtime configuration. Co-Authored-By: Claude Fable 5 --- docs/plan/23-init-owns-the-files.md | 68 +++++++++++++++++++++++++++++ docs/plan/README.md | 5 +++ 2 files changed, 73 insertions(+) create mode 100644 docs/plan/23-init-owns-the-files.md diff --git a/docs/plan/23-init-owns-the-files.md b/docs/plan/23-init-owns-the-files.md new file mode 100644 index 0000000..7c55a77 --- /dev/null +++ b/docs/plan/23-init-owns-the-files.md @@ -0,0 +1,68 @@ +# Step 23: Init Owns the Files, `tools/remote` Wraps + +Prerequisites: step 21 session D (the role-named `tools/remote`). +Read `README.md` first. + +## The Problem + +`werkator init` and `tools/remote` overlap: both write the machine config — init as a commented template, the script by appending heredoc blocks (`bwrap`, `server`) and patching values with `sed`. +The script re-implements configuration knowledge Werkator owns (YAML shape, indentation, key names), outside the three-places sync invariant — the indentation-mismatch of one append guard produced nine duplicate `bwrap` blocks on mih34 before it was found. +Smaller duplications of the same kind: the script re-implements control-token generation in bash (`ControlTokenService` owns it), and `check-prerequisites` still pipes the bash script whose generic half exists as `werkdock doctor`. + +## The Decision (2026-09-01) + +Werkator becomes the executing app wherever possible; `tools/remote` shrinks to a wrapper: build artifacts locally, transport them, execute Werkator/werkdock remotely, switch services. + +Parameters travel as an **env file**, not as many CLI options: + +```bash +tools/remote --env .env.mih34 werkator repo-init +werkator --env .env.mih34 init +``` + +- `tools/remote --env FILE` selects the target (default: `.env`), so several instances (`.env.mih34`, `.env.vm4006`, later a Werkbaum instance) are files, not edits. +- `werkator --env FILE` loads the same file; `init` reads the `WERKATOR_*` values from it (or from the process environment) and writes **real values** into the files it owns, instead of commented templates the script then patches. +- The wrapper uploads the env file alongside the artifacts and calls `werkator --env … init` remotely — the heredocs and `sed` calls in `tools/remote` disappear. + +Not a relapse into the removed legacy env-to-YAML conversion: the env file is an *input to init at setup time*, written once into the machine config — the server never reads `WERKATOR_*` at runtime, and the YAML stays the single source of truth. + +## Env Keys and Their Targets + +Consumed by `werkator init` (written into the machine config; re-runs update these managed values in place, schema-aware instead of `sed`): + +| Env key | Config key | +|---|---| +| `WERKATOR_PORT` | `server.port` | +| `WERKATOR_DOMAIN` | `server.publicBaseUrl` (`https:///`) | +| `WERKATOR_MEMORY_MAX` / `WERKATOR_TASKS_MAX` | `server.systemd.memoryMax` / `tasksMax` | +| `WERKATOR_ROOTFS` (remote path) | `builds.default.bwrap.rootfs` (+ `bwrap.enabled: true`) | +| `WERKATOR_WERKDOCK` | `builds.default.bwrap.werkdock` | + +Transport-only keys (`WERKATOR_REMOTE`, `WERKATOR_PATH`, `WERKATOR_LOCAL_PORT`, `WERKATOR_REPO_URL`) stay the wrapper's business; init ignores them, documented. +Secrets (`git.token`, `gitea` keys) stay out of env files on purpose — they are entered in the machine config on the host, as today. + +## The Sessions + +### A — Werkator side + +- Root-level picocli option `--env FILE`: loads `KEY=VALUE` lines into the command's parameter environment; unknown keys are ignored (they belong to the wrapper). +- `init` writes real values for the keys above when they are set — creating the sections when missing, updating the managed values when present, never duplicating (the duplication class dies here). +- `init --systemd` keeps generating the units; decide in the session whether the Apache `.htaccess` becomes part of the host-integration output when `WERKATOR_PORT`/`WERKATOR_DOMAIN` are set (proposal: yes, under `init --systemd`, since it is generated host integration exactly like the units). +- New subcommand `werkator control-token`: print the token, creating it exactly like `ControlTokenService` does — the bash duplication in the wrapper dies. +- Tests per the writing-tests conventions; `docs/configuration.md` and `docs/bootstrapping.md` document the env keys. + +### B — Wrapper side + +- `tools/remote --env FILE` (default `.env`); the file is uploaded and every remote `werkator` call gets `--env`. +- `repo-init` and `instance-start` lose their heredoc/`sed` config writing; `control-token` delegates to the new subcommand. +- `check-prerequisites` uploads the werkdock binary first and runs `werkdock doctor`; `tools/werkator-build-prerequisites.sh` retires (its werkdock port is the survivor). + +### C — Live verification and docs + +- Run the full wrapper flow against mih34 (`instance-update`, `repo-init`, `instance-start` as no-op re-runs); `docs/deployment.md`'s webspace section switches to the `--env` invocations. + +## Acceptance Criteria + +- Session A: `werkator --env … init` writes and updates the managed config values idempotently; `werkator control-token` exists; full suite green. +- Session B: `tools/remote` contains no YAML heredocs and no `sed` into the machine config; the prerequisites bash script is gone. +- Session C: the mih34 re-runs change nothing on a configured host and the deployment docs show only `--env`-style calls. diff --git a/docs/plan/README.md b/docs/plan/README.md index a6a0d00..0df0ed1 100644 --- a/docs/plan/README.md +++ b/docs/plan/README.md @@ -95,6 +95,10 @@ Added to correct the bwrap prototype's drift toward self-building on the webspac - [ ] `21-werkdock-extraction-and-webspace-install.md` — roadmap in four sessions: close step 17's open ends, grow the sandbox tooling into **Werkdock** (a docker-like filesystem-only sandbox CLI, developed in the `werkdock/` subdirectory, later its own repository), let Werkator consume it, and replace the webspace self-build with the local-build-plus-install path of ADR 0006 +Added after step 21 session D exposed that `tools/remote` re-implements configuration Werkator owns (2026-09-01): + +- [ ] `23-init-owns-the-files.md` — Werkator becomes the executing app, `tools/remote` a thin wrapper: parameters travel as env files (`remote --env .env.mih34 werkator …`, `werkator --env … init`), `init` writes real values idempotently instead of templates the script patches, `werkator control-token` and `werkdock doctor` replace the bash duplications + Added for surfacing build time as a trend (2026-08-31): - [ ] `20-build-duration-tracking.md` — a per-name duration trend over the existing history, derived on read in the History view: series, window average/min/max, and a visible marker when the latest build is slower than its window average (grouped by the history's own `name`, so branch builds and named jobs stay separate — complements Step 14, which owns phase timing) @@ -110,3 +114,4 @@ Step 18 depends on nothing in code but on the watched repository having migrated Step 19 depends on nothing; `WatcherState` and `/api/watcher` already carry everything it needs to render. Step 20 depends on nothing; the duration is already recorded, and the trend is derived read-only from `repository.history()`. Step 21 depends on 17; its sessions B and C grow Werkdock in the `werkdock/` subdirectory (later its own repository), and session D supersedes the self-build prototype in `tools/remote`. +Step 23 depends on 21 session D; its env-file convention (one file per instance) also feeds step 22's instance setup and should land before Werkbaum rolls out. From 7a24ad1d7f91b381b39a341cc33f6a84e73faf11 Mon Sep 17 00:00:00 2001 From: mhoennig Date: Tue, 1 Sep 2026 18:32:29 +0200 Subject: [PATCH 2/3] Step 23 refined: init takes a YAML fragment in the config schema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refinement decided 2026-09-01: each side gets its native format — the wrapper keeps a bash-sourceable transport env file, Werkator takes a YAML fragment in its own config schema via 'init --apply FILE', deep-merged idempotently. The env-to-config mapping table disappears entirely: the fragment IS configuration in the one schema, validated by the existing binding, documented by the existing reference. The env file names the fragment (WERKATOR_INIT_CONFIG), keeping one entry point per instance. Co-Authored-By: Claude Fable 5 --- docs/plan/23-init-owns-the-files.md | 45 ++++++++++------------------- docs/plan/README.md | 4 +-- 2 files changed, 18 insertions(+), 31 deletions(-) diff --git a/docs/plan/23-init-owns-the-files.md b/docs/plan/23-init-owns-the-files.md index 7c55a77..2a0fc17 100644 --- a/docs/plan/23-init-owns-the-files.md +++ b/docs/plan/23-init-owns-the-files.md @@ -13,47 +13,34 @@ Smaller duplications of the same kind: the script re-implements control-token ge Werkator becomes the executing app wherever possible; `tools/remote` shrinks to a wrapper: build artifacts locally, transport them, execute Werkator/werkdock remotely, switch services. -Parameters travel as an **env file**, not as many CLI options: +Parameters travel as **files**, not as many CLI options — and each side gets the format that is native to it (refined 2026-09-01): -```bash -tools/remote --env .env.mih34 werkator repo-init -werkator --env .env.mih34 init -``` +- The **wrapper** keeps a small, bash-sourceable env file with the transport values only: `tools/remote --env .env.mih34 werkator repo-init` selects the target (default: `.env`), so several instances (`.env.mih34`, `.env.vm4006`, later a Werkbaum instance) are files, not edits. +- **Werkator** takes a **YAML fragment in its own config schema**: `werkator init --apply mih34.yml` deep-merges the fragment into the machine config, idempotently — creating sections that are missing, updating the given values, never duplicating. + No mapping table exists: the fragment says `server: {port: …}` and `builds: {default: {bwrap: …}}` directly, is validated by the existing schema binding, and is documented by the existing `docs/configuration.md`. +- The wrapper uploads the fragment alongside the artifacts and calls `werkator init --apply …` remotely — the heredocs and `sed` calls in `tools/remote` disappear. +- The env file names the fragment (`WERKATOR_INIT_CONFIG=mih34.yml`), keeping one entry point per instance. -- `tools/remote --env FILE` selects the target (default: `.env`), so several instances (`.env.mih34`, `.env.vm4006`, later a Werkbaum instance) are files, not edits. -- `werkator --env FILE` loads the same file; `init` reads the `WERKATOR_*` values from it (or from the process environment) and writes **real values** into the files it owns, instead of commented templates the script then patches. -- The wrapper uploads the env file alongside the artifacts and calls `werkator --env … init` remotely — the heredocs and `sed` calls in `tools/remote` disappear. +The removed legacy env-to-YAML conversion stays removed — there is no conversion at all anymore: the fragment already *is* configuration in the one schema, applied once at setup time; the server reads nothing but its YAML at runtime. -Not a relapse into the removed legacy env-to-YAML conversion: the env file is an *input to init at setup time*, written once into the machine config — the server never reads `WERKATOR_*` at runtime, and the YAML stays the single source of truth. +## The Files per Instance -## Env Keys and Their Targets - -Consumed by `werkator init` (written into the machine config; re-runs update these managed values in place, schema-aware instead of `sed`): - -| Env key | Config key | -|---|---| -| `WERKATOR_PORT` | `server.port` | -| `WERKATOR_DOMAIN` | `server.publicBaseUrl` (`https:///`) | -| `WERKATOR_MEMORY_MAX` / `WERKATOR_TASKS_MAX` | `server.systemd.memoryMax` / `tasksMax` | -| `WERKATOR_ROOTFS` (remote path) | `builds.default.bwrap.rootfs` (+ `bwrap.enabled: true`) | -| `WERKATOR_WERKDOCK` | `builds.default.bwrap.werkdock` | - -Transport-only keys (`WERKATOR_REMOTE`, `WERKATOR_PATH`, `WERKATOR_LOCAL_PORT`, `WERKATOR_REPO_URL`) stay the wrapper's business; init ignores them, documented. -Secrets (`git.token`, `gitea` keys) stay out of env files on purpose — they are entered in the machine config on the host, as today. +- `.env.mih34` (wrapper): `WERKATOR_REMOTE`, `WERKATOR_PATH`, `WERKATOR_LOCAL_PORT`, `WERKATOR_REPO_URL`, `WERKATOR_ROOTFS` (the *local* archive to upload), `WERKATOR_INIT_CONFIG`. +- `mih34.yml` (init fragment): `server.*` (port, publicBaseUrl, systemd limits) and `builds.default.bwrap.*` (enabled, the *remote* rootfs path, werkdock path) — exactly the blocks the script used to append. +- Secrets (`git.token`, `gitea` keys) stay out of both files on purpose — they are entered in the machine config on the host, as today. ## The Sessions ### A — Werkator side -- Root-level picocli option `--env FILE`: loads `KEY=VALUE` lines into the command's parameter environment; unknown keys are ignored (they belong to the wrapper). -- `init` writes real values for the keys above when they are set — creating the sections when missing, updating the managed values when present, never duplicating (the duplication class dies here). -- `init --systemd` keeps generating the units; decide in the session whether the Apache `.htaccess` becomes part of the host-integration output when `WERKATOR_PORT`/`WERKATOR_DOMAIN` are set (proposal: yes, under `init --systemd`, since it is generated host integration exactly like the units). +- `init --apply FILE`: deep-merge the YAML fragment into the machine config — reusing the loader's merge, creating missing sections, updating given values, never duplicating (the duplication class dies here); a fragment that fails the schema binding or carries unknown keys is refused loudly. +- `init --systemd` keeps generating the units; decide in the session whether the Apache `.htaccess` becomes part of the host-integration output when the applied config carries `server.port` and a public domain (proposal: yes, under `init --systemd`, since it is generated host integration exactly like the units). - New subcommand `werkator control-token`: print the token, creating it exactly like `ControlTokenService` does — the bash duplication in the wrapper dies. -- Tests per the writing-tests conventions; `docs/configuration.md` and `docs/bootstrapping.md` document the env keys. +- Tests per the writing-tests conventions; `docs/bootstrapping.md` documents `--apply` (the fragment keys need no new reference — they are ordinary `docs/configuration.md` keys). ### B — Wrapper side -- `tools/remote --env FILE` (default `.env`); the file is uploaded and every remote `werkator` call gets `--env`. +- `tools/remote --env FILE` (default `.env`); the init fragment named by `WERKATOR_INIT_CONFIG` is uploaded, and the remote init runs with `--apply`. - `repo-init` and `instance-start` lose their heredoc/`sed` config writing; `control-token` delegates to the new subcommand. - `check-prerequisites` uploads the werkdock binary first and runs `werkdock doctor`; `tools/werkator-build-prerequisites.sh` retires (its werkdock port is the survivor). @@ -63,6 +50,6 @@ Secrets (`git.token`, `gitea` keys) stay out of env files on purpose — they ar ## Acceptance Criteria -- Session A: `werkator --env … init` writes and updates the managed config values idempotently; `werkator control-token` exists; full suite green. +- Session A: `werkator init --apply …` merges and re-merges a fragment idempotently; `werkator control-token` exists; full suite green. - Session B: `tools/remote` contains no YAML heredocs and no `sed` into the machine config; the prerequisites bash script is gone. - Session C: the mih34 re-runs change nothing on a configured host and the deployment docs show only `--env`-style calls. diff --git a/docs/plan/README.md b/docs/plan/README.md index 0df0ed1..4cfadd0 100644 --- a/docs/plan/README.md +++ b/docs/plan/README.md @@ -97,7 +97,7 @@ Added to correct the bwrap prototype's drift toward self-building on the webspac Added after step 21 session D exposed that `tools/remote` re-implements configuration Werkator owns (2026-09-01): -- [ ] `23-init-owns-the-files.md` — Werkator becomes the executing app, `tools/remote` a thin wrapper: parameters travel as env files (`remote --env .env.mih34 werkator …`, `werkator --env … init`), `init` writes real values idempotently instead of templates the script patches, `werkator control-token` and `werkdock doctor` replace the bash duplications +- [ ] `23-init-owns-the-files.md` — Werkator becomes the executing app, `tools/remote` a thin wrapper: the wrapper takes a transport env file (`remote --env .env.mih34 werkator …`), init takes a YAML fragment in the real config schema (`werkator init --apply mih34.yml`, deep-merged idempotently — no mapping table, no heredocs), `werkator control-token` and `werkdock doctor` replace the bash duplications Added for surfacing build time as a trend (2026-08-31): @@ -114,4 +114,4 @@ Step 18 depends on nothing in code but on the watched repository having migrated Step 19 depends on nothing; `WatcherState` and `/api/watcher` already carry everything it needs to render. Step 20 depends on nothing; the duration is already recorded, and the trend is derived read-only from `repository.history()`. Step 21 depends on 17; its sessions B and C grow Werkdock in the `werkdock/` subdirectory (later its own repository), and session D supersedes the self-build prototype in `tools/remote`. -Step 23 depends on 21 session D; its env-file convention (one file per instance) also feeds step 22's instance setup and should land before Werkbaum rolls out. +Step 23 depends on 21 session D; its per-instance file convention (transport env + init fragment) also feeds step 22's instance setup and should land before Werkbaum rolls out. From bc358900d77753f5f89b7c53dcdda33ea1605338 Mon Sep 17 00:00:00 2001 From: mhoennig Date: Wed, 2 Sep 2026 07:11:26 +0200 Subject: [PATCH 3/3] PR-doc for PR #8: step 23 plan, init owns the files Co-Authored-By: Claude Sonnet 5 --- ...2-PR#8-step-23-plan-init-owns-the-files.md | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 docs/prs/2026-09-02-PR#8-step-23-plan-init-owns-the-files.md diff --git a/docs/prs/2026-09-02-PR#8-step-23-plan-init-owns-the-files.md b/docs/prs/2026-09-02-PR#8-step-23-plan-init-owns-the-files.md new file mode 100644 index 0000000..b407b55 --- /dev/null +++ b/docs/prs/2026-09-02-PR#8-step-23-plan-init-owns-the-files.md @@ -0,0 +1,29 @@ +> **WARNING:** This document describes only the change applied in this PR. +> It may already be outdated once the next PR is merged. +> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation. + +## The Problem + +`werkator init` and `tools/remote` overlap: both write the machine config — init as a commented template, the script by appending heredoc blocks and patching values with `sed`. +The script re-implements configuration knowledge Werkator already owns (YAML shape, indentation, key names) outside the three-places sync invariant; an indentation mismatch in one append guard produced nine duplicate `bwrap` blocks on mih34 (step 21 session D) before it was found. +Smaller duplications of the same kind: the script re-implements control-token generation in bash, and `check-prerequisites` still pipes a bash script whose generic half now exists as `werkdock doctor`. + +## Non-Goals + +- Implementing the change — this PR is the plan only; PR #9 implements it. +- Multi-repository support for one Werkator instance (step 22, PR #10). + +## The Solution + +`docs/plan/23-init-owns-the-files.md` records the decision: Werkator becomes the executing app wherever possible, `tools/remote` shrinks to a wrapper. +Parameters travel as files, each side getting the format native to it: the wrapper keeps a small env file with transport-only values (`--env-file FILE`, mirroring Docker's flag naming since `--env` there means a single variable); Werkator takes a YAML fragment in its own config schema, applied via a new `init --apply FILE`, validated by the existing schema binding and needing no separate mapping table. +The plan was refined once during review: the first sketch proposed an env-file-only transport; the fragment being a first-class YAML file in Werkator's own schema replaced that, so there is no env-key-to-config-key conversion table to maintain at all. +Three sessions are laid out: A (Werkator side: `init --apply`, `control-token` subcommand), B (wrapper side: `tools/remote` loses its heredocs), C (live verification on mih34 and doc updates). + +## Prerequisite PRs + +- PR #7 (webspace install path) — this plan corrects the remaining duplication that PR left in place. + +## Follow-up PRs + +- PR #9: implements sessions A, B, and C of this plan.