Merge main into the deflake-maxconcurrent-test branch
This commit is contained in:
mhoennig
@@ -0,0 +1,98 @@
|
||||
> **WARNING:** This document describes only the change applied in this PR.
|
||||
> It may already be outdated once the next PR is merged.
|
||||
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
||||
|
||||
## The Problem
|
||||
|
||||
`tools/remote` grew with the Hostsharing Managed Webspace rollout (steps 21 and 23) and encoded that rollout's layout as if it were the only one.
|
||||
The watched repository had to be `$WERKATOR_PATH/werkator`, the runtime `$WERKATOR_PATH/.werkator/werkator`, and the systemd unit was spelled out as `werkator-werkator.service`; a werkdock binary and a bwrap rootfs archive were built and uploaded unconditionally.
|
||||
`vm4006`, the Docker host that has been running Werkator since long before the script existed, matches none of that: its watched repository is `~/hs.hsadmin.ng` (so its unit is `werkator-hs.hsadmin.ng.service`), its runtime lives in `~/opt/werkator`, and it has neither werkdock nor a rootfs because its builds run in Docker.
|
||||
So that host could only be deployed by hand, and had drifted nine releases behind — the question that started this PR was whether the Werkdock and multi-repo work had broken it, which it had not: what was broken was the deployment tooling.
|
||||
|
||||
The second problem surfaced while deploying: `instance-update` stopped the systemd unit and *then* started the upload.
|
||||
The 66 MB transfer to `vm4006` died with `scp: Connection closed`, leaving the host with no running Werkator and nothing new to start.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- `instance-start` for a Docker host: it places a Hostsharing `.htaccess` into a `doms/<domain>/` docroot, which only exists on a Managed Webspace. `vm4006` uses the managed nginx container instead, and keeps its existing units.
|
||||
- `repo-init` for `vm4006` — the repository has been cloned and configured there for months; only `instance-update` was needed.
|
||||
- Making the machine configuration of `vm4006` current: it still carries the pre-rename `gitTally:` meta key, which is read by nothing today. Harmless while `ConfigVersions.FORMAT_BROKE_IN` is empty, a trap on the day it is not.
|
||||
|
||||
## The Scenarios
|
||||
|
||||
### Feature: one deployment command for every host layout
|
||||
|
||||
#### Background
|
||||
|
||||
- The env file carries transport values only; the three new keys describe *where* things are on the host, not what Werkator does.
|
||||
- The defaults are exactly the layout `instance-install` creates, so an env file naming none of them resolves as before.
|
||||
|
||||
#### Scenario#18.01: A host that predates the script can be deployed with it
|
||||
|
||||
So that an installation is not condemned to hand-typed `scp` sequences because it was set up before the tooling existed.
|
||||
|
||||
- **Given** an env file with `WERKATOR_REPO_DIR=hs.hsadmin.ng`, `WERKATOR_INSTALL_DIR=/home/tallyman/opt` and `WERKATOR_SANDBOX=docker`
|
||||
- **When** `tools/remote --env-file .env.vm4006 werkator instance-update` runs
|
||||
- **Then** it addresses `werkator-hs.hsadmin.ng.service`, unpacks into `~/opt`, and uploads neither a werkdock binary nor a rootfs archive
|
||||
|
||||
##### Verified by
|
||||
|
||||
- Live on `vm4006` (2026-09-03): `check-prerequisites` reported the Docker daemon instead of running `werkdock doctor`; the update swapped `~/opt/werkator` from v1.0.1 to **v1.1.2**, the unit came up `active`, `/api/watcher` polls without errors, and `/api/system` still reports real disk figures (`diskSource.kind: volume`) rather than the quota shape from PR#16.
|
||||
|
||||
#### Scenario#18.02: The webspace hosts are unaffected
|
||||
|
||||
So that making the script layout-aware does not break the deployment path that is actually in production.
|
||||
|
||||
- **Given** `.env.mih09` and `.env.mih34`, neither naming any of the new keys
|
||||
- **When** the layout is resolved
|
||||
- **Then** repository directory, install directory, unit name and sandbox are identical to the hardcoded values they replace
|
||||
|
||||
##### Verified by
|
||||
|
||||
- Resolution measured for both env files (2026-09-03): `REPO_DIR=$WERKATOR_PATH/werkator`, `INSTALL_DIR=$WERKATOR_PATH/.werkator`, `UNIT=werkator-werkator.service`, `SANDBOX=bwrap`.
|
||||
- Live on `mih09` (2026-09-03): a full `instance-update` ran through the bwrap path — werkdock uploaded, `werkdock 0.1.0-dev` reported after the swap, service `active`, `https://werkator.javagil.de/` answering 200.
|
||||
|
||||
#### Scenario#18.03: A failed transfer does not take the service down
|
||||
|
||||
So that a dropped connection costs the upload and nothing else.
|
||||
|
||||
- **Given** an instance whose service is running
|
||||
- **When** the runtime bundle cannot be transferred
|
||||
- **Then** the service is still running, because the upload happens before the stop, and a partially transferred file is never moved into place
|
||||
|
||||
##### Verified by
|
||||
|
||||
- The failure itself on `vm4006` (2026-09-03), which is what this scenario is written from: with the old order, `scp: Connection closed` left the unit stopped and the host without Werkator.
|
||||
- Live on `mih09` (2026-09-03): the bundle already on the host was recognised by its sha256 and skipped, so the upload step cost nothing and the stop followed only after it.
|
||||
|
||||
## The Solution
|
||||
|
||||
Three optional env keys replace three hardcoded assumptions.
|
||||
`WERKATOR_REPO_DIR` and `WERKATOR_INSTALL_DIR` are resolved absolute-or-relative-to-`WERKATOR_PATH`, and the unit name is now *derived* from the repository directory the way `SystemdServiceFiles.unitName` derives it (basename, every character outside `[A-Za-z0-9_.-]` replaced by a dash) instead of being spelled out — one rule, in two places, with the script naming the Kotlin function it mirrors.
|
||||
`WERKATOR_SANDBOX=docker` skips everything bwrap-shaped: no werkdock build, no werkdock upload, no rootfs archive, and `check-prerequisites` asks the Docker daemon instead of running `werkdock doctor`.
|
||||
`repo-add` clones beside the watched repository rather than into `WERKATOR_PATH`, which is the same directory whenever the default layout is used.
|
||||
|
||||
`deploy_instance` is split into `upload_instance_artifacts` and `swap_instance_runtime`, and `instance_update` calls the first *before* stopping the unit.
|
||||
Every artifact is uploaded to `<name>.part`, compared by sha256 with the local file, and only then moved into place; three attempts, and an unchanged artifact is skipped entirely.
|
||||
The checksum is not belt-and-braces: a truncated tarball would unpack into a runtime that starts and misbehaves, which is far worse than the failed transfer it came from.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **Should `instance-start` learn the Docker-host shape too?** Not answered here. `vm4006` keeps its existing units and its managed nginx; the day it needs regenerating, `init --systemd` on the host is the documented path.
|
||||
|
||||
## Additional Changes
|
||||
|
||||
- None beyond the feature itself.
|
||||
|
||||
## Follow-up work discovered while deploying this
|
||||
|
||||
- `https://vm4006.hostsharing.net:8443/` is not reachable from outside the host, while `https://127.0.0.1:8443/` answers 200 and the `werkator-nginx-hs.hsadmin.ng` container publishes both ports. Pre-existing and unrelated to this PR — the host's own firewall, not Werkator.
|
||||
- The machine configuration on `vm4006` still declares `gitTally: version: since: "0.9.20"`; the current code reads only `werkator:`, so the file's version claim is silently ignored (`werkator.version.since` prints empty). It costs nothing while `ConfigVersions.FORMAT_BROKE_IN` is `""`, and stops protecting that host the moment plan step 18 sets it.
|
||||
|
||||
## Prerequisite PRs
|
||||
|
||||
- None; it changes only `tools/remote` and the documentation.
|
||||
|
||||
## Follow-up PRs
|
||||
|
||||
- None planned.
|
||||
@@ -0,0 +1,98 @@
|
||||
> **WARNING:** This document describes only the change applied in this PR.
|
||||
> It may already be outdated once the next PR is merged.
|
||||
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
||||
|
||||
## The Problem
|
||||
|
||||
The build sandbox for hosts without Docker was configured as `bwrap`, named after the mechanism rather than after the thing Werkator runs.
|
||||
Since step 21 session C (v1.0.0) Werkator does not invoke `bwrap` at all: it shells out to the [werkdock](https://git.javagil.de/mi/werkdock) CLI, which assembles the bubblewrap invocation and owns the image store.
|
||||
The name outlived its truth, and the clearest symptom was the key `bwrap.werkdock` — a section naming its own executor.
|
||||
|
||||
It also leaked outward.
|
||||
`tools/remote` gained a `WERKATOR_SANDBOX` key in PR#18 whose value had to be `bwrap` while the very thing it switches on is *uploading the werkdock binary*, and the question that prompted this PR — "is there also `WERKATOR_SANDBOX=werkdock`?" — is one nobody would ask about a name that matched.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Refusing the old name. That belongs to the release which sets `ConfigVersions.FORMAT_BROKE_IN` (plan step 18): only there can a file that declares no version be caught by name at all, and only there is one migration asked of the operator instead of two.
|
||||
- Renaming `tools/build-bwrap-rootfs.sh` or `docs/plan/17-bwrap-build-runtime.md`. The script really does build a bubblewrap rootfs, and plan documents are historic records.
|
||||
- Touching ADR 0008, which decided the *runtime* and is a snapshot of that decision.
|
||||
|
||||
## The Scenarios
|
||||
|
||||
### Feature: the sandbox is named after the tool that runs it
|
||||
|
||||
#### Background
|
||||
|
||||
- `builds.<name>.werkdock` replaces `builds.<name>.bwrap`, and `werkdock.binary` replaces `bwrap.werkdock`.
|
||||
- The pinned set is unchanged in meaning: `enabled`, `rootfs` and the binary stay host-pinned, under their new names.
|
||||
|
||||
#### Scenario#19.01: A configuration written for the old name keeps working
|
||||
|
||||
So that no installation has to be edited before it can be updated — the section lives in machine configurations that no repository tracks.
|
||||
|
||||
- **Given** a configuration writing `builds.default.bwrap` with `enabled`, `rootfs`, `werkdock` and `env`
|
||||
- **When** the configuration is loaded
|
||||
- **Then** the settings appear as `werkdock.enabled`, `werkdock.rootfs`, `werkdock.binary` and `werkdock.env`, and the file is named once in a warning
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [ConfigLoaderTest — "the legacy bwrap section is read as werkdock, its werkdock key as binary"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
|
||||
#### Scenario#19.02: The old name is not a way around the pinning
|
||||
|
||||
So that a branch cannot escape its sandbox by writing the section a branch is not allowed to write under its previous name.
|
||||
|
||||
- **Given** a host configuration with `werkdock.enabled: true` and a rootfs
|
||||
- **When** a branch's committed config sets `bwrap.enabled: false` with a foreign rootfs
|
||||
- **Then** the sandbox stays enabled and the host's rootfs is used
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [ConfigLoaderTest — "a legacy bwrap section on a branch is pinned exactly like the new name"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||
|
||||
#### Scenario#19.03: The new name behaves exactly as the old one did
|
||||
|
||||
So that the rename is a rename, not a change of behavior.
|
||||
|
||||
- **Given** configurations using `werkdock` throughout
|
||||
- **When** builds are dispatched, pinned keys stripped, and both sandboxes enabled at once
|
||||
- **Then** the werkdock runner is selected, the branch cannot override the pinned keys, and enabling docker and werkdock together is rejected naming both
|
||||
|
||||
##### Verified by
|
||||
|
||||
- [DispatchingBuildRunnerTest — "runs in the werkdock sandbox when the branch enables it (and not Docker)"](../../src/test/kotlin/de/hoennig/werkator/build/DispatchingBuildRunnerTest.kt)
|
||||
- [ConfigLoaderTest — "a branch cannot disable its werkdock sandbox …"](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt) and "enabling both docker and werkdock on a build is rejected, not picked silently"
|
||||
- [WerkdockBuildRunnerTest](../../src/test/kotlin/de/hoennig/werkator/build/WerkdockBuildRunnerTest.kt), unchanged in substance and renamed with the runner
|
||||
|
||||
## The Solution
|
||||
|
||||
`BwrapConfig` → `WerkdockConfig` (field `werkdock` → `binary`), `BwrapOverrides` → `WerkdockOverrides`, `BranchConfig.bwrap` → `.werkdock`, `BwrapBuildRunner` → `WerkdockBuildRunner`, and `PINNED_BWRAP_KEYS` → `PINNED_WERKDOCK_KEYS` with `binary` in place of `werkdock`.
|
||||
|
||||
The compatibility lives in exactly one function, `ConfigLoader.renameLegacySandbox`, applied in `loadFile` and `parseYaml` — the two places a raw layer enters — so it runs before merging, before pinning and before binding, and every consumer downstream knows one name.
|
||||
That placement is what makes Scenario#19.02 hold without a second thought: the branch layer is normalised *before* `stripPinned` reads it, so the old name cannot smuggle a pinned key past a check that looks for the new one.
|
||||
Where a file writes both sections, the explicit `werkdock` one wins, because it is the name that is meant.
|
||||
The warning is emitted once per file (`warnedSections`), like the other section-level warnings, since the config is re-read on every poll cycle.
|
||||
|
||||
The version is bumped to 1.2.0 with a release note: the minor, because this changes the configuration schema, and a deployment must be identifiable as the one that introduced it.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **Should `WERKATOR_SANDBOX` keep accepting `bwrap`?** It does today, normalised on read and documented as the former name. The env files are local and gitignored, so this alias costs one line and can go whenever the config alias does.
|
||||
|
||||
## Additional Changes
|
||||
|
||||
- None beyond the rename and its documentation.
|
||||
|
||||
## Deployment note
|
||||
|
||||
The order matters, and only in one direction: deploy v1.2.0 to a host *before* rewriting its init fragment (`.env.<instance>.yml`) to the new key names.
|
||||
A fragment carrying `werkdock:` applied by an older Werkator fails the fragment's strict schema validation — which is the safe outcome, but a failed `repo-init` nonetheless.
|
||||
The reverse never breaks: v1.2.0 reads every existing `bwrap:` fragment and machine config as before.
|
||||
|
||||
## Prerequisite PRs
|
||||
|
||||
- [PR#18](2026-09-03-PR%2318-remote-host-layout.md) introduced `WERKATOR_SANDBOX`, whose value this PR renames.
|
||||
|
||||
## Follow-up PRs
|
||||
|
||||
- Plan step 18 (removing the legacy `branches` section) sets `ConfigVersions.FORMAT_BROKE_IN`; the `bwrap` alias should be dropped in the same release, refusing the key by name.
|
||||
Reference in New Issue
Block a user