4 Commits
Author SHA1 Message Date
mhoennigandClaude Opus 5 2c64329d22 docs(prs): PR#18 — tools/remote drives any host layout
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 19:59:40 +02:00
mhoennigandClaude Opus 5 e1477eb8d0 docs(deployment): tools/remote drives any host layout
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 19:58:11 +02:00
mhoennigandClaude Opus 5 389fae388e fix(remote): upload before stopping the service, and verify the transfer
instance-update stopped the unit and only then started the upload, so a transfer
that dies mid-way leaves the host with no running Werkator and nothing to start
again. That is not theoretical: deploying to vm4006 on 2026-09-03 failed with
"scp: Connection closed" with the service already stopped.

The upload now happens before the stop, and each artifact is transferred to a
.part file whose sha256 is compared with the local one before it is moved into
place, retrying twice. A truncated archive would otherwise unpack into a broken
runtime, which is worse than the failed transfer it came from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 19:55:23 +02:00
mhoennigandClaude Opus 5 506e817c82 feat(remote): the host layout is configurable, not the mih convention
tools/remote assumed the layout instance-install creates: the watched repository
in $WERKATOR_PATH/werkator, the runtime in $WERKATOR_PATH/.werkator, a werkdock
binary and a rootfs beside it, and the unit hardcoded as werkator-werkator.service.
An installation that predates the script — vm4006, a docker host with the repository
in ~/hs.hsadmin.ng and the runtime in ~/opt — could not be deployed with it at all.

WERKATOR_REPO_DIR, WERKATOR_INSTALL_DIR and WERKATOR_SANDBOX name the three values
that actually differ; their defaults are what instance-install writes, so the
existing env files resolve to exactly the same paths as before. The unit name is
derived from the repository directory the way SystemdServiceFiles.unitName does it,
instead of being spelled out. With WERKATOR_SANDBOX=docker the werkdock binary and
the rootfs archive are neither built nor uploaded — a docker host has no sandbox to
install, and check-prerequisites asks the docker daemon instead of werkdock doctor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 19:53:04 +02:00
3 changed files with 250 additions and 55 deletions
+6
View File
@@ -184,6 +184,11 @@ ssh <user>@<host>
The tarball unpacks to a `werkator/` directory, so it must not be extracted over `~/opt` directly — unpack it in `/tmp` and move it into place, as above. The tarball unpacks to a `werkator/` directory, so it must not be extracted over `~/opt` directly — unpack it in `/tmp` and move it into place, as above.
Rollback is the reverse: stop, remove the new directory (or jar), move `.bak` back, start. Rollback is the reverse: stop, remove the new directory (or jar), move `.bak` back, start.
`tools/remote --env-file .env.<instance> werkator instance-update` does the same sequence for any host, not only the webspace layout it was written for.
Three optional keys in the env file name what differs (see the script's header): `WERKATOR_REPO_DIR` (directory of the watched repository, which also names the systemd unit), `WERKATOR_INSTALL_DIR` (where the runtime bundle is unpacked), and `WERKATOR_SANDBOX` (`bwrap`, the default, or `docker` — a Docker host has no werkdock binary and no rootfs archive to upload).
Their defaults are the layout `instance-install` creates, so an env file that names none of them behaves exactly as before.
The upload happens before the service is stopped and every artifact is checksum-verified after the transfer, so a dropped connection costs the transfer and not the running service.
Then check `https://<public-url>/` for the new version in the footer, and `journalctl --user -u werkator-<repo-name>.service -n 50` for a clean start. Then check `https://<public-url>/` for the new version in the footer, and `journalctl --user -u werkator-<repo-name>.service -n 50` for a clean start.
Config file changes are not needed for an update; new keys take their defaults. Config file changes are not needed for an update; new keys take their defaults.
@@ -335,6 +340,7 @@ tools/remote --env-file .env.mih34 port-forward start # browser tunne
``` ```
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`. Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation).
The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user. The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`. Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
@@ -0,0 +1,98 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
`tools/remote` grew with the Hostsharing Managed Webspace rollout (steps 21 and 23) and encoded that rollout's layout as if it were the only one.
The watched repository had to be `$WERKATOR_PATH/werkator`, the runtime `$WERKATOR_PATH/.werkator/werkator`, and the systemd unit was spelled out as `werkator-werkator.service`; a werkdock binary and a bwrap rootfs archive were built and uploaded unconditionally.
`vm4006`, the Docker host that has been running Werkator since long before the script existed, matches none of that: its watched repository is `~/hs.hsadmin.ng` (so its unit is `werkator-hs.hsadmin.ng.service`), its runtime lives in `~/opt/werkator`, and it has neither werkdock nor a rootfs because its builds run in Docker.
So that host could only be deployed by hand, and had drifted nine releases behind — the question that started this PR was whether the Werkdock and multi-repo work had broken it, which it had not: what was broken was the deployment tooling.
The second problem surfaced while deploying: `instance-update` stopped the systemd unit and *then* started the upload.
The 66 MB transfer to `vm4006` died with `scp: Connection closed`, leaving the host with no running Werkator and nothing new to start.
## Non-Goals
- `instance-start` for a Docker host: it places a Hostsharing `.htaccess` into a `doms/<domain>/` docroot, which only exists on a Managed Webspace. `vm4006` uses the managed nginx container instead, and keeps its existing units.
- `repo-init` for `vm4006` — the repository has been cloned and configured there for months; only `instance-update` was needed.
- Making the machine configuration of `vm4006` current: it still carries the pre-rename `gitTally:` meta key, which is read by nothing today. Harmless while `ConfigVersions.FORMAT_BROKE_IN` is empty, a trap on the day it is not.
## The Scenarios
### Feature: one deployment command for every host layout
#### Background
- The env file carries transport values only; the three new keys describe *where* things are on the host, not what Werkator does.
- The defaults are exactly the layout `instance-install` creates, so an env file naming none of them resolves as before.
#### Scenario#18.01: A host that predates the script can be deployed with it
So that an installation is not condemned to hand-typed `scp` sequences because it was set up before the tooling existed.
- **Given** an env file with `WERKATOR_REPO_DIR=hs.hsadmin.ng`, `WERKATOR_INSTALL_DIR=/home/tallyman/opt` and `WERKATOR_SANDBOX=docker`
- **When** `tools/remote --env-file .env.vm4006 werkator instance-update` runs
- **Then** it addresses `werkator-hs.hsadmin.ng.service`, unpacks into `~/opt`, and uploads neither a werkdock binary nor a rootfs archive
##### Verified by
- Live on `vm4006` (2026-09-03): `check-prerequisites` reported the Docker daemon instead of running `werkdock doctor`; the update swapped `~/opt/werkator` from v1.0.1 to **v1.1.2**, the unit came up `active`, `/api/watcher` polls without errors, and `/api/system` still reports real disk figures (`diskSource.kind: volume`) rather than the quota shape from PR#16.
#### Scenario#18.02: The webspace hosts are unaffected
So that making the script layout-aware does not break the deployment path that is actually in production.
- **Given** `.env.mih09` and `.env.mih34`, neither naming any of the new keys
- **When** the layout is resolved
- **Then** repository directory, install directory, unit name and sandbox are identical to the hardcoded values they replace
##### Verified by
- Resolution measured for both env files (2026-09-03): `REPO_DIR=$WERKATOR_PATH/werkator`, `INSTALL_DIR=$WERKATOR_PATH/.werkator`, `UNIT=werkator-werkator.service`, `SANDBOX=bwrap`.
- Live on `mih09` (2026-09-03): a full `instance-update` ran through the bwrap path — werkdock uploaded, `werkdock 0.1.0-dev` reported after the swap, service `active`, `https://werkator.javagil.de/` answering 200.
#### Scenario#18.03: A failed transfer does not take the service down
So that a dropped connection costs the upload and nothing else.
- **Given** an instance whose service is running
- **When** the runtime bundle cannot be transferred
- **Then** the service is still running, because the upload happens before the stop, and a partially transferred file is never moved into place
##### Verified by
- The failure itself on `vm4006` (2026-09-03), which is what this scenario is written from: with the old order, `scp: Connection closed` left the unit stopped and the host without Werkator.
- Live on `mih09` (2026-09-03): the bundle already on the host was recognised by its sha256 and skipped, so the upload step cost nothing and the stop followed only after it.
## The Solution
Three optional env keys replace three hardcoded assumptions.
`WERKATOR_REPO_DIR` and `WERKATOR_INSTALL_DIR` are resolved absolute-or-relative-to-`WERKATOR_PATH`, and the unit name is now *derived* from the repository directory the way `SystemdServiceFiles.unitName` derives it (basename, every character outside `[A-Za-z0-9_.-]` replaced by a dash) instead of being spelled out — one rule, in two places, with the script naming the Kotlin function it mirrors.
`WERKATOR_SANDBOX=docker` skips everything bwrap-shaped: no werkdock build, no werkdock upload, no rootfs archive, and `check-prerequisites` asks the Docker daemon instead of running `werkdock doctor`.
`repo-add` clones beside the watched repository rather than into `WERKATOR_PATH`, which is the same directory whenever the default layout is used.
`deploy_instance` is split into `upload_instance_artifacts` and `swap_instance_runtime`, and `instance_update` calls the first *before* stopping the unit.
Every artifact is uploaded to `<name>.part`, compared by sha256 with the local file, and only then moved into place; three attempts, and an unchanged artifact is skipped entirely.
The checksum is not belt-and-braces: a truncated tarball would unpack into a runtime that starts and misbehaves, which is far worse than the failed transfer it came from.
## Open Questions
- **Should `instance-start` learn the Docker-host shape too?** Not answered here. `vm4006` keeps its existing units and its managed nginx; the day it needs regenerating, `init --systemd` on the host is the documented path.
## Additional Changes
- None beyond the feature itself.
## Follow-up work discovered while deploying this
- `https://vm4006.hostsharing.net:8443/` is not reachable from outside the host, while `https://127.0.0.1:8443/` answers 200 and the `werkator-nginx-hs.hsadmin.ng` container publishes both ports. Pre-existing and unrelated to this PR — the host's own firewall, not Werkator.
- The machine configuration on `vm4006` still declares `gitTally: version: since: "0.9.20"`; the current code reads only `werkator:`, so the file's version claim is silently ignored (`werkator.version.since` prints empty). It costs nothing while `ConfigVersions.FORMAT_BROKE_IN` is `""`, and stops protecting that host the moment plan step 18 sets it.
## Prerequisite PRs
- None; it changes only `tools/remote` and the documentation.
## Follow-up PRs
- None planned.
+136 -45
View File
@@ -45,13 +45,22 @@
# WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start) # WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start)
# WERKATOR_REPO_URL https clone URL of the watched repository # WERKATOR_REPO_URL https clone URL of the watched repository
# (default: https://github.com/mhoennig/werkator.git) # (default: https://github.com/mhoennig/werkator.git)
# WERKATOR_REPO_DIR directory of the watched repository, absolute or relative to
# WERKATOR_PATH (default: werkator); it also names the systemd
# unit, exactly as `init --systemd` derives it
# WERKATOR_INSTALL_DIR directory holding the unpacked runtime bundle, absolute or
# relative to WERKATOR_PATH (default: .werkator)
# WERKATOR_SANDBOX build runtime of the host: bwrap (default) or docker; a docker
# host needs neither the werkdock binary nor a rootfs archive
# WERKDOCK_REPO checkout of the werkdock repository, whose binary the # WERKDOCK_REPO checkout of the werkdock repository, whose binary the
# instance runs (default: <repo>/../werkdock) # instance runs (default: <repo>/../werkdock)
# WERKDOCK_BINARY the built werkdock binary (default: $WERKDOCK_REPO/dist/werkdock) # WERKDOCK_BINARY the built werkdock binary (default: $WERKDOCK_REPO/dist/werkdock)
# WERKATOR_ROOTFS rootfs archive path for repo-init # WERKATOR_ROOTFS rootfs archive path for repo-init
# (default: <repo>/build/werkator-buildenv-trixie-java-go-node.tar.zst) # (default: <repo>/build/werkator-buildenv-trixie-java-go-node.tar.zst)
# #
# Install layout on the host: # Install layout on the host — the default, which the three keys above bend to an
# installation that predates this script (e.g. the docker host vm4006: the watched
# repository is ~/hs.hsadmin.ng, the runtime lives in ~/opt, there is no werkdock):
# $WERKATOR_PATH/werkator/ the watched repository (clone) # $WERKATOR_PATH/werkator/ the watched repository (clone)
# $WERKATOR_PATH/.werkator/werkator/ the unpacked runtime bundle # $WERKATOR_PATH/.werkator/werkator/ the unpacked runtime bundle
# $WERKATOR_PATH/.werkator/bin/ the werkdock binary # $WERKATOR_PATH/.werkator/bin/ the werkdock binary
@@ -113,9 +122,33 @@ HOST="$WERKATOR_REMOTE"
TARGET_DIR="$WERKATOR_PATH" TARGET_DIR="$WERKATOR_PATH"
ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}" ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}"
REPO_URL="${WERKATOR_REPO_URL:-https://github.com/mhoennig/werkator.git}" REPO_URL="${WERKATOR_REPO_URL:-https://github.com/mhoennig/werkator.git}"
MACHINE_CONFIG="$TARGET_DIR/werkator/.git/werkator/.werkator.yml"
WERKATOR_BIN="$TARGET_DIR/.werkator/werkator/bin/werkator" # The host layout is three values, not one convention: an installation that grew
UNIT="werkator-werkator.service" # before this script existed puts them elsewhere, and the defaults are exactly what
# `instance-install` creates, so an env file that names none of them behaves as before.
# Both directories may be absolute; a bare name is taken relative to WERKATOR_PATH.
resolve_dir() {
case "$1" in
/*) echo "$1" ;;
*) echo "$TARGET_DIR/$1" ;;
esac
}
REPO_DIR="$(resolve_dir "${WERKATOR_REPO_DIR:-werkator}")"
INSTALL_DIR="$(resolve_dir "${WERKATOR_INSTALL_DIR:-.werkator}")"
# where `repo-add` puts a further repository of the registry: beside the watched one
SIBLING_DIR="$(dirname "$REPO_DIR")"
SANDBOX="${WERKATOR_SANDBOX:-bwrap}"
case "$SANDBOX" in
bwrap|docker) ;;
*) die "WERKATOR_SANDBOX is 'bwrap' or 'docker', not '$SANDBOX'" ;;
esac
MACHINE_CONFIG="$REPO_DIR/.git/werkator/.werkator.yml"
WERKATOR_BIN="$INSTALL_DIR/werkator/bin/werkator"
# mirrors SystemdServiceFiles.unitName: the repository's directory name, every
# character outside [A-Za-z0-9_.-] replaced by a dash — the unit `init --systemd`
# writes, which is the one this script may stop and start.
UNIT="werkator-$(basename "$REPO_DIR" | sed 's/[^A-Za-z0-9_.-]/-/g').service"
ssh_present() { ssh_present() {
ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST" true 2>/dev/null ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST" true 2>/dev/null
@@ -133,13 +166,21 @@ ensure_ssh() {
# Werkdock owns the host checks (`werkdock doctor` ports the old prerequisites # Werkdock owns the host checks (`werkdock doctor` ports the old prerequisites
# script); the binary is uploaded first, so the check works pre-install. # script); the binary is uploaded first, so the check works pre-install.
# On a docker host there is no werkdock and no sandbox to check: the build runtime
# is the docker daemon, so the check is that the daemon answers this user.
check_prerequisites() { check_prerequisites() {
if [ "$SANDBOX" = "docker" ]; then
echo "==> Checking the docker build runtime on $HOST (WERKATOR_SANDBOX=docker)"
ssh "$HOST" "docker info >/dev/null" || die "docker is not usable by this user on $HOST"
ssh "$HOST" "docker --version"
return 0
fi
ensure_werkdock_binary ensure_werkdock_binary
echo "==> Uploading werkdock and running its doctor on $HOST (target dir: $TARGET_DIR)" echo "==> Uploading werkdock and running its doctor on $HOST (target dir: $TARGET_DIR)"
ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator/bin'" ssh "$HOST" "mkdir -p '$INSTALL_DIR/bin'"
scp -q "$WERKDOCK_BINARY" "$HOST:$TARGET_DIR/.werkator/bin/werkdock.new" scp -q "$WERKDOCK_BINARY" "$HOST:$INSTALL_DIR/bin/werkdock.new"
ssh "$HOST" "mv '$TARGET_DIR/.werkator/bin/werkdock.new' '$TARGET_DIR/.werkator/bin/werkdock' && chmod 755 '$TARGET_DIR/.werkator/bin/werkdock'" ssh "$HOST" "mv '$INSTALL_DIR/bin/werkdock.new' '$INSTALL_DIR/bin/werkdock' && chmod 755 '$INSTALL_DIR/bin/werkdock'"
if ! ssh "$HOST" "'$TARGET_DIR/.werkator/bin/werkdock' doctor '$TARGET_DIR'"; then if ! ssh "$HOST" "'$INSTALL_DIR/bin/werkdock' doctor '$TARGET_DIR'"; then
die "werkdock doctor failed on $HOST — install aborted" die "werkdock doctor failed on $HOST — install aborted"
fi fi
} }
@@ -160,7 +201,7 @@ next to this repository, or point WERKDOCK_REPO/WERKDOCK_BINARY at your checkout
upload_fragment() { upload_fragment() {
[ -n "${WERKATOR_INIT_CONFIG:-}" ] || { echo ""; return 0; } [ -n "${WERKATOR_INIT_CONFIG:-}" ] || { echo ""; return 0; }
[ -f "$WERKATOR_INIT_CONFIG" ] || die "init fragment missing: $WERKATOR_INIT_CONFIG" [ -f "$WERKATOR_INIT_CONFIG" ] || die "init fragment missing: $WERKATOR_INIT_CONFIG"
local remote="$TARGET_DIR/.werkator/$(basename "$WERKATOR_INIT_CONFIG")" local remote="$INSTALL_DIR/$(basename "$WERKATOR_INIT_CONFIG")"
scp -q "$WERKATOR_INIT_CONFIG" "$HOST:$remote" scp -q "$WERKATOR_INIT_CONFIG" "$HOST:$remote"
echo "$remote" echo "$remote"
} }
@@ -174,43 +215,85 @@ ensure_instance_artifacts() {
(cd "$REPO_ROOT" && ./gradlew runtimeBundle --console=plain -q) (cd "$REPO_ROOT" && ./gradlew runtimeBundle --console=plain -q)
fi fi
[ -f "$RUNTIME_BUNDLE" ] || die "runtime bundle missing: $RUNTIME_BUNDLE" [ -f "$RUNTIME_BUNDLE" ] || die "runtime bundle missing: $RUNTIME_BUNDLE"
ensure_werkdock_binary [ "$SANDBOX" = "docker" ] || ensure_werkdock_binary
} }
# Uploads and unpacks the instance artifacts. The previous runtime stays as # Uploads one file and verifies it arrived whole: a transfer that dies mid-way
# werkator.prev for one deployment as the rollback asset. # (scp: Connection closed) otherwise leaves a truncated archive that unpacks into
deploy_instance() { # a broken runtime. Retries twice, because a dropped WAN connection is not a reason
# to abort a deployment.
upload_verified() {
local src="$1" dest="$2"
local local_sha remote_sha attempt
local_sha="$(sha256sum "$src" | cut -d' ' -f1)"
remote_sha="$(ssh "$HOST" "sha256sum '$dest' 2>/dev/null | cut -d' ' -f1" || true)"
if [ "$local_sha" = "$remote_sha" ]; then
echo " $(basename "$src"): already on the host, skipping"
return 0
fi
for attempt in 1 2 3; do
if scp -q "$src" "$HOST:$dest.part"; then
remote_sha="$(ssh "$HOST" "sha256sum '$dest.part' 2>/dev/null | cut -d' ' -f1" || true)"
if [ "$local_sha" = "$remote_sha" ]; then
ssh "$HOST" "mv '$dest.part' '$dest'"
return 0
fi
echo " checksum mismatch after transfer $attempt of $(basename "$src")" >&2
else
echo " transfer $attempt of $(basename "$src") failed" >&2
fi
done
ssh "$HOST" "rm -f '$dest.part'" || true
die "cannot upload $src to $HOST:$dest — three attempts failed"
}
# Uploads the instance artifacts, without touching the installed runtime: the
# service keeps running until swap_instance_runtime replaces it, so a failed
# transfer costs nothing but the transfer.
upload_instance_artifacts() {
if [ "$SANDBOX" = "docker" ]; then
echo "==> Uploading runtime bundle"
else
echo "==> Uploading runtime bundle and werkdock binary" echo "==> Uploading runtime bundle and werkdock binary"
ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator/bin'" fi
scp -q "$RUNTIME_BUNDLE" "$HOST:$TARGET_DIR/.werkator/" ssh "$HOST" "mkdir -p '$INSTALL_DIR/bin'"
scp -q "$WERKDOCK_BINARY" "$HOST:$TARGET_DIR/.werkator/bin/werkdock.new" upload_verified "$RUNTIME_BUNDLE" "$INSTALL_DIR/$(basename "$RUNTIME_BUNDLE")"
if [ "$SANDBOX" != "docker" ]; then
upload_verified "$WERKDOCK_BINARY" "$INSTALL_DIR/bin/werkdock.new"
fi
}
# Swaps in the uploaded artifacts. The previous runtime stays as werkator.prev
# for one deployment as the rollback asset.
swap_instance_runtime() {
echo "==> Unpacking" echo "==> Unpacking"
ssh "$HOST" "set -e ssh "$HOST" "set -e
cd '$TARGET_DIR/.werkator' cd '$INSTALL_DIR'
mv bin/werkdock.new bin/werkdock && chmod 755 bin/werkdock [ ! -f bin/werkdock.new ] || { mv bin/werkdock.new bin/werkdock && chmod 755 bin/werkdock; }
rm -rf werkator.prev rm -rf werkator.prev
[ ! -d werkator ] || mv werkator werkator.prev [ ! -d werkator ] || mv werkator werkator.prev
tar xzf '$(basename "$RUNTIME_BUNDLE")' tar xzf '$(basename "$RUNTIME_BUNDLE")'
'./werkator/bin/werkator' --version './werkator/bin/werkator' --version
'./bin/werkdock' version" [ ! -x bin/werkdock ] || './bin/werkdock' version"
} }
instance_install() { instance_install() {
ensure_ssh ensure_ssh
check_prerequisites check_prerequisites
ensure_instance_artifacts ensure_instance_artifacts
deploy_instance upload_instance_artifacts
swap_instance_runtime
echo echo
echo "==> Instance installed." echo "==> Instance installed."
echo " Runtime: $WERKATOR_BIN" echo " Runtime: $WERKATOR_BIN"
echo " werkdock: $TARGET_DIR/.werkator/bin/werkdock" [ "$SANDBOX" = "docker" ] || echo " werkdock: $INSTALL_DIR/bin/werkdock"
echo " Next: tools/remote werkator repo-init, then instance-start" echo " Next: tools/remote werkator repo-init, then instance-start"
} }
# Refuse to swap the runtime under a running build; FORCE=1 overrides. # Refuse to swap the runtime under a running build; FORCE=1 overrides.
require_idle() { require_idle() {
local port local port
port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' 2>/dev/null && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"' || true)" port="$(ssh "$HOST" "cd '$REPO_DIR' 2>/dev/null && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"' || true)"
[ -n "$port" ] || return 0 [ -n "$port" ] || return 0
local current local current
current="$(ssh "$HOST" "curl -s --max-time 5 http://127.0.0.1:$port/api/builds/current" || true)" current="$(ssh "$HOST" "curl -s --max-time 5 http://127.0.0.1:$port/api/builds/current" || true)"
@@ -224,6 +307,9 @@ instance_update() {
ensure_ssh ensure_ssh
ensure_instance_artifacts ensure_instance_artifacts
require_idle require_idle
# upload first, stop second: a transfer that fails must not leave the host
# without a running service (measured on vm4006, 2026-09-03)
upload_instance_artifacts
local was_active=0 local was_active=0
if ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user is-active --quiet '$UNIT'"; then if ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user is-active --quiet '$UNIT'"; then
was_active=1 was_active=1
@@ -232,7 +318,7 @@ instance_update() {
echo "==> Stopping $UNIT" echo "==> Stopping $UNIT"
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user stop '$UNIT'" ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user stop '$UNIT'"
fi fi
deploy_instance swap_instance_runtime
if [ "$was_active" = "1" ]; then if [ "$was_active" = "1" ]; then
echo "==> Starting $UNIT" echo "==> Starting $UNIT"
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user start '$UNIT' && sleep 3 && systemctl --user is-active '$UNIT'" ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user start '$UNIT' && sleep 3 && systemctl --user is-active '$UNIT'"
@@ -249,18 +335,22 @@ instance_update() {
# writing is init's — this script transports and invokes (step 23). # writing is init's — this script transports and invokes (step 23).
repo_init() { repo_init() {
ensure_ssh ensure_ssh
[ -f "$ROOTFS" ] || die "rootfs archive missing: $ROOTFS — build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS" [ "$SANDBOX" = "docker" ] || [ -f "$ROOTFS" ] ||
die "rootfs archive missing: $ROOTFS — build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS"
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning the watched repository" echo "==> Cloning the watched repository"
if ssh "$HOST" "test -d '$TARGET_DIR/werkator/.git'"; then if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then
echo " (already cloned, skipping)" echo " (already cloned, skipping)"
else else
ssh "$HOST" "git clone '$REPO_URL' '$TARGET_DIR/werkator'" ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
fi fi
if [ "$SANDBOX" = "docker" ]; then
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
else
echo "==> Uploading the rootfs archive (skipped when unchanged)" echo "==> Uploading the rootfs archive (skipped when unchanged)"
local rootfs_remote="$TARGET_DIR/.werkator/$(basename "$ROOTFS")" local rootfs_remote="$INSTALL_DIR/$(basename "$ROOTFS")"
local local_sha remote_sha local local_sha remote_sha
local_sha="$(sha256sum "$ROOTFS" | cut -d' ' -f1)" local_sha="$(sha256sum "$ROOTFS" | cut -d' ' -f1)"
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' 2>/dev/null | cut -d' ' -f1" || true)" remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' 2>/dev/null | cut -d' ' -f1" || true)"
@@ -271,18 +361,19 @@ repo_init() {
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' | cut -d' ' -f1")" remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' | cut -d' ' -f1")"
[ "$local_sha" = "$remote_sha" ] || die "rootfs upload checksum mismatch" [ "$local_sha" = "$remote_sha" ] || die "rootfs upload checksum mismatch"
fi fi
fi
echo "==> Running werkator init${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}" echo "==> Running werkator init${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
local fragment_remote local fragment_remote
fragment_remote="$(upload_fragment)" fragment_remote="$(upload_fragment)"
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}" ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
echo "==> Verifying the effective configuration" echo "==> Verifying the effective configuration"
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null | grep -A4 'bwrap:' | head -5" ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' config:print 2>/dev/null | grep -A4 '$SANDBOX:' | head -5"
echo echo
echo "==> Repository ready." echo "==> Repository ready."
echo " Repo: $TARGET_DIR/werkator" echo " Repo: $REPO_DIR"
echo " Next: fill git.account/git.token in $MACHINE_CONFIG if the origin is private," echo " Next: fill git.account/git.token in $MACHINE_CONFIG if the origin is private,"
echo " then tools/remote werkator instance-start" echo " then tools/remote werkator instance-start"
} }
@@ -306,10 +397,10 @@ repo_add() {
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first" ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
echo "==> Cloning $url as '$name'" echo "==> Cloning $url as '$name'"
if ssh "$HOST" "test -d '$TARGET_DIR/$name/.git'"; then if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then
echo " (already cloned, skipping)" echo " (already cloned, skipping)"
else else
ssh "$HOST" "git clone '$url' '$TARGET_DIR/$name'" ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
fi fi
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock # The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
@@ -318,26 +409,26 @@ repo_add() {
echo "==> Running werkator init in $name${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}" echo "==> Running werkator init in $name${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
local fragment_remote local fragment_remote
fragment_remote="$(upload_fragment)" fragment_remote="$(upload_fragment)"
ssh "$HOST" "cd '$TARGET_DIR/$name' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}" ssh "$HOST" "cd '$SIBLING_DIR/$name' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
echo "==> Checking the registry" echo "==> Checking the registry"
# Grepped locally: the entry may name the path absolute or as ~/<name>, and # Grepped locally: the entry may name the path absolute or as ~/<name>, and
# matching both is easier without a second layer of remote shell quoting. # matching both is easier without a second layer of remote shell quoting.
if ssh "$HOST" "cat ~/.werkator.yml 2>/dev/null" | if ssh "$HOST" "cat ~/.werkator.yml 2>/dev/null" |
grep -qE "path: *(~|$TARGET_DIR)/$name[[:space:]]*$"; then grep -qE "path: *(~|$SIBLING_DIR)/$name[[:space:]]*$"; then
echo " (~/.werkator.yml already names this path)" echo " (~/.werkator.yml already names this path)"
else else
echo " not registered yet — add this entry to ~/.werkator.yml on $HOST:" echo " not registered yet — add this entry to ~/.werkator.yml on $HOST:"
echo echo
echo " repositories:" echo " repositories:"
echo " - path: $TARGET_DIR/$name" echo " - path: $SIBLING_DIR/$name"
echo " name: $name" echo " name: $name"
echo echo
fi fi
echo "==> Repository prepared." echo "==> Repository prepared."
echo " Repo: $TARGET_DIR/$name" echo " Repo: $SIBLING_DIR/$name"
echo " Next: fill git.account/git.token in $TARGET_DIR/$name/.git/werkator/.werkator.yml if the origin is private" echo " Next: fill git.account/git.token in $SIBLING_DIR/$name/.git/werkator/.werkator.yml if the origin is private"
echo " (or once for all repositories in the 'defaults' block of ~/.werkator.yml)," echo " (or once for all repositories in the 'defaults' block of ~/.werkator.yml),"
echo " then restart the service — the registry is read at start." echo " then restart the service — the registry is read at start."
} }
@@ -353,11 +444,11 @@ instance_start() {
echo "==> Applying the instance fragment and generating the host integration (init --systemd)" echo "==> Applying the instance fragment and generating the host integration (init --systemd)"
local fragment_remote local fragment_remote
fragment_remote="$(upload_fragment)" fragment_remote="$(upload_fragment)"
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'} --systemd" ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'} --systemd"
local htaccess_src="$TARGET_DIR/werkator/.git/werkator/werkator.htaccess" local htaccess_src="$REPO_DIR/.git/werkator/werkator.htaccess"
local htaccess="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/.htaccess" local htaccess="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/.htaccess"
local maintenance_src="$TARGET_DIR/werkator/.git/werkator/werkator-maintenance.html" local maintenance_src="$REPO_DIR/.git/werkator/werkator-maintenance.html"
local maintenance="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/werkator-maintenance.html" local maintenance="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/werkator-maintenance.html"
if ssh "$HOST" "test -f '$htaccess_src'"; then if ssh "$HOST" "test -f '$htaccess_src'"; then
echo "==> Placing the generated Apache reverse proxy at $htaccess" echo "==> Placing the generated Apache reverse proxy at $htaccess"
@@ -368,9 +459,9 @@ instance_start() {
echo "==> Linking the units into ~/.config/systemd/user and enabling the service" echo "==> Linking the units into ~/.config/systemd/user and enabling the service"
ssh "$HOST" "mkdir -p ~/.config/systemd/user && \ ssh "$HOST" "mkdir -p ~/.config/systemd/user && \
ln -sf '$TARGET_DIR/werkator/.git/werkator/$UNIT' ~/.config/systemd/user/ && \ ln -sf '$REPO_DIR/.git/werkator/$UNIT' ~/.config/systemd/user/ && \
ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.service' ~/.config/systemd/user/ && \ ln -sf '$REPO_DIR/.git/werkator/werkator-docker-prune.service' ~/.config/systemd/user/ && \
ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.timer' ~/.config/systemd/user/ && \ ln -sf '$REPO_DIR/.git/werkator/werkator-docker-prune.timer' ~/.config/systemd/user/ && \
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user daemon-reload && \ XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user daemon-reload && \
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user restart '$UNIT' && \ XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user restart '$UNIT' && \
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user status '$UNIT' --no-pager -l | head -12" XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user status '$UNIT' --no-pager -l | head -12"
@@ -388,7 +479,7 @@ port_forward() {
# the effective port, wherever it is configured (machine config or applied # the effective port, wherever it is configured (machine config or applied
# fragment) — config:print is the single answer, not this script's parser # fragment) — config:print is the single answer, not this script's parser
local remote_port local remote_port
remote_port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"')" remote_port="$(ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"')"
[ -n "$remote_port" ] || die "no server.port configured — run 'tools/remote werkator instance-start' first" [ -n "$remote_port" ] || die "no server.port configured — run 'tools/remote werkator instance-start' first"
case "$COMMAND" in case "$COMMAND" in
@@ -430,7 +521,7 @@ port_forward() {
# CLI owns creation and format (step 23), this script only invokes it. # CLI owns creation and format (step 23), this script only invokes it.
control_token() { control_token() {
ensure_ssh ensure_ssh
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' control-token" ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' control-token"
} }
case "$REPO" in case "$REPO" in