Compare commits
4
Commits
main
...
95564587d3
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
95564587d3 | ||
|
|
618a2acb9f | ||
|
|
2ffc14a20c | ||
|
|
b5df75f844 |
@@ -184,6 +184,11 @@ ssh <user>@<host>
|
|||||||
The tarball unpacks to a `werkator/` directory, so it must not be extracted over `~/opt` directly — unpack it in `/tmp` and move it into place, as above.
|
The tarball unpacks to a `werkator/` directory, so it must not be extracted over `~/opt` directly — unpack it in `/tmp` and move it into place, as above.
|
||||||
Rollback is the reverse: stop, remove the new directory (or jar), move `.bak` back, start.
|
Rollback is the reverse: stop, remove the new directory (or jar), move `.bak` back, start.
|
||||||
|
|
||||||
|
`tools/remote --env-file .env.<instance> werkator instance-update` does the same sequence for any host, not only the webspace layout it was written for.
|
||||||
|
Three optional keys in the env file name what differs (see the script's header): `WERKATOR_REPO_DIR` (directory of the watched repository, which also names the systemd unit), `WERKATOR_INSTALL_DIR` (where the runtime bundle is unpacked), and `WERKATOR_SANDBOX` (`bwrap`, the default, or `docker` — a Docker host has no werkdock binary and no rootfs archive to upload).
|
||||||
|
Their defaults are the layout `instance-install` creates, so an env file that names none of them behaves exactly as before.
|
||||||
|
The upload happens before the service is stopped and every artifact is checksum-verified after the transfer, so a dropped connection costs the transfer and not the running service.
|
||||||
|
|
||||||
Then check `https://<public-url>/` for the new version in the footer, and `journalctl --user -u werkator-<repo-name>.service -n 50` for a clean start.
|
Then check `https://<public-url>/` for the new version in the footer, and `journalctl --user -u werkator-<repo-name>.service -n 50` for a clean start.
|
||||||
Config file changes are not needed for an update; new keys take their defaults.
|
Config file changes are not needed for an update; new keys take their defaults.
|
||||||
|
|
||||||
@@ -335,6 +340,7 @@ tools/remote --env-file .env.mih34 port-forward start # browser tunne
|
|||||||
```
|
```
|
||||||
|
|
||||||
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
|
Layout on the host: the watched repository at `$WERKATOR_PATH/werkator/`, the unpacked runtime at `$WERKATOR_PATH/.werkator/werkator/`, the werkdock binary at `$WERKATOR_PATH/.werkator/bin/werkdock`.
|
||||||
|
That is the default, not a requirement: `WERKATOR_REPO_DIR`, `WERKATOR_INSTALL_DIR` and `WERKATOR_SANDBOX` bend it to an installation that predates the script, see [Updating an Existing Installation](#updating-an-existing-installation).
|
||||||
The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
|
The rootfs archive is loaded once per source into werkdock's image store (`~/.werkdock`), shared by every repository of the user.
|
||||||
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
|
Fill `git.account`/`git.token` in the machine config when the origin is private, and make the user's services survive logout with `loginctl enable-linger`.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,98 @@
|
|||||||
|
> **WARNING:** This document describes only the change applied in this PR.
|
||||||
|
> It may already be outdated once the next PR is merged.
|
||||||
|
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
||||||
|
|
||||||
|
## The Problem
|
||||||
|
|
||||||
|
`tools/remote` grew with the Hostsharing Managed Webspace rollout (steps 21 and 23) and encoded that rollout's layout as if it were the only one.
|
||||||
|
The watched repository had to be `$WERKATOR_PATH/werkator`, the runtime `$WERKATOR_PATH/.werkator/werkator`, and the systemd unit was spelled out as `werkator-werkator.service`; a werkdock binary and a bwrap rootfs archive were built and uploaded unconditionally.
|
||||||
|
`vm4006`, the Docker host that has been running Werkator since long before the script existed, matches none of that: its watched repository is `~/hs.hsadmin.ng` (so its unit is `werkator-hs.hsadmin.ng.service`), its runtime lives in `~/opt/werkator`, and it has neither werkdock nor a rootfs because its builds run in Docker.
|
||||||
|
So that host could only be deployed by hand, and had drifted nine releases behind — the question that started this PR was whether the Werkdock and multi-repo work had broken it, which it had not: what was broken was the deployment tooling.
|
||||||
|
|
||||||
|
The second problem surfaced while deploying: `instance-update` stopped the systemd unit and *then* started the upload.
|
||||||
|
The 66 MB transfer to `vm4006` died with `scp: Connection closed`, leaving the host with no running Werkator and nothing new to start.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- `instance-start` for a Docker host: it places a Hostsharing `.htaccess` into a `doms/<domain>/` docroot, which only exists on a Managed Webspace. `vm4006` uses the managed nginx container instead, and keeps its existing units.
|
||||||
|
- `repo-init` for `vm4006` — the repository has been cloned and configured there for months; only `instance-update` was needed.
|
||||||
|
- Making the machine configuration of `vm4006` current: it still carries the pre-rename `gitTally:` meta key, which is read by nothing today. Harmless while `ConfigVersions.FORMAT_BROKE_IN` is empty, a trap on the day it is not.
|
||||||
|
|
||||||
|
## The Scenarios
|
||||||
|
|
||||||
|
### Feature: one deployment command for every host layout
|
||||||
|
|
||||||
|
#### Background
|
||||||
|
|
||||||
|
- The env file carries transport values only; the three new keys describe *where* things are on the host, not what Werkator does.
|
||||||
|
- The defaults are exactly the layout `instance-install` creates, so an env file naming none of them resolves as before.
|
||||||
|
|
||||||
|
#### Scenario#18.01: A host that predates the script can be deployed with it
|
||||||
|
|
||||||
|
So that an installation is not condemned to hand-typed `scp` sequences because it was set up before the tooling existed.
|
||||||
|
|
||||||
|
- **Given** an env file with `WERKATOR_REPO_DIR=hs.hsadmin.ng`, `WERKATOR_INSTALL_DIR=/home/tallyman/opt` and `WERKATOR_SANDBOX=docker`
|
||||||
|
- **When** `tools/remote --env-file .env.vm4006 werkator instance-update` runs
|
||||||
|
- **Then** it addresses `werkator-hs.hsadmin.ng.service`, unpacks into `~/opt`, and uploads neither a werkdock binary nor a rootfs archive
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- Live on `vm4006` (2026-09-03): `check-prerequisites` reported the Docker daemon instead of running `werkdock doctor`; the update swapped `~/opt/werkator` from v1.0.1 to **v1.1.2**, the unit came up `active`, `/api/watcher` polls without errors, and `/api/system` still reports real disk figures (`diskSource.kind: volume`) rather than the quota shape from PR#16.
|
||||||
|
|
||||||
|
#### Scenario#18.02: The webspace hosts are unaffected
|
||||||
|
|
||||||
|
So that making the script layout-aware does not break the deployment path that is actually in production.
|
||||||
|
|
||||||
|
- **Given** `.env.mih09` and `.env.mih34`, neither naming any of the new keys
|
||||||
|
- **When** the layout is resolved
|
||||||
|
- **Then** repository directory, install directory, unit name and sandbox are identical to the hardcoded values they replace
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- Resolution measured for both env files (2026-09-03): `REPO_DIR=$WERKATOR_PATH/werkator`, `INSTALL_DIR=$WERKATOR_PATH/.werkator`, `UNIT=werkator-werkator.service`, `SANDBOX=bwrap`.
|
||||||
|
- Live on `mih09` (2026-09-03): a full `instance-update` ran through the bwrap path — werkdock uploaded, `werkdock 0.1.0-dev` reported after the swap, service `active`, `https://werkator.javagil.de/` answering 200.
|
||||||
|
|
||||||
|
#### Scenario#18.03: A failed transfer does not take the service down
|
||||||
|
|
||||||
|
So that a dropped connection costs the upload and nothing else.
|
||||||
|
|
||||||
|
- **Given** an instance whose service is running
|
||||||
|
- **When** the runtime bundle cannot be transferred
|
||||||
|
- **Then** the service is still running, because the upload happens before the stop, and a partially transferred file is never moved into place
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- The failure itself on `vm4006` (2026-09-03), which is what this scenario is written from: with the old order, `scp: Connection closed` left the unit stopped and the host without Werkator.
|
||||||
|
- Live on `mih09` (2026-09-03): the bundle already on the host was recognised by its sha256 and skipped, so the upload step cost nothing and the stop followed only after it.
|
||||||
|
|
||||||
|
## The Solution
|
||||||
|
|
||||||
|
Three optional env keys replace three hardcoded assumptions.
|
||||||
|
`WERKATOR_REPO_DIR` and `WERKATOR_INSTALL_DIR` are resolved absolute-or-relative-to-`WERKATOR_PATH`, and the unit name is now *derived* from the repository directory the way `SystemdServiceFiles.unitName` derives it (basename, every character outside `[A-Za-z0-9_.-]` replaced by a dash) instead of being spelled out — one rule, in two places, with the script naming the Kotlin function it mirrors.
|
||||||
|
`WERKATOR_SANDBOX=docker` skips everything bwrap-shaped: no werkdock build, no werkdock upload, no rootfs archive, and `check-prerequisites` asks the Docker daemon instead of running `werkdock doctor`.
|
||||||
|
`repo-add` clones beside the watched repository rather than into `WERKATOR_PATH`, which is the same directory whenever the default layout is used.
|
||||||
|
|
||||||
|
`deploy_instance` is split into `upload_instance_artifacts` and `swap_instance_runtime`, and `instance_update` calls the first *before* stopping the unit.
|
||||||
|
Every artifact is uploaded to `<name>.part`, compared by sha256 with the local file, and only then moved into place; three attempts, and an unchanged artifact is skipped entirely.
|
||||||
|
The checksum is not belt-and-braces: a truncated tarball would unpack into a runtime that starts and misbehaves, which is far worse than the failed transfer it came from.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- **Should `instance-start` learn the Docker-host shape too?** Not answered here. `vm4006` keeps its existing units and its managed nginx; the day it needs regenerating, `init --systemd` on the host is the documented path.
|
||||||
|
|
||||||
|
## Additional Changes
|
||||||
|
|
||||||
|
- None beyond the feature itself.
|
||||||
|
|
||||||
|
## Follow-up work discovered while deploying this
|
||||||
|
|
||||||
|
- `https://vm4006.hostsharing.net:8443/` is not reachable from outside the host, while `https://127.0.0.1:8443/` answers 200 and the `werkator-nginx-hs.hsadmin.ng` container publishes both ports. Pre-existing and unrelated to this PR — the host's own firewall, not Werkator.
|
||||||
|
- The machine configuration on `vm4006` still declares `gitTally: version: since: "0.9.20"`; the current code reads only `werkator:`, so the file's version claim is silently ignored (`werkator.version.since` prints empty). It costs nothing while `ConfigVersions.FORMAT_BROKE_IN` is `""`, and stops protecting that host the moment plan step 18 sets it.
|
||||||
|
|
||||||
|
## Prerequisite PRs
|
||||||
|
|
||||||
|
- None; it changes only `tools/remote` and the documentation.
|
||||||
|
|
||||||
|
## Follow-up PRs
|
||||||
|
|
||||||
|
- None planned.
|
||||||
+146
-55
@@ -45,13 +45,22 @@
|
|||||||
# WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start)
|
# WERKATOR_INIT_CONFIG the init fragment to apply (repo-init, instance-start)
|
||||||
# WERKATOR_REPO_URL https clone URL of the watched repository
|
# WERKATOR_REPO_URL https clone URL of the watched repository
|
||||||
# (default: https://github.com/mhoennig/werkator.git)
|
# (default: https://github.com/mhoennig/werkator.git)
|
||||||
|
# WERKATOR_REPO_DIR directory of the watched repository, absolute or relative to
|
||||||
|
# WERKATOR_PATH (default: werkator); it also names the systemd
|
||||||
|
# unit, exactly as `init --systemd` derives it
|
||||||
|
# WERKATOR_INSTALL_DIR directory holding the unpacked runtime bundle, absolute or
|
||||||
|
# relative to WERKATOR_PATH (default: .werkator)
|
||||||
|
# WERKATOR_SANDBOX build runtime of the host: bwrap (default) or docker; a docker
|
||||||
|
# host needs neither the werkdock binary nor a rootfs archive
|
||||||
# WERKDOCK_REPO checkout of the werkdock repository, whose binary the
|
# WERKDOCK_REPO checkout of the werkdock repository, whose binary the
|
||||||
# instance runs (default: <repo>/../werkdock)
|
# instance runs (default: <repo>/../werkdock)
|
||||||
# WERKDOCK_BINARY the built werkdock binary (default: $WERKDOCK_REPO/dist/werkdock)
|
# WERKDOCK_BINARY the built werkdock binary (default: $WERKDOCK_REPO/dist/werkdock)
|
||||||
# WERKATOR_ROOTFS rootfs archive path for repo-init
|
# WERKATOR_ROOTFS rootfs archive path for repo-init
|
||||||
# (default: <repo>/build/werkator-buildenv-trixie-java-go-node.tar.zst)
|
# (default: <repo>/build/werkator-buildenv-trixie-java-go-node.tar.zst)
|
||||||
#
|
#
|
||||||
# Install layout on the host:
|
# Install layout on the host — the default, which the three keys above bend to an
|
||||||
|
# installation that predates this script (e.g. the docker host vm4006: the watched
|
||||||
|
# repository is ~/hs.hsadmin.ng, the runtime lives in ~/opt, there is no werkdock):
|
||||||
# $WERKATOR_PATH/werkator/ the watched repository (clone)
|
# $WERKATOR_PATH/werkator/ the watched repository (clone)
|
||||||
# $WERKATOR_PATH/.werkator/werkator/ the unpacked runtime bundle
|
# $WERKATOR_PATH/.werkator/werkator/ the unpacked runtime bundle
|
||||||
# $WERKATOR_PATH/.werkator/bin/ the werkdock binary
|
# $WERKATOR_PATH/.werkator/bin/ the werkdock binary
|
||||||
@@ -113,9 +122,33 @@ HOST="$WERKATOR_REMOTE"
|
|||||||
TARGET_DIR="$WERKATOR_PATH"
|
TARGET_DIR="$WERKATOR_PATH"
|
||||||
ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}"
|
ROOTFS="${WERKATOR_ROOTFS:-$REPO_ROOT/build/werkator-buildenv-trixie-java-go-node.tar.zst}"
|
||||||
REPO_URL="${WERKATOR_REPO_URL:-https://github.com/mhoennig/werkator.git}"
|
REPO_URL="${WERKATOR_REPO_URL:-https://github.com/mhoennig/werkator.git}"
|
||||||
MACHINE_CONFIG="$TARGET_DIR/werkator/.git/werkator/.werkator.yml"
|
|
||||||
WERKATOR_BIN="$TARGET_DIR/.werkator/werkator/bin/werkator"
|
# The host layout is three values, not one convention: an installation that grew
|
||||||
UNIT="werkator-werkator.service"
|
# before this script existed puts them elsewhere, and the defaults are exactly what
|
||||||
|
# `instance-install` creates, so an env file that names none of them behaves as before.
|
||||||
|
# Both directories may be absolute; a bare name is taken relative to WERKATOR_PATH.
|
||||||
|
resolve_dir() {
|
||||||
|
case "$1" in
|
||||||
|
/*) echo "$1" ;;
|
||||||
|
*) echo "$TARGET_DIR/$1" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
REPO_DIR="$(resolve_dir "${WERKATOR_REPO_DIR:-werkator}")"
|
||||||
|
INSTALL_DIR="$(resolve_dir "${WERKATOR_INSTALL_DIR:-.werkator}")"
|
||||||
|
# where `repo-add` puts a further repository of the registry: beside the watched one
|
||||||
|
SIBLING_DIR="$(dirname "$REPO_DIR")"
|
||||||
|
SANDBOX="${WERKATOR_SANDBOX:-bwrap}"
|
||||||
|
case "$SANDBOX" in
|
||||||
|
bwrap|docker) ;;
|
||||||
|
*) die "WERKATOR_SANDBOX is 'bwrap' or 'docker', not '$SANDBOX'" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
MACHINE_CONFIG="$REPO_DIR/.git/werkator/.werkator.yml"
|
||||||
|
WERKATOR_BIN="$INSTALL_DIR/werkator/bin/werkator"
|
||||||
|
# mirrors SystemdServiceFiles.unitName: the repository's directory name, every
|
||||||
|
# character outside [A-Za-z0-9_.-] replaced by a dash — the unit `init --systemd`
|
||||||
|
# writes, which is the one this script may stop and start.
|
||||||
|
UNIT="werkator-$(basename "$REPO_DIR" | sed 's/[^A-Za-z0-9_.-]/-/g').service"
|
||||||
|
|
||||||
ssh_present() {
|
ssh_present() {
|
||||||
ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST" true 2>/dev/null
|
ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST" true 2>/dev/null
|
||||||
@@ -133,13 +166,21 @@ ensure_ssh() {
|
|||||||
|
|
||||||
# Werkdock owns the host checks (`werkdock doctor` ports the old prerequisites
|
# Werkdock owns the host checks (`werkdock doctor` ports the old prerequisites
|
||||||
# script); the binary is uploaded first, so the check works pre-install.
|
# script); the binary is uploaded first, so the check works pre-install.
|
||||||
|
# On a docker host there is no werkdock and no sandbox to check: the build runtime
|
||||||
|
# is the docker daemon, so the check is that the daemon answers this user.
|
||||||
check_prerequisites() {
|
check_prerequisites() {
|
||||||
|
if [ "$SANDBOX" = "docker" ]; then
|
||||||
|
echo "==> Checking the docker build runtime on $HOST (WERKATOR_SANDBOX=docker)"
|
||||||
|
ssh "$HOST" "docker info >/dev/null" || die "docker is not usable by this user on $HOST"
|
||||||
|
ssh "$HOST" "docker --version"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
ensure_werkdock_binary
|
ensure_werkdock_binary
|
||||||
echo "==> Uploading werkdock and running its doctor on $HOST (target dir: $TARGET_DIR)"
|
echo "==> Uploading werkdock and running its doctor on $HOST (target dir: $TARGET_DIR)"
|
||||||
ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator/bin'"
|
ssh "$HOST" "mkdir -p '$INSTALL_DIR/bin'"
|
||||||
scp -q "$WERKDOCK_BINARY" "$HOST:$TARGET_DIR/.werkator/bin/werkdock.new"
|
scp -q "$WERKDOCK_BINARY" "$HOST:$INSTALL_DIR/bin/werkdock.new"
|
||||||
ssh "$HOST" "mv '$TARGET_DIR/.werkator/bin/werkdock.new' '$TARGET_DIR/.werkator/bin/werkdock' && chmod 755 '$TARGET_DIR/.werkator/bin/werkdock'"
|
ssh "$HOST" "mv '$INSTALL_DIR/bin/werkdock.new' '$INSTALL_DIR/bin/werkdock' && chmod 755 '$INSTALL_DIR/bin/werkdock'"
|
||||||
if ! ssh "$HOST" "'$TARGET_DIR/.werkator/bin/werkdock' doctor '$TARGET_DIR'"; then
|
if ! ssh "$HOST" "'$INSTALL_DIR/bin/werkdock' doctor '$TARGET_DIR'"; then
|
||||||
die "werkdock doctor failed on $HOST — install aborted"
|
die "werkdock doctor failed on $HOST — install aborted"
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
@@ -160,7 +201,7 @@ next to this repository, or point WERKDOCK_REPO/WERKDOCK_BINARY at your checkout
|
|||||||
upload_fragment() {
|
upload_fragment() {
|
||||||
[ -n "${WERKATOR_INIT_CONFIG:-}" ] || { echo ""; return 0; }
|
[ -n "${WERKATOR_INIT_CONFIG:-}" ] || { echo ""; return 0; }
|
||||||
[ -f "$WERKATOR_INIT_CONFIG" ] || die "init fragment missing: $WERKATOR_INIT_CONFIG"
|
[ -f "$WERKATOR_INIT_CONFIG" ] || die "init fragment missing: $WERKATOR_INIT_CONFIG"
|
||||||
local remote="$TARGET_DIR/.werkator/$(basename "$WERKATOR_INIT_CONFIG")"
|
local remote="$INSTALL_DIR/$(basename "$WERKATOR_INIT_CONFIG")"
|
||||||
scp -q "$WERKATOR_INIT_CONFIG" "$HOST:$remote"
|
scp -q "$WERKATOR_INIT_CONFIG" "$HOST:$remote"
|
||||||
echo "$remote"
|
echo "$remote"
|
||||||
}
|
}
|
||||||
@@ -174,43 +215,85 @@ ensure_instance_artifacts() {
|
|||||||
(cd "$REPO_ROOT" && ./gradlew runtimeBundle --console=plain -q)
|
(cd "$REPO_ROOT" && ./gradlew runtimeBundle --console=plain -q)
|
||||||
fi
|
fi
|
||||||
[ -f "$RUNTIME_BUNDLE" ] || die "runtime bundle missing: $RUNTIME_BUNDLE"
|
[ -f "$RUNTIME_BUNDLE" ] || die "runtime bundle missing: $RUNTIME_BUNDLE"
|
||||||
ensure_werkdock_binary
|
[ "$SANDBOX" = "docker" ] || ensure_werkdock_binary
|
||||||
}
|
}
|
||||||
|
|
||||||
# Uploads and unpacks the instance artifacts. The previous runtime stays as
|
# Uploads one file and verifies it arrived whole: a transfer that dies mid-way
|
||||||
# werkator.prev for one deployment as the rollback asset.
|
# (scp: Connection closed) otherwise leaves a truncated archive that unpacks into
|
||||||
deploy_instance() {
|
# a broken runtime. Retries twice, because a dropped WAN connection is not a reason
|
||||||
echo "==> Uploading runtime bundle and werkdock binary"
|
# to abort a deployment.
|
||||||
ssh "$HOST" "mkdir -p '$TARGET_DIR/.werkator/bin'"
|
upload_verified() {
|
||||||
scp -q "$RUNTIME_BUNDLE" "$HOST:$TARGET_DIR/.werkator/"
|
local src="$1" dest="$2"
|
||||||
scp -q "$WERKDOCK_BINARY" "$HOST:$TARGET_DIR/.werkator/bin/werkdock.new"
|
local local_sha remote_sha attempt
|
||||||
|
local_sha="$(sha256sum "$src" | cut -d' ' -f1)"
|
||||||
|
remote_sha="$(ssh "$HOST" "sha256sum '$dest' 2>/dev/null | cut -d' ' -f1" || true)"
|
||||||
|
if [ "$local_sha" = "$remote_sha" ]; then
|
||||||
|
echo " $(basename "$src"): already on the host, skipping"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
for attempt in 1 2 3; do
|
||||||
|
if scp -q "$src" "$HOST:$dest.part"; then
|
||||||
|
remote_sha="$(ssh "$HOST" "sha256sum '$dest.part' 2>/dev/null | cut -d' ' -f1" || true)"
|
||||||
|
if [ "$local_sha" = "$remote_sha" ]; then
|
||||||
|
ssh "$HOST" "mv '$dest.part' '$dest'"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
echo " checksum mismatch after transfer $attempt of $(basename "$src")" >&2
|
||||||
|
else
|
||||||
|
echo " transfer $attempt of $(basename "$src") failed" >&2
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
ssh "$HOST" "rm -f '$dest.part'" || true
|
||||||
|
die "cannot upload $src to $HOST:$dest — three attempts failed"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Uploads the instance artifacts, without touching the installed runtime: the
|
||||||
|
# service keeps running until swap_instance_runtime replaces it, so a failed
|
||||||
|
# transfer costs nothing but the transfer.
|
||||||
|
upload_instance_artifacts() {
|
||||||
|
if [ "$SANDBOX" = "docker" ]; then
|
||||||
|
echo "==> Uploading runtime bundle"
|
||||||
|
else
|
||||||
|
echo "==> Uploading runtime bundle and werkdock binary"
|
||||||
|
fi
|
||||||
|
ssh "$HOST" "mkdir -p '$INSTALL_DIR/bin'"
|
||||||
|
upload_verified "$RUNTIME_BUNDLE" "$INSTALL_DIR/$(basename "$RUNTIME_BUNDLE")"
|
||||||
|
if [ "$SANDBOX" != "docker" ]; then
|
||||||
|
upload_verified "$WERKDOCK_BINARY" "$INSTALL_DIR/bin/werkdock.new"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Swaps in the uploaded artifacts. The previous runtime stays as werkator.prev
|
||||||
|
# for one deployment as the rollback asset.
|
||||||
|
swap_instance_runtime() {
|
||||||
echo "==> Unpacking"
|
echo "==> Unpacking"
|
||||||
ssh "$HOST" "set -e
|
ssh "$HOST" "set -e
|
||||||
cd '$TARGET_DIR/.werkator'
|
cd '$INSTALL_DIR'
|
||||||
mv bin/werkdock.new bin/werkdock && chmod 755 bin/werkdock
|
[ ! -f bin/werkdock.new ] || { mv bin/werkdock.new bin/werkdock && chmod 755 bin/werkdock; }
|
||||||
rm -rf werkator.prev
|
rm -rf werkator.prev
|
||||||
[ ! -d werkator ] || mv werkator werkator.prev
|
[ ! -d werkator ] || mv werkator werkator.prev
|
||||||
tar xzf '$(basename "$RUNTIME_BUNDLE")'
|
tar xzf '$(basename "$RUNTIME_BUNDLE")'
|
||||||
'./werkator/bin/werkator' --version
|
'./werkator/bin/werkator' --version
|
||||||
'./bin/werkdock' version"
|
[ ! -x bin/werkdock ] || './bin/werkdock' version"
|
||||||
}
|
}
|
||||||
|
|
||||||
instance_install() {
|
instance_install() {
|
||||||
ensure_ssh
|
ensure_ssh
|
||||||
check_prerequisites
|
check_prerequisites
|
||||||
ensure_instance_artifacts
|
ensure_instance_artifacts
|
||||||
deploy_instance
|
upload_instance_artifacts
|
||||||
|
swap_instance_runtime
|
||||||
echo
|
echo
|
||||||
echo "==> Instance installed."
|
echo "==> Instance installed."
|
||||||
echo " Runtime: $WERKATOR_BIN"
|
echo " Runtime: $WERKATOR_BIN"
|
||||||
echo " werkdock: $TARGET_DIR/.werkator/bin/werkdock"
|
[ "$SANDBOX" = "docker" ] || echo " werkdock: $INSTALL_DIR/bin/werkdock"
|
||||||
echo " Next: tools/remote werkator repo-init, then instance-start"
|
echo " Next: tools/remote werkator repo-init, then instance-start"
|
||||||
}
|
}
|
||||||
|
|
||||||
# Refuse to swap the runtime under a running build; FORCE=1 overrides.
|
# Refuse to swap the runtime under a running build; FORCE=1 overrides.
|
||||||
require_idle() {
|
require_idle() {
|
||||||
local port
|
local port
|
||||||
port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' 2>/dev/null && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"' || true)"
|
port="$(ssh "$HOST" "cd '$REPO_DIR' 2>/dev/null && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"' || true)"
|
||||||
[ -n "$port" ] || return 0
|
[ -n "$port" ] || return 0
|
||||||
local current
|
local current
|
||||||
current="$(ssh "$HOST" "curl -s --max-time 5 http://127.0.0.1:$port/api/builds/current" || true)"
|
current="$(ssh "$HOST" "curl -s --max-time 5 http://127.0.0.1:$port/api/builds/current" || true)"
|
||||||
@@ -224,6 +307,9 @@ instance_update() {
|
|||||||
ensure_ssh
|
ensure_ssh
|
||||||
ensure_instance_artifacts
|
ensure_instance_artifacts
|
||||||
require_idle
|
require_idle
|
||||||
|
# upload first, stop second: a transfer that fails must not leave the host
|
||||||
|
# without a running service (measured on vm4006, 2026-09-03)
|
||||||
|
upload_instance_artifacts
|
||||||
local was_active=0
|
local was_active=0
|
||||||
if ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user is-active --quiet '$UNIT'"; then
|
if ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user is-active --quiet '$UNIT'"; then
|
||||||
was_active=1
|
was_active=1
|
||||||
@@ -232,7 +318,7 @@ instance_update() {
|
|||||||
echo "==> Stopping $UNIT"
|
echo "==> Stopping $UNIT"
|
||||||
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user stop '$UNIT'"
|
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user stop '$UNIT'"
|
||||||
fi
|
fi
|
||||||
deploy_instance
|
swap_instance_runtime
|
||||||
if [ "$was_active" = "1" ]; then
|
if [ "$was_active" = "1" ]; then
|
||||||
echo "==> Starting $UNIT"
|
echo "==> Starting $UNIT"
|
||||||
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user start '$UNIT' && sleep 3 && systemctl --user is-active '$UNIT'"
|
ssh "$HOST" "XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user start '$UNIT' && sleep 3 && systemctl --user is-active '$UNIT'"
|
||||||
@@ -249,40 +335,45 @@ instance_update() {
|
|||||||
# writing is init's — this script transports and invokes (step 23).
|
# writing is init's — this script transports and invokes (step 23).
|
||||||
repo_init() {
|
repo_init() {
|
||||||
ensure_ssh
|
ensure_ssh
|
||||||
[ -f "$ROOTFS" ] || die "rootfs archive missing: $ROOTFS — build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS"
|
[ "$SANDBOX" = "docker" ] || [ -f "$ROOTFS" ] ||
|
||||||
|
die "rootfs archive missing: $ROOTFS — build it with tools/build-bwrap-rootfs.sh or set WERKATOR_ROOTFS"
|
||||||
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
||||||
|
|
||||||
echo "==> Cloning the watched repository"
|
echo "==> Cloning the watched repository"
|
||||||
if ssh "$HOST" "test -d '$TARGET_DIR/werkator/.git'"; then
|
if ssh "$HOST" "test -d '$REPO_DIR/.git'"; then
|
||||||
echo " (already cloned, skipping)"
|
echo " (already cloned, skipping)"
|
||||||
else
|
else
|
||||||
ssh "$HOST" "git clone '$REPO_URL' '$TARGET_DIR/werkator'"
|
ssh "$HOST" "git clone '$REPO_URL' '$REPO_DIR'"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "==> Uploading the rootfs archive (skipped when unchanged)"
|
if [ "$SANDBOX" = "docker" ]; then
|
||||||
local rootfs_remote="$TARGET_DIR/.werkator/$(basename "$ROOTFS")"
|
echo "==> No rootfs needed (WERKATOR_SANDBOX=docker) — the build image is the repository's own Dockerfile"
|
||||||
local local_sha remote_sha
|
|
||||||
local_sha="$(sha256sum "$ROOTFS" | cut -d' ' -f1)"
|
|
||||||
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' 2>/dev/null | cut -d' ' -f1" || true)"
|
|
||||||
if [ "$local_sha" = "$remote_sha" ]; then
|
|
||||||
echo " (already on the host, skipping)"
|
|
||||||
else
|
else
|
||||||
scp -q "$ROOTFS" "$HOST:$rootfs_remote"
|
echo "==> Uploading the rootfs archive (skipped when unchanged)"
|
||||||
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' | cut -d' ' -f1")"
|
local rootfs_remote="$INSTALL_DIR/$(basename "$ROOTFS")"
|
||||||
[ "$local_sha" = "$remote_sha" ] || die "rootfs upload checksum mismatch"
|
local local_sha remote_sha
|
||||||
|
local_sha="$(sha256sum "$ROOTFS" | cut -d' ' -f1)"
|
||||||
|
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' 2>/dev/null | cut -d' ' -f1" || true)"
|
||||||
|
if [ "$local_sha" = "$remote_sha" ]; then
|
||||||
|
echo " (already on the host, skipping)"
|
||||||
|
else
|
||||||
|
scp -q "$ROOTFS" "$HOST:$rootfs_remote"
|
||||||
|
remote_sha="$(ssh "$HOST" "sha256sum '$rootfs_remote' | cut -d' ' -f1")"
|
||||||
|
[ "$local_sha" = "$remote_sha" ] || die "rootfs upload checksum mismatch"
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "==> Running werkator init${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
|
echo "==> Running werkator init${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
|
||||||
local fragment_remote
|
local fragment_remote
|
||||||
fragment_remote="$(upload_fragment)"
|
fragment_remote="$(upload_fragment)"
|
||||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
|
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
|
||||||
|
|
||||||
echo "==> Verifying the effective configuration"
|
echo "==> Verifying the effective configuration"
|
||||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null | grep -A4 'bwrap:' | head -5"
|
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' config:print 2>/dev/null | grep -A4 '$SANDBOX:' | head -5"
|
||||||
|
|
||||||
echo
|
echo
|
||||||
echo "==> Repository ready."
|
echo "==> Repository ready."
|
||||||
echo " Repo: $TARGET_DIR/werkator"
|
echo " Repo: $REPO_DIR"
|
||||||
echo " Next: fill git.account/git.token in $MACHINE_CONFIG if the origin is private,"
|
echo " Next: fill git.account/git.token in $MACHINE_CONFIG if the origin is private,"
|
||||||
echo " then tools/remote werkator instance-start"
|
echo " then tools/remote werkator instance-start"
|
||||||
}
|
}
|
||||||
@@ -306,10 +397,10 @@ repo_add() {
|
|||||||
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
ssh "$HOST" "test -x '$WERKATOR_BIN'" || die "no instance on $HOST — run instance-install first"
|
||||||
|
|
||||||
echo "==> Cloning $url as '$name'"
|
echo "==> Cloning $url as '$name'"
|
||||||
if ssh "$HOST" "test -d '$TARGET_DIR/$name/.git'"; then
|
if ssh "$HOST" "test -d '$SIBLING_DIR/$name/.git'"; then
|
||||||
echo " (already cloned, skipping)"
|
echo " (already cloned, skipping)"
|
||||||
else
|
else
|
||||||
ssh "$HOST" "git clone '$url' '$TARGET_DIR/$name'"
|
ssh "$HOST" "git clone '$url' '$SIBLING_DIR/$name'"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
|
# The instance fragment carries the sandbox policy (bwrap rootfs and werkdock
|
||||||
@@ -318,26 +409,26 @@ repo_add() {
|
|||||||
echo "==> Running werkator init in $name${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
|
echo "==> Running werkator init in $name${WERKATOR_INIT_CONFIG:+ --apply $(basename "${WERKATOR_INIT_CONFIG}")}"
|
||||||
local fragment_remote
|
local fragment_remote
|
||||||
fragment_remote="$(upload_fragment)"
|
fragment_remote="$(upload_fragment)"
|
||||||
ssh "$HOST" "cd '$TARGET_DIR/$name' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
|
ssh "$HOST" "cd '$SIBLING_DIR/$name' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'}"
|
||||||
|
|
||||||
echo "==> Checking the registry"
|
echo "==> Checking the registry"
|
||||||
# Grepped locally: the entry may name the path absolute or as ~/<name>, and
|
# Grepped locally: the entry may name the path absolute or as ~/<name>, and
|
||||||
# matching both is easier without a second layer of remote shell quoting.
|
# matching both is easier without a second layer of remote shell quoting.
|
||||||
if ssh "$HOST" "cat ~/.werkator.yml 2>/dev/null" |
|
if ssh "$HOST" "cat ~/.werkator.yml 2>/dev/null" |
|
||||||
grep -qE "path: *(~|$TARGET_DIR)/$name[[:space:]]*$"; then
|
grep -qE "path: *(~|$SIBLING_DIR)/$name[[:space:]]*$"; then
|
||||||
echo " (~/.werkator.yml already names this path)"
|
echo " (~/.werkator.yml already names this path)"
|
||||||
else
|
else
|
||||||
echo " not registered yet — add this entry to ~/.werkator.yml on $HOST:"
|
echo " not registered yet — add this entry to ~/.werkator.yml on $HOST:"
|
||||||
echo
|
echo
|
||||||
echo " repositories:"
|
echo " repositories:"
|
||||||
echo " - path: $TARGET_DIR/$name"
|
echo " - path: $SIBLING_DIR/$name"
|
||||||
echo " name: $name"
|
echo " name: $name"
|
||||||
echo
|
echo
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "==> Repository prepared."
|
echo "==> Repository prepared."
|
||||||
echo " Repo: $TARGET_DIR/$name"
|
echo " Repo: $SIBLING_DIR/$name"
|
||||||
echo " Next: fill git.account/git.token in $TARGET_DIR/$name/.git/werkator/.werkator.yml if the origin is private"
|
echo " Next: fill git.account/git.token in $SIBLING_DIR/$name/.git/werkator/.werkator.yml if the origin is private"
|
||||||
echo " (or once for all repositories in the 'defaults' block of ~/.werkator.yml),"
|
echo " (or once for all repositories in the 'defaults' block of ~/.werkator.yml),"
|
||||||
echo " then restart the service — the registry is read at start."
|
echo " then restart the service — the registry is read at start."
|
||||||
}
|
}
|
||||||
@@ -353,11 +444,11 @@ instance_start() {
|
|||||||
echo "==> Applying the instance fragment and generating the host integration (init --systemd)"
|
echo "==> Applying the instance fragment and generating the host integration (init --systemd)"
|
||||||
local fragment_remote
|
local fragment_remote
|
||||||
fragment_remote="$(upload_fragment)"
|
fragment_remote="$(upload_fragment)"
|
||||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'} --systemd"
|
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' init ${fragment_remote:+--apply '$fragment_remote'} --systemd"
|
||||||
|
|
||||||
local htaccess_src="$TARGET_DIR/werkator/.git/werkator/werkator.htaccess"
|
local htaccess_src="$REPO_DIR/.git/werkator/werkator.htaccess"
|
||||||
local htaccess="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/.htaccess"
|
local htaccess="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/.htaccess"
|
||||||
local maintenance_src="$TARGET_DIR/werkator/.git/werkator/werkator-maintenance.html"
|
local maintenance_src="$REPO_DIR/.git/werkator/werkator-maintenance.html"
|
||||||
local maintenance="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/werkator-maintenance.html"
|
local maintenance="$TARGET_DIR/doms/$WERKATOR_DOMAIN/subs/www/werkator-maintenance.html"
|
||||||
if ssh "$HOST" "test -f '$htaccess_src'"; then
|
if ssh "$HOST" "test -f '$htaccess_src'"; then
|
||||||
echo "==> Placing the generated Apache reverse proxy at $htaccess"
|
echo "==> Placing the generated Apache reverse proxy at $htaccess"
|
||||||
@@ -368,9 +459,9 @@ instance_start() {
|
|||||||
|
|
||||||
echo "==> Linking the units into ~/.config/systemd/user and enabling the service"
|
echo "==> Linking the units into ~/.config/systemd/user and enabling the service"
|
||||||
ssh "$HOST" "mkdir -p ~/.config/systemd/user && \
|
ssh "$HOST" "mkdir -p ~/.config/systemd/user && \
|
||||||
ln -sf '$TARGET_DIR/werkator/.git/werkator/$UNIT' ~/.config/systemd/user/ && \
|
ln -sf '$REPO_DIR/.git/werkator/$UNIT' ~/.config/systemd/user/ && \
|
||||||
ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.service' ~/.config/systemd/user/ && \
|
ln -sf '$REPO_DIR/.git/werkator/werkator-docker-prune.service' ~/.config/systemd/user/ && \
|
||||||
ln -sf '$TARGET_DIR/werkator/.git/werkator/werkator-docker-prune.timer' ~/.config/systemd/user/ && \
|
ln -sf '$REPO_DIR/.git/werkator/werkator-docker-prune.timer' ~/.config/systemd/user/ && \
|
||||||
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user daemon-reload && \
|
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user daemon-reload && \
|
||||||
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user restart '$UNIT' && \
|
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user restart '$UNIT' && \
|
||||||
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user status '$UNIT' --no-pager -l | head -12"
|
XDG_RUNTIME_DIR=/run/user/\$(id -u) systemctl --user status '$UNIT' --no-pager -l | head -12"
|
||||||
@@ -388,7 +479,7 @@ port_forward() {
|
|||||||
# the effective port, wherever it is configured (machine config or applied
|
# the effective port, wherever it is configured (machine config or applied
|
||||||
# fragment) — config:print is the single answer, not this script's parser
|
# fragment) — config:print is the single answer, not this script's parser
|
||||||
local remote_port
|
local remote_port
|
||||||
remote_port="$(ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"')"
|
remote_port="$(ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' config:print 2>/dev/null" | awk '/^server:/{f=1;next} f && /^ port:/{print $2; exit}' | tr -d '"')"
|
||||||
[ -n "$remote_port" ] || die "no server.port configured — run 'tools/remote werkator instance-start' first"
|
[ -n "$remote_port" ] || die "no server.port configured — run 'tools/remote werkator instance-start' first"
|
||||||
|
|
||||||
case "$COMMAND" in
|
case "$COMMAND" in
|
||||||
@@ -430,7 +521,7 @@ port_forward() {
|
|||||||
# CLI owns creation and format (step 23), this script only invokes it.
|
# CLI owns creation and format (step 23), this script only invokes it.
|
||||||
control_token() {
|
control_token() {
|
||||||
ensure_ssh
|
ensure_ssh
|
||||||
ssh "$HOST" "cd '$TARGET_DIR/werkator' && '$WERKATOR_BIN' control-token"
|
ssh "$HOST" "cd '$REPO_DIR' && '$WERKATOR_BIN' control-token"
|
||||||
}
|
}
|
||||||
|
|
||||||
case "$REPO" in
|
case "$REPO" in
|
||||||
|
|||||||
Reference in New Issue
Block a user