werkator/docs/deployment.md

Werkator Deployment

This document describes how to run Werkator as a permanent service. The recommended setup is a systemd user service behind an existing reverse proxy. By default Werkator does not manage nginx or TLS certificates itself; it relies on the host's existing web server and certbot. For hosts without one, an opt-in managed nginx/TLS container is available, see Hosts Without a Reverse Proxy.

Prerequisites

• Linux with systemd.
• Java runtime (JRE 21) — or none, when using the self-contained runtime bundle, see Hosts Without a Java Runtime.
• git CLI on the PATH.
• docker CLI on the PATH, only if any branch uses docker.enabled (see configuration.md).
• A checked-out working tree of the repository to watch, with a remote named origin.

Jar Location Convention

Build the executable jar once:

./gradlew build
ls build/libs/werkator.jar

Copy the jar to a stable path outside any watched repository, by convention ~/bin/werkator.jar:

mkdir -p ~/bin
cp build/libs/werkator.jar ~/bin/werkator.jar

The systemd unit generated below points at the jar that was used to run init --systemd. So always run it via the stable path, not via build/libs/.

Install the Service

Initialize Werkator in the repository to watch (see bootstrapping.md for details):

cd /path/to/repo
java -jar ~/bin/werkator.jar init
# fill in git.account and git.token in .git/werkator/.werkator.yml
# review .werkator.yml

Generate the systemd user unit:

java -jar ~/bin/werkator.jar init --systemd

This writes .git/werkator/werkator-<repo-name>.service, .git/werkator/werkator.env, and the nightly Docker cleanup units (werkator-docker-prune.service/.timer), and prints the install commands:

ln -sf /path/to/repo/.git/werkator/werkator-<repo-name>.service ~/.config/systemd/user/werkator-<repo-name>.service
ln -sf /path/to/repo/.git/werkator/werkator-docker-prune.service ~/.config/systemd/user/werkator-docker-prune.service
ln -sf /path/to/repo/.git/werkator/werkator-docker-prune.timer ~/.config/systemd/user/werkator-docker-prune.timer
systemctl --user daemon-reload
systemctl --user enable --now werkator-<repo-name>.service
systemctl --user enable --now werkator-docker-prune.timer

The unit runs java -jar ~/bin/werkator.jar server with the repository as working directory and Restart=always. It also declares SuccessExitStatus=143, because a stop sends SIGTERM and the JVM exits 143 for it — without that line systemd journals every clean stop, a deployment's swap included, as Failed with result 'exit-code'. An instance installed before this was added keeps its old unit until init --systemd writes it again. The unit name contains the repository name, so several repositories can be served by one host, each with its own service and port.

Nightly Docker Cleanup

The werkator-docker-prune.timer runs docker system prune -af every night at 02:00 (host time), before the usual auto-build slots. It removes stopped containers, unused images, unused networks, and dangling build cache. The build images Werkator built itself (label org.werkator.build-inputs-sha256) survive six nights a week. A nightly build then does not start with a rebuild of its image, which can take minutes and counts toward the build timeout. The night to Sunday takes them as well, so the base image and its packages are at most a week old: Werkator builds an image without --pull, and keeps it until its Dockerfile changes. A build image replaced by a rebuild loses its tag but keeps the label; a docker image prune -f after the main step removes it. Unlike the legacy cleanup it does not prune named volumes — the per-repository Gradle cache volumes survive. A second step removes the anonymous volumes no container uses any more, told by the 64 hex digits Docker names them with: a build that removes a database container without -v leaves one behind every time. It does not use docker volume prune, because before Docker 23 that command takes every unused volume, the Gradle caches included. A host set up before v1.23.2 gets both steps by running init --systemd again and systemctl --user daemon-reload. The units are host-global (no repository name): with several Werkator instances on one host, every init --systemd generates the same files and the symlinks coincide. On hosts without a docker CLI the service is skipped, not failed (ExecCondition). Persistent=true catches up a missed run after downtime.

Expect a burst of builds right after the very first start: in a fresh clone every origin branch counts as new, so each branch with commits younger than watcher.newBranchMaxAge (default 5d) is built once — one build per branch, executed serially up to builds.maxConcurrent. Lower watcher.newBranchMaxAge before the first start, or enable requirePullRequest, to limit the initial backlog.

User services stop at logout unless lingering is enabled once per user:

loginctl enable-linger "$USER"

Serving Several Repositories

One instance serves a set of repositories (ADR 0009): one service, one port, one UI, one watcher loop, one control token. Adding a repository is editing a registry entry — never a data migration, because everything repository-specific already lives inside the repository (.git/werkator/: machine config with secrets, build results, auto-build slots, worktrees).

1. Clone it on the host, next to the ones already served:

   git clone https://gitea.example.com/<owner>/<repo>.git ~/repos/<repo>
   

2. Prepare it like any watched repository — the machine config with the git credentials, and the sandbox policy if the host needs one:

   cd ~/repos/<repo> && java -jar ~/bin/werkator.jar init
   # fill in git.account and git.token in .git/werkator/.werkator.yml
   

   Credentials shared by every repository of the same forge can live once in the instance file's defaults block instead (docs/configuration.md); they merge below each repository's own layers, so a repository may still override them. The repository's own .werkator.yml — its builds — is committed and comes with the clone.

3. Register it in ~/.werkator.yml of the user running the service:

   repositories:
     - path: ~/repos/werkator
     - path: ~/repos/<repo>
       name: <short-name>      # optional; default is the directory basename
   

   The name is the route segment (/repos/<name>/…) and the UI's switcher entry, so it must be unique: a duplicate aborts the start naming this file, and so does an entry that is no git repository. A repository whose configuration Werkator must not read (a version violation) is skipped with an error — the others keep building.

   Steps 1 and 2 are mechanical and can be done from the workstation:

   tools/remote --env-file .env.<instance> werkator repo-add https://gitea.example.com/<owner>/<repo>.git [<name>]
   

   It clones the repository next to the ones already served, runs init in it, and prints the registry entry. It does not write ~/.werkator.yml: that file is the instance's own — port, global concurrency, possibly shared credentials — and a script editing it in place would rewrite the operator's configuration behind their back. Cloning and initialising is mechanical; registering is a decision. A private https origin authenticates with the shared defaults.git.account/defaults.git.token of ~/.werkator.yml (the token travels via a one-shot GIT_ASKPASS on the host, never in a URL or process list); enter those once before cloning a private repository — without them only public origins clone.

4. Restart the service; startup recovery re-enqueues what was in flight:

   systemctl --user restart werkator-<repo-name>.service
   

The service unit keeps the name it was installed with — it is the instance's unit, not one repository's. With one registered repository every URL stays what it was; with several, the pages and the API carry the repository (/repos/<name>/…, /api/repos/<name>/…), the unscoped paths keep meaning the repository the instance was started in, and the navigation shows a switcher. The watcher polls every repository in its own guard: an unreachable origin is that repository's report in the health banner, and the others are polled regardless. executor.maxConcurrent is the global cap across all of them, and builds take slots in enqueue order (FIFO). executor.maxConcurrentPerRepository caps one repository below that, for a machine that can take a heavy build next to a light one but not two heavy ones.

Operating the Service

systemctl --user status werkator-<repo-name>.service     # state and last log lines
journalctl --user -u werkator-<repo-name>.service -f     # follow the log
systemctl --user restart werkator-<repo-name>.service    # restart (e.g. after config changes)
systemctl --user stop werkator-<repo-name>.service       # stop

Updating an Existing Installation

A restart is safe at any time: an in-flight build is recorded as INTERRUPTED and re-enqueued by the startup recovery. Waiting for an idle queue is still nicer, because the interrupted build starts over from scratch.

curl -s http://127.0.0.1:18080/api/builds/current      # ideally [] — nothing running

With a jar installation:

./gradlew build                                        # on the dev machine
scp build/libs/werkator.jar <user>@<host>:~/bin/werkator.jar.new
ssh <user>@<host>
  systemctl --user stop werkator-<repo-name>.service
  mv ~/bin/werkator.jar ~/bin/werkator.jar.bak         # rollback copy
  mv ~/bin/werkator.jar.new ~/bin/werkator.jar
  systemctl --user start werkator-<repo-name>.service
  systemctl --user is-active werkator-<repo-name>.service

With a runtime bundle (hosts without Java, see below):

./gradlew runtimeBundle                                # on the dev machine
scp build/distributions/werkator-runtime-linux-x64.tar.gz <user>@<host>:/tmp/werkator-new.tar.gz
ssh <user>@<host>
  systemctl --user stop werkator-<repo-name>.service
  mv ~/opt/werkator ~/opt/werkator.bak                 # rollback copy
  tar xzf /tmp/werkator-new.tar.gz -C /tmp/ && mv /tmp/werkator ~/opt/werkator
  ~/opt/werkator/bin/werkator --version                # expected: the new version
  systemctl --user start werkator-<repo-name>.service
  systemctl --user is-active werkator-<repo-name>.service
  rm -f /tmp/werkator-new.tar.gz

The tarball unpacks to a werkator/ directory, so it must not be extracted over ~/opt directly — unpack it in /tmp and move it into place, as above. Rollback is the reverse: stop, remove the new directory (or jar), move .bak back, start.

tools/remote --env-file .env.<instance> werkator instance-update does the same sequence for any host, not only the webspace layout it was written for. Three optional keys in the env file name what differs (see the script's header): WERKATOR_REPO_DIR (directory of the watched repository, which also names the systemd unit), WERKATOR_INSTALL_DIR (where the runtime bundle is unpacked), and WERKATOR_SANDBOX (werkdock, the default, or docker — a Docker host has no werkdock binary and no rootfs archive to upload). Their defaults are the layout instance-install creates, so an env file that names none of them behaves exactly as before. The upload happens before the service is stopped and every artifact is checksum-verified after the transfer, so a dropped connection costs the transfer and not the running service.

Then check https://<public-url>/ for the new version in the footer, and journalctl --user -u werkator-<repo-name>.service -n 50 for a clean start. Config file changes are not needed for an update; new keys take their defaults.

Self-Deployment

An instance that watches Werkator's own repository can install its merged commits itself, as a follow-up build (RFC 0005, step 1). Every other repository deploys with an ordinary afterSuccessOf definition; this one cannot, because its deploy build is a child of the process it replaces. Stopping the service ends the build, the build is recorded INTERRUPTED, and the startup recovery re-runs it — deploying again, restarting again. So the build never installs anything: it stages the new runtime beside the running one, proves it starts, and hands the rest to a transient systemd unit that the stop cannot end. Since step 2 the instance ends itself: a build with restartsInstance makes the executor stop taking builds, finish and record what is still executing, and then close its context — the transient unit only waits for the process to go, swaps into the gap, and starts the new version. That is the difference between asking and telling: the instance no longer has to be judged idle from outside, it decides when it is done.

The configuration is split the way follow-up builds always are. The repository's committed .werkator.yml says what the deployment does:

builds:
  default:
    buildCommand: ./gradlew --console=plain --no-daemon test runtimeBundleDir
  deploy:
    cleanCommand: ""                  # keeps the bundle the test build just produced
    buildCommand: tools/self-deploy
    artifactDirs: []

The host's part — when, for which branches, and that this one build runs natively — goes into the instance fragment installed with init --apply:

builds:
  deploy:
    trigger:
      afterSuccessOf: default
      branches: ["main"]
    statusContext: werkator/deploy
    restartsInstance: true
    werkdock:
      enabled: false

All four are pinned, so a branch can neither add the trigger nor widen it, nor hand its own build the host's filesystem.

On a multi-repository instance this block belongs to the watched repository alone (PR#87). Keep it in its own fragment, named by WERKATOR_SELF_DEPLOY_CONFIG in the env file: repo-init and instance-start apply it on top of WERKATOR_INIT_CONFIG, repo-add never does, and repo-add refuses a WERKATOR_INIT_CONFIG that sets restartsInstance. Mark the watched repository's registry entry in ~/.werkator.yml with selfDeploys: true; Werkator refuses restartsInstance in every other repository. The bundle has to be built in the sandbox, because only the sandbox has a JDK and therefore jlink; it travels to the deployment through the shared worktree, which is why deploy sets cleanCommand: "". runtimeBundleDir builds it as a directory and runtimeBundle packs that directory as a tarball: a self-deployment unpacks the bundle on the machine that built it, so it takes the directory and no archive is written at all, while a transfer from a developer machine takes the tarball. It is deliberately not an artifact: 91 MiB in the worktree, kept retentionPerBranch times, with nobody to read it.

tools/self-deploy takes no parameters and derives everything from where it runs — the primary checkout from the common git directory, the unit name from its basename, the installation directory from that unit's ExecStart, the port from config:print. It refuses to run in the primary checkout, refuses an installation that is not a runtime bundle, refuses a unit that watches a different repository, and refuses to install anything when the worktree holds no bundle, which is what a restart of deploy alone looks like.

The build stages and verifies, and stops there: the bundle is put beside the running runtime as werkator.new and its launcher is run natively, which is what catches a jlinked JRE that will not start on the host. Nothing of the installation is touched, so a broken bundle costs a red build and no downtime.

The swap itself happens between the stop and the start, never while the instance is running. A JVM whose runtime directory is replaced underneath it keeps resolving its own java.home to what is now a different runtime, and from that moment it cannot start a single child process — no git fetch, no build command. The runtime that is replaced stays as werkator.prev, the last known good version.

All of it runs in a transient unit scheduled with systemd-run --user, which is also the watchdog: it waits for the instance to end itself (at most ten minutes, then it proceeds anyway and says so), stops the service, swaps, starts it, waits up to two minutes for http://127.0.0.1:<port>/api/watcher to answer, and when it does not, moves the new runtime aside as werkator.failed, puts werkator.prev back and restarts again. Both outcomes are in the journal:

journalctl --user -u werkator-self-restart-<repo-name> -n 50

The transient unit is named after the service it restarts, because transient unit names are global to the user manager and two instances under one account must not collide on one name.

How it waits depends on what the instance can do for itself. The executor sets WERKATOR_RESTARTS_INSTANCE=true in the environment of a build that restarts the instance, so tools/self-deploy knows without asking for a version; it then reads the service's MainPID while the instance is still the running one and hands it to the transient unit, which waits for that pid to go. The systemctl stop afterwards is not what ends the instance — it is what keeps Restart=always from bringing the old runtime back into the middle of the swap, since RestartSec=30 is exactly the gap the swap needs. Against an instance without the setting, the unit falls back to the old condition and waits for the running and pending counts of /api/repos to reach zero.

The rollback stays outside Werkator either way: an instance that does not start cannot put the previous runtime back itself.

tools/remote … instance-update runs the same script for the swap, only without scheduling a restart, since it stops and starts the service itself. The two therefore install a bundle exactly alike.

A host without a systemd user manager has no self-deployment: the script says so and fails.

Control Token

Viewing is public by design: build states, logs and artifacts are readable without any login, so they can be linked from Gitea, chats or tickets. That is safe as long as the builds themselves handle no real secrets — Werkator has no per-endpoint gating, so an installation whose build output could contain credentials must stay off the public internet (reverse proxy with access control, or server.bindAddress: 127.0.0.1). Only the three mutating actions — restart, cancel, delete — require the control token from .git/werkator/control-token, a random secret the server generates on first start (mode 0600; delete the file to rotate it).

The token is never embedded in a page. The first time you press one of the control buttons, the browser asks for it once and keeps it in localStorage for that browser; a rejected token is dropped and asked for again. Read it on the host:

cat ~/<repo>/.git/werkator/control-token

For scripts, pass it as a header — it is not accepted as a query parameter, because URLs end up in access logs and browser history:

curl -X POST -H "X-werkator-Token: $(cat .git/werkator/control-token)" \
  "https://ci.example.org/api/builds/restart?branch=main&atOriginHead=true"

A restart comes in the two forms every build row offers (PR#150):

• POST /api/builds/<artifactKey>/restart builds that run again: its commit and its build definition.
• POST /api/builds/restart?branch=<name>&atOriginHead=true builds the newest commit of the branch with the build definition of <name>.

A restart that cannot run answers 409 with the reason in error. The newest commit is refused for a pull-request target build, for a follow-up build (trigger.afterSuccessOf) and for a branch gone from origin; the same commit is refused for a branch gone from origin. Without atOriginHead the second form repeats the latest recorded commit of <name>, as before.

Environment File

.git/werkator/werkator.env is loaded by the unit as EnvironmentFile. It only tunes the JVM process, e.g. JAVA_OPTS=-Xmx256m. All Werkator configuration lives in the YAML files described in configuration.md, not in environment variables. init --systemd never overwrites an existing environment file.

Reverse Proxy (nginx)

Bind Werkator to localhost — the default since v0.9.9 — and set the public URL in .werkator.yml:

server:
  bindAddress: 127.0.0.1
  port: 18080
  publicBaseUrl: "https://ci.example.org/"

publicBaseUrl is used for all links posted to Gitea, so it must be the externally reachable URL.

Add a server block to the host's nginx:

server {
    listen 443 ssl;
    server_name ci.example.org;

    ssl_certificate     /etc/letsencrypt/live/ci.example.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/ci.example.org/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:18080;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Obtain and renew the certificate with the host's existing certbot, e.g.:

sudo certbot --nginx -d ci.example.org

This replaces the legacy script's managed nginx/Let's Encrypt Docker container for hosts that have their own web server.

Hosts Without a Java Runtime (Runtime Bundle)

Some hosts provide git and Docker but no Java runtime and no way to install one, e.g. Hostsharing container servers. For these, Werkator ships as a self-contained runtime bundle: a jlink-trimmed JRE, werkator.jar, and a launcher script in one tarball (ADR 0006).

Build the bundle on a Linux x86_64 machine whose glibc is not newer than the target's:

./gradlew runtimeBundle
ls build/distributions/werkator-runtime-linux-x64.tar.gz

Copy and unpack it on the target host, by convention to ~/opt/werkator:

scp build/distributions/werkator-runtime-linux-x64.tar.gz user@host:/tmp/
ssh user@host 'mkdir -p ~/opt && tar -xzf /tmp/werkator-runtime-linux-x64.tar.gz -C ~/opt'

Then use ~/opt/werkator/bin/werkator wherever this document says java -jar ~/bin/werkator.jar:

cd /path/to/repo
~/opt/werkator/bin/werkator init
~/opt/werkator/bin/werkator init --systemd

init --systemd detects the bundle automatically: the generated unit's ExecStart points at the bundle's jre/bin/java and lib/werkator.jar, so the install commands printed by init --systemd work unchanged. JAVA_OPTS from the environment file applies as usual.

To update Werkator, stop the service, unpack the new bundle over ~/opt/werkator, and restart the service.

Hosts Without a Reverse Proxy (Managed nginx/TLS)

Some hosts provide Docker but no root access and no host web server, e.g. Hostsharing managed container environments. For these, Werkator can manage its own nginx+certbot Docker container (ADR 0005). This is opt-in; where a host web server exists, prefer the reverse-proxy setup above.

Enable it in the server section of the configuration:

server:
  port: 18080
  nginx:
    enabled: true
    serverName: ci.example.org
    httpPort: 8080
    httpsPort: 8443
    letsencryptEmail: admin@example.org

On server start, Werkator writes the nginx configuration, starts a labelled nginx container publishing httpPort and httpsPort, obtains a Let's Encrypt certificate via a certbot container (webroot mode), and restarts nginx with the full HTTPS configuration. A renewal check runs daily; certificates and nginx state persist in server.nginx.stateDir across restarts. On shutdown the container is removed. All nginx and certificate failures are non-fatal warnings — the plain HTTP server keeps running without the proxy.

serverName must be a public DNS name pointing at the host, reachable from the internet on port 80/443 (directly or via a port forward to httpPort/httpsPort), otherwise the ACME challenge fails. The nginx container cannot reach localhost of the host, so the proxy upstream defaults to serverName; set server.nginx.upstreamHost if the host is reachable under a different name from inside containers. With the managed nginx, set server.bindAddress: 0.0.0.0 explicitly (or an address reachable from the Docker network) — the default 127.0.0.1 makes Werkator unreachable for the proxy container. See configuration.md for all server.nginx.* keys.

Hostsharing Managed Webspace

The third deployment variant (plan step 21, verified live on a real webspace): no root, no Docker daemon, no own reverse proxy. Werkator runs as a systemd user service on the assigned localhost port ("eigener Serverdienst"), the platform's managed Apache terminates TLS and proxies via .htaccess, and builds run in the bubblewrap sandbox executed by the werkdock CLI (ADR 0008, step 21 session C).

Werkator is never built on the webspace: the runtime bundle and the werkdock binary are built locally and uploaded (ADR 0006). The werkdock binary is rebuilt from its checkout whenever a source there is newer than it, and the version going up is printed before the upload — otherwise a binary built weeks ago outlives every git pull and is shipped unnoticed. All steps are driven by tools/remote; commands name their role — instance-* manages the installed Werkator, repo-* the repository it watches. The script multiplexes all of its SSH and SCP calls over one connection (an OpenSSH control master under $XDG_RUNTIME_DIR, kept for ten minutes), because a Hostsharing host refuses rapid successive connections from one address and a single instance-update used to open about ten of them. Each instance is a pair of files (step 23): a transport env file selected with --env-file (default .env), and a YAML fragment in the configuration schema, named by its WERKATOR_INIT_CONFIG key and installed remotely via werkator init --apply — e.g. .env.mih34 + .env.mih34.yml, both gitignored. The fragment carries the Werkator configuration (server.port, publicBaseUrl, systemd limits, builds.default.werkdock.*); the env file only says where and how to reach the host. A self-deploying instance adds a second fragment, WERKATOR_SELF_DEPLOY_CONFIG, for the watched repository alone, see Self-Deployment.

tools/remote --env-file .env.mih34 werkator check-prerequisites  # uploads werkdock, runs its doctor
tools/remote --env-file .env.mih34 werkator instance-install     # upload + unpack the runtime bundle and werkdock
tools/remote --env-file .env.mih34 werkator repo-init            # clone the watched repo, rootfs archive, init --apply
tools/remote --env-file .env.mih34 werkator instance-start       # init --apply --systemd, place .htaccess, enable the unit
tools/remote --env-file .env.mih34 port-forward start            # browser tunnel while no public domain is set up

Layout on the host: the watched repository at $WERKATOR_PATH/werkator/, the unpacked runtime at $WERKATOR_PATH/.werkator/werkator/, the werkdock binary at $WERKATOR_PATH/.werkator/bin/werkdock. That is the default, not a requirement: WERKATOR_REPO_DIR, WERKATOR_INSTALL_DIR and WERKATOR_SANDBOX bend it to an installation that predates the script, see Updating an Existing Installation. The rootfs archive is imported once per source into werkdock's image store (~/.werkdock), shared by every repository of the user. Fill git.account/git.token in the machine config when the origin is private, and make the user's services survive logout with loginctl enable-linger.

Updates are one command, refused while a build runs (FORCE=1 overrides):

tools/remote --env-file .env.mih34 werkator instance-update

The previous runtime stays as .werkator/werkator.prev for one deployment as the rollback asset. For the few seconds instance-update restarts the service, nothing listens on its port; the generated .htaccess maps that refused connection (ErrorDocument 502/503/504) to a static werkator-maintenance.html placed next to it by instance-start, so the page reads "Werkator is restarting" instead of Apache's default error page or a hung request.

Werkator serves its own /robots.txt, which keeps crawlers off the artifact and branch-file routes (test reports, coverage, binaries are an unbounded URL space) and the JSON API; no proxy needs a rule for it. Crawlers that ignore robots.txt are a matter for the proxy in front of Werkator, e.g. a user-agent block in the .htaccess.

The /system page (PR#27) states whether the machine is healthy, under pressure or overloaded — from RAM and disk headroom, the load average per core, the kernel's pressure-stall counters and swap, all of which describe the whole shared host, not only Werkator's builds — and keeps four hours of 15-second samples, a day of minutes and a month of quarter hours with their peaks, so a resource that was ever critical in the last month is named as evidence for booking more RAM, CPU or disk. From seven days of history on, the verdict panel also advises on the booking: one chip per resource, green when the size fits, red with the recommended size when an upgrade is due (a RAM or disk peak at 90 %, a disk filling towards it within a month, a load that keeps exceeding the cores), yellow with the smaller size when a downsize is possible (RAM as soon as a smaller bookable size holds the peak with its headroom, CPU and disk only from a peak that never reached 40 %); the chip's tooltip names the numbers behind it. The history lives in system-metrics-state.json in the instance's state home, ${XDG_STATE_HOME:-~/.local/state}/werkator/ (a few hundred kilobytes, rewritten every minute); an older installation's file in the artifact root is moved there at the first start. A host with several filesystems — on a Managed Webspace the SSD home directory and the HDD storage tree — lists them under metrics.disks in ~/.werkator.yml (see the configuration reference) to get one gauge, one history, and one sizing chip per filesystem. The /system page's disk metric is quota-aware (PR#16): on a Managed Webspace the binding limit is usually the package's group quota, not the free space of the shared host volume, so a disk total there is the quota's soft limit — the info line names it (group quota <package>, hard limit … GiB) instead of showing the host's full disk size.