werkator/AGENTS.md

Werkator — Agent Instructions

This file holds the shared, tool-agnostic instructions for all AI coding agents. Claude Code imports it from CLAUDE.md via @AGENTS.md; Claude-Code-specific instructions belong in CLAUDE.md, everything else here. Detailed guides live as Agent Skills under .claude/skills/ (SKILL.md format); they load on demand — see Skills.

Build and Test Commands

./gradlew build          # compile + ktlintCheck + test
./gradlew ktlintFormat   # auto-format before committing
./gradlew test           # run all tests (can be slow, prefer single test)
./gradlew unitTest       # only *UnitTest and *RestTest: no full Spring context, no real git or process
./gradlew integrationTest  # only *IntegrationTest
./gradlew pitest mutationCrapCheck mutationCrapCodeCharta  # mutation testing, hours on a first run; -PpitestThreads=4 for more minions
./gradlew test --tests "de.javagil.werkator.ApplicationContextIntegrationTest"  # example for running a single test class

Run the JAR directly:

java -jar build/libs/werkator.jar --help
java -jar build/libs/werkator.jar init

ktlintFormat must be run before build passes — the formatter is enforced as part of the check lifecycle.

Architecture Overview

Werkator is a lightweight, declarative CI/CD build system: git-centric, one instance per repository, builds native or in Docker, statuses reported to Gitea. It is a dual-mode application: CLI (interactive, status, config) and Server (HTTP, persistent web UI + JSON API). IMPORTANT: Before designing or modifying code in any production package, load the architecture skill — it holds the subsystem details (CLI wiring, server mode, web UI, config system, git access, build execution, watcher, metrics).

Package Structure

All production code lives under de.javagil.werkator, with sub-packages commands (picocli subcommands), config (YAML config loading and schema), git (git CLI access), gitea (Gitea commit-status API client), build (build execution, results, workspaces), artifacts (filesystem artifact store), repo (the RepoContext a repository is worked on through: checkout, results, artifact store, name), watcher (branch polling, auto-builds, startup recovery), metrics (system resource sampling and aggregation), forgejo (converting the committed build definitions into Forgejo Actions workflows, RFC 0006), and server (JSON API controllers, Thymeleaf UI, artifact serving, control token, watcher and metrics lifecycles). Tests mirror this structure under src/test/kotlin.

Hard Invariants

• exitProcess is called only from main() — never inside CliRunner.run(); this keeps the Spring context alive during tests.
• Nothing is scheduled during CLI runs or tests: the watcher poll loop and metrics sampling start only via an explicit start() in the server profile.
• Builds run detached in worktrees under .git/werkator/worktrees/<branchKey>; the primary checkout is never used for builds; never assume a single running build.
• Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a RepoContext, never through an implicit current directory: the executor serializes per (context, branch) under one global maxConcurrent and the optional per-repository maxConcurrentPerRepository (a build takes its repository's slot before a global one); pull-request target builds take no slot of these but one of their own pool, maxConcurrentPullRequestTargets and maxConcurrentPullRequestTargetsPerRepository (BuildPool, PR#119), each pool with its own queue, and they are serialized per (context, <head-branch>@<build>) instead of with their head branch, since they run in a fresh workspace; the watcher polls every context in its own guard. RepoRegistry opens one context per entry of the instance configuration ~/.werkator.yml (ADR 0009), or the current directory without one; the instance-level keys (server, executor, watcher.pollInterval) and the defaults block are folded into every repository's effective config by ConfigLoader itself, so no consumer reads the home file directly. Server routes carry the repository as /repos/<name>/… and /api/repos/<name>/…, with the unscoped form (/latest, /branches, /history, /api/builds/…) permanently meaning the served repository; / belongs to the instance since RFC 0003 and carries the home page; the pages stay per repository; what makes them one UI is the home page, the Repositories page (/repos, GET /api/repos, newest build first) and the repository line below the header naming the current and the conspicuous repositories (RFC 0001).
• When config keys change, three places must stay in sync: the WerkatorConfig data classes, the InitCommand templates, and docs/configuration.md.
• The build configuration stays expressible as a Forgejo Actions workflow (ADR 0010): a new key or concept in a build definition needs a workflow counterpart, named in its design; a feature request without one is reshaped until it has one, or stays out. Host-side keys (instance, server, executor, watcher, credentials, the pinned sandbox and trust policy) are exempt. The first step towards one workflow for both systems is RFC 0006.
• Every config file may declare werkator.version.since/below (the Werkator it is written for, never a format version — no API is involved). since is enforced in both directions, using ConfigVersions.FORMAT_BROKE_IN for "file predates a breaking change"; below only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
• A branch describes its own CI: its committed .werkator.yml is the branch layer (ConfigLoader.loadWithBranchLayer, used by the watcher per origin branch and by loadForWorktree at build time) and takes precedence over .git/project — including the whole builds section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (git), host/repository sections (server, gitea, executor, watcher), the docker (docker.enabled, docker.network, docker.socket) and werkdock (werkdock.enabled, werkdock.rootfs, werkdock.binary, werkdock.gradleHome) sandbox policies, the trust gate (requirePullRequest), the restart of the instance itself (restartsInstance), and explicitly host-authored triggers and privileged follow-up triggers (host execution, Docker socket access, or instance restart); ordinary sandboxed trigger.afterSuccessOf definitions remain branch-configurable. A branch must never reach credentials, disable its container or sandbox, change its network, hand itself the Docker socket, substitute a foreign rootfs, raise global concurrency, bypass its own pull-request gate, end the instance, or deploy itself; a branch's definitions apply to that branch alone. Stripping alone did not carry that: a branch layer writing a non-map where a block belongs (docker: null) had nothing to strip from while the merge replaced the host's block wholesale, so the shape is refused (checkLayerShape) and the pinned values are written back from the host's layers after merging (enforcePinned). builds.<name>: null stays allowed — a branch may say it runs no such build — and the host's builds.default is the fallback the pinned values are taken from when the branch empties it.
• A pull-request target build (trigger.onPullRequestTarget, Forgejo's pull_request_target) is pinned whole: it holds the host's secrets while it judges a change, so a branch can never change its command, environment or secrets, switch it off, add one, or fake its status. ConfigLoader.stripPinned drops every such definition from the branch layer (the host's and the branch's own) and enforcePinnedPullRequestTargets puts the host's back as the primary configuration resolves them, so not even a branch-rewritten builds.default reaches it; the executor resolves its settings from the primary configuration alone (BuildSettingsSource) and refuses the definition without its pull request. It runs at the base commit the forge reports, in a fresh workspace of its own (BranchWorkspaces.prepareFresh), never on the head's files, and publishes its status on the head commit; the watcher decides it per open, non-draft, non-fork pull request and head commit (PullRequestTargets), and re-runs keep the recorded BuildResult.pullRequest. Combined with another trigger key, requirePullRequest, builds.default or a follow-up it is refused at start. It shares nothing the repository's other builds write: the werkdock runner binds a fresh home beside its fresh workspace as /root (GitWorktreeWorkspaces.pullRequestHomeOf), never .git/werkator/buildenv/home, and no werkdock.gradleHome; the Docker runner mounts neither the Gradle volume nor the socket; it never runs natively (BuildRunner.start(pullRequestTarget = true)). A target without a sandbox or with docker.socket is refused at start.
• A build definition carries the complete description of its build, split in two: the trigger block (onPush, atTimes, afterSuccessOf, onPullRequestTarget, branches, activeWithin) says when and for which branches it runs, everything else what it does. afterSuccessOf makes a definition the follow-up of another one — a deployment is a build that follows a green build (PR#23): it runs at the predecessor's commit after every green run of it, whoever started that run, and the FollowUpTrigger that enqueues it is armed only by Watcher.start(). builds.default is the base every other definition inherits its settings — never its trigger — from. The split is structural so that a selector added to TriggerConfig later is non-inheritable by construction; writing a trigger key flat is refused, never ignored, because ignoring it leaves a build that silently stops running. A ! prefix in trigger.branches excludes and always wins.
• The inheritance is applied after all layers are merged: that order is what makes a build a branch invents inherit the host's sandbox policy instead of the data-class default, so the pinning also holds for a build the host has never heard of. Pinned are requirePullRequest, statusContext, restartsInstance, docker.enabled, docker.network, docker.socket, werkdock.enabled, werkdock.rootfs, werkdock.binary, werkdock.gradleHome, and explicitly host-authored or privileged follow-up triggers. restartsInstance says this build installs the version the instance is to run next (RFC 0005): on SUCCESS the executor drains — nothing new starts, everything executing is finished and recorded — and then closes the server context, so Restart=always brings the new version up. It is refused at start on builds.default and on a definition without trigger.branches, because a restart must never come from whatever branch went green last, and — with a registry — in every repository whose entry in ~/.werkator.yml lacks selfDeploys: true (PR#87), because a deployment block copied into a foreign repository must not restart the instance. Docker and werkdock are mutually exclusive per branch — enabling both is rejected at start. The section was called bwrap until v1.2.0 and is still read under that name, with a warning; the hard refusal waits for the release that sets ConfigVersions.FORMAT_BROKE_IN.
• builds or the legacy branches, never both: branches is read only while the merged config defines no build at all (builds.maxConcurrent is not one), and ignored with a warning as soon as one exists. The section is deprecated and goes away once the repositories have migrated; then ConfigVersions.FORMAT_BROKE_IN gets set and a leftover branches: key must be rejected by name — the version check alone cannot catch a file that declares no version.
• A resource is sized by whether anything waited for it, never by how full it is (RFC 0004): CPU from load above the threads, RAM from the share of quarter hours whose average memoryPressure reached SystemHealth.MEMORY_OVERLOAD_PERCENT — the two share the constant, so a machine the health verdict calls healthy cannot be red in the sizing. A fill level is named in the reason and may still prove slack for a downsize, but it never asks for a raise; swap occupancy asks for nothing at all. A disk is sized by the level it holds — the 95th percentile of the last week's quarter-hour maxima — and by the growth it shows on most days — the median of the daily deltas —, never by a single quarter hour or a step (PR#78); its window maximum is named and may veto a downsize, never ask for a raise. Every advised size is one that can be booked (BookableSteps, derived from the HostingEnvironment by the caller, no configuration key), and a downsize is advised only when the next bookable size down would still leave the target headroom.
• A build result outlives its artifacts and the branch it was built from (PR#59, PR#84): retentionPerBranch and retentionMaxAge decide over the artifacts only. A result beyond them is marked with BuildResult.artifactsPrunedAt, a result whose branch is gone from origin with BuildResult.branchGoneAt, and both stay in the history as the record of a build that ran; records go only beyond the newest BuildResultRepository.RECORDS_PER_NAME (100) of a name. The gone mark is cleared should the branch return; the pruned mark never is. What goes with the mark is what needs the artifacts: the stored artifacts and the log (the artifact store is pruned to history().withArtifacts()), the row's artifact link, and for a gone branch the exemptions that keep a dormant branch's last status and its latest green build, and the Gitea branch link. latestFor, latestGreenFor and latestPerName skip marked entries; history() is the one place that returns them, and the History answers with its newest 200 rows.
• A private repository gives out its metadata, never its content (RFC 0003, D8). Content is everything that names the work — the repository's name, its branch and build-definition names, commits, artifacts, logs and what stands in them — and it is redacted, or the route answers 404. Metadata says nothing about the content — that a build ran, when, how long, with what outcome, plus storage sizes and every other aggregate figure of the machine — and it is served to anyone. A route named after a repository (/repos/<name>/…, /api/repos/<name>/…) therefore answers a visitor without the control token exactly like an unserved name, since the name itself is content (RepoRouting); the unscoped form means the served repository and answers redacted.
• Web UI: server-rendered Thymeleaf plus one hand-written static/werkator.js — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; UiFormats and werkator.js must produce identical display formats.
• Git and Docker access shells out to the CLIs (GitCommandRunner, docker) — no JGit, no Docker SDK.

Testing

Tests use Kotest FunSpec with MockK; SpringExtension is registered globally in io.kotest.provided.ProjectConfig — do not add it per-spec. Every spec name ends in what it needs: *UnitTest (no Spring, no real git or process), *RestTest (a @WebMvcTest slice) or *IntegrationTest (full context, real git, processes, WireMock, containers); TestNamingConventionUnitTest enforces it. IMPORTANT: Before writing or changing tests, load the writing-tests skill — it holds the spec structure and the Spring-slice mocking patterns.

The nightly build mutation measures every method's Mutation-CRAP (werkcrappit): C² × (1 − MS)³ + C, C the complexity, MS the share of killed mutants. The hard limit is 15 (a method may score 15, not more); the gate maxMutationCrap in build.gradle.kts is a ratchet, lowered with every refactoring until it reaches 15, and never raised. Red in the code city from 12 is an urgent warning, not a limit: a method there is to be looked at, and a new or changed method should not land there without a reason. Since the score never falls below C, a method above 15 needs a smaller method, not more tests: split it along its domain steps, never by technical cuts such as "first half, second half".

File-Formatting

Markdown

Write documentation in English in Markdown files. In Markdown, use a single line per sentence. Keep sentences short. Column-align Markdown tables in the file itself (pad cells, align the separator row), so a table reads as a table in plain text and in diffs.

Documentation

• docs/Werkator-Konzept.md — product concept and target architecture (in German): git-centric CI, builds in Docker, one instance per repository, status reported back to Gitea.
• docs/configuration.md — configuration reference; keep in sync with WerkatorConfig and the init templates.
• docs/bootstrapping.md — how init prepares a repository.
• docs/deployment.md — running Werkator as a systemd user service behind an existing reverse proxy (init --systemd generates the unit).
• docs/werkator-migrationsplan.md — renaming a running installation from GitTally to Werkator: what the name fallback covers and what has to be moved by hand.
• docs/plan/ — the step-by-step rewrite plan; docs/plan/README.md explains how to execute a step, docs/plan/00-legacy-analysis.md summarizes the legacy bash script.
• docs/rfcs/ — requests for comments: proposals that are larger than one PR and not yet a decision (an accepted RFC becomes an ADR or a plan step).
• docs/prs/ — one document per pull request; every PR needs one, except a pure release PR (version bump plus release notes, no code). IMPORTANT: Before opening or finishing a pull request, load the pr-doc skill and write the PR-doc.

Every PR bumps version in build.gradle.kts and adds its own entry to the release notes (see Workflow). The two must agree — ReleaseVersionConsistencyUnitTest fails the build otherwise. A version is what identifies a running instance, so no change reaches main without one of its own.

Workflow

Follow docs/agentic-workflow.md, the workflow of all werk repositories: roles, steps of a PR, review rounds, PR documents, versions and the writing style. In addition, for werkator:

• The branch protection of main requires the statuses werkator, ai-review/gate and ai-review/codex and one approval, from the review agent ai-reviewer or from mi.
• Every PR adds its own <h2>vX.Y.Z</h2> entry at the top of src/main/resources/templates/releases.html; a change that users do not notice gets an entry that starts with "Internal:".

Key Architectural Decisions

All major decisions are in docs/adrs/. Run adr-status (after source .envrc) for a one-line summary of each. Decisions in force:

• Test framework: Kotest + MockK + WireMock + Testcontainers (ADR 0001)
• Gradle: 8.14.5 (ADR 0002)
• Spring Boot: 4.0.6 (ADR 0003)
• Rewrite architecture: JSON-file persistence behind a repository interface, server-rendered UI with JSON polling, no managed nginx — systemd unit behind the host's reverse proxy (ADR 0004)
• Managed nginx/TLS: revises ADR 0004 — an opt-in nginx+certbot container for hosts without a reverse proxy (e.g. Hostsharing), planned as docs/plan/13-nginx-tls.md (ADR 0005)
• Runtime bundle distribution: ./gradlew runtimeBundle builds a jlink-trimmed JRE + jar tarball for hosts without a Java runtime; GraalVM native image and a containerized runtime were rejected (ADR 0006)
• Build definitions: a top-level builds section of named builds with trigger blocks replaces the branch-owned autoBuild schedules (ADR 0007)
• bwrap build runtime: on hosts without root and without Docker (Hostsharing Managed Webspaces), builds run in a bwrap user-namespace sandbox over a prepared rootfs — filesystem isolation only, network and uid shared with the host; proot/fakechroot and unisolated native builds were rejected (ADR 0008)
• Multi-repo instance: one instance serves a registry of self-contained repositories (instance config in ~/.werkator.yml, repo config in each repo); revises the one-instance-per-repository tenet, implementation planned as docs/plan/22-multi-repo.md (ADR 0009)
• Forgejo Actions compatible configuration: the build configuration moves towards the Forgejo Actions workflow format; no new build-definition key without a workflow counterpart; one workflow for both systems as the first step (RFC 0006) (ADR 0010)

Skills

On-demand guides in the cross-tool SKILL.md format; agents with skill support load them automatically by description, all others should read the linked files when the topic comes up:

• architecture — subsystem details: CLI wiring, server mode, web UI, config, git access, build execution, watcher, metrics.
• writing-tests — Kotest/MockK conventions and Spring slice-test mocking patterns.
• pr-doc — how to write the mandatory per-PR documentation in docs/prs/.