mi/werkator
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Step 22 C: metrics over the registry, documentation, PR-doc

Browse Source 
The metrics page sums the registered repositories' sizes; the instance
configuration is documented in docs/configuration.md, the architecture skill,
AGENTS.md and the step 22 plan (session C ticked, fairness decided FIFO).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
mhoennig
2026-09-02 09:06:13 +02:00
co-authored by Claude Fable 5.1
parent b66d26a03a
commit 58799ed0ed
8 changed files with 199 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/configuration.md
+48
View File
@@ -6,6 +6,7 @@ Werkator is configured via YAML files. Settings are merged from several sources
| Layer | Path | Committed to Git | Purpose |
|--------------------------|----------------------------|------------------|----------------------------------------------|
| Instance config | `~/.werkator.yml` | No | The repository registry and the instance-level settings; optional `defaults` below every repository (see below) |
| Project config | `.werkator.yml` | Yes | Shared team settings |
| Applied instance fragment | `.git/werkator/.werkator.applied.yml` | No | Instance parameters installed by `init --apply` |
| Repo installation config | `.git/werkator/.werkator.yml` | No | Machine- or user-specific overrides, secrets |
@@ -115,6 +116,53 @@ The pinned settings are stripped wherever they appear, in a build definition as
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
install/project config only, and only while nothing defines a build at all.
## `~/.werkator.yml` — the instance configuration
One Werkator instance serves a *set* of repositories (ADR 0009): one service, one port, one UI, one watcher loop.
The set and everything shared by it live in `.werkator.yml` in the home directory of the user running Werkator — one instance per OS user.
The file name is deliberately the same everywhere; only the location carries the meaning: home is the instance, the repository root is the project, `.git` is the machine.
`WERKATOR_HOME` overrides the directory the file is looked up in.
Without this file Werkator serves the current working directory exactly as before.
With it, the registry wins over the current directory: `werkator server` serves the registered repositories wherever it is started.
```yaml
werkator:
version:
since: "0.9.16" # like every other config file
repositories: # the registry; each entry is one served repository
- path: ~/repos/werkator # absolute, or relative to the home directory (~ expands)
- path: ~/repos/werkbaum
name: baum # optional; default is the directory basename
server: # the instance's server section — port, bind address, public URL, nginx
port: 18080
executor:
maxConcurrent: 2 # the global cap over all repositories
watcher:
pollInterval: 10s # one loop, one delay
defaults: # optional: repository-level keys merged BELOW every repository's own layers
git:
account: ci-bot
token: "…" # secrets may then live here instead of in each repository
gitea:
baseUrl: https://git.example.org
```
Key ownership once this file exists:
- **Instance-level** — read from this file alone: the whole `server` section, `executor.maxConcurrent`, and `watcher.pollInterval`.
A repository file still carrying one of them is ignored on that key, with one warning naming both files; it is never merged silently.
- **Repository defaults** — the `defaults` block, in the repository config schema: merged below each repository's project config, applied fragment, machine config, and branch layer, so a repository's own value always wins.
Pinning is unchanged: home defaults and the repository's machine config are both host-side layers, and a branch still cannot reach a pinned key.
- **Repository-level** — everything else stays in the repository's own files: `gitea.*`, `git.*` credentials, `builds`, retention, the other `watcher` keys (`pullRequestGate`, `newBranchMaxAge`, `fastForwardLocalRefs`), and the sandbox policy.
Repository names are the `--repo` selector of `werkator build`, `retry`, and `status` (without it a command means the current directory when served, otherwise the first registered repository) and will become the route segment of the web UI.
Two entries resolving to the same name abort the start, as does an entry that is not a git repository.
A repository whose configuration this Werkator must not read (see the version declaration) is skipped with an error; the others are served.
## Inspect the Effective Config
```bash
docs/plan/22-multi-repo.md
+8 -6
View File
@@ -55,10 +55,12 @@ The pinning model is untouched: pinned keys still come from each repo's machine
### C — The registry and N repositories
- Load the registry, build one `RepoContext` per entry; fail the start loudly on duplicate names or unreadable repos (config-version violations abort only that repo's registration, like branch-config violations fail only that branch).
- Watcher multiplexing: one poll cycle iterates the contexts (fetch, enqueue, prune per repo) with per-repo error isolation — one unreachable origin must not starve the others; `WatcherState` gains the repo dimension for the health banner.
- Startup recovery per repo; auto-build slots stay in each repo's `.git/werkator/`.
- CLI commands gain an optional repo selector and default to the current working directory, so `werkator status` inside a repo behaves as today.
- ~~Load the registry, build one `RepoContext` per entry; fail the start loudly on duplicate names or unreadable repos (config-version violations abort only that repo's registration, like branch-config violations fail only that branch).~~ — done 2026-09-02: `InstanceConfig` binds `~/.werkator.yml` (`ConfigLoader.homeDir`, `WERKATOR_HOME` overrides); `RepoRegistry` opens the contexts. The instance keys and the `defaults` block are applied inside `ConfigLoader.loadRaw`, so every `load(dir)` consumer sees them without knowing the file — the repository's copies of instance keys are dropped with one warning naming both files.
- ~~Watcher multiplexing: one poll cycle iterates the contexts (fetch, enqueue, prune per repo) with per-repo error isolation — one unreachable origin must not starve the others; `WatcherState` gains the repo dimension for the health banner.~~ — done 2026-09-02: `pollAll(repos)`, one guard per repository, `WatcherState.repositories`; the top-level fields aggregate (unchanged with one repository, name-prefixed with several). Isolation proven by test (one unreachable origin, the other still enqueues).
- ~~Startup recovery per repo; auto-build slots stay in each repo's `.git/werkator/`.~~ — done: `start(repos)` recovers each in its own guard; slots unchanged.
- ~~CLI commands gain an optional repo selector and default to the current working directory, so `werkator status` inside a repo behaves as today.~~ — done: `--repo <name>` (`RepoOption` mixin) on `build`, `retry`, `status`; default is the cwd when served, else the first registered repository.
- Also done: the pre-rename state-dir migration runs per opened repository; the metrics page's repository size sums the registered repositories (the disk metric is the first one's file store).
- Carried over to session D: `RunningBuild` still carries no repository (the "current builds" view and the worktree pruning cannot tell repositories apart); the controllers still serve `registry.current()` only; `docs/deployment.md` gets the registry setup with session E.
### D — Server, API, and UI scoping
@@ -74,7 +76,7 @@ The pinning model is untouched: pinned keys still come from each repo's machine
## Open Questions
- Fairness across repos when the global concurrency cap is contended (round-robin per repo vs. FIFO) — decide in session C with the real queue behavior at hand.
- ~~Fairness across repos when the global concurrency cap is contended (round-robin per repo vs. FIFO) — decide in session C with the real queue behavior at hand.~~ — decided 2026-09-02: FIFO. The executor's slot semaphore is already fair, so builds take slots in enqueue order across repositories; the watcher enqueues in registry order within one cycle, which is a fixed and inspectable bias rather than a scheduler. Round-robin per repository only when a real queue shows starvation.
- Whether buildenv rootfs trees should be shared across repos (today each repo unpacks its own under `.git/werkator/buildenv/`) — the natural answer is Werkdock's image store (step 21 session C), not instance-level state; until then duplicate unpacked rootfs trees are the accepted cost.
- ~~Whether `artifactKey` needs a repo prefix or stays globally unique by construction (random suffix) — decide in session B when the routes are designed.~~ — decided 2026-09-02: no prefix. The key is derived from pool name and start time, and both the results file and the artifact store are per repository, so it only ever has to be unique within one; the repo dimension enters through the route segment in session D, never through the key. A prefix would also change every existing artifact directory name.
@@ -82,6 +84,6 @@ The pinning model is untouched: pinned keys still come from each repo's machine
- Session A: ADR 0009 written (done 2026-09-01); the registry and key ownership land in `docs/configuration.md` together with the implementing sessions, since that reference describes implemented configuration only.
- ~~Session B: full suite green with `RepoContext` threaded through; no route or behavior change observable.~~ — done 2026-09-02.
- Session C: an instance with two registered repos builds pushes in both, with per-repo error isolation proven by a test (one broken origin, the other keeps building).
- ~~Session C: an instance with two registered repos builds pushes in both, with per-repo error isolation proven by a test (one broken origin, the other keeps building).~~ — done 2026-09-02 (`WatcherTest`: "one repository's unreachable origin neither stops nor silences the other").
- Session D: both repos browsable in one UI; single-repo installations keep their existing URLs.
- Session E: mih34 builds Werkator and Werkbaum from one service; `docs/deployment.md` describes the registry setup.
docs/prs/2026-09-02-PR#000-repo-registry.md
+132
View File
@@ -0,0 +1,132 @@
> **WARNING:** This document describes only the change applied in this PR.
> It may already be outdated once the next PR is merged.
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
## The Problem
PR #11 gave every repository-scoped code path an explicit `RepoContext`, but exactly one exists: the current working directory.
ADR 0009 wants one instance to serve a *set* of repositories from a registry in the instance configuration, with the instance-level settings (server, global concurrency, poll interval) owned by that file and optional defaults shared by every repository — and the watcher must poll every repository in a way that one unreachable origin cannot starve or silence the others.
Step 22 session C is that: the registry, N contexts, the multiplexed watcher, and a `--repo` selector for the CLI.
## Non-Goals
- Repository-scoped routes, API paths, and UI grouping (session D): the controllers still serve the current repository only.
- The rollout on mih34 with Werkbaum and the deployment documentation of the registry (session E).
- A per-repository concurrency cap below the global one, and round-robin fairness across repositories (decided FIFO, see The Solution).
## The Scenarios
### Feature: the instance configuration `~/.werkator.yml`
#### Background
- The file lives in the home directory of the user running Werkator — one instance per OS user; `WERKATOR_HOME` overrides the directory.
- It carries the registry (`repositories`), the instance-level keys (`server`, `executor.maxConcurrent`, `watcher.pollInterval`), and an optional `defaults` block in the repository config schema.
#### Scenario#000.01: The defaults block sits below every repository's own layers
So that one `git.account`/`git.token` or one `gitea.baseUrl` can be written once for every repository of the same forge, while a repository's own value always wins.
- **Given** a home config with `defaults`
- **When** a repository's effective config is loaded
- **Then** every key the repository's layers do not set comes from `defaults`, and every key they do set stays the repository's.
##### Verified by
- [the home config carries the registry, and its defaults sit below every repository layer](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
#### Scenario#000.02: Instance-level keys come from the home file alone
So that a `server.port` left in a repository's machine config can never silently compete with the instance's.
- **Given** a home config and a repository file still carrying `server`, `executor`, or `watcher.pollInterval`
- **When** the repository's effective config is loaded
- **Then** the whole `server` section, `executor` and `watcher.pollInterval` are the home file's
- **and** the repository's copies are dropped with one warning naming both files
- **and** the other `watcher` keys (the gates) stay the repository's.
##### Verified by
- [with a home config the instance keys come from it alone, and a repository's copies are ignored](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
- [the home config is version-checked like every other file](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
### Feature: the registry
#### Scenario#000.03: Every entry becomes a context, the start fails loudly on what cannot be served
So that an instance serving the wrong set never comes up looking healthy.
- **Given** a home config with `repositories`
- **When** the registry is opened
- **Then** each entry yields a `RepoContext` named after its directory unless the entry names it
- **and** an entry that is not a git repository or two entries resolving to the same name abort the start with a message naming the home file
- **and** a repository whose configuration this Werkator must not read is skipped with an error while the others are served
- **and** without a home config the registry is the current directory alone, exactly as before.
##### Verified by
- [RepoRegistryTest](../../src/test/kotlin/de/hoennig/werkator/repo/RepoRegistryTest.kt) (all five tests)
### Feature: the watcher over N repositories
#### Scenario#000.04: One repository's failure neither stops nor silences the others
So that a wrong token in one repository cannot stall the builds of every other one.
- **Given** two registered repositories, one with an unreachable origin
- **When** a poll cycle runs
- **Then** the other repository's due branches are enqueued
- **and** the state reports the failure under the failing repository's name, and per repository in `repositories`
- **and** a repository whose poll crashes reports it the same way and the cycle continues.
##### Verified by
- [one repository's unreachable origin neither stops nor silences the other](../../src/test/kotlin/de/hoennig/werkator/watcher/WatcherTest.kt)
- [a repository whose poll crashes reports it by name and the cycle goes on](../../src/test/kotlin/de/hoennig/werkator/watcher/WatcherTest.kt)
- the existing single-repository `WatcherTest` tests, proving the top-level state fields read as before
### Feature: the CLI selects a repository
#### Scenario#000.05: `--repo <name>` selects a registered repository; without it a command means the current directory
So that `werkator status` inside a repository behaves as it always did, and a registered repository can be addressed from anywhere.
- **Given** a registry with two repositories
- **When** `werkator status --repo second` runs
- **Then** the second repository's results are printed
- **and** an unknown name is a usage error (exit code 2) naming the registered repositories.
##### Verified by
- [--repo selects a registered repository; an unknown name is a usage error naming the registered ones](../../src/test/kotlin/de/hoennig/werkator/commands/StatusCommandTest.kt)
- [BuildCommandTest](../../src/test/kotlin/de/hoennig/werkator/commands/BuildCommandTest.kt), [RetryCommandTest](../../src/test/kotlin/de/hoennig/werkator/commands/RetryCommandTest.kt) (the default: the registry's current repository)
## The Solution
`InstanceConfig` binds the home file; `ConfigLoader` gained `homeDir`, `instanceFile()`, and `loadInstance()`.
The instance keys and the `defaults` block are folded in by `ConfigLoader.loadRaw` itself — `defaults` below the repository layers, `server`/`executor`/`watcher.pollInterval` overlaid on top and stripped from the repository files with one warning — so every existing consumer of `load(dir)` (the server command, the executor's slot count, the watcher's interval) sees the instance values without knowing the file exists.
`RepoRegistry` opens the contexts lazily on first use; `RepoConfiguration` provides `registry.current()` as the `RepoContext` bean the still-unscoped controllers use.
`Watcher.start(repos)` recovers each repository in its own guard, and `pollAll(repos)` polls each in its own guard, aggregating the per-repository reports into `WatcherState` (new `repositories` list; the top-level fields unchanged with one repository, name-prefixed with several, so the health banner needs no change).
`RepoOption` is a picocli mixin shared by `build`, `retry`, and `status`.
The pre-rename state-dir migration moved from the CLI runner into `RepoContexts.open`, so it runs per served repository; the metrics collector sums the registered repositories' sizes.
Fairness across repositories is decided FIFO: the executor's slot semaphore is already fair, and the watcher enqueues in registry order — a fixed, inspectable bias instead of a scheduler; round-robin only when a real queue shows starvation.
## Open Questions
- `RunningBuild` still carries no repository, so the current-builds view and the worktree pruning cannot tell repositories apart — session D, with the routes.
- With several repositories the `queuedBranches` list mixes their branch names unprefixed; session D scopes it with the UI.
## Additional Changes
- `docs/configuration.md`: the instance configuration documented, the layer table gained the home file.
- Architecture skill and AGENTS.md: registry, instance config folding, watcher multiplexing.
- `docs/plan/22-multi-repo.md`: session C ticked, fairness decided, carry-overs to session D.
## Prerequisite PRs
- PR #11 (`RepoContext` refactor).
## Follow-up PRs
- Session D: server/API/UI repo scoping.
- Session E: rollout on mih34 with Werkbaum, registry setup in `docs/deployment.md`.