Merge PR #12: Repo-Registry — mehrere Repositories je Instanz
Die Instanz liest ihre Repositories aus ~/.werkator.yml (ADR 0009); Watcher, Build-Ausführung und Artefakte arbeiten je Repository. Bisher lief sie auf einem Vorab-Build dieses Branches — mit dem Merge entspricht main wieder dem, was produktiv läuft. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
mhoennig
@@ -63,7 +63,7 @@ Three places must stay in sync when config keys change: the `WerkatorConfig` dat
|
|||||||
|
|
||||||
## Repository Context
|
## Repository Context
|
||||||
|
|
||||||
Everything repository-scoped goes through a `RepoContext` (`repo` package, ADR 0009): the primary checkout (`workingDir`), the repository's `BuildResultRepository` (`.git/werkator/build-results.json`), its `ArtifactStore` (keyed by the repository path), and a short `name` defaulting to the directory basename — the future route segment. `RepoContexts.open(dir)` builds one; `RepoConfiguration` provides the single current-directory context as a bean, and the `BuildResultRepository`/`ArtifactStore` beans are that context's, so code that still injects them sees the same objects. Git access and config loading stay path-based services taking `repo.workingDir`. The context object is the identity (executor pools, watcher memory are keyed by it), so exactly one is opened per repository; the registry of step 22 session C opens one per entry. Not yet repository-scoped and left for that session: `StateDirMigration` (once per process on the cwd), the metrics collector's repository size, `ServerCommand`'s config, and `RunningBuild`, which carries no repository yet.
|
Everything repository-scoped goes through a `RepoContext` (`repo` package, ADR 0009): the primary checkout (`workingDir`), the repository's `BuildResultRepository` (`.git/werkator/build-results.json`), its `ArtifactStore` (keyed by the repository path), and a short `name` defaulting to the directory basename — the future route segment. `RepoContexts.open(dir)` builds one (running the pre-rename state-dir migration for that repository on the way); `RepoRegistry` opens one per entry of the instance configuration's `repositories` — or the current directory without a registry — lazily on first use and loudly: a non-repository entry or a duplicate name aborts the start naming the home file, a repository whose config must not be read (`ConfigException`) is skipped with an error. `RepoConfiguration` provides `registry.current()` (the cwd when served, else the first entry) as the `RepoContext` bean for the still-unscoped controllers, and the `BuildResultRepository`/`ArtifactStore` beans are that context's. The `--repo` mixin (`RepoOption`) selects by name in `build`, `retry`, and `status`. Git access and config loading stay path-based services taking `repo.workingDir`; the instance configuration (`~/.werkator.yml`, `ConfigLoader.homeDir`/`WERKATOR_HOME`, bound as `InstanceConfig`) is folded in by `ConfigLoader.loadRaw` itself — its `defaults` below every repository layer, its `server`/`executor`/`watcher.pollInterval` overlaid on top and stripped from the repository files with one warning — so every consumer of `load(dir)` sees the instance values without knowing the file. The context object is the identity (executor pools, watcher memory are keyed by it), so exactly one is opened per repository. Not yet repository-scoped: `RunningBuild` carries no repository, so `currentBuilds()` and the worktree pruning cannot tell repositories apart (session D, with the routes).
|
||||||
|
|
||||||
## Build Execution
|
## Build Execution
|
||||||
|
|
||||||
@@ -77,7 +77,7 @@ The runtime is selected per build behind the `BuildRunner` interface: `Dispatchi
|
|||||||
|
|
||||||
## Watcher
|
## Watcher
|
||||||
|
|
||||||
`Watcher` replaces the legacy blocking main loop with a non-blocking fixed-delay poll cycle over a `RepoContext` (`start(repo)`, `poll(repo)`, `recoverOnStartup(repo)`; what it remembers per repository — the logged fetch error, the deprecation warning, the cached branch definitions — lives in a `RepoWatch` keyed by context, while `WatcherState` is still one per instance): fetch origin, enqueue due branches (changed local, recent new origin, due auto-build slots) via `BuildExecutor`, then prune results, artifacts, and stale worktrees. Branches whose build has `requirePullRequest` are enqueued only while their head commit matches a pull-request head, detected without an API token by listing `refs/pull/*/head` via `git ls-remote` (lazily, at most once per poll cycle); manual `build` commands bypass this gate, and `watcher.pullRequestGate: false` disables it globally for plain-git origins without pull-request refs. Which builds are due is decided per branch from that branch's own definitions (`definitionsFor`): the primary config with the branch's committed `.werkator.yml` merged on top, cached per branch by its head commit *and* the primary config it was merged with, so the `git show` runs only when the branch moved while an edited machine or project config still takes effect on the next poll, and falling back to the primary definitions when that config is unreadable. Nothing is scheduled until `Watcher.start()` is called explicitly (server/watch mode) — CLI commands and tests never start the loop. "Already built" is tracked via the result repository, not by moving local branch refs.
|
`Watcher` replaces the legacy blocking main loop with a non-blocking fixed-delay poll cycle over the registry's contexts (`start(repos)`, `pollAll(repos)`, `poll(repo)` for one; `recoverOnStartup(repo)` per repository, each in its own guard). Every repository is polled in its own guard — a crash or an unreachable origin is that repository's report in `WatcherState.repositories`, and the next one is polled regardless; the top-level `WatcherState` fields aggregate the reports, reading exactly as before with one repository and prefixed by the repository name with several. What the watcher remembers per repository — the logged fetch error, the deprecation warning, the cached branch definitions — lives in a `RepoWatch` keyed by context. Per repository the cycle is: fetch origin, enqueue due branches (changed local, recent new origin, due auto-build slots) via `BuildExecutor`, then prune results, artifacts, and stale worktrees. Branches whose build has `requirePullRequest` are enqueued only while their head commit matches a pull-request head, detected without an API token by listing `refs/pull/*/head` via `git ls-remote` (lazily, at most once per poll cycle); manual `build` commands bypass this gate, and `watcher.pullRequestGate: false` disables it globally for plain-git origins without pull-request refs. Which builds are due is decided per branch from that branch's own definitions (`definitionsFor`): the primary config with the branch's committed `.werkator.yml` merged on top, cached per branch by its head commit *and* the primary config it was merged with, so the `git show` runs only when the branch moved while an edited machine or project config still takes effect on the next poll, and falling back to the primary definitions when that config is unreadable. Nothing is scheduled until `Watcher.start()` is called explicitly (server/watch mode) — CLI commands and tests never start the loop. "Already built" is tracked via the result repository, not by moving local branch refs.
|
||||||
After the enqueue decision — and only after it, because a local ref lagging behind origin *is* the change signal — the cycle fast-forwards the primary checkout's local branch refs to their origin counterparts (`watcher.fastForwardLocalRefs`, `GitService.fastForwardLocalBranches`), so build tools reading the shared `.git` from a worktree see the refs they expect; diverged or ahead branches are never touched. Auto-build slot state lives in `.git/werkator/auto-builds.json`; watcher health is exposed via `Watcher.state()`.
|
After the enqueue decision — and only after it, because a local ref lagging behind origin *is* the change signal — the cycle fast-forwards the primary checkout's local branch refs to their origin counterparts (`watcher.fastForwardLocalRefs`, `GitService.fastForwardLocalBranches`), so build tools reading the shared `.git` from a worktree see the refs they expect; diverged or ahead branches are never touched. Auto-build slot state lives in `.git/werkator/auto-builds.json`; watcher health is exposed via `Watcher.state()`.
|
||||||
|
|
||||||
## System Metrics
|
## System Metrics
|
||||||
|
|||||||
@@ -37,7 +37,7 @@ All production code lives under `de.hoennig.werkator`, with sub-packages `comman
|
|||||||
- `exitProcess` is called only from `main()` — never inside `CliRunner.run()`; this keeps the Spring context alive during tests.
|
- `exitProcess` is called only from `main()` — never inside `CliRunner.run()`; this keeps the Spring context alive during tests.
|
||||||
- Nothing is scheduled during CLI runs or tests: the watcher poll loop and metrics sampling start only via an explicit `start()` in the `server` profile.
|
- Nothing is scheduled during CLI runs or tests: the watcher poll loop and metrics sampling start only via an explicit `start()` in the `server` profile.
|
||||||
- Builds run detached in worktrees under `.git/werkator/worktrees/<branchKey>`; the primary checkout is never used for builds; never assume a single running build.
|
- Builds run detached in worktrees under `.git/werkator/worktrees/<branchKey>`; the primary checkout is never used for builds; never assume a single running build.
|
||||||
- Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls a context. Today exactly one context exists, the current working directory; the registry (step 22 session C) opens one per entry.
|
- Everything repository-scoped (results, artifacts, worktrees, git and config access) goes through a `RepoContext`, never through an implicit current directory: the executor serializes per (context, branch) under one global `maxConcurrent`, the watcher polls every context in its own guard. `RepoRegistry` opens one context per entry of the instance configuration `~/.werkator.yml` (ADR 0009), or the current directory without one; the instance-level keys (`server`, `executor`, `watcher.pollInterval`) and the `defaults` block are folded into every repository's effective config by `ConfigLoader` itself, so no consumer reads the home file directly.
|
||||||
- When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
|
- When config keys change, three places must stay in sync: the `WerkatorConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
|
||||||
- Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
|
- Every config file may declare `werkator.version.since`/`below` (the Werkator it is written for, never a format version — no API is involved). `since` is enforced in both directions, using `ConfigVersions.FORMAT_BROKE_IN` for "file predates a breaking change"; `below` only warns. A violation aborts the start for the machine and project config, but fails only that branch's builds for a branch config.
|
||||||
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and bubblewrap (`bwrap.enabled`, `bwrap.rootfs`, `bwrap.werkdock`) sandbox policies, and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone.
|
- A branch describes its own CI: its committed `.werkator.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker (`docker.enabled`, `docker.network`) and bubblewrap (`bwrap.enabled`, `bwrap.rootfs`, `bwrap.werkdock`) sandbox policies, and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container or sandbox, change its network, substitute a foreign rootfs, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone.
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ Werkator is configured via YAML files. Settings are merged from several sources
|
|||||||
|
|
||||||
| Layer | Path | Committed to Git | Purpose |
|
| Layer | Path | Committed to Git | Purpose |
|
||||||
|--------------------------|----------------------------|------------------|----------------------------------------------|
|
|--------------------------|----------------------------|------------------|----------------------------------------------|
|
||||||
|
| Instance config | `~/.werkator.yml` | No | The repository registry and the instance-level settings; optional `defaults` below every repository (see below) |
|
||||||
| Project config | `.werkator.yml` | Yes | Shared team settings |
|
| Project config | `.werkator.yml` | Yes | Shared team settings |
|
||||||
| Applied instance fragment | `.git/werkator/.werkator.applied.yml` | No | Instance parameters installed by `init --apply` |
|
| Applied instance fragment | `.git/werkator/.werkator.applied.yml` | No | Instance parameters installed by `init --apply` |
|
||||||
| Repo installation config | `.git/werkator/.werkator.yml` | No | Machine- or user-specific overrides, secrets |
|
| Repo installation config | `.git/werkator/.werkator.yml` | No | Machine- or user-specific overrides, secrets |
|
||||||
@@ -115,6 +116,53 @@ The pinned settings are stripped wherever they appear, in a build definition as
|
|||||||
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
|
a legacy `branches` entry. The deprecated `branches` section itself is read from the repo
|
||||||
install/project config only, and only while nothing defines a build at all.
|
install/project config only, and only while nothing defines a build at all.
|
||||||
|
|
||||||
|
## `~/.werkator.yml` — the instance configuration
|
||||||
|
|
||||||
|
One Werkator instance serves a *set* of repositories (ADR 0009): one service, one port, one UI, one watcher loop.
|
||||||
|
The set and everything shared by it live in `.werkator.yml` in the home directory of the user running Werkator — one instance per OS user.
|
||||||
|
The file name is deliberately the same everywhere; only the location carries the meaning: home is the instance, the repository root is the project, `.git` is the machine.
|
||||||
|
`WERKATOR_HOME` overrides the directory the file is looked up in.
|
||||||
|
|
||||||
|
Without this file Werkator serves the current working directory exactly as before.
|
||||||
|
With it, the registry wins over the current directory: `werkator server` serves the registered repositories wherever it is started.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
werkator:
|
||||||
|
version:
|
||||||
|
since: "0.9.16" # like every other config file
|
||||||
|
|
||||||
|
repositories: # the registry; each entry is one served repository
|
||||||
|
- path: ~/repos/werkator # absolute, or relative to the home directory (~ expands)
|
||||||
|
- path: ~/repos/werkbaum
|
||||||
|
name: baum # optional; default is the directory basename
|
||||||
|
|
||||||
|
server: # the instance's server section — port, bind address, public URL, nginx
|
||||||
|
port: 18080
|
||||||
|
executor:
|
||||||
|
maxConcurrent: 2 # the global cap over all repositories
|
||||||
|
watcher:
|
||||||
|
pollInterval: 10s # one loop, one delay
|
||||||
|
|
||||||
|
defaults: # optional: repository-level keys merged BELOW every repository's own layers
|
||||||
|
git:
|
||||||
|
account: ci-bot
|
||||||
|
token: "…" # secrets may then live here instead of in each repository
|
||||||
|
gitea:
|
||||||
|
baseUrl: https://git.example.org
|
||||||
|
```
|
||||||
|
|
||||||
|
Key ownership once this file exists:
|
||||||
|
|
||||||
|
- **Instance-level** — read from this file alone: the whole `server` section, `executor.maxConcurrent`, and `watcher.pollInterval`.
|
||||||
|
A repository file still carrying one of them is ignored on that key, with one warning naming both files; it is never merged silently.
|
||||||
|
- **Repository defaults** — the `defaults` block, in the repository config schema: merged below each repository's project config, applied fragment, machine config, and branch layer, so a repository's own value always wins.
|
||||||
|
Pinning is unchanged: home defaults and the repository's machine config are both host-side layers, and a branch still cannot reach a pinned key.
|
||||||
|
- **Repository-level** — everything else stays in the repository's own files: `gitea.*`, `git.*` credentials, `builds`, retention, the other `watcher` keys (`pullRequestGate`, `newBranchMaxAge`, `fastForwardLocalRefs`), and the sandbox policy.
|
||||||
|
|
||||||
|
Repository names are the `--repo` selector of `werkator build`, `retry`, and `status` (without it a command means the current directory when served, otherwise the first registered repository) and will become the route segment of the web UI.
|
||||||
|
Two entries resolving to the same name abort the start, as does an entry that is not a git repository.
|
||||||
|
A repository whose configuration this Werkator must not read (see the version declaration) is skipped with an error; the others are served.
|
||||||
|
|
||||||
## Inspect the Effective Config
|
## Inspect the Effective Config
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -55,10 +55,12 @@ The pinning model is untouched: pinned keys still come from each repo's machine
|
|||||||
|
|
||||||
### C — The registry and N repositories
|
### C — The registry and N repositories
|
||||||
|
|
||||||
- Load the registry, build one `RepoContext` per entry; fail the start loudly on duplicate names or unreadable repos (config-version violations abort only that repo's registration, like branch-config violations fail only that branch).
|
- ~~Load the registry, build one `RepoContext` per entry; fail the start loudly on duplicate names or unreadable repos (config-version violations abort only that repo's registration, like branch-config violations fail only that branch).~~ — done 2026-09-02: `InstanceConfig` binds `~/.werkator.yml` (`ConfigLoader.homeDir`, `WERKATOR_HOME` overrides); `RepoRegistry` opens the contexts. The instance keys and the `defaults` block are applied inside `ConfigLoader.loadRaw`, so every `load(dir)` consumer sees them without knowing the file — the repository's copies of instance keys are dropped with one warning naming both files.
|
||||||
- Watcher multiplexing: one poll cycle iterates the contexts (fetch, enqueue, prune per repo) with per-repo error isolation — one unreachable origin must not starve the others; `WatcherState` gains the repo dimension for the health banner.
|
- ~~Watcher multiplexing: one poll cycle iterates the contexts (fetch, enqueue, prune per repo) with per-repo error isolation — one unreachable origin must not starve the others; `WatcherState` gains the repo dimension for the health banner.~~ — done 2026-09-02: `pollAll(repos)`, one guard per repository, `WatcherState.repositories`; the top-level fields aggregate (unchanged with one repository, name-prefixed with several). Isolation proven by test (one unreachable origin, the other still enqueues).
|
||||||
- Startup recovery per repo; auto-build slots stay in each repo's `.git/werkator/`.
|
- ~~Startup recovery per repo; auto-build slots stay in each repo's `.git/werkator/`.~~ — done: `start(repos)` recovers each in its own guard; slots unchanged.
|
||||||
- CLI commands gain an optional repo selector and default to the current working directory, so `werkator status` inside a repo behaves as today.
|
- ~~CLI commands gain an optional repo selector and default to the current working directory, so `werkator status` inside a repo behaves as today.~~ — done: `--repo <name>` (`RepoOption` mixin) on `build`, `retry`, `status`; default is the cwd when served, else the first registered repository.
|
||||||
|
- Also done: the pre-rename state-dir migration runs per opened repository; the metrics page's repository size sums the registered repositories (the disk metric is the first one's file store).
|
||||||
|
- Carried over to session D: `RunningBuild` still carries no repository (the "current builds" view and the worktree pruning cannot tell repositories apart); the controllers still serve `registry.current()` only; `docs/deployment.md` gets the registry setup with session E.
|
||||||
|
|
||||||
### D — Server, API, and UI scoping
|
### D — Server, API, and UI scoping
|
||||||
|
|
||||||
@@ -74,7 +76,7 @@ The pinning model is untouched: pinned keys still come from each repo's machine
|
|||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
- Fairness across repos when the global concurrency cap is contended (round-robin per repo vs. FIFO) — decide in session C with the real queue behavior at hand.
|
- ~~Fairness across repos when the global concurrency cap is contended (round-robin per repo vs. FIFO) — decide in session C with the real queue behavior at hand.~~ — decided 2026-09-02: FIFO. The executor's slot semaphore is already fair, so builds take slots in enqueue order across repositories; the watcher enqueues in registry order within one cycle, which is a fixed and inspectable bias rather than a scheduler. Round-robin per repository only when a real queue shows starvation.
|
||||||
- Whether buildenv rootfs trees should be shared across repos (today each repo unpacks its own under `.git/werkator/buildenv/`) — the natural answer is Werkdock's image store (step 21 session C), not instance-level state; until then duplicate unpacked rootfs trees are the accepted cost.
|
- Whether buildenv rootfs trees should be shared across repos (today each repo unpacks its own under `.git/werkator/buildenv/`) — the natural answer is Werkdock's image store (step 21 session C), not instance-level state; until then duplicate unpacked rootfs trees are the accepted cost.
|
||||||
- ~~Whether `artifactKey` needs a repo prefix or stays globally unique by construction (random suffix) — decide in session B when the routes are designed.~~ — decided 2026-09-02: no prefix. The key is derived from pool name and start time, and both the results file and the artifact store are per repository, so it only ever has to be unique within one; the repo dimension enters through the route segment in session D, never through the key. A prefix would also change every existing artifact directory name.
|
- ~~Whether `artifactKey` needs a repo prefix or stays globally unique by construction (random suffix) — decide in session B when the routes are designed.~~ — decided 2026-09-02: no prefix. The key is derived from pool name and start time, and both the results file and the artifact store are per repository, so it only ever has to be unique within one; the repo dimension enters through the route segment in session D, never through the key. A prefix would also change every existing artifact directory name.
|
||||||
|
|
||||||
@@ -82,6 +84,6 @@ The pinning model is untouched: pinned keys still come from each repo's machine
|
|||||||
|
|
||||||
- Session A: ADR 0009 written (done 2026-09-01); the registry and key ownership land in `docs/configuration.md` together with the implementing sessions, since that reference describes implemented configuration only.
|
- Session A: ADR 0009 written (done 2026-09-01); the registry and key ownership land in `docs/configuration.md` together with the implementing sessions, since that reference describes implemented configuration only.
|
||||||
- ~~Session B: full suite green with `RepoContext` threaded through; no route or behavior change observable.~~ — done 2026-09-02.
|
- ~~Session B: full suite green with `RepoContext` threaded through; no route or behavior change observable.~~ — done 2026-09-02.
|
||||||
- Session C: an instance with two registered repos builds pushes in both, with per-repo error isolation proven by a test (one broken origin, the other keeps building).
|
- ~~Session C: an instance with two registered repos builds pushes in both, with per-repo error isolation proven by a test (one broken origin, the other keeps building).~~ — done 2026-09-02 (`WatcherTest`: "one repository's unreachable origin neither stops nor silences the other").
|
||||||
- Session D: both repos browsable in one UI; single-repo installations keep their existing URLs.
|
- Session D: both repos browsable in one UI; single-repo installations keep their existing URLs.
|
||||||
- Session E: mih34 builds Werkator and Werkbaum from one service; `docs/deployment.md` describes the registry setup.
|
- Session E: mih34 builds Werkator and Werkbaum from one service; `docs/deployment.md` describes the registry setup.
|
||||||
|
|||||||
@@ -0,0 +1,132 @@
|
|||||||
|
> **WARNING:** This document describes only the change applied in this PR.
|
||||||
|
> It may already be outdated once the next PR is merged.
|
||||||
|
> Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.
|
||||||
|
|
||||||
|
## The Problem
|
||||||
|
|
||||||
|
PR #11 gave every repository-scoped code path an explicit `RepoContext`, but exactly one exists: the current working directory.
|
||||||
|
ADR 0009 wants one instance to serve a *set* of repositories from a registry in the instance configuration, with the instance-level settings (server, global concurrency, poll interval) owned by that file and optional defaults shared by every repository — and the watcher must poll every repository in a way that one unreachable origin cannot starve or silence the others.
|
||||||
|
Step 22 session C is that: the registry, N contexts, the multiplexed watcher, and a `--repo` selector for the CLI.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- Repository-scoped routes, API paths, and UI grouping (session D): the controllers still serve the current repository only.
|
||||||
|
- The rollout on mih34 with Werkbaum and the deployment documentation of the registry (session E).
|
||||||
|
- A per-repository concurrency cap below the global one, and round-robin fairness across repositories (decided FIFO, see The Solution).
|
||||||
|
|
||||||
|
## The Scenarios
|
||||||
|
|
||||||
|
### Feature: the instance configuration `~/.werkator.yml`
|
||||||
|
|
||||||
|
#### Background
|
||||||
|
|
||||||
|
- The file lives in the home directory of the user running Werkator — one instance per OS user; `WERKATOR_HOME` overrides the directory.
|
||||||
|
- It carries the registry (`repositories`), the instance-level keys (`server`, `executor.maxConcurrent`, `watcher.pollInterval`), and an optional `defaults` block in the repository config schema.
|
||||||
|
|
||||||
|
#### Scenario#12.01: The defaults block sits below every repository's own layers
|
||||||
|
|
||||||
|
So that one `git.account`/`git.token` or one `gitea.baseUrl` can be written once for every repository of the same forge, while a repository's own value always wins.
|
||||||
|
|
||||||
|
- **Given** a home config with `defaults`
|
||||||
|
- **When** a repository's effective config is loaded
|
||||||
|
- **Then** every key the repository's layers do not set comes from `defaults`, and every key they do set stays the repository's.
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [the home config carries the registry, and its defaults sit below every repository layer](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||||
|
|
||||||
|
#### Scenario#12.02: Instance-level keys come from the home file alone
|
||||||
|
|
||||||
|
So that a `server.port` left in a repository's machine config can never silently compete with the instance's.
|
||||||
|
|
||||||
|
- **Given** a home config and a repository file still carrying `server`, `executor`, or `watcher.pollInterval`
|
||||||
|
- **When** the repository's effective config is loaded
|
||||||
|
- **Then** the whole `server` section, `executor` and `watcher.pollInterval` are the home file's
|
||||||
|
- **and** the repository's copies are dropped with one warning naming both files
|
||||||
|
- **and** the other `watcher` keys (the gates) stay the repository's.
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [with a home config the instance keys come from it alone, and a repository's copies are ignored](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||||
|
- [the home config is version-checked like every other file](../../src/test/kotlin/de/hoennig/werkator/config/ConfigLoaderTest.kt)
|
||||||
|
|
||||||
|
### Feature: the registry
|
||||||
|
|
||||||
|
#### Scenario#12.03: Every entry becomes a context, the start fails loudly on what cannot be served
|
||||||
|
|
||||||
|
So that an instance serving the wrong set never comes up looking healthy.
|
||||||
|
|
||||||
|
- **Given** a home config with `repositories`
|
||||||
|
- **When** the registry is opened
|
||||||
|
- **Then** each entry yields a `RepoContext` named after its directory unless the entry names it
|
||||||
|
- **and** an entry that is not a git repository or two entries resolving to the same name abort the start with a message naming the home file
|
||||||
|
- **and** a repository whose configuration this Werkator must not read is skipped with an error while the others are served
|
||||||
|
- **and** without a home config the registry is the current directory alone, exactly as before.
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [RepoRegistryTest](../../src/test/kotlin/de/hoennig/werkator/repo/RepoRegistryTest.kt) (all five tests)
|
||||||
|
|
||||||
|
### Feature: the watcher over N repositories
|
||||||
|
|
||||||
|
#### Scenario#12.04: One repository's failure neither stops nor silences the others
|
||||||
|
|
||||||
|
So that a wrong token in one repository cannot stall the builds of every other one.
|
||||||
|
|
||||||
|
- **Given** two registered repositories, one with an unreachable origin
|
||||||
|
- **When** a poll cycle runs
|
||||||
|
- **Then** the other repository's due branches are enqueued
|
||||||
|
- **and** the state reports the failure under the failing repository's name, and per repository in `repositories`
|
||||||
|
- **and** a repository whose poll crashes reports it the same way and the cycle continues.
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [one repository's unreachable origin neither stops nor silences the other](../../src/test/kotlin/de/hoennig/werkator/watcher/WatcherTest.kt)
|
||||||
|
- [a repository whose poll crashes reports it by name and the cycle goes on](../../src/test/kotlin/de/hoennig/werkator/watcher/WatcherTest.kt)
|
||||||
|
- the existing single-repository `WatcherTest` tests, proving the top-level state fields read as before
|
||||||
|
|
||||||
|
### Feature: the CLI selects a repository
|
||||||
|
|
||||||
|
#### Scenario#12.05: `--repo <name>` selects a registered repository; without it a command means the current directory
|
||||||
|
|
||||||
|
So that `werkator status` inside a repository behaves as it always did, and a registered repository can be addressed from anywhere.
|
||||||
|
|
||||||
|
- **Given** a registry with two repositories
|
||||||
|
- **When** `werkator status --repo second` runs
|
||||||
|
- **Then** the second repository's results are printed
|
||||||
|
- **and** an unknown name is a usage error (exit code 2) naming the registered repositories.
|
||||||
|
|
||||||
|
##### Verified by
|
||||||
|
|
||||||
|
- [--repo selects a registered repository; an unknown name is a usage error naming the registered ones](../../src/test/kotlin/de/hoennig/werkator/commands/StatusCommandTest.kt)
|
||||||
|
- [BuildCommandTest](../../src/test/kotlin/de/hoennig/werkator/commands/BuildCommandTest.kt), [RetryCommandTest](../../src/test/kotlin/de/hoennig/werkator/commands/RetryCommandTest.kt) (the default: the registry's current repository)
|
||||||
|
|
||||||
|
## The Solution
|
||||||
|
|
||||||
|
`InstanceConfig` binds the home file; `ConfigLoader` gained `homeDir`, `instanceFile()`, and `loadInstance()`.
|
||||||
|
The instance keys and the `defaults` block are folded in by `ConfigLoader.loadRaw` itself — `defaults` below the repository layers, `server`/`executor`/`watcher.pollInterval` overlaid on top and stripped from the repository files with one warning — so every existing consumer of `load(dir)` (the server command, the executor's slot count, the watcher's interval) sees the instance values without knowing the file exists.
|
||||||
|
`RepoRegistry` opens the contexts lazily on first use; `RepoConfiguration` provides `registry.current()` as the `RepoContext` bean the still-unscoped controllers use.
|
||||||
|
`Watcher.start(repos)` recovers each repository in its own guard, and `pollAll(repos)` polls each in its own guard, aggregating the per-repository reports into `WatcherState` (new `repositories` list; the top-level fields unchanged with one repository, name-prefixed with several, so the health banner needs no change).
|
||||||
|
`RepoOption` is a picocli mixin shared by `build`, `retry`, and `status`.
|
||||||
|
The pre-rename state-dir migration moved from the CLI runner into `RepoContexts.open`, so it runs per served repository; the metrics collector sums the registered repositories' sizes.
|
||||||
|
Fairness across repositories is decided FIFO: the executor's slot semaphore is already fair, and the watcher enqueues in registry order — a fixed, inspectable bias instead of a scheduler; round-robin only when a real queue shows starvation.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- `RunningBuild` still carries no repository, so the current-builds view and the worktree pruning cannot tell repositories apart — session D, with the routes.
|
||||||
|
- With several repositories the `queuedBranches` list mixes their branch names unprefixed; session D scopes it with the UI.
|
||||||
|
|
||||||
|
## Additional Changes
|
||||||
|
|
||||||
|
- `docs/configuration.md`: the instance configuration documented, the layer table gained the home file.
|
||||||
|
- Architecture skill and AGENTS.md: registry, instance config folding, watcher multiplexing.
|
||||||
|
- `docs/plan/22-multi-repo.md`: session C ticked, fairness decided, carry-overs to session D.
|
||||||
|
|
||||||
|
## Prerequisite PRs
|
||||||
|
|
||||||
|
- PR #11 (`RepoContext` refactor).
|
||||||
|
|
||||||
|
## Follow-up PRs
|
||||||
|
|
||||||
|
- Session D: server/API/UI repo scoping.
|
||||||
|
- Session E: rollout on mih34 with Werkbaum, registry setup in `docs/deployment.md`.
|
||||||
@@ -10,7 +10,6 @@ import org.springframework.context.annotation.Profile
|
|||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
import picocli.CommandLine
|
import picocli.CommandLine
|
||||||
import picocli.CommandLine.IFactory
|
import picocli.CommandLine.IFactory
|
||||||
import java.nio.file.Paths
|
|
||||||
import kotlin.system.exitProcess
|
import kotlin.system.exitProcess
|
||||||
|
|
||||||
@SpringBootApplication
|
@SpringBootApplication
|
||||||
@@ -27,8 +26,6 @@ class CliRunner(
|
|||||||
private var exitCode = 0
|
private var exitCode = 0
|
||||||
|
|
||||||
override fun run(vararg args: String) {
|
override fun run(vararg args: String) {
|
||||||
// before any command resolves a path under it, and once per process
|
|
||||||
StateDirMigration.migrateIfNeeded(Paths.get("."))
|
|
||||||
exitCode =
|
exitCode =
|
||||||
CommandLine(rootCommand, factory)
|
CommandLine(rootCommand, factory)
|
||||||
.setExecutionExceptionHandler { exception, commandLine, _ ->
|
.setExecutionExceptionHandler { exception, commandLine, _ ->
|
||||||
|
|||||||
@@ -3,9 +3,11 @@ package de.hoennig.werkator.commands
|
|||||||
import de.hoennig.werkator.build.BuildStatus
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
import de.hoennig.werkator.git.GitService
|
import de.hoennig.werkator.git.GitService
|
||||||
import de.hoennig.werkator.repo.RepoContext
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
import picocli.CommandLine.Command
|
import picocli.CommandLine.Command
|
||||||
import picocli.CommandLine.ExitCode
|
import picocli.CommandLine.ExitCode
|
||||||
|
import picocli.CommandLine.Mixin
|
||||||
import picocli.CommandLine.Parameters
|
import picocli.CommandLine.Parameters
|
||||||
import java.nio.file.Path
|
import java.nio.file.Path
|
||||||
import java.util.concurrent.Callable
|
import java.util.concurrent.Callable
|
||||||
@@ -24,9 +26,11 @@ import java.util.concurrent.Callable
|
|||||||
class BuildCommand(
|
class BuildCommand(
|
||||||
private val gitService: GitService,
|
private val gitService: GitService,
|
||||||
private val consoleBuildRunner: ConsoleBuildRunner,
|
private val consoleBuildRunner: ConsoleBuildRunner,
|
||||||
/** The repository to build: the current working directory (a repo selector comes with the registry). */
|
private val registry: RepoRegistry,
|
||||||
var repo: RepoContext,
|
|
||||||
) : Callable<Int> {
|
) : Callable<Int> {
|
||||||
|
@Mixin
|
||||||
|
var repoOption = RepoOption()
|
||||||
|
|
||||||
@Parameters(
|
@Parameters(
|
||||||
index = "0",
|
index = "0",
|
||||||
arity = "0..1",
|
arity = "0..1",
|
||||||
@@ -35,6 +39,8 @@ class BuildCommand(
|
|||||||
)
|
)
|
||||||
var branchFragment: String? = null
|
var branchFragment: String? = null
|
||||||
|
|
||||||
|
private lateinit var repo: RepoContext
|
||||||
|
|
||||||
private val workingDir: Path
|
private val workingDir: Path
|
||||||
get() = repo.workingDir
|
get() = repo.workingDir
|
||||||
|
|
||||||
@@ -42,6 +48,7 @@ class BuildCommand(
|
|||||||
val branch: String
|
val branch: String
|
||||||
val commit: String
|
val commit: String
|
||||||
try {
|
try {
|
||||||
|
repo = repoOption.select(registry)
|
||||||
fetchBestEffort()
|
fetchBestEffort()
|
||||||
branch = resolveBranch() ?: return ExitCode.USAGE
|
branch = resolveBranch() ?: return ExitCode.USAGE
|
||||||
commit = commitToBuild(branch) ?: return ExitCode.USAGE
|
commit = commitToBuild(branch) ?: return ExitCode.USAGE
|
||||||
|
|||||||
@@ -0,0 +1,28 @@
|
|||||||
|
package de.hoennig.werkator.commands
|
||||||
|
|
||||||
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
|
import picocli.CommandLine.Option
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The `--repo` selector of the repository-scoped commands (ADR 0009): names an entry
|
||||||
|
* of the instance registry. Without it a command means the current working directory
|
||||||
|
* when that is served, otherwise the first registered repository — so inside a
|
||||||
|
* repository every command behaves exactly as it did with one.
|
||||||
|
*/
|
||||||
|
class RepoOption {
|
||||||
|
@Option(
|
||||||
|
names = ["--repo"],
|
||||||
|
paramLabel = "<name>",
|
||||||
|
description = ["registered repository to act on (default: the current directory)"],
|
||||||
|
)
|
||||||
|
var name: String? = null
|
||||||
|
|
||||||
|
fun select(registry: RepoRegistry): RepoContext {
|
||||||
|
val wanted = name?.trim()?.takeIf { it.isNotEmpty() } ?: return registry.current()
|
||||||
|
return registry.byName(wanted)
|
||||||
|
?: throw IllegalArgumentException(
|
||||||
|
"no repository named '$wanted' is registered (registered: ${registry.all().joinToString(", ") { it.name }})",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -4,9 +4,11 @@ import de.hoennig.werkator.build.BuildResult
|
|||||||
import de.hoennig.werkator.build.BuildStatus
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
import de.hoennig.werkator.git.GitService
|
import de.hoennig.werkator.git.GitService
|
||||||
import de.hoennig.werkator.repo.RepoContext
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
import picocli.CommandLine.Command
|
import picocli.CommandLine.Command
|
||||||
import picocli.CommandLine.ExitCode
|
import picocli.CommandLine.ExitCode
|
||||||
|
import picocli.CommandLine.Mixin
|
||||||
import java.nio.file.Path
|
import java.nio.file.Path
|
||||||
import java.util.concurrent.Callable
|
import java.util.concurrent.Callable
|
||||||
|
|
||||||
@@ -25,15 +27,20 @@ import java.util.concurrent.Callable
|
|||||||
class RetryCommand(
|
class RetryCommand(
|
||||||
private val gitService: GitService,
|
private val gitService: GitService,
|
||||||
private val consoleBuildRunner: ConsoleBuildRunner,
|
private val consoleBuildRunner: ConsoleBuildRunner,
|
||||||
/** The repository to retry in: the current working directory (a repo selector comes with the registry). */
|
private val registry: RepoRegistry,
|
||||||
var repo: RepoContext,
|
|
||||||
) : Callable<Int> {
|
) : Callable<Int> {
|
||||||
|
@Mixin
|
||||||
|
var repoOption = RepoOption()
|
||||||
|
|
||||||
|
private lateinit var repo: RepoContext
|
||||||
|
|
||||||
private val workingDir: Path
|
private val workingDir: Path
|
||||||
get() = repo.workingDir
|
get() = repo.workingDir
|
||||||
|
|
||||||
override fun call(): Int {
|
override fun call(): Int {
|
||||||
val failed: List<BuildResult>
|
val failed: List<BuildResult>
|
||||||
try {
|
try {
|
||||||
|
repo = repoOption.select(registry)
|
||||||
fetchBestEffort()
|
fetchBestEffort()
|
||||||
failed = repo.results.latestPerName().filter { it.status == BuildStatus.FAILED }
|
failed = repo.results.latestPerName().filter { it.status == BuildStatus.FAILED }
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
package de.hoennig.werkator.commands
|
package de.hoennig.werkator.commands
|
||||||
|
|
||||||
import de.hoennig.werkator.build.BuildResult
|
import de.hoennig.werkator.build.BuildResult
|
||||||
import de.hoennig.werkator.build.BuildResultRepository
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import de.hoennig.werkator.server.UiFormats
|
import de.hoennig.werkator.server.UiFormats
|
||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
import picocli.CommandLine.Command
|
import picocli.CommandLine.Command
|
||||||
import picocli.CommandLine.ExitCode
|
import picocli.CommandLine.ExitCode
|
||||||
|
import picocli.CommandLine.Mixin
|
||||||
import picocli.CommandLine.Option
|
import picocli.CommandLine.Option
|
||||||
import java.util.concurrent.Callable
|
import java.util.concurrent.Callable
|
||||||
|
|
||||||
@@ -20,12 +21,22 @@ import java.util.concurrent.Callable
|
|||||||
mixinStandardHelpOptions = true,
|
mixinStandardHelpOptions = true,
|
||||||
)
|
)
|
||||||
class StatusCommand(
|
class StatusCommand(
|
||||||
private val repository: BuildResultRepository,
|
private val registry: RepoRegistry,
|
||||||
) : Callable<Int> {
|
) : Callable<Int> {
|
||||||
@Option(names = ["--history"], description = ["Print all recorded builds, not only the latest per branch"])
|
@Option(names = ["--history"], description = ["Print all recorded builds, not only the latest per branch"])
|
||||||
var history: Boolean = false
|
var history: Boolean = false
|
||||||
|
|
||||||
|
@Mixin
|
||||||
|
var repoOption = RepoOption()
|
||||||
|
|
||||||
override fun call(): Int {
|
override fun call(): Int {
|
||||||
|
val repository =
|
||||||
|
try {
|
||||||
|
repoOption.select(registry).results
|
||||||
|
} catch (e: IllegalArgumentException) {
|
||||||
|
System.err.println("error: ${e.message}")
|
||||||
|
return ExitCode.USAGE
|
||||||
|
}
|
||||||
val results = if (history) repository.history() else repository.latestPerName()
|
val results = if (history) repository.history() else repository.latestPerName()
|
||||||
if (results.isEmpty()) {
|
if (results.isEmpty()) {
|
||||||
println("(no builds recorded)")
|
println("(no builds recorded)")
|
||||||
|
|||||||
@@ -19,6 +19,14 @@ object ConfigFiles {
|
|||||||
/** The machine-specific configuration inside `.git`; secrets live here. */
|
/** The machine-specific configuration inside `.git`; secrets live here. */
|
||||||
const val REPO_INSTALL = ".git/werkator/$COMMITTED"
|
const val REPO_INSTALL = ".git/werkator/$COMMITTED"
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The instance configuration (ADR 0009), relative to the home directory of the user
|
||||||
|
* running Werkator. Deliberately the same file name: only the location carries the
|
||||||
|
* meaning — home is the instance, the repository root is the project, `.git` is the
|
||||||
|
* machine.
|
||||||
|
*/
|
||||||
|
const val INSTANCE = COMMITTED
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The applied instance fragment (`init --apply`, step 23): a config-schema YAML
|
* The applied instance fragment (`init --apply`, step 23): a config-schema YAML
|
||||||
* fragment installed verbatim as its own layer — above the committed project
|
* fragment installed verbatim as its own layer — above the committed project
|
||||||
|
|||||||
@@ -50,6 +50,35 @@ class ConfigLoader(
|
|||||||
/** Section-level warnings already reported, keyed by a fixed slug; the config is loaded on every poll cycle. */
|
/** Section-level warnings already reported, keyed by a fixed slug; the config is loaded on every poll cycle. */
|
||||||
private val warnedSections = ConcurrentHashMap.newKeySet<String>()
|
private val warnedSections = ConcurrentHashMap.newKeySet<String>()
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Where the instance configuration lives (ADR 0009): the home directory of the user
|
||||||
|
* running Werkator, `WERKATOR_HOME` overriding it for tests and unusual layouts.
|
||||||
|
*/
|
||||||
|
@Volatile
|
||||||
|
var homeDir: Path = Paths.get(System.getenv("WERKATOR_HOME")?.takeIf { it.isNotBlank() } ?: System.getProperty("user.home"))
|
||||||
|
|
||||||
|
/** The instance configuration file, whether or not it exists. */
|
||||||
|
fun instanceFile(): Path = homeDir.resolve(ConfigFiles.INSTANCE)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The instance configuration, or null without a home file — the single-repository
|
||||||
|
* case, in which the current directory is served exactly as before ADR 0009.
|
||||||
|
*/
|
||||||
|
fun loadInstance(): InstanceConfig? {
|
||||||
|
val raw = loadInstanceRaw()
|
||||||
|
if (raw.isEmpty()) {
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
return yaml.convertValue(raw, InstanceConfig::class.java)
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun loadInstanceRaw(): Map<String, Any?> {
|
||||||
|
val file = instanceFile()
|
||||||
|
val raw = loadFile(file.toFile())
|
||||||
|
checkVersion(raw, file.toString(), ROLLBACK_HINT)
|
||||||
|
return raw
|
||||||
|
}
|
||||||
|
|
||||||
fun load(workingDir: Path = Paths.get(".")): WerkatorConfig = toConfig(loadRaw(workingDir))
|
fun load(workingDir: Path = Paths.get(".")): WerkatorConfig = toConfig(loadRaw(workingDir))
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -307,11 +336,79 @@ class ConfigLoader(
|
|||||||
checkTriggerBlocks(project, projectName, ROLLBACK_HINT)
|
checkTriggerBlocks(project, projectName, ROLLBACK_HINT)
|
||||||
checkTriggerBlocks(applied, ConfigFiles.APPLIED, ROLLBACK_HINT)
|
checkTriggerBlocks(applied, ConfigFiles.APPLIED, ROLLBACK_HINT)
|
||||||
checkTriggerBlocks(repoInstall, repoInstallName, ROLLBACK_HINT)
|
checkTriggerBlocks(repoInstall, repoInstallName, ROLLBACK_HINT)
|
||||||
// the applied instance fragment sits above the committed project config and
|
val instance = loadInstanceRaw()
|
||||||
// below the hand-edited machine config, which always has the last word
|
if (instance.isEmpty()) {
|
||||||
return deepMerge(deepMerge(project, applied), repoInstall)
|
// the applied instance fragment sits above the committed project config and
|
||||||
|
// below the hand-edited machine config, which always has the last word
|
||||||
|
return deepMerge(deepMerge(project, applied), repoInstall)
|
||||||
|
}
|
||||||
|
// with a home config (ADR 0009): its `defaults` sit below every repository layer,
|
||||||
|
// and the instance-level keys come from it alone — a repository file still
|
||||||
|
// carrying them is told so, never merged silently
|
||||||
|
val repoLayers =
|
||||||
|
deepMerge(
|
||||||
|
deepMerge(
|
||||||
|
withoutInstanceKeys(project, workingDir.resolve(projectName)),
|
||||||
|
withoutInstanceKeys(applied, workingDir.resolve(ConfigFiles.APPLIED)),
|
||||||
|
),
|
||||||
|
withoutInstanceKeys(repoInstall, workingDir.resolve(repoInstallName)),
|
||||||
|
)
|
||||||
|
|
||||||
|
@Suppress("UNCHECKED_CAST")
|
||||||
|
val defaults = instance["defaults"] as? Map<String, Any?> ?: emptyMap()
|
||||||
|
return deepMerge(deepMerge(defaults, repoLayers), instanceKeysOf(instance))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Drops the instance-level keys from one repository layer, saying so once per file
|
||||||
|
* with both file names: the setting the operator wrote there is not in effect, and
|
||||||
|
* the message must name where it is read from instead.
|
||||||
|
*/
|
||||||
|
@Suppress("UNCHECKED_CAST")
|
||||||
|
private fun withoutInstanceKeys(
|
||||||
|
layer: Map<String, Any?>,
|
||||||
|
file: Path,
|
||||||
|
): Map<String, Any?> {
|
||||||
|
val instanceKeys = instanceKeysOf(layer)
|
||||||
|
if (instanceKeys.isEmpty()) {
|
||||||
|
return layer
|
||||||
|
}
|
||||||
|
if (warnedSections.add("instance-keys:$file")) {
|
||||||
|
log.warn(
|
||||||
|
"ignoring {} in {}: these are instance settings and come from {} now",
|
||||||
|
describeKeys(instanceKeys),
|
||||||
|
file,
|
||||||
|
instanceFile(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
val result = layer.toMutableMap()
|
||||||
|
for (key in INSTANCE_SECTIONS) {
|
||||||
|
result.remove(key)
|
||||||
|
}
|
||||||
|
val watcher = (layer["watcher"] as? Map<String, Any?>)?.minus(INSTANCE_WATCHER_KEYS)
|
||||||
|
if (watcher == null || watcher.isEmpty()) result.remove("watcher") else result["watcher"] = watcher
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The instance-level part of a raw configuration map: the sections and keys owned by the instance. */
|
||||||
|
@Suppress("UNCHECKED_CAST")
|
||||||
|
private fun instanceKeysOf(raw: Map<String, Any?>): Map<String, Any?> {
|
||||||
|
val result = mutableMapOf<String, Any?>()
|
||||||
|
for (key in INSTANCE_SECTIONS) {
|
||||||
|
raw[key]?.let { result[key] = it }
|
||||||
|
}
|
||||||
|
val watcher = (raw["watcher"] as? Map<String, Any?>)?.filterKeys { it in INSTANCE_WATCHER_KEYS }
|
||||||
|
if (!watcher.isNullOrEmpty()) {
|
||||||
|
result["watcher"] = watcher
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun describeKeys(instanceKeys: Map<String, Any?>): String =
|
||||||
|
instanceKeys.keys.joinToString(", ") { key ->
|
||||||
|
if (key == "watcher") INSTANCE_WATCHER_KEYS.joinToString(", ") { "watcher.$it" } else key
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Validates and installs an instance fragment (`init --apply`, step 23): the file
|
* Validates and installs an instance fragment (`init --apply`, step 23): the file
|
||||||
* must be non-empty, pass the version and trigger checks, and bind *strictly*
|
* must be non-empty, pass the version and trigger checks, and bind *strictly*
|
||||||
@@ -467,6 +564,16 @@ class ConfigLoader(
|
|||||||
/** The keys that moved into [TRIGGER_KEYS]; still writing them flat is refused, not ignored. */
|
/** The keys that moved into [TRIGGER_KEYS]; still writing them flat is refused, not ignored. */
|
||||||
private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin")
|
private val FLAT_TRIGGER_KEYS = setOf("onPush", "atTimes", "branches", "activeWithin")
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Top-level sections owned by the instance once a home config exists (ADR 0009):
|
||||||
|
* the one server and the one global concurrency cap. Read from the home file,
|
||||||
|
* ignored with a warning in a repository's files.
|
||||||
|
*/
|
||||||
|
private val INSTANCE_SECTIONS = setOf("server", "executor")
|
||||||
|
|
||||||
|
/** The `watcher` keys owned by the instance: one loop, one delay; the gates stay per repository. */
|
||||||
|
private val INSTANCE_WATCHER_KEYS = setOf("pollInterval")
|
||||||
|
|
||||||
private const val LEGACY_BRANCHES_WARNING = "legacy-branches-ignored"
|
private const val LEGACY_BRANCHES_WARNING = "legacy-branches-ignored"
|
||||||
|
|
||||||
private const val NO_TRIGGER_WARNING = "no-build-triggered"
|
private const val NO_TRIGGER_WARNING = "no-build-triggered"
|
||||||
|
|||||||
@@ -0,0 +1,45 @@
|
|||||||
|
package de.hoennig.werkator.config
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The instance configuration (ADR 0009): `~/.werkator.yml` in the home directory of the
|
||||||
|
* user running Werkator — one instance per OS user. It owns what is shared by every
|
||||||
|
* repository the instance serves: the repository registry, the `server` section, the
|
||||||
|
* global `executor.maxConcurrent`, and the watcher poll interval. Everything else in
|
||||||
|
* the file is either `defaults` — a fragment in the repository config schema merged
|
||||||
|
* *below* every repository's own layers — or ignored.
|
||||||
|
*
|
||||||
|
* Without this file, Werkator serves the current working directory exactly as before.
|
||||||
|
* With it, the registry wins over the current directory (`werkator server` serves the
|
||||||
|
* registered repositories wherever it is started), and the instance-level keys of a
|
||||||
|
* repository's own files are ignored with a warning naming both files, never merged.
|
||||||
|
*/
|
||||||
|
data class InstanceConfig(
|
||||||
|
/** What this file declares about the Werkator that reads it; see [VersionRequirement]. */
|
||||||
|
val werkator: WerkatorMeta = WerkatorMeta(),
|
||||||
|
val server: ServerConfig = ServerConfig(),
|
||||||
|
val executor: ExecutorConfig = ExecutorConfig(),
|
||||||
|
val watcher: InstanceWatcherConfig = InstanceWatcherConfig(),
|
||||||
|
/** The registry: the repositories this instance serves; empty means the current directory. */
|
||||||
|
val repositories: List<RepositoryEntry> = emptyList(),
|
||||||
|
/**
|
||||||
|
* Repository-level keys in the repository config schema (e.g. one `git.account`/`git.token`
|
||||||
|
* for every repository of the same forge), merged below each repository's own layers.
|
||||||
|
* Raw on purpose: it is a fragment, not a configuration, and binds through the same
|
||||||
|
* path as every other layer.
|
||||||
|
*/
|
||||||
|
val defaults: Map<String, Any?> = emptyMap(),
|
||||||
|
)
|
||||||
|
|
||||||
|
/** The watcher settings that are the instance's, not a repository's: one loop, one delay. */
|
||||||
|
data class InstanceWatcherConfig(
|
||||||
|
/** Delay between poll cycles over all repositories, e.g. `10s` or `1m`. */
|
||||||
|
val pollInterval: String = "10s",
|
||||||
|
)
|
||||||
|
|
||||||
|
/** One registry entry: a repository directory and the name it is known by. */
|
||||||
|
data class RepositoryEntry(
|
||||||
|
/** The repository's primary checkout, absolute or relative to the home directory; `~` expands. */
|
||||||
|
val path: String = "",
|
||||||
|
/** Short unique name for display and routes; empty means the directory basename. */
|
||||||
|
val name: String = "",
|
||||||
|
)
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
package de.hoennig.werkator.metrics
|
package de.hoennig.werkator.metrics
|
||||||
|
|
||||||
import de.hoennig.werkator.build.ArtifactStore
|
import de.hoennig.werkator.build.ArtifactStore
|
||||||
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import org.springframework.context.annotation.Bean
|
import org.springframework.context.annotation.Bean
|
||||||
import org.springframework.context.annotation.Configuration
|
import org.springframework.context.annotation.Configuration
|
||||||
import java.time.Clock
|
import java.time.Clock
|
||||||
@@ -16,10 +17,12 @@ class MetricsConfiguration {
|
|||||||
@Bean
|
@Bean
|
||||||
fun systemMetricsCollector(
|
fun systemMetricsCollector(
|
||||||
artifactStore: ArtifactStore,
|
artifactStore: ArtifactStore,
|
||||||
|
registry: RepoRegistry,
|
||||||
clock: Clock,
|
clock: Clock,
|
||||||
): SystemMetricsCollector =
|
): SystemMetricsCollector =
|
||||||
SystemMetricsCollector(
|
SystemMetricsCollector(
|
||||||
stateFile = { artifactStore.rootDir().resolve(SystemMetricsCollector.STATE_FILE_NAME) },
|
stateFile = { artifactStore.rootDir().resolve(SystemMetricsCollector.STATE_FILE_NAME) },
|
||||||
|
repoDirs = { registry.all().map { it.workingDir } },
|
||||||
clock = clock,
|
clock = clock,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -35,7 +35,8 @@ data class PersistedMetricsState(
|
|||||||
*/
|
*/
|
||||||
class SystemMetricsCollector(
|
class SystemMetricsCollector(
|
||||||
private val stateFile: () -> Path,
|
private val stateFile: () -> Path,
|
||||||
private val workingDir: Path = Paths.get("."),
|
/** The served repositories: the disk metric is the first one's file store, the repository size their sum. */
|
||||||
|
private val repoDirs: () -> List<Path> = { listOf(Paths.get(".")) },
|
||||||
private val clock: Clock = Clock.systemUTC(),
|
private val clock: Clock = Clock.systemUTC(),
|
||||||
private val procStat: Path = Paths.get("/proc/stat"),
|
private val procStat: Path = Paths.get("/proc/stat"),
|
||||||
private val procMeminfo: Path = Paths.get("/proc/meminfo"),
|
private val procMeminfo: Path = Paths.get("/proc/meminfo"),
|
||||||
@@ -239,7 +240,7 @@ class SystemMetricsCollector(
|
|||||||
.removeSuffix(" kB")
|
.removeSuffix(" kB")
|
||||||
.toLong()
|
.toLong()
|
||||||
|
|
||||||
private fun readDisk(): DiskSpace? = readSource("disk") { diskSpace(workingDir) }
|
private fun readDisk(): DiskSpace? = readSource("disk") { diskSpace(repoDirs().first()) }
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The repository size is expensive to determine (a full file walk), so unlike
|
* The repository size is expensive to determine (a full file walk), so unlike
|
||||||
@@ -251,7 +252,7 @@ class SystemMetricsCollector(
|
|||||||
samplesSinceRepoSizeProbe++
|
samplesSinceRepoSizeProbe++
|
||||||
return lastRepoSizeGib
|
return lastRepoSizeGib
|
||||||
}
|
}
|
||||||
lastRepoSizeGib = readSource("repo size") { repoSizeBytes(workingDir) / BYTES_PER_GIB }
|
lastRepoSizeGib = readSource("repo size") { repoDirs().sumOf(repoSizeBytes) / BYTES_PER_GIB }
|
||||||
samplesSinceRepoSizeProbe = 1
|
samplesSinceRepoSizeProbe = 1
|
||||||
return lastRepoSizeGib
|
return lastRepoSizeGib
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,15 +2,14 @@ package de.hoennig.werkator.repo
|
|||||||
|
|
||||||
import org.springframework.context.annotation.Bean
|
import org.springframework.context.annotation.Bean
|
||||||
import org.springframework.context.annotation.Configuration
|
import org.springframework.context.annotation.Configuration
|
||||||
import java.nio.file.Paths
|
|
||||||
|
|
||||||
@Configuration
|
@Configuration
|
||||||
class RepoConfiguration {
|
class RepoConfiguration {
|
||||||
/**
|
/**
|
||||||
* The single-repository case: the current working directory, which is how every
|
* The repository the unscoped code paths mean — the current working directory
|
||||||
* CLI command and the server resolve their files. Only paths are computed here, so
|
* when it is served, see [RepoRegistry.current]. Without a registry only paths are
|
||||||
* the bean is safe outside a git repository.
|
* computed here, so the bean is safe outside a git repository.
|
||||||
*/
|
*/
|
||||||
@Bean
|
@Bean
|
||||||
fun currentRepo(repoContexts: RepoContexts): RepoContext = repoContexts.open(Paths.get("."))
|
fun currentRepo(registry: RepoRegistry): RepoContext = registry.current()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,12 +1,17 @@
|
|||||||
package de.hoennig.werkator.repo
|
package de.hoennig.werkator.repo
|
||||||
|
|
||||||
|
import de.hoennig.werkator.StateDirMigration
|
||||||
import de.hoennig.werkator.artifacts.FileArtifactStore
|
import de.hoennig.werkator.artifacts.FileArtifactStore
|
||||||
import de.hoennig.werkator.build.FileBuildResultRepository
|
import de.hoennig.werkator.build.FileBuildResultRepository
|
||||||
import de.hoennig.werkator.config.ConfigLoader
|
import de.hoennig.werkator.config.ConfigLoader
|
||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
import java.nio.file.Path
|
import java.nio.file.Path
|
||||||
|
|
||||||
/** Opens a [RepoContext] over a repository directory; nothing is touched until the first build. */
|
/**
|
||||||
|
* Opens a [RepoContext] over a repository directory. Nothing is created until the first
|
||||||
|
* build; only a pre-rename state directory is moved to its current name on the way, per
|
||||||
|
* repository, before any path under it is resolved.
|
||||||
|
*/
|
||||||
@Component
|
@Component
|
||||||
class RepoContexts(
|
class RepoContexts(
|
||||||
private val configLoader: ConfigLoader,
|
private val configLoader: ConfigLoader,
|
||||||
@@ -14,13 +19,15 @@ class RepoContexts(
|
|||||||
fun open(
|
fun open(
|
||||||
workingDir: Path,
|
workingDir: Path,
|
||||||
name: String = defaultName(workingDir),
|
name: String = defaultName(workingDir),
|
||||||
): RepoContext =
|
): RepoContext {
|
||||||
RepoContext(
|
StateDirMigration.migrateIfNeeded(workingDir)
|
||||||
|
return RepoContext(
|
||||||
name = name,
|
name = name,
|
||||||
workingDir = workingDir,
|
workingDir = workingDir,
|
||||||
results = FileBuildResultRepository(workingDir.resolve(RESULTS_FILE)),
|
results = FileBuildResultRepository(workingDir.resolve(RESULTS_FILE)),
|
||||||
artifactStore = FileArtifactStore(configLoader, workingDir),
|
artifactStore = FileArtifactStore(configLoader, workingDir),
|
||||||
)
|
)
|
||||||
|
}
|
||||||
|
|
||||||
companion object {
|
companion object {
|
||||||
/** Results file relative to the repository, next to the machine config in `.git/werkator/`. */
|
/** Results file relative to the repository, next to the machine config in `.git/werkator/`. */
|
||||||
|
|||||||
@@ -0,0 +1,100 @@
|
|||||||
|
package de.hoennig.werkator.repo
|
||||||
|
|
||||||
|
import de.hoennig.werkator.config.ConfigException
|
||||||
|
import de.hoennig.werkator.config.ConfigLoader
|
||||||
|
import de.hoennig.werkator.config.RepositoryEntry
|
||||||
|
import org.slf4j.LoggerFactory
|
||||||
|
import org.springframework.stereotype.Component
|
||||||
|
import java.nio.file.Files
|
||||||
|
import java.nio.file.Path
|
||||||
|
import java.nio.file.Paths
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The repositories this instance serves (ADR 0009): one [RepoContext] per entry of the
|
||||||
|
* home configuration's `repositories`, or — without a home config or with an empty
|
||||||
|
* registry — the current working directory, exactly as before.
|
||||||
|
*
|
||||||
|
* Opened once, on first use, and loudly: an entry that is no git repository or a name
|
||||||
|
* used twice aborts the start with a message naming the home file, because an instance
|
||||||
|
* serving the wrong set is worse than one that does not come up. A repository whose
|
||||||
|
* configuration Werkator must not read (version or format violation) is the exception:
|
||||||
|
* it is skipped with an error, like a branch config violation fails only that branch —
|
||||||
|
* the other repositories keep building.
|
||||||
|
*/
|
||||||
|
@Component
|
||||||
|
class RepoRegistry(
|
||||||
|
private val configLoader: ConfigLoader,
|
||||||
|
private val repoContexts: RepoContexts,
|
||||||
|
) {
|
||||||
|
private val log = LoggerFactory.getLogger(RepoRegistry::class.java)
|
||||||
|
|
||||||
|
private val contexts: List<RepoContext> by lazy { open() }
|
||||||
|
|
||||||
|
/** Every served repository, in registry order. */
|
||||||
|
fun all(): List<RepoContext> = contexts
|
||||||
|
|
||||||
|
/** The repository registered under [name], or null. */
|
||||||
|
fun byName(name: String): RepoContext? = contexts.firstOrNull { it.name == name }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The repository a command without a selector means: the current working directory
|
||||||
|
* when it is served (so `werkator status` inside a repository behaves as today),
|
||||||
|
* otherwise the first registered one.
|
||||||
|
*/
|
||||||
|
fun current(): RepoContext {
|
||||||
|
val cwd = Paths.get(".").toAbsolutePath().normalize()
|
||||||
|
return contexts.firstOrNull { it.workingDir.toAbsolutePath().normalize() == cwd } ?: contexts.first()
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun open(): List<RepoContext> {
|
||||||
|
val entries = configLoader.loadInstance()?.repositories.orEmpty()
|
||||||
|
if (entries.isEmpty()) {
|
||||||
|
return listOf(repoContexts.open(Paths.get(".")))
|
||||||
|
}
|
||||||
|
val home = configLoader.instanceFile()
|
||||||
|
val opened = entries.mapNotNull { openEntry(it, home) }
|
||||||
|
val duplicates = opened.groupBy { it.name }.filterValues { it.size > 1 }
|
||||||
|
if (duplicates.isNotEmpty()) {
|
||||||
|
val listed =
|
||||||
|
duplicates.entries.joinToString(
|
||||||
|
"; ",
|
||||||
|
) { (name, repos) -> "$name: ${repos.joinToString(", ") { it.workingDir.toString() }}" }
|
||||||
|
throw IllegalStateException("$home registers the same repository name more than once ($listed); set a distinct name per entry")
|
||||||
|
}
|
||||||
|
check(opened.isNotEmpty()) { "$home registers no readable repository" }
|
||||||
|
return opened
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun openEntry(
|
||||||
|
entry: RepositoryEntry,
|
||||||
|
home: Path,
|
||||||
|
): RepoContext? {
|
||||||
|
val dir = resolve(entry.path)
|
||||||
|
if (!Files.isDirectory(dir) || !Files.exists(dir.resolve(".git"))) {
|
||||||
|
throw IllegalStateException(
|
||||||
|
"$home registers ${entry.path.ifBlank { "an entry without a path" }}, which is not a git repository ($dir)",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
val name = entry.name.trim().ifEmpty { RepoContexts.defaultName(dir) }
|
||||||
|
try {
|
||||||
|
// the configuration is read here only to find out whether Werkator may read it at all
|
||||||
|
configLoader.load(dir)
|
||||||
|
} catch (e: ConfigException) {
|
||||||
|
log.error("not serving repository {} ({}): {}", name, dir, e.message)
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
return repoContexts.open(dir, name)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `~` expands to the home directory; a relative path is relative to the home directory, not the cwd. */
|
||||||
|
private fun resolve(path: String): Path {
|
||||||
|
val home = configLoader.homeDir
|
||||||
|
val expanded =
|
||||||
|
when {
|
||||||
|
path == "~" -> home
|
||||||
|
path.startsWith("~/") -> home.resolve(path.removePrefix("~/"))
|
||||||
|
else -> home.resolve(path)
|
||||||
|
}
|
||||||
|
return expanded.toAbsolutePath().normalize()
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
package de.hoennig.werkator.server
|
package de.hoennig.werkator.server
|
||||||
|
|
||||||
import de.hoennig.werkator.repo.RepoContext
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import de.hoennig.werkator.watcher.Watcher
|
import de.hoennig.werkator.watcher.Watcher
|
||||||
import jakarta.annotation.PreDestroy
|
import jakarta.annotation.PreDestroy
|
||||||
import org.springframework.boot.context.event.ApplicationReadyEvent
|
import org.springframework.boot.context.event.ApplicationReadyEvent
|
||||||
@@ -9,7 +9,7 @@ import org.springframework.context.event.EventListener
|
|||||||
import org.springframework.stereotype.Component
|
import org.springframework.stereotype.Component
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Starts the watcher poll loop over the served repository once the server context
|
* Starts the watcher poll loop over the served repositories once the server context
|
||||||
* is ready and stops it on shutdown. Only in the `server` profile — CLI commands
|
* is ready and stops it on shutdown. Only in the `server` profile — CLI commands
|
||||||
* and tests never start the loop (see [Watcher]).
|
* and tests never start the loop (see [Watcher]).
|
||||||
*/
|
*/
|
||||||
@@ -17,11 +17,11 @@ import org.springframework.stereotype.Component
|
|||||||
@Profile("server")
|
@Profile("server")
|
||||||
class ServerWatcherLifecycle(
|
class ServerWatcherLifecycle(
|
||||||
private val watcher: Watcher,
|
private val watcher: Watcher,
|
||||||
private val repo: RepoContext,
|
private val registry: RepoRegistry,
|
||||||
) {
|
) {
|
||||||
@EventListener(ApplicationReadyEvent::class)
|
@EventListener(ApplicationReadyEvent::class)
|
||||||
fun onApplicationReady() {
|
fun onApplicationReady() {
|
||||||
watcher.start(repo)
|
watcher.start(registry.all())
|
||||||
}
|
}
|
||||||
|
|
||||||
@PreDestroy
|
@PreDestroy
|
||||||
|
|||||||
@@ -26,10 +26,12 @@ import java.util.concurrent.ScheduledExecutorService
|
|||||||
import java.util.concurrent.TimeUnit
|
import java.util.concurrent.TimeUnit
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Replaces the legacy blocking main loop: a non-blocking fixed-delay poll cycle that
|
* Replaces the legacy blocking main loop: a non-blocking fixed-delay poll cycle that,
|
||||||
* fetches origin, enqueues due branches via the async [BuildExecutor], and prunes
|
* for every served repository, fetches origin, enqueues due branches via the async
|
||||||
* retention — it never waits for a build and never builds in the primary checkout
|
* [BuildExecutor], and prunes retention — it never waits for a build and never builds
|
||||||
* (whose branch refs it does fast-forward, see [fastForwardLocalRefs]).
|
* in the primary checkout (whose branch refs it does fast-forward, see
|
||||||
|
* [fastForwardLocalRefs]). One repository's failure never reaches another: each is
|
||||||
|
* polled in its own guard and reports on its own in [WatcherState.repositories].
|
||||||
* The loop only runs after an explicit [start] (server/watch mode, step 07);
|
* The loop only runs after an explicit [start] (server/watch mode, step 07);
|
||||||
* nothing is scheduled during CLI commands or tests.
|
* nothing is scheduled during CLI commands or tests.
|
||||||
*/
|
*/
|
||||||
@@ -78,24 +80,35 @@ class Watcher(
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Runs the startup recovery and schedules the poll loop with the fixed delay
|
* Runs the startup recovery of every repository and schedules the poll loop with the
|
||||||
* `watcher.pollInterval`; the first poll runs immediately.
|
* fixed delay `watcher.pollInterval` — one loop, one delay: the instance's setting,
|
||||||
|
* which every repository's effective config carries; the first poll runs immediately.
|
||||||
*/
|
*/
|
||||||
@Synchronized
|
@Synchronized
|
||||||
fun start(repo: RepoContext) {
|
fun start(repos: List<RepoContext>) {
|
||||||
check(scheduler == null) { "watcher is already running" }
|
check(scheduler == null) { "watcher is already running" }
|
||||||
recoverOnStartup(repo)
|
require(repos.isNotEmpty()) { "no repository to watch" }
|
||||||
val interval = DurationParser.parse(configLoader.load(repo.workingDir).watcher.pollInterval)
|
repos.forEach { recoverSafely(it) }
|
||||||
|
val interval = DurationParser.parse(configLoader.load(repos.first().workingDir).watcher.pollInterval)
|
||||||
scheduler =
|
scheduler =
|
||||||
Executors
|
Executors
|
||||||
.newSingleThreadScheduledExecutor { runnable ->
|
.newSingleThreadScheduledExecutor { runnable ->
|
||||||
Thread(runnable, "werkator-watcher").apply { isDaemon = true }
|
Thread(runnable, "werkator-watcher").apply { isDaemon = true }
|
||||||
}.also {
|
}.also {
|
||||||
it.scheduleWithFixedDelay({ pollSafely(repo) }, 0, interval.toMillis(), TimeUnit.MILLISECONDS)
|
it.scheduleWithFixedDelay({ pollAll(repos) }, 0, interval.toMillis(), TimeUnit.MILLISECONDS)
|
||||||
}
|
}
|
||||||
state = state.copy(running = true)
|
state = state.copy(running = true)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** A repository whose recovery crashes is still polled; the others' recovery is never skipped. */
|
||||||
|
private fun recoverSafely(repo: RepoContext) {
|
||||||
|
try {
|
||||||
|
recoverOnStartup(repo)
|
||||||
|
} catch (e: Exception) {
|
||||||
|
log.error("[{}] startup recovery failed", repo.name, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@Synchronized
|
@Synchronized
|
||||||
fun stop() {
|
fun stop() {
|
||||||
scheduler?.shutdownNow()
|
scheduler?.shutdownNow()
|
||||||
@@ -144,31 +157,73 @@ class Watcher(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** One poll cycle over a single repository; see [pollAll]. */
|
||||||
|
fun poll(repo: RepoContext) = pollAll(listOf(repo))
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* One poll cycle, never blocking on a build: fetch origin (on failure: log once per
|
* One poll cycle over all served repositories, never blocking on a build. Each
|
||||||
* message, expose in [state], retry next cycle), enqueue due branches — changed local branches
|
* repository is polled in its own guard — a crash or an unreachable origin is that
|
||||||
* first, then recent new origin branches, then due auto-build slots — then
|
* repository's report, and the next one is polled regardless — and the cycle's
|
||||||
* fast-forward the local branch refs, and finally prune results, artifacts, and
|
* [state] aggregates the reports: the top-level fields read as before with one
|
||||||
* worktrees of branches gone from origin.
|
* repository, and name the repository in front of every message with several.
|
||||||
*/
|
*/
|
||||||
fun poll(repo: RepoContext) {
|
fun pollAll(repos: List<RepoContext>) {
|
||||||
val startedAt = clock.instant()
|
val startedAt = clock.instant()
|
||||||
|
val reports = repos.map { pollSafely(it, startedAt) }
|
||||||
|
val several = repos.size > 1
|
||||||
|
|
||||||
|
fun named(
|
||||||
|
report: RepoWatcherState,
|
||||||
|
message: String,
|
||||||
|
): String = if (several) "${report.name}: $message" else message
|
||||||
|
state =
|
||||||
|
state.copy(
|
||||||
|
lastPollAt = startedAt,
|
||||||
|
lastFetchError = reports.mapNotNull { report -> report.lastFetchError?.let { named(report, it) } }.joinOrNull(),
|
||||||
|
lastPollError = reports.mapNotNull { report -> report.lastPollError?.let { named(report, it) } }.joinOrNull(),
|
||||||
|
queuedBranches = reports.flatMap { it.queuedBranches },
|
||||||
|
repositories = reports,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun List<String>.joinOrNull(): String? = takeIf { it.isNotEmpty() }?.joinToString("; ")
|
||||||
|
|
||||||
|
private fun pollSafely(
|
||||||
|
repo: RepoContext,
|
||||||
|
startedAt: Instant,
|
||||||
|
): RepoWatcherState =
|
||||||
|
try {
|
||||||
|
pollRepo(repo, startedAt)
|
||||||
|
} catch (e: Exception) {
|
||||||
|
log.error("[{}] poll cycle failed", repo.name, e)
|
||||||
|
RepoWatcherState(repo.name, lastPollAt = startedAt, lastPollError = e.message ?: e.javaClass.simpleName)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One repository's poll: fetch origin (on failure: log once per message, report it,
|
||||||
|
* retry next cycle), enqueue due branches — changed local branches first, then recent
|
||||||
|
* new origin branches, then due auto-build slots — then fast-forward the local branch
|
||||||
|
* refs, and finally prune results, artifacts, and worktrees of branches gone from origin.
|
||||||
|
*/
|
||||||
|
private fun pollRepo(
|
||||||
|
repo: RepoContext,
|
||||||
|
startedAt: Instant,
|
||||||
|
): RepoWatcherState {
|
||||||
val workingDir = repo.workingDir
|
val workingDir = repo.workingDir
|
||||||
val watch = watchOf(repo)
|
val watch = watchOf(repo)
|
||||||
try {
|
try {
|
||||||
gitService.fetchOrigin(workingDir)
|
gitService.fetchOrigin(workingDir)
|
||||||
if (watch.loggedFetchError != null) {
|
if (watch.loggedFetchError != null) {
|
||||||
log.info("fetching origin succeeded again")
|
log.info("[{}] fetching origin succeeded again", repo.name)
|
||||||
watch.loggedFetchError = null
|
watch.loggedFetchError = null
|
||||||
}
|
}
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
val failure = e.message ?: e.javaClass.simpleName
|
val failure = e.message ?: e.javaClass.simpleName
|
||||||
if (watch.loggedFetchError != failure) {
|
if (watch.loggedFetchError != failure) {
|
||||||
log.warn("fetching origin failed; retrying every cycle until it succeeds: {}", failure)
|
log.warn("[{}] fetching origin failed; retrying every cycle until it succeeds: {}", repo.name, failure)
|
||||||
watch.loggedFetchError = failure
|
watch.loggedFetchError = failure
|
||||||
}
|
}
|
||||||
state = state.copy(lastPollAt = startedAt, lastFetchError = failure)
|
return RepoWatcherState(repo.name, lastPollAt = startedAt, lastFetchError = failure)
|
||||||
return
|
|
||||||
}
|
}
|
||||||
val config = configLoader.load(workingDir)
|
val config = configLoader.load(workingDir)
|
||||||
val originBranches = gitService.originBranches(workingDir)
|
val originBranches = gitService.originBranches(workingDir)
|
||||||
@@ -177,26 +232,15 @@ class Watcher(
|
|||||||
fastForwardLocalRefs(workingDir)
|
fastForwardLocalRefs(workingDir)
|
||||||
}
|
}
|
||||||
prune(repo, config, originBranches)
|
prune(repo, config, originBranches)
|
||||||
state =
|
return RepoWatcherState(
|
||||||
state.copy(
|
repo.name,
|
||||||
lastPollAt = startedAt,
|
lastPollAt = startedAt,
|
||||||
lastFetchError = null,
|
queuedBranches =
|
||||||
lastPollError = null,
|
repo.results
|
||||||
queuedBranches =
|
.latestPerName()
|
||||||
repo.results
|
.filter { it.status == BuildStatus.PENDING || it.status == BuildStatus.RUNNING }
|
||||||
.latestPerName()
|
.map { it.name },
|
||||||
.filter { it.status == BuildStatus.PENDING || it.status == BuildStatus.RUNNING }
|
)
|
||||||
.map { it.name },
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun pollSafely(repo: RepoContext) {
|
|
||||||
try {
|
|
||||||
poll(repo)
|
|
||||||
} catch (e: Exception) {
|
|
||||||
log.error("poll cycle failed", e)
|
|
||||||
state = state.copy(lastPollAt = clock.instant(), lastPollError = e.message ?: e.javaClass.simpleName)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -343,7 +387,7 @@ class Watcher(
|
|||||||
log.info("not enqueueing branch {}: no pull request has head commit {}", branch, commit)
|
log.info("not enqueueing branch {}: no pull request has head commit {}", branch, commit)
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
log.info("enqueueing build {} of branch {} at commit {}", build, branch, commit)
|
log.info("[{}] enqueueing build {} of branch {} at commit {}", repo.name, build, branch, commit)
|
||||||
buildExecutor.startBuild(repo, branch, commit, build)
|
buildExecutor.startBuild(repo, branch, commit, build)
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,16 +2,32 @@ package de.hoennig.werkator.watcher
|
|||||||
|
|
||||||
import java.time.Instant
|
import java.time.Instant
|
||||||
|
|
||||||
/** Observable watcher health for status endpoints (step 07) and the UI (step 08). */
|
/**
|
||||||
|
* Observable watcher health for status endpoints (step 07) and the UI (step 08).
|
||||||
|
* The top-level fields describe the whole poll cycle — with one repository they are
|
||||||
|
* that repository's, with several they aggregate [repositories], where each served
|
||||||
|
* repository reports on its own (ADR 0009).
|
||||||
|
*/
|
||||||
data class WatcherState(
|
data class WatcherState(
|
||||||
/** Whether the poll loop is scheduled. */
|
/** Whether the poll loop is scheduled. */
|
||||||
val running: Boolean = false,
|
val running: Boolean = false,
|
||||||
/** When the last poll cycle started, successful or not. */
|
/** When the last poll cycle started, successful or not. */
|
||||||
val lastPollAt: Instant? = null,
|
val lastPollAt: Instant? = null,
|
||||||
/** Why the last `fetchOrigin` failed; null after a successful fetch. */
|
/** Why the last `fetchOrigin` failed; null after a successful fetch. With several repositories, `<name>: <reason>` per failure. */
|
||||||
|
val lastFetchError: String? = null,
|
||||||
|
/** Why the last poll cycle crashed after a successful fetch; null after a clean cycle. Named per repository like [lastFetchError]. */
|
||||||
|
val lastPollError: String? = null,
|
||||||
|
/** Branches whose latest build was PENDING or RUNNING at the end of the last poll, over all repositories. */
|
||||||
|
val queuedBranches: List<String> = emptyList(),
|
||||||
|
/** The same per served repository, in registry order. */
|
||||||
|
val repositories: List<RepoWatcherState> = emptyList(),
|
||||||
|
)
|
||||||
|
|
||||||
|
/** One repository's part of the last poll cycle. */
|
||||||
|
data class RepoWatcherState(
|
||||||
|
val name: String,
|
||||||
|
val lastPollAt: Instant? = null,
|
||||||
val lastFetchError: String? = null,
|
val lastFetchError: String? = null,
|
||||||
/** Why the last poll cycle crashed after a successful fetch; null after a clean cycle. */
|
|
||||||
val lastPollError: String? = null,
|
val lastPollError: String? = null,
|
||||||
/** Branches whose latest build was PENDING or RUNNING at the end of the last poll. */
|
|
||||||
val queuedBranches: List<String> = emptyList(),
|
val queuedBranches: List<String> = emptyList(),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package de.hoennig.werkator.commands
|
|||||||
import de.hoennig.werkator.build.BuildStatus
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
import de.hoennig.werkator.git.GitService
|
import de.hoennig.werkator.git.GitService
|
||||||
import de.hoennig.werkator.repo.RepoContext
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import io.kotest.core.spec.style.FunSpec
|
import io.kotest.core.spec.style.FunSpec
|
||||||
import io.kotest.matchers.shouldBe
|
import io.kotest.matchers.shouldBe
|
||||||
import io.kotest.matchers.string.shouldContain
|
import io.kotest.matchers.string.shouldContain
|
||||||
@@ -20,9 +21,10 @@ class BuildCommandTest : FunSpec() {
|
|||||||
private val consoleBuildRunner = mockk<ConsoleBuildRunner>()
|
private val consoleBuildRunner = mockk<ConsoleBuildRunner>()
|
||||||
private val dir: Path = Paths.get(".")
|
private val dir: Path = Paths.get(".")
|
||||||
private val repo = RepoContext("test", dir, mockk(), mockk())
|
private val repo = RepoContext("test", dir, mockk(), mockk())
|
||||||
|
private val registry = mockk<RepoRegistry>().also { every { it.current() } returns repo }
|
||||||
|
|
||||||
private fun command(fragment: String? = null) =
|
private fun command(fragment: String? = null) =
|
||||||
BuildCommand(gitService, consoleBuildRunner, repo).apply {
|
BuildCommand(gitService, consoleBuildRunner, registry).apply {
|
||||||
branchFragment = fragment
|
branchFragment = fragment
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import de.hoennig.werkator.build.BuildResultRepository
|
|||||||
import de.hoennig.werkator.build.BuildStatus
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
import de.hoennig.werkator.git.GitService
|
import de.hoennig.werkator.git.GitService
|
||||||
import de.hoennig.werkator.repo.RepoContext
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import io.kotest.core.spec.style.FunSpec
|
import io.kotest.core.spec.style.FunSpec
|
||||||
import io.kotest.matchers.shouldBe
|
import io.kotest.matchers.shouldBe
|
||||||
import io.kotest.matchers.string.shouldContain
|
import io.kotest.matchers.string.shouldContain
|
||||||
@@ -24,8 +25,9 @@ class RetryCommandTest : FunSpec() {
|
|||||||
private val consoleBuildRunner = mockk<ConsoleBuildRunner>()
|
private val consoleBuildRunner = mockk<ConsoleBuildRunner>()
|
||||||
private val dir: Path = Paths.get(".")
|
private val dir: Path = Paths.get(".")
|
||||||
private val repo = RepoContext("test", dir, repository, mockk())
|
private val repo = RepoContext("test", dir, repository, mockk())
|
||||||
|
private val registry = mockk<RepoRegistry>().also { every { it.current() } returns repo }
|
||||||
|
|
||||||
private fun command() = RetryCommand(gitService, consoleBuildRunner, repo)
|
private fun command() = RetryCommand(gitService, consoleBuildRunner, registry)
|
||||||
|
|
||||||
private fun result(
|
private fun result(
|
||||||
branch: String,
|
branch: String,
|
||||||
|
|||||||
@@ -3,6 +3,8 @@ package de.hoennig.werkator.commands
|
|||||||
import de.hoennig.werkator.build.BuildResult
|
import de.hoennig.werkator.build.BuildResult
|
||||||
import de.hoennig.werkator.build.BuildResultRepository
|
import de.hoennig.werkator.build.BuildResultRepository
|
||||||
import de.hoennig.werkator.build.BuildStatus
|
import de.hoennig.werkator.build.BuildStatus
|
||||||
|
import de.hoennig.werkator.repo.RepoContext
|
||||||
|
import de.hoennig.werkator.repo.RepoRegistry
|
||||||
import io.kotest.core.spec.style.FunSpec
|
import io.kotest.core.spec.style.FunSpec
|
||||||
import io.kotest.matchers.shouldBe
|
import io.kotest.matchers.shouldBe
|
||||||
import io.kotest.matchers.string.shouldContain
|
import io.kotest.matchers.string.shouldContain
|
||||||
@@ -16,6 +18,30 @@ import java.time.Instant
|
|||||||
|
|
||||||
class StatusCommandTest : FunSpec() {
|
class StatusCommandTest : FunSpec() {
|
||||||
private val repository = mockk<BuildResultRepository>()
|
private val repository = mockk<BuildResultRepository>()
|
||||||
|
private val other = mockk<BuildResultRepository>()
|
||||||
|
private val registry =
|
||||||
|
mockk<RepoRegistry>().also {
|
||||||
|
val current =
|
||||||
|
RepoContext(
|
||||||
|
"current",
|
||||||
|
java.nio.file.Paths
|
||||||
|
.get("."),
|
||||||
|
repository,
|
||||||
|
mockk(),
|
||||||
|
)
|
||||||
|
val second =
|
||||||
|
RepoContext(
|
||||||
|
"second",
|
||||||
|
java.nio.file.Paths
|
||||||
|
.get("second"),
|
||||||
|
other,
|
||||||
|
mockk(),
|
||||||
|
)
|
||||||
|
every { it.current() } returns current
|
||||||
|
every { it.all() } returns listOf(current, second)
|
||||||
|
every { it.byName("second") } returns second
|
||||||
|
every { it.byName("nope") } returns null
|
||||||
|
}
|
||||||
|
|
||||||
private fun result(
|
private fun result(
|
||||||
branch: String,
|
branch: String,
|
||||||
@@ -32,7 +58,21 @@ class StatusCommandTest : FunSpec() {
|
|||||||
|
|
||||||
init {
|
init {
|
||||||
beforeEach {
|
beforeEach {
|
||||||
clearMocks(repository)
|
clearMocks(repository, other)
|
||||||
|
}
|
||||||
|
|
||||||
|
test("--repo selects a registered repository; an unknown name is a usage error naming the registered ones") {
|
||||||
|
every { other.latestPerName() } returns listOf(result("main", BuildStatus.SUCCESS))
|
||||||
|
|
||||||
|
var exitCode = -1
|
||||||
|
val console = captureConsole { exitCode = StatusCommand(registry).apply { repoOption.name = "second" }.call() }
|
||||||
|
exitCode shouldBe 0
|
||||||
|
console.stdout shouldContain "main"
|
||||||
|
verify(exactly = 0) { repository.latestPerName() }
|
||||||
|
|
||||||
|
val failed = captureConsole { exitCode = StatusCommand(registry).apply { repoOption.name = "nope" }.call() }
|
||||||
|
exitCode shouldBe 2
|
||||||
|
failed.stderr shouldContain "current, second"
|
||||||
}
|
}
|
||||||
|
|
||||||
test("prints the latest build per branch as a table with short commits and legacy duration format") {
|
test("prints the latest build per branch as a table with short commits and legacy duration format") {
|
||||||
@@ -43,7 +83,7 @@ class StatusCommandTest : FunSpec() {
|
|||||||
)
|
)
|
||||||
|
|
||||||
var exitCode = -1
|
var exitCode = -1
|
||||||
val console = captureConsole { exitCode = StatusCommand(repository).call() }
|
val console = captureConsole { exitCode = StatusCommand(registry).call() }
|
||||||
|
|
||||||
exitCode shouldBe 0
|
exitCode shouldBe 0
|
||||||
console.stdout shouldContain "BRANCH"
|
console.stdout shouldContain "BRANCH"
|
||||||
@@ -64,7 +104,7 @@ class StatusCommandTest : FunSpec() {
|
|||||||
result("main", BuildStatus.FAILED),
|
result("main", BuildStatus.FAILED),
|
||||||
)
|
)
|
||||||
|
|
||||||
val command = StatusCommand(repository).apply { history = true }
|
val command = StatusCommand(registry).apply { history = true }
|
||||||
var exitCode = -1
|
var exitCode = -1
|
||||||
val console = captureConsole { exitCode = command.call() }
|
val console = captureConsole { exitCode = command.call() }
|
||||||
|
|
||||||
@@ -78,7 +118,7 @@ class StatusCommandTest : FunSpec() {
|
|||||||
every { repository.latestPerName() } returns emptyList()
|
every { repository.latestPerName() } returns emptyList()
|
||||||
|
|
||||||
var exitCode = -1
|
var exitCode = -1
|
||||||
val console = captureConsole { exitCode = StatusCommand(repository).call() }
|
val console = captureConsole { exitCode = StatusCommand(registry).call() }
|
||||||
|
|
||||||
exitCode shouldBe 0
|
exitCode shouldBe 0
|
||||||
console.stdout shouldContain "(no builds recorded)"
|
console.stdout shouldContain "(no builds recorded)"
|
||||||
|
|||||||
@@ -110,6 +110,95 @@ class ConfigLoaderTest : FunSpec() {
|
|||||||
"./gradlew fromBranch"
|
"./gradlew fromBranch"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test("without a home config the instance is null and the repository layers are read as before") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
val loader = ConfigLoader().apply { homeDir = home }
|
||||||
|
|
||||||
|
loader.loadInstance() shouldBe null
|
||||||
|
loader.instanceFile() shouldBe home.resolve(".werkator.yml")
|
||||||
|
}
|
||||||
|
|
||||||
|
test("the home config carries the registry, and its defaults sit below every repository layer") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
home.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
repositories:
|
||||||
|
- path: ~/repos/werkator
|
||||||
|
- path: /srv/werkbaum
|
||||||
|
name: baum
|
||||||
|
defaults:
|
||||||
|
git:
|
||||||
|
account: shared-bot
|
||||||
|
token: shared-secret
|
||||||
|
gitea:
|
||||||
|
baseUrl: https://git.example.org
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
val loader = ConfigLoader().apply { homeDir = home }
|
||||||
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
|
dir.resolve(".werkator.yml").toFile().writeText("gitea:\n owner: my-org\n")
|
||||||
|
Files.createDirectories(dir.resolve(".git/werkator"))
|
||||||
|
dir.resolve(".git/werkator/.werkator.yml").toFile().writeText("git:\n token: own-secret\n")
|
||||||
|
|
||||||
|
val instance = loader.loadInstance().shouldNotBeNull()
|
||||||
|
instance.repositories.map { it.path to it.name } shouldBe listOf("~/repos/werkator" to "", "/srv/werkbaum" to "baum")
|
||||||
|
|
||||||
|
val config = loader.load(dir)
|
||||||
|
// the repository's own layers win over the defaults, untouched keys fall through
|
||||||
|
config.git.account shouldBe "shared-bot"
|
||||||
|
config.git.token shouldBe "own-secret"
|
||||||
|
config.gitea.baseUrl shouldBe "https://git.example.org"
|
||||||
|
config.gitea.owner shouldBe "my-org"
|
||||||
|
}
|
||||||
|
|
||||||
|
test("with a home config the instance keys come from it alone, and a repository's copies are ignored") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
home.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
server:
|
||||||
|
port: 18088
|
||||||
|
executor:
|
||||||
|
maxConcurrent: 3
|
||||||
|
watcher:
|
||||||
|
pollInterval: 1m
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
val loader = ConfigLoader().apply { homeDir = home }
|
||||||
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
|
dir.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
server:
|
||||||
|
port: 1000
|
||||||
|
bindAddress: 0.0.0.0
|
||||||
|
watcher:
|
||||||
|
pollInterval: 5s
|
||||||
|
pullRequestGate: false
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
Files.createDirectories(dir.resolve(".git/werkator"))
|
||||||
|
dir.resolve(".git/werkator/.werkator.yml").toFile().writeText("executor:\n maxConcurrent: 9\n")
|
||||||
|
|
||||||
|
val config = loader.load(dir)
|
||||||
|
|
||||||
|
// the whole server section is the instance's, not merged key by key
|
||||||
|
config.server.port shouldBe 18088
|
||||||
|
config.server.bindAddress shouldBe "127.0.0.1"
|
||||||
|
config.executor.maxConcurrent shouldBe 3
|
||||||
|
config.watcher.pollInterval shouldBe "1m"
|
||||||
|
// the per-repository watcher gates stay the repository's
|
||||||
|
config.watcher.pullRequestGate.shouldBeFalse()
|
||||||
|
}
|
||||||
|
|
||||||
|
test("the home config is version-checked like every other file") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
home.resolve(".werkator.yml").toFile().writeText("werkator:\n version:\n since: \"9.9\"\n")
|
||||||
|
val loader = loaderRunning("1.0.0").apply { homeDir = home }
|
||||||
|
|
||||||
|
val error = shouldThrow<ConfigVersionException> { loader.loadInstance() }
|
||||||
|
|
||||||
|
error.message shouldContain home.resolve(".werkator.yml").toString()
|
||||||
|
}
|
||||||
|
|
||||||
test("the applied instance fragment layers above the project config and below the machine config") {
|
test("the applied instance fragment layers above the project config and below the machine config") {
|
||||||
val dir = Files.createTempDirectory("werkator-test")
|
val dir = Files.createTempDirectory("werkator-test")
|
||||||
dir.resolve(".werkator.yml").toFile().writeText("server:\n port: 1000\n publicBaseUrl: \"https://project/\"\n")
|
dir.resolve(".werkator.yml").toFile().writeText("server:\n port: 1000\n publicBaseUrl: \"https://project/\"\n")
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ class SystemMetricsCollectorTest : FunSpec() {
|
|||||||
repoSizeBytes: (Path) -> Long = { HALF_GIB_BYTES },
|
repoSizeBytes: (Path) -> Long = { HALF_GIB_BYTES },
|
||||||
) = SystemMetricsCollector(
|
) = SystemMetricsCollector(
|
||||||
stateFile = { tempDir.resolve("system-metrics-state.json") },
|
stateFile = { tempDir.resolve("system-metrics-state.json") },
|
||||||
workingDir = tempDir,
|
repoDirs = { listOf(tempDir) },
|
||||||
clock = Clock.fixed(now, ZoneOffset.UTC),
|
clock = Clock.fixed(now, ZoneOffset.UTC),
|
||||||
procStat = tempDir.resolve("stat"),
|
procStat = tempDir.resolve("stat"),
|
||||||
procMeminfo = tempDir.resolve("meminfo"),
|
procMeminfo = tempDir.resolve("meminfo"),
|
||||||
|
|||||||
@@ -0,0 +1,100 @@
|
|||||||
|
package de.hoennig.werkator.repo
|
||||||
|
|
||||||
|
import de.hoennig.werkator.config.ConfigLoader
|
||||||
|
import io.kotest.assertions.throwables.shouldThrow
|
||||||
|
import io.kotest.core.spec.style.FunSpec
|
||||||
|
import io.kotest.matchers.shouldBe
|
||||||
|
import io.kotest.matchers.string.shouldContain
|
||||||
|
import io.mockk.every
|
||||||
|
import io.mockk.mockk
|
||||||
|
import org.springframework.beans.factory.ObjectProvider
|
||||||
|
import org.springframework.boot.info.BuildProperties
|
||||||
|
import java.nio.file.Files
|
||||||
|
import java.nio.file.Path
|
||||||
|
import java.nio.file.Paths
|
||||||
|
import java.util.Properties
|
||||||
|
|
||||||
|
class RepoRegistryTest : FunSpec() {
|
||||||
|
private fun loaderWithHome(
|
||||||
|
home: Path,
|
||||||
|
version: String = "1.0.0",
|
||||||
|
): ConfigLoader {
|
||||||
|
val provider = mockk<ObjectProvider<BuildProperties>>()
|
||||||
|
every { provider.getIfAvailable() } returns BuildProperties(Properties().apply { setProperty("version", version) })
|
||||||
|
return ConfigLoader(provider).apply { homeDir = home }
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun gitRepo(
|
||||||
|
parent: Path,
|
||||||
|
name: String,
|
||||||
|
): Path = Files.createDirectories(parent.resolve(name)).also { Files.createDirectories(it.resolve(".git")) }
|
||||||
|
|
||||||
|
private fun registry(loader: ConfigLoader) = RepoRegistry(loader, RepoContexts(loader))
|
||||||
|
|
||||||
|
init {
|
||||||
|
test("without a home config the registry is the current directory alone") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
val registry = registry(loaderWithHome(home))
|
||||||
|
|
||||||
|
registry.all().map { it.workingDir } shouldBe listOf(Paths.get("."))
|
||||||
|
registry.current().workingDir shouldBe Paths.get(".")
|
||||||
|
}
|
||||||
|
|
||||||
|
test("every registry entry becomes a context, named after its directory unless the entry says otherwise") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
val one = gitRepo(home, "repos/werkator")
|
||||||
|
val two = gitRepo(home, "repos/werkbaum")
|
||||||
|
home.resolve(".werkator.yml").toFile().writeText(
|
||||||
|
"""
|
||||||
|
repositories:
|
||||||
|
- path: ~/repos/werkator
|
||||||
|
- path: $two
|
||||||
|
name: baum
|
||||||
|
""".trimIndent(),
|
||||||
|
)
|
||||||
|
|
||||||
|
val registry = registry(loaderWithHome(home))
|
||||||
|
|
||||||
|
registry.all().map { it.name to it.workingDir } shouldBe listOf("werkator" to one, "baum" to two)
|
||||||
|
registry.byName("baum")?.workingDir shouldBe two
|
||||||
|
registry.byName("nope") shouldBe null
|
||||||
|
// the cwd is not registered, so the first entry is the default
|
||||||
|
registry.current().name shouldBe "werkator"
|
||||||
|
}
|
||||||
|
|
||||||
|
test("an entry that is not a git repository aborts the start, naming the home file") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
Files.createDirectories(home.resolve("not-a-repo"))
|
||||||
|
home.resolve(".werkator.yml").toFile().writeText("repositories:\n - path: ~/not-a-repo\n")
|
||||||
|
|
||||||
|
val error = shouldThrow<IllegalStateException> { registry(loaderWithHome(home)).all() }
|
||||||
|
|
||||||
|
error.message shouldContain home.resolve(".werkator.yml").toString()
|
||||||
|
error.message shouldContain "not a git repository"
|
||||||
|
}
|
||||||
|
|
||||||
|
test("two entries resolving to the same name abort the start") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
gitRepo(home, "a/werkator")
|
||||||
|
gitRepo(home, "b/werkator")
|
||||||
|
home.resolve(".werkator.yml").toFile().writeText("repositories:\n - path: ~/a/werkator\n - path: ~/b/werkator\n")
|
||||||
|
|
||||||
|
val error = shouldThrow<IllegalStateException> { registry(loaderWithHome(home)).all() }
|
||||||
|
|
||||||
|
error.message shouldContain "werkator"
|
||||||
|
error.message shouldContain "distinct name"
|
||||||
|
}
|
||||||
|
|
||||||
|
test("a repository whose configuration must not be read is skipped, the others are served") {
|
||||||
|
val home = Files.createTempDirectory("werkator-home")
|
||||||
|
val fine = gitRepo(home, "fine")
|
||||||
|
val broken = gitRepo(home, "broken")
|
||||||
|
broken.resolve(".werkator.yml").toFile().writeText("werkator:\n version:\n since: \"9.9\"\n")
|
||||||
|
home.resolve(".werkator.yml").toFile().writeText("repositories:\n - path: ~/broken\n - path: ~/fine\n")
|
||||||
|
|
||||||
|
val registry = registry(loaderWithHome(home, version = "1.0.0"))
|
||||||
|
|
||||||
|
registry.all().map { it.workingDir } shouldBe listOf(fine)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -767,19 +767,62 @@ class WatcherTest : FunSpec() {
|
|||||||
Files.exists(busyWorktree).shouldBeTrue()
|
Files.exists(busyWorktree).shouldBeTrue()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test("one repository's unreachable origin neither stops nor silences the other") {
|
||||||
|
val harness = Harness()
|
||||||
|
val otherDir = Files.createTempDirectory("werkator-watcher-other")
|
||||||
|
val other =
|
||||||
|
RepoContext(
|
||||||
|
"other",
|
||||||
|
otherDir,
|
||||||
|
FileBuildResultRepository(otherDir.resolve(".git/werkator/build-results.json")),
|
||||||
|
harness.artifactStore,
|
||||||
|
)
|
||||||
|
every { harness.gitService.fetchOrigin(harness.workingDir) } throws RuntimeException("origin unreachable")
|
||||||
|
every { harness.gitService.originBranches(otherDir) } returns listOf("main")
|
||||||
|
every { harness.gitService.originBranchHeads(otherDir) } returns mapOf("main" to "commit-other")
|
||||||
|
every { harness.gitService.localBranches(otherDir) } returns listOf("main")
|
||||||
|
every { harness.gitService.hasNewCommits("main", otherDir) } returns true
|
||||||
|
every { harness.gitService.originHeadCommit("main", otherDir) } returns "commit-other"
|
||||||
|
|
||||||
|
harness.watcher.pollAll(listOf(harness.repo, other))
|
||||||
|
|
||||||
|
verify { harness.buildExecutor.startBuild(other, "main", "commit-other", BuildDefinition.DEFAULT) }
|
||||||
|
verify(exactly = 0) { harness.buildExecutor.startBuild(harness.repo, any(), any(), any()) }
|
||||||
|
val state = harness.watcher.state()
|
||||||
|
state.lastFetchError shouldBe "test: origin unreachable"
|
||||||
|
state.lastPollError shouldBe null
|
||||||
|
state.repositories.map { it.name } shouldBe listOf("test", "other")
|
||||||
|
state.repositories[0].lastFetchError shouldBe "origin unreachable"
|
||||||
|
state.repositories[1].lastFetchError shouldBe null
|
||||||
|
}
|
||||||
|
|
||||||
|
test("a repository whose poll crashes reports it by name and the cycle goes on") {
|
||||||
|
val harness = Harness()
|
||||||
|
val otherDir = Files.createTempDirectory("werkator-watcher-other")
|
||||||
|
val other = RepoContext("other", otherDir, harness.repository, harness.artifactStore)
|
||||||
|
every { harness.gitService.originBranches(otherDir) } throws IllegalStateException("corrupt refs")
|
||||||
|
|
||||||
|
harness.watcher.pollAll(listOf(harness.repo, other))
|
||||||
|
|
||||||
|
val state = harness.watcher.state()
|
||||||
|
state.lastPollError shouldBe "other: corrupt refs"
|
||||||
|
state.repositories[1].lastPollError shouldBe "corrupt refs"
|
||||||
|
state.repositories[0].lastPollError shouldBe null
|
||||||
|
}
|
||||||
|
|
||||||
test("start runs recovery plus an immediate first poll; stop halts the loop") {
|
test("start runs recovery plus an immediate first poll; stop halts the loop") {
|
||||||
val harness = Harness()
|
val harness = Harness()
|
||||||
val fetches = CountDownLatch(2)
|
val fetches = CountDownLatch(2)
|
||||||
every { harness.gitService.fetchOrigin(any()) } answers { fetches.countDown() }
|
every { harness.gitService.fetchOrigin(any()) } answers { fetches.countDown() }
|
||||||
|
|
||||||
harness.watcher.start(harness.repo)
|
harness.watcher.start(listOf(harness.repo))
|
||||||
|
|
||||||
fetches.await(5, TimeUnit.SECONDS).shouldBeTrue()
|
fetches.await(5, TimeUnit.SECONDS).shouldBeTrue()
|
||||||
harness.watcher
|
harness.watcher
|
||||||
.state()
|
.state()
|
||||||
.running
|
.running
|
||||||
.shouldBeTrue()
|
.shouldBeTrue()
|
||||||
shouldThrow<IllegalStateException> { harness.watcher.start(harness.repo) }
|
shouldThrow<IllegalStateException> { harness.watcher.start(listOf(harness.repo)) }
|
||||||
|
|
||||||
harness.watcher.stop()
|
harness.watcher.stop()
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user