Build definitions with onPush/atTimes replace branch-owned schedules

ADR 0007: the YAML builds section (next to the reserved maxConcurrent
key) defines named builds (jobs) with onPush/atTimes triggers, branch
selectors (name globs, activeWithin age filter), and build-setting
overrides applied last over the merged branch config. The implicit
default build (onPush over all branches) preserves the previous
behavior; the section is pinned against the worktree layer.

Results record the job name; restart, retry, and startup recovery
re-run by it, resolving settings from the current config. A non-default
build records under the <branch>@<build> pool with its own row,
retention count, and permanent latest-green link.

branches.*.autoBuild stays as a deprecated alias (plain times only);
the unreleased-in-practice v0.9.13 per-slot buildCommand/name syntax is
removed again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
mhoennig
2026-08-28 20:06:02 +02:00
co-authored by Claude Fable 5
parent 5051c7bb99
commit 0e8db18e69
23 changed files with 595 additions and 405 deletions
+1 -1
View File
@@ -54,7 +54,7 @@ Three places must stay in sync when config keys change: the `GitTallyConfig` dat
## Build Execution ## Build Execution
`BuildExecutor` runs builds asynchronously: up to `builds.maxConcurrent` branches concurrently (default 1), but never more than one build per branch at a time. Each branch builds in its own reusable git worktree at `.git/gittally/worktrees/<branchKey>` (`BranchWorkspaces`), checked out detached at the requested commit — the primary checkout is never used for builds. Status transitions are persisted via `BuildResultRepository` (JSON file under `.git/gittally/`), published to Gitea non-fatally, and emitted as `BuildStatusChangedEvent`s. An auto-build slot (`autoBuild.times` entry) may carry its own `buildCommand` and `name`; the watcher passes them to `startBuild` as `buildCommandOverride` and `name`, persisted in the build result — UI restart and startup recovery pass the recorded values on, so a build always repeats with the command and name it originally ran under. Both are resolved watcher-side from the repo install/project config; the worktree layer cannot change them. `BuildResult.name` (default: the branch) keys everything display- and retention-side repository grouping (`latestPerName`), retention pools, branches-view rows, permanent latest-green links while `BuildResult.branch` keys everything git-side: origin lookups, gone-from-origin pruning, worktrees (a named slot builds in its branch's worktree, serialized with the branch's other builds), and Gitea links/statuses. Cancellation addresses a build by artifact key and terminates the whole process tree. Future code (watcher, server, UI) must not assume a single running build. `BuildExecutor` runs builds asynchronously: up to `builds.maxConcurrent` branches concurrently (default 1), but never more than one build per branch at a time. Each branch builds in its own reusable git worktree at `.git/gittally/worktrees/<branchKey>` (`BranchWorkspaces`), checked out detached at the requested commit — the primary checkout is never used for builds. Status transitions are persisted via `BuildResultRepository` (JSON file under `.git/gittally/`), published to Gitea non-fatally, and emitted as `BuildStatusChangedEvent`s. Every run belongs to a named build definition (job, ADR 0007): the YAML `builds` section (reserved key `maxConcurrent` split off by `ConfigLoader`, section pinned against the worktree layer) defines triggers (`onPush`, `atTimes`), branch selectors (`branches` globs, `activeWithin`), and build-setting overrides applied last over the merged branch config; the implicit `default` build (`onPush`, all branches) preserves the job-less behavior. `BuildResult.build` records the job; restart, retry, and startup recovery re-run by that name, resolving settings from the *current* config. `BuildResult.name` — the pool, `<branch>@<build>` for non-default builds — keys everything display- and retention-side (repository grouping via `latestPerName`, retention pools, branches-view rows, permanent latest-green links), while `BuildResult.branch` keys everything git-side: origin lookups, gone-from-origin pruning, worktrees (every build runs in its branch's worktree, serialized per branch), and Gitea links/statuses. `branches.*.autoBuild` survives as a deprecated alias for a scheduled default-pool rebuild. Cancellation addresses a build by artifact key and terminates the whole process tree. Future code (watcher, server, UI) must not assume a single running build.
On context close (e.g. systemd SIGTERM), a `ContextClosedEvent` listener in `BuildExecutor` terminates the process trees of all executing builds and waits (bounded) until their results are persisted as INTERRUPTED — a shutdown is never recorded as FAILED. Builds still queued stay PENDING and start no process. Both are re-enqueued by the watcher's startup recovery; INTERRUPTED therefore publishes as Gitea state `pending`, not `failure` (`GiteaStateMapping`). On context close (e.g. systemd SIGTERM), a `ContextClosedEvent` listener in `BuildExecutor` terminates the process trees of all executing builds and waits (bounded) until their results are persisted as INTERRUPTED — a shutdown is never recorded as FAILED. Builds still queued stay PENDING and start no process. Both are re-enqueued by the watcher's startup recovery; INTERRUPTED therefore publishes as Gitea state `pending`, not `failure` (`GiteaStateMapping`).
@@ -2,11 +2,11 @@
**Status:** **Status:**
- proposed: 2026-08-28 - proposed: 2026-08-28
- accepted: - - accepted: 2026-08-28
- rejected: - - rejected: -
- superseded: - - superseded: -
**Decision [proposed]:** A top-level `builds` section defines named builds (jobs) with `onPush`/`atTimes` triggers and a branch selector — the branch-owned `autoBuild` schedule and the v0.9.13 per-slot `buildCommand`/`name` syntax are replaced by it. **Decision [accepted]:** A top-level `builds` section defines named builds (jobs) with `onPush`/`atTimes` triggers and a branch selector — the branch-owned `autoBuild` schedule and the v0.9.13 per-slot `buildCommand`/`name` syntax are replaced by it.
`branches` stays what it is: per-branch build settings that every build inherits. `branches` stays what it is: per-branch build settings that every build inherits.
## Context and Problem Statement ## Context and Problem Statement
@@ -82,7 +82,7 @@ Semantics:
Compatibility and migration: Compatibility and migration:
- No `builds` section, or no `default` entry: the implicit `default` build (`onPush: true`, all branches) preserves today's behavior exactly. Defining other builds does not disable it; `builds.default.onPush: false` does. - No `builds` section, or no `default` entry: the implicit `default` build (`onPush: true`, all branches) preserves today's behavior exactly. Defining other builds does not disable it; `builds.default.onPush: false` does.
- `branches.<name>.autoBuild` (`enabled` + plain `times`) keeps working for a transition period, internally mapped to a scheduled build of the branch's own pool, with a deprecation warning in the log; `docs/configuration.md` documents only `builds`. - `branches.<name>.autoBuild` (`enabled` + plain `times`) keeps working for compatibility, internally mapped to a scheduled build of the branch's own pool, with a deprecation warning in the log; removal is not scheduled.
- The v0.9.13 per-slot `buildCommand` and `name` are **removed** (not deprecated): released one day ago, configured nowhere. - The v0.9.13 per-slot `buildCommand` and `name` are **removed** (not deprecated): released one day ago, configured nowhere.
#### Advantages #### Advantages
@@ -112,5 +112,5 @@ Keep `autoBuild` (including the v0.9.13 slot syntax) forever next to `builds`.
## Decision Outcome ## Decision Outcome
Top-level `builds` with `onPush`/`atTimes`, as specified above. Top-level `builds` with `onPush`/`atTimes`, as specified above; the reserved key `maxConcurrent` stays in the same section for compatibility.
`branches.*.autoBuild` survives one deprecation period as a mapped alias; the v0.9.13 slot extras are reverted. `branches.*.autoBuild` stays as a deprecated, mapped alias; the v0.9.13 slot extras are reverted.
+56 -41
View File
@@ -24,12 +24,13 @@ This layer applies **only** to the build itself. A pinned set is always taken fr
install/project config and can never be set from the worktree: install/project config and can never be set from the worktree:
- secrets and server-side settings: the whole `git`, `gitea`, and `server` sections; - secrets and server-side settings: the whole `git`, `gitea`, and `server` sections;
- the whole `builds` section: build definitions and `maxConcurrent`;
- the container sandbox policy: `docker.enabled` and `docker.network`. - the container sandbox policy: `docker.enabled` and `docker.network`.
This keeps a branch from disabling its own build container, changing its network mode, or This keeps a branch from disabling its own build container, changing its network mode, or
reaching credentials. Watcher decisions that happen before a build exists — the whole reaching credentials. The whole `builds` section (build definitions and `maxConcurrent`)
`autoBuild` section (schedule and slot commands) and the `requirePullRequest` gate — are and the deprecated `autoBuild` schedules are pinned too — jobs and their triggers are
read from the repo install/project config, because there is no worktree at that point. server-side decisions made before a build worktree exists.
## Inspect the Effective Config ## Inspect the Effective Config
@@ -84,13 +85,30 @@ gitea:
repo: my-repo # repository name repo: my-repo # repository name
statusContext: GitTally # label shown on Gitea commit status checks (default: GitTally) statusContext: GitTally # label shown on Gitea commit status checks (default: GitTally)
# Build execution. # Build execution and named build definitions (jobs, see notes below).
# "maxConcurrent" is a reserved key; every other key names a build definition.
builds: builds:
# How many branches may build at the same time. # How many branches may build at the same time.
# At most one build per branch runs regardless; each branch builds in its own # At most one build per branch runs regardless; each branch builds in its own
# git worktree under .git/gittally/worktrees/, never in the primary checkout. # git worktree under .git/gittally/worktrees/, never in the primary checkout.
# Changing this value requires a restart. # Changing this value requires a restart.
maxConcurrent: 1 maxConcurrent: 1
# Implicit unless overridden: the default build runs on push over all branches
# with the branch's regular settings — exactly the behavior without any
# build definitions. Set onPush: false here to disable on-push builds.
# default:
# onPush: true
#
# Example of a named build definition; all keys except its name are optional:
# pitest:
# onPush: false # trigger: build every new commit (default: false)
# atTimes: ["01:00"] # trigger: daily UTC times HH:MM (default: none)
# branches: ["master", "release/*"] # selector: names or glob patterns (default: all)
# activeWithin: 24h # selector: only branches with commits in the last 24h
# buildCommand: ./gradlew piTestFull # overrides; unset keys fall back to the
# cleanCommand: rm -rf build # merged branch settings (also available:
# artifactDirs: [build/reports] # stdoutLog, stderrLog, and docker
# # image/dockerfile/context/env)
# Build artifact storage and retention. # Build artifact storage and retention.
artifacts: artifacts:
@@ -145,15 +163,11 @@ branches:
# Build this branch only while its head commit matches a pull-request head on origin # Build this branch only while its head commit matches a pull-request head on origin
# (refs/pull/*/head — read via plain git, no API token needed; see notes below). # (refs/pull/*/head — read via plain git, no API token needed; see notes below).
requirePullRequest: false requirePullRequest: false
# DEPRECATED: define a build with atTimes in the builds section instead.
# Kept for compatibility: rebuilds this branch on schedule with its regular command.
autoBuild: autoBuild:
enabled: false # whether to rebuild on schedule enabled: false # whether to rebuild on schedule
# UTC times HH:MM for scheduled builds. An entry may carry its own build times: ["01:00"] # UTC times HH:MM for scheduled builds
# command, so a nightly slot can run a fuller check than the on-commit
# builds, and a name recording its builds in a separate pool (see notes below):
# - time: "01:00"
# buildCommand: ./gradlew fullCheck
# name: main@nightly
times: ["01:00"]
# Optional Docker build runtime; when enabled, the clean and build commands # Optional Docker build runtime; when enabled, the clean and build commands
# run inside a container instead of natively (see notes below). # run inside a container instead of natively (see notes below).
docker: docker:
@@ -170,27 +184,19 @@ branches:
# additional environment variables set inside the build container # additional environment variables set inside the build container
env: {} env: {}
main:
autoBuild:
enabled: true
master: master:
buildCommand: ./gradlew --console=plain --no-daemon quickCheck buildCommand: ./gradlew --console=plain --no-daemon quickCheck
autoBuild:
enabled: true
times:
# the nightly rebuild runs the full check instead of the quick on-commit
# one, recorded separately as master@nightly
- time: "01:00"
buildCommand: ./gradlew --console=plain --no-daemon completeCheck
name: master@nightly
release: release:
buildCommand: ./gradlew --console=plain --no-daemon --no-build-cache test jacocoReport buildCommand: ./gradlew --console=plain --no-daemon --no-build-cache test jacocoReport
autoBuild:
enabled: true builds:
times: # the nightly rebuild runs the full check instead of the quick on-commit one,
- "04:00" # recorded separately as master@pitest
pitest:
atTimes: ["01:00"]
branches: ["master"]
buildCommand: ./gradlew -PfullPitTest --console=plain --no-daemon piTestFull
``` ```
### Notes on `server.bindAddress` ### Notes on `server.bindAddress`
@@ -237,24 +243,33 @@ Without the `main` override, direct pushes and merges to `main` would never buil
A plain git origin (no Gitea/GitHub) serves no `refs/pull/*/head` at all, so gated branches would never build there. A plain git origin (no Gitea/GitHub) serves no `refs/pull/*/head` at all, so gated branches would never build there.
For such origins, disable all gates globally with `watcher.pullRequestGate: false` — typically in the machine-specific `.git/gittally/.gittally.yml`, so the committed configuration keeps the gates for forge-backed environments. For such origins, disable all gates globally with `watcher.pullRequestGate: false` — typically in the machine-specific `.git/gittally/.gittally.yml`, so the committed configuration keeps the gates for forge-backed environments.
### Notes on `branches.<name>.autoBuild.times` ### Notes on `builds` (build definitions)
Each entry is either a plain `HH:MM` string or an object with `time` and optional `buildCommand` and `name`; both forms mix freely in one list. Next to the reserved execution key `maxConcurrent`, every key of the `builds` section names a build definition (a job) over the branches — ADR 0007.
A slot without its own command runs the branch's regular `buildCommand`. A build definition has triggers, a branch selector, and build-setting overrides.
The typical use is a quick check on every commit and a fuller, slower check in the nightly slot of the same branch.
A slot's command is recorded in the build result. Triggers: `onPush: true` builds every new commit of the selected branches; `atTimes: ["HH:MM", …]` rebuilds their heads once per day and slot (UTC).
Restarting such a build from the UI re-runs it with the slot's command, and the startup recovery re-enqueues an interrupted one likewise — a build is always repeated with the command it originally ran. A definition may have both; one with neither never triggers automatically.
Manual `gittally build <branch>` runs and watcher builds for new commits always use the regular `buildCommand`.
Without a `name`, a slot's builds share the branch's history, retention pool, and permanent latest-green link — on a busy branch, the regular builds can displace the nightly build and its artifacts within a day. Selector: `branches` lists branch names or glob patterns (`*` matches any characters, also across `/`); empty selects all origin branches.
A slot `name` (e.g. `master@nightly`) records the slot's builds in their own pool instead: an own row in the branches view (sorted after its branch), an own `retentionPerBranch` count, an own latest status, and an own permanent artifact link. `activeWithin` (e.g. `24h`) additionally keeps only branches whose origin head commit is younger than the duration — useful to run a nightly deep check over all recently active branches.
The URL key is the sanitized name — `master@nightly` is served as `/branches/master_nightly/…`. Both parts combine as an intersection.
The builds still run in the branch's worktree, one build per branch at a time, and the Gitea commit status is still reported per commit in the shared status context, so the last build of a commit wins there regardless of its name. The `branches.<name>.requirePullRequest` gate stays a branch property and gates all watcher-triggered builds of that branch.
Do not name a slot like an existing branch — the pools would merge.
The name's results live as long as the underlying branch exists on origin.
The whole `autoBuild` section is a watcher decision made before a build worktree exists, so — unlike `buildCommand` itself — it is read from the repo install/project config and cannot be changed by the `.gittally.yml` committed on the branch being built. Overrides: `buildCommand`, `cleanCommand`, `artifactDirs`, `stdoutLog`/`stderrLog`, and the docker image keys (`image`, `dockerfile`, `context`, `env`).
The effective settings of one build on one branch merge in this order: defaults → `branches.default``branches.<branch>` → the worktree's committed `.gittally.yml` → the build definition's overrides.
Unset keys fall back; the definition wins last because it is the job.
The whole `builds` section is pinned: it always comes from the repo install/project config, and the `.gittally.yml` committed on a branch can neither define jobs nor change `maxConcurrent`.
The implicit `default` build (`onPush: true`, all branches) preserves the behavior without any definitions; defining other builds does not disable it, `builds.default.onPush: false` does.
The `default` build records under the plain branch name; every other build records under `<branch>@<name>` with its own row in the branches view (sorted after its branch), its own `retentionPerBranch` count, latest status, and permanent latest-green artifact link.
The URL key is the sanitized pool name — `master@pitest` is served as `/branches/master_pitest/…`.
The pools live as long as the underlying branch exists on origin.
Restart, `gittally retry`, and the startup recovery re-run a build under its recorded definition, resolving the settings from the current configuration — the job definition is the source of truth, not the historical run.
The builds still run in their branch's worktree, one build per branch at a time, and the Gitea commit status is reported per commit in the shared status context (the last build of a commit wins there).
`branches.<name>.autoBuild` (`enabled` + `times`) is the deprecated pre-ADR-0007 schedule, kept for compatibility: it rebuilds the branch's own pool with its regular command and logs a deprecation warning.
`autoBuild.times` entries carrying their own `buildCommand`/`name` (a short-lived v0.9.13 syntax) are no longer supported — use a build definition.
### Notes on `watcher.fastForwardLocalRefs` ### Notes on `watcher.fastForwardLocalRefs`
@@ -1,6 +1,7 @@
package de.hoennig.gittally.build package de.hoennig.gittally.build
import de.hoennig.gittally.config.BranchConfig import de.hoennig.gittally.config.BranchConfig
import de.hoennig.gittally.config.BuildDefinition
import de.hoennig.gittally.config.ConfigLoader import de.hoennig.gittally.config.ConfigLoader
import de.hoennig.gittally.gitea.GiteaClient import de.hoennig.gittally.gitea.GiteaClient
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
@@ -67,27 +68,25 @@ class BuildExecutor(
* not cancel-requested), that build is returned instead of stacking a duplicate — * not cancel-requested), that build is returned instead of stacking a duplicate —
* a double-triggered UI restart must not queue the same commit twice. Re-running * a double-triggered UI restart must not queue the same commit twice. Re-running
* a *finished* build stays possible; this only guards the active queue. * a *finished* build stays possible; this only guards the active queue.
* A [buildCommandOverride] (from an auto-build slot with its own command) replaces * The [build] names the build definition (job, ADR 0007) this run belongs to; its
* the branch's configured `buildCommand`; since the command differs, such a build * settings — command overrides and the result pool `<branch>@<build>` — are
* never counts as a duplicate of a regular build of the same commit. * resolved from the current configuration when the build starts executing. A
* A [name] (from a named auto-build slot) records the result under that name * non-default build has its own pool, so it never counts as a duplicate of the
* instead of the branch name, giving the slot its own history and retention pool; * branch's regular build of the same commit; it still runs in the branch's
* the build still runs in the branch's worktree, serialized with the branch's * worktree, serialized with the branch's other builds.
* other builds.
*/ */
fun startBuild( fun startBuild(
branch: String, branch: String,
commit: String, commit: String,
workingDir: Path = Paths.get("."), workingDir: Path = Paths.get("."),
buildCommandOverride: String? = null, build: String = BuildDefinition.DEFAULT,
name: String = branch,
): RunningBuild { ): RunningBuild {
val name = BuildDefinition.poolName(branch, build)
val duplicate = val duplicate =
builds.values.firstOrNull { builds.values.firstOrNull {
!it.cancelled.get() && !it.cancelled.get() &&
it.runningBuild.name == name && it.runningBuild.name == name &&
it.runningBuild.commit == commit && it.runningBuild.commit == commit
it.runningBuild.buildCommandOverride == buildCommandOverride
} }
if (duplicate != null) { if (duplicate != null) {
log.info("build of branch {} at commit {} is already queued or running; not queueing a duplicate", branch, commit) log.info("build of branch {} at commit {} is already queued or running; not queueing a duplicate", branch, commit)
@@ -98,23 +97,21 @@ class BuildExecutor(
val runningBuild = val runningBuild =
RunningBuild( RunningBuild(
branch = branch, branch = branch,
name = name, build = build,
commit = commit, commit = commit,
artifactKey = ArtifactKeys.buildKey(name, startedAt), artifactKey = ArtifactKeys.buildKey(name, startedAt),
startedAt = startedAt, startedAt = startedAt,
stagingDir = stagingDir, stagingDir = stagingDir,
liveLogFile = stagingDir.resolve(LIVE_LOG_FILE), liveLogFile = stagingDir.resolve(LIVE_LOG_FILE),
buildCommandOverride = buildCommandOverride,
) )
val pending = val pending =
BuildResult( BuildResult(
branch = branch, branch = branch,
name = name, build = build,
commit = commit, commit = commit,
status = BuildStatus.PENDING, status = BuildStatus.PENDING,
startedAt = startedAt, startedAt = startedAt,
duration = null, duration = null,
buildCommandOverride = buildCommandOverride,
artifactKey = runningBuild.artifactKey, artifactKey = runningBuild.artifactKey,
) )
repository.append(pending) repository.append(pending)
@@ -259,8 +256,8 @@ class BuildExecutor(
build: ActiveBuild, build: ActiveBuild,
workspace: Path, workspace: Path,
): Int { ): Int {
val branchConfig = branchConfig(build.runningBuild.branch, build.workingDir, workspace) val branchConfig = buildConfig(build.runningBuild, build.workingDir, workspace)
val buildCommand = build.runningBuild.buildCommandOverride ?: branchConfig.buildCommand val buildCommand = branchConfig.buildCommand
val stagingDir = build.runningBuild.stagingDir val stagingDir = build.runningBuild.stagingDir
Files.newOutputStream(stagingDir.resolve(branchConfig.stdoutLog)).use { stdoutLog -> Files.newOutputStream(stagingDir.resolve(branchConfig.stdoutLog)).use { stdoutLog ->
Files.newOutputStream(stagingDir.resolve(branchConfig.stderrLog)).use { stderrLog -> Files.newOutputStream(stagingDir.resolve(branchConfig.stderrLog)).use { stderrLog ->
@@ -374,13 +371,12 @@ class BuildExecutor(
) )
} ?: BuildResult( } ?: BuildResult(
branch = runningBuild.branch, branch = runningBuild.branch,
name = runningBuild.name, build = runningBuild.build,
commit = runningBuild.commit, commit = runningBuild.commit,
status = status, status = status,
startedAt = runningBuild.startedAt, startedAt = runningBuild.startedAt,
runningSince = runningBuild.runningSince, runningSince = runningBuild.runningSince,
duration = duration, duration = duration,
buildCommandOverride = runningBuild.buildCommandOverride,
artifactKey = runningBuild.artifactKey, artifactKey = runningBuild.artifactKey,
).also { repository.append(it) } ).also { repository.append(it) }
eventPublisher.publishEvent(BuildStatusChangedEvent(updated)) eventPublisher.publishEvent(BuildStatusChangedEvent(updated))
@@ -433,15 +429,12 @@ class BuildExecutor(
val header = val header =
buildString { buildString {
appendLine("building branch: ${runningBuild.branch}") appendLine("building branch: ${runningBuild.branch}")
if (runningBuild.name != runningBuild.branch) { if (runningBuild.build != BuildDefinition.DEFAULT) {
appendLine("build name: ${runningBuild.name}") appendLine("build: ${runningBuild.build} (recorded as ${runningBuild.name})")
} }
appendLine("commit: ${runningBuild.commit}") appendLine("commit: ${runningBuild.commit}")
appendLine("started: ${runningBuild.startedAt}") appendLine("started: ${runningBuild.startedAt}")
appendLine("workspace: $workspace") appendLine("workspace: $workspace")
if (runningBuild.buildCommandOverride != null) {
appendLine("triggered by: auto-build slot with its own build command")
}
appendLine("build command: $buildCommand") appendLine("build command: $buildCommand")
if (branchConfig.cleanCommand.isNotBlank()) { if (branchConfig.cleanCommand.isNotBlank()) {
appendLine("clean command: ${branchConfig.cleanCommand}") appendLine("clean command: ${branchConfig.cleanCommand}")
@@ -470,14 +463,22 @@ class BuildExecutor(
} }
} }
/** The build config for [branch], with the build [worktree]'s `.gittally.yml` layered on top (see [ConfigLoader.loadForWorktree]). */ /**
private fun branchConfig( * The effective settings of this run: the branch config with the build [worktree]'s
branch: String, * `.gittally.yml` layered on top (see [ConfigLoader.loadForWorktree]), then the
* build definition's overrides applied last — the job wins, and it always comes
* from the primary config (`builds` is a pinned section). An unknown build name
* (a stale result whose job was removed) falls back to the plain branch settings.
*/
private fun buildConfig(
runningBuild: RunningBuild,
workingDir: Path, workingDir: Path,
worktree: Path, worktree: Path,
): BranchConfig { ): BranchConfig {
val branches = configLoader.loadForWorktree(workingDir, worktree).branches val config = configLoader.loadForWorktree(workingDir, worktree)
return branches[branch] ?: branches["default"] ?: BranchConfig() val branchConfig = config.branches[runningBuild.branch] ?: config.branches["default"] ?: BranchConfig()
val definition = config.effectiveBuildDefinitions()[runningBuild.build] ?: return branchConfig
return definition.applyTo(branchConfig)
} }
private class ActiveBuild( private class ActiveBuild(
@@ -1,5 +1,6 @@
package de.hoennig.gittally.build package de.hoennig.gittally.build
import de.hoennig.gittally.config.BuildDefinition
import java.time.Duration import java.time.Duration
import java.time.Instant import java.time.Instant
@@ -7,12 +8,18 @@ data class BuildResult(
/** The git branch that was built — Gitea links and origin lookups always use this. */ /** The git branch that was built — Gitea links and origin lookups always use this. */
val branch: String, val branch: String,
/** /**
* The build name this result is recorded under: the branch name, unless a named * The build definition (job) this result belongs to; re-runs (restart, retry,
* auto-build slot (`autoBuild.times[].name`, e.g. `master@nightly`) set its own. * startup recovery) resolve their settings from the current configuration by
* History rows, retention pools, latest status, and the permanent latest-green * this name the job definition is the source of truth, not the recorded run.
* artifact links are all keyed by this name.
*/ */
val name: String = branch, val build: String = BuildDefinition.DEFAULT,
/**
* The pool this result is recorded under: the branch name for the default build,
* `<branch>@<build>` otherwise (see [BuildDefinition.poolName]). History rows,
* retention pools, latest status, and the permanent latest-green artifact links
* are all keyed by this name.
*/
val name: String = BuildDefinition.poolName(branch, build),
val commit: String, val commit: String,
val status: BuildStatus, val status: BuildStatus,
/** When the build was accepted (enqueued); the time until [runningSince] is queue wait. */ /** When the build was accepted (enqueued); the time until [runningSince] is queue wait. */
@@ -21,11 +28,5 @@ data class BuildResult(
val runningSince: Instant? = null, val runningSince: Instant? = null,
/** Pure build execution time (from [runningSince]), without the queue wait. */ /** Pure build execution time (from [runningSince]), without the queue wait. */
val duration: Duration? = null, val duration: Duration? = null,
/**
* Command dictated by the auto-build slot that triggered this build; null means the
* branch's configured `buildCommand` was used. A restart or startup-recovery re-run
* repeats the build with this command, so a build always reruns what it originally ran.
*/
val buildCommandOverride: String? = null,
val artifactKey: String, val artifactKey: String,
) )
@@ -1,5 +1,6 @@
package de.hoennig.gittally.build package de.hoennig.gittally.build
import de.hoennig.gittally.config.BuildDefinition
import java.nio.file.Path import java.nio.file.Path
import java.time.Instant import java.time.Instant
@@ -7,8 +8,10 @@ import java.time.Instant
data class RunningBuild( data class RunningBuild(
/** The git branch being built. */ /** The git branch being built. */
val branch: String, val branch: String,
/** The build name the result is recorded under; the branch name unless a named auto-build slot set its own. */ /** The build definition (job) this build runs; its settings are resolved from config at run time. */
val name: String = branch, val build: String = BuildDefinition.DEFAULT,
/** The pool the result is recorded under: the branch, or `<branch>@<build>` for non-default builds. */
val name: String = BuildDefinition.poolName(branch, build),
val commit: String, val commit: String,
val artifactKey: String, val artifactKey: String,
val startedAt: Instant, val startedAt: Instant,
@@ -6,6 +6,7 @@ import de.hoennig.gittally.build.BuildResult
import de.hoennig.gittally.build.BuildResultRepository import de.hoennig.gittally.build.BuildResultRepository
import de.hoennig.gittally.build.BuildStatus import de.hoennig.gittally.build.BuildStatus
import de.hoennig.gittally.build.RunningBuild import de.hoennig.gittally.build.RunningBuild
import de.hoennig.gittally.config.BuildDefinition
import de.hoennig.gittally.server.UiFormats import de.hoennig.gittally.server.UiFormats
import org.springframework.stereotype.Component import org.springframework.stereotype.Component
import java.io.IOException import java.io.IOException
@@ -37,10 +38,9 @@ class ConsoleBuildRunner(
branch: String, branch: String,
commit: String, commit: String,
workingDir: Path = Paths.get("."), workingDir: Path = Paths.get("."),
buildCommandOverride: String? = null, buildDefinition: String = BuildDefinition.DEFAULT,
name: String = branch,
): BuildStatus { ): BuildStatus {
val build = buildExecutor.startBuild(branch, commit, workingDir, buildCommandOverride, name) val build = buildExecutor.startBuild(branch, commit, workingDir, buildDefinition)
var printed = 0L var printed = 0L
var result: BuildResult? = null var result: BuildResult? = null
while (result?.status?.isTerminal != true) { while (result?.status?.isTerminal != true) {
@@ -150,10 +150,18 @@ class InitCommand(
repo: ${detected.repo} # repository name repo: ${detected.repo} # repository name
statusContext: GitTally # label shown on Gitea commit status checks (default: GitTally) statusContext: GitTally # label shown on Gitea commit status checks (default: GitTally)
# Build execution. # Build execution and named build definitions (jobs); "maxConcurrent" is a
# reserved key, every other key names a build definition over the branches.
builds: builds:
# how many branches may build at the same time (at most one build per branch regardless) # how many branches may build at the same time (at most one build per branch regardless)
maxConcurrent: 1 maxConcurrent: 1
# Example definition triggers (onPush/atTimes), branch selector
# (branches/activeWithin), and overrides of the branch settings:
# pitest:
# atTimes: ["01:00"] # daily UTC times HH:MM
# branches: ["master"] # names or glob patterns; default: all branches
# activeWithin: 24h # only branches with recent commits
# buildCommand: ./gradlew piTestFull
# Build artifact storage and retention. # Build artifact storage and retention.
artifacts: artifacts:
@@ -197,14 +205,10 @@ class InitCommand(
# build only while the branch head matches a pull-request head on origin # build only while the branch head matches a pull-request head on origin
# (refs/pull/*/head read via plain git, no API token needed) # (refs/pull/*/head read via plain git, no API token needed)
requirePullRequest: false requirePullRequest: false
# DEPRECATED: define a build with atTimes in the builds section instead
autoBuild: autoBuild:
enabled: false # whether to rebuild on schedule enabled: false # whether to rebuild on schedule
# UTC times HH:MM for scheduled builds; an entry may carry its own times: ["01:00"] # UTC times HH:MM for scheduled builds
# command and a name for a separate history/artifact pool:
# - time: "01:00"
# buildCommand: ./gradlew fullCheck
# name: main@nightly
times: ["01:00"]
docker: docker:
enabled: false # run clean/build commands in a Docker container instead of natively enabled: false # run clean/build commands in a Docker container instead of natively
image: "" # image for the build container; required when enabled image: "" # image for the build container; required when enabled
@@ -50,9 +50,9 @@ class RetryCommand(
println("skipping branch ${result.branch}: gone from origin") println("skipping branch ${result.branch}: gone from origin")
continue continue
} }
println("retrying branch ${result.branch} at commit ${commit.take(12)}") println("retrying build ${result.name} at commit ${commit.take(12)}")
// a failed auto-slot build retries with its recorded command, under its name // a failed build retries its recorded build definition (settings from the current config)
val status = consoleBuildRunner.buildAndStream(result.branch, commit, workingDir, result.buildCommandOverride, result.name) val status = consoleBuildRunner.buildAndStream(result.branch, commit, workingDir, result.build)
if (status != BuildStatus.SUCCESS) { if (status != BuildStatus.SUCCESS) {
anyFailed = true anyFailed = true
} }
@@ -1,62 +0,0 @@
package de.hoennig.gittally.config
import com.fasterxml.jackson.core.JsonGenerator
import com.fasterxml.jackson.core.JsonParser
import com.fasterxml.jackson.databind.DeserializationContext
import com.fasterxml.jackson.databind.JsonDeserializer
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.JsonSerializer
import com.fasterxml.jackson.databind.SerializerProvider
/**
* Accepts both YAML forms of an [AutoBuildSlot] entry: a plain `HH:MM` string, or an
* object with `time` and optional `buildCommand` and `name`.
*/
class AutoBuildSlotDeserializer : JsonDeserializer<AutoBuildSlot>() {
override fun deserialize(
parser: JsonParser,
context: DeserializationContext,
): AutoBuildSlot {
val node = parser.readValueAsTree<JsonNode>()
if (node.isTextual) {
return AutoBuildSlot(time = node.asText())
}
if (node.isObject) {
val time =
node.get("time")?.takeIf { it.isTextual }?.asText()
?: throw context.instantiationException(AutoBuildSlot::class.java, "auto-build slot object needs a 'time' (HH:MM)")
return AutoBuildSlot(
time = time,
buildCommand = node.get("buildCommand")?.asText() ?: "",
name = node.get("name")?.asText() ?: "",
)
}
throw context.instantiationException(
AutoBuildSlot::class.java,
"auto-build slot must be an HH:MM string or an object with 'time' and optional 'buildCommand' and 'name'",
)
}
}
/** Writes the compact form back: a plain string unless the slot carries its own command or name. */
class AutoBuildSlotSerializer : JsonSerializer<AutoBuildSlot>() {
override fun serialize(
slot: AutoBuildSlot,
generator: JsonGenerator,
provider: SerializerProvider,
) {
if (slot.buildCommand.isBlank() && slot.name.isBlank()) {
generator.writeString(slot.time)
return
}
generator.writeStartObject()
generator.writeStringField("time", slot.time)
if (slot.buildCommand.isNotBlank()) {
generator.writeStringField("buildCommand", slot.buildCommand)
}
if (slot.name.isNotBlank()) {
generator.writeStringField("name", slot.name)
}
generator.writeEndObject()
}
}
@@ -0,0 +1,111 @@
package de.hoennig.gittally.config
import java.time.Duration
import java.time.Instant
/**
* A named build (job) over the branches ADR 0007. In YAML these live in the
* top-level `builds` section next to the reserved execution key `maxConcurrent`
* (split apart by [ConfigLoader]); a build definition always comes from the repo
* install/project config, never from a build worktree.
*
* The `default` build records its results under the plain branch name; every other
* build records under `<branch>@<name>` with its own history, retention pool, and
* permanent latest-green link.
*/
data class BuildDefinition(
/** Build every new commit of the selected branches. */
val onPush: Boolean = false,
/** Daily UTC times `HH:MM`; each slot rebuilds the selected branches' heads once per day. */
val atTimes: List<String> = emptyList(),
/**
* Branch names or glob patterns (`*` matches any characters, also across `/`);
* empty selects all origin branches.
*/
val branches: List<String> = emptyList(),
/**
* Only branches whose origin head commit is younger than this (e.g. `24h`);
* empty applies no age filter. Combines with [branches] as an intersection.
*/
val activeWithin: String = "",
/** Overrides the branch's build command; null inherits it. */
val buildCommand: String? = null,
/** Overrides the branch's clean command; null inherits it. */
val cleanCommand: String? = null,
/** Overrides the branch's artifact directories; null inherits them. */
val artifactDirs: List<String>? = null,
/** Overrides the branch's stdout log file name; null inherits it. */
val stdoutLog: String? = null,
/** Overrides the branch's stderr log file name; null inherits it. */
val stderrLog: String? = null,
/** Overrides of the branch's docker image settings; the sandbox policy (`enabled`, `network`) is not overridable. */
val docker: DockerOverrides? = null,
) {
/** True when [branch] matches the [branches] patterns (or none are configured). */
fun selectsByName(branch: String): Boolean = branches.isEmpty() || branches.any { globToRegex(it).matches(branch) }
/**
* True when [branch] passes both selector parts; [headCommittedAt] is the branch
* head's committer time, only consulted while [activeWithin] is set (null then
* deselects the branch).
*/
fun selects(
branch: String,
headCommittedAt: () -> Instant?,
now: Instant,
): Boolean {
if (!selectsByName(branch)) {
return false
}
if (activeWithin.isBlank()) {
return true
}
val committedAt = headCommittedAt() ?: return false
return committedAt >= now.minus(maxAge())
}
fun maxAge(): Duration = DurationParser.parse(activeWithin)
/** The branch settings with this build's overrides applied; unset values fall through. */
fun applyTo(branchConfig: BranchConfig): BranchConfig =
branchConfig.copy(
buildCommand = buildCommand ?: branchConfig.buildCommand,
cleanCommand = cleanCommand ?: branchConfig.cleanCommand,
artifactDirs = artifactDirs ?: branchConfig.artifactDirs,
stdoutLog = stdoutLog ?: branchConfig.stdoutLog,
stderrLog = stderrLog ?: branchConfig.stderrLog,
docker =
branchConfig.docker.copy(
image = docker?.image ?: branchConfig.docker.image,
dockerfile = docker?.dockerfile ?: branchConfig.docker.dockerfile,
context = docker?.context ?: branchConfig.docker.context,
env = docker?.env ?: branchConfig.docker.env,
),
)
companion object {
/** Name of the implicit build that preserves the pre-ADR-0007 behavior: `onPush` over all branches. */
const val DEFAULT = "default"
/** The result-pool name of [build] on [branch]: the plain branch for the default build. */
fun poolName(
branch: String,
build: String,
): String = if (build == DEFAULT) branch else "$branch@$build"
private fun globToRegex(pattern: String): Regex =
Regex(
pattern
.split('*')
.joinToString(".*") { Regex.escape(it) },
)
}
}
/** Nullable docker image overrides of a [BuildDefinition]; null values inherit the branch's setting. */
data class DockerOverrides(
val image: String? = null,
val dockerfile: String? = null,
val context: String? = null,
val env: Map<String, String>? = null,
)
@@ -29,10 +29,11 @@ class ConfigLoader {
* `docker.image`/`env`, ). * `docker.image`/`env`, ).
* *
* The [pinned][stripPinned] keys are the exception: secrets (`git`), `gitea`/`server` * The [pinned][stripPinned] keys are the exception: secrets (`git`), `gitea`/`server`
* settings, and the docker sandbox policy (`docker.enabled`/`docker.network`) always * settings, the whole `builds` section (job definitions and execution settings), and
* come from `.git`/primary a branch must never be able to disable its own container, * the docker sandbox policy (`docker.enabled`/`docker.network`) always come from
* change its network mode, or reach the credentials. They are stripped from the * `.git`/primary a branch must never be able to disable its own container, change
* worktree layer before it is merged, so a worktree cannot set them at all. * its network mode, redefine jobs, or reach the credentials. They are stripped from
* the worktree layer before it is merged, so a worktree cannot set them at all.
* *
* With no worktree `.gittally.yml` this is identical to [load]. * With no worktree `.gittally.yml` this is identical to [load].
*/ */
@@ -50,11 +51,30 @@ class ConfigLoader {
if (raw.isEmpty()) { if (raw.isEmpty()) {
GitTallyConfig() GitTallyConfig()
} else { } else {
yaml.convertValue(mergeBranchDefaults(raw), GitTallyConfig::class.java) yaml.convertValue(splitBuildsSection(mergeBranchDefaults(raw)), GitTallyConfig::class.java)
} }
return defaultPublicBaseUrl(config) return defaultPublicBaseUrl(config)
} }
/**
* The YAML `builds` section carries the reserved execution key `maxConcurrent`
* next to the named build definitions (ADR 0007); the schema separates them into
* [GitTallyConfig.builds] and [GitTallyConfig.buildDefinitions].
*/
@Suppress("UNCHECKED_CAST")
private fun splitBuildsSection(raw: Map<String, Any?>): Map<String, Any?> {
val builds = raw["builds"] as? Map<String, Any?> ?: return raw
val definitions = builds.filterKeys { it !in RESERVED_BUILDS_KEYS }
if (definitions.isEmpty()) {
return raw
}
return raw +
mapOf(
"builds" to builds.filterKeys { it in RESERVED_BUILDS_KEYS },
"buildDefinitions" to definitions,
)
}
/** /**
* Removes the keys a build worktree must never override: the secret/server-side * Removes the keys a build worktree must never override: the secret/server-side
* top-level sections and the per-branch docker sandbox policy. See [loadForWorktree]. * top-level sections and the per-branch docker sandbox policy. See [loadForWorktree].
@@ -141,8 +161,15 @@ class ConfigLoader {
} }
companion object { companion object {
/** Top-level sections a build worktree must never override: secrets and server-side settings. */ /**
private val PINNED_TOP_LEVEL_KEYS = setOf("git", "gitea", "server") * Top-level sections a build worktree must never override: secrets, server-side
* settings, and the build definitions with their execution settings (a branch
* must not be able to redefine jobs or raise concurrency).
*/
private val PINNED_TOP_LEVEL_KEYS = setOf("git", "gitea", "server", "builds")
/** Keys of the YAML `builds` section that are execution settings, not build definitions. */
private val RESERVED_BUILDS_KEYS = setOf("maxConcurrent")
/** Per-branch `docker` keys the worktree must never override: the sandbox policy. */ /** Per-branch `docker` keys the worktree must never override: the sandbox policy. */
private val PINNED_DOCKER_KEYS = setOf("enabled", "network") private val PINNED_DOCKER_KEYS = setOf("enabled", "network")
@@ -1,8 +1,5 @@
package de.hoennig.gittally.config package de.hoennig.gittally.config
import com.fasterxml.jackson.databind.annotation.JsonDeserialize
import com.fasterxml.jackson.databind.annotation.JsonSerialize
data class GitTallyConfig( data class GitTallyConfig(
val server: ServerConfig = ServerConfig(), val server: ServerConfig = ServerConfig(),
val git: GitConfig = GitConfig(), val git: GitConfig = GitConfig(),
@@ -11,7 +8,17 @@ data class GitTallyConfig(
val artifacts: ArtifactsConfig = ArtifactsConfig(), val artifacts: ArtifactsConfig = ArtifactsConfig(),
val watcher: WatcherConfig = WatcherConfig(), val watcher: WatcherConfig = WatcherConfig(),
val branches: Map<String, BranchConfig> = mapOf("default" to BranchConfig()), val branches: Map<String, BranchConfig> = mapOf("default" to BranchConfig()),
) /**
* Named build definitions (jobs) over the branches (ADR 0007). The implicit
* [BuildDefinition.DEFAULT] build (`onPush` over all branches) applies unless this
* map overrides it; see [effectiveBuildDefinitions].
*/
val buildDefinitions: Map<String, BuildDefinition> = emptyMap(),
) {
/** The configured [buildDefinitions] plus the implicit `default` build unless overridden. */
fun effectiveBuildDefinitions(): Map<String, BuildDefinition> =
mapOf(BuildDefinition.DEFAULT to BuildDefinition(onPush = true)) + buildDefinitions
}
data class ServerConfig( data class ServerConfig(
/** /**
@@ -149,32 +156,13 @@ data class DockerConfig(
val env: Map<String, String> = emptyMap(), val env: Map<String, String> = emptyMap(),
) )
/**
* Deprecated per-branch schedule (pre-ADR-0007), kept for compatibility: mapped to a
* daily rebuild of the branch's own pool with its regular command. New configurations
* define a build with `atTimes` in the top-level `builds` section instead.
*/
data class AutoBuildConfig( data class AutoBuildConfig(
val enabled: Boolean = false, val enabled: Boolean = false,
/** /** Daily UTC times `HH:MM`. */
* Daily UTC slots. Each entry is either a plain `HH:MM` time or an object with val times: List<String> = listOf("01:00"),
* `time` and its own `buildCommand` so a nightly slot can run a fuller check
* than the on-commit builds of the same branch.
*/
val times: List<AutoBuildSlot> = listOf(AutoBuildSlot("01:00")),
)
/**
* One scheduled auto-build slot; in YAML either a plain `HH:MM` string or an object
* (`time` plus optional `buildCommand` and `name`) see [AutoBuildSlotDeserializer].
*/
@JsonDeserialize(using = AutoBuildSlotDeserializer::class)
@JsonSerialize(using = AutoBuildSlotSerializer::class)
data class AutoBuildSlot(
/** UTC time of day, `HH:MM`. */
val time: String,
/** Command this slot's build runs; empty runs the branch's [BranchConfig.buildCommand]. */
val buildCommand: String = "",
/**
* Build name this slot's builds are recorded under (e.g. `master@nightly`); empty
* records them under the branch name. A named slot gets its own history, retention
* pool, latest status, and permanent latest-green artifact link, so its builds are
* not displaced by the branch's regular builds.
*/
val name: String = "",
) )
@@ -140,6 +140,19 @@ class GitService(
}.map { (branch, _) -> branch } }.map { (branch, _) -> branch }
} }
/** Committer timestamps of all origin branch heads, in one git call; used by `activeWithin` build selectors. */
fun originBranchCommitTimes(workingDir: Path = Paths.get(".")): Map<String, Instant> =
runner
.runOrThrow(
listOf("git", "for-each-ref", "--format=%(refname:strip=3) %(committerdate:unix)", "refs/remotes/origin"),
workingDir,
).lines()
.mapNotNull { line ->
val branch = line.substringBeforeLast(' ')
val epochSeconds = line.substringAfterLast(' ').toLongOrNull() ?: return@mapNotNull null
if (branch == "HEAD") null else branch to Instant.ofEpochSecond(epochSeconds)
}.toMap()
/** Switches to an existing local branch, or creates a tracking branch from origin. */ /** Switches to an existing local branch, or creates a tracking branch from origin. */
fun checkout( fun checkout(
branch: String, branch: String,
@@ -5,6 +5,7 @@ import de.hoennig.gittally.build.BuildExecutor
import de.hoennig.gittally.build.BuildResult import de.hoennig.gittally.build.BuildResult
import de.hoennig.gittally.build.BuildResultRepository import de.hoennig.gittally.build.BuildResultRepository
import de.hoennig.gittally.build.BuildStatus import de.hoennig.gittally.build.BuildStatus
import de.hoennig.gittally.config.BuildDefinition
import de.hoennig.gittally.git.GitService import de.hoennig.gittally.git.GitService
import org.springframework.http.HttpStatus import org.springframework.http.HttpStatus
import org.springframework.http.ResponseEntity import org.springframework.http.ResponseEntity
@@ -103,13 +104,12 @@ class BuildsApiController(
latest?.commit latest?.commit
?: gitService.originHeadCommit(branch, workingDir) ?: gitService.originHeadCommit(branch, workingDir)
?: return notFound("branch '$branch' has no recorded build and no origin counterpart") ?: return notFound("branch '$branch' has no recorded build and no origin counterpart")
// a restarted build repeats what it originally ran: same branch, name, and command // a restarted build re-runs its recorded build definition (settings from the current config)
val running = val running =
buildExecutor.startBuild( buildExecutor.startBuild(
branch = latest?.branch ?: branch, branch = latest?.branch ?: branch,
commit = commit, commit = commit,
buildCommandOverride = latest?.buildCommandOverride, build = latest?.build ?: BuildDefinition.DEFAULT,
name = latest?.name ?: branch,
) )
return ResponseEntity.accepted().body( return ResponseEntity.accepted().body(
BuildResultDto( BuildResultDto(
@@ -5,7 +5,6 @@ import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.databind.SerializationFeature import com.fasterxml.jackson.databind.SerializationFeature
import com.fasterxml.jackson.module.kotlin.readValue import com.fasterxml.jackson.module.kotlin.readValue
import com.fasterxml.jackson.module.kotlin.registerKotlinModule import com.fasterxml.jackson.module.kotlin.registerKotlinModule
import de.hoennig.gittally.config.AutoBuildSlot
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import java.nio.file.Files import java.nio.file.Files
import java.nio.file.Path import java.nio.file.Path
@@ -14,28 +13,33 @@ import java.time.LocalDate
import java.time.LocalTime import java.time.LocalTime
import java.time.format.DateTimeParseException import java.time.format.DateTimeParseException
/** One recorded auto-build trigger: [branch] was enqueued for the [slot] (UTC `HH:MM`) of [date] (ISO). */ /**
* One recorded scheduled-build trigger: the result pool [branch] (a branch, or
* `<branch>@<build>` of a named build definition) was enqueued for the [slot]
* (UTC `HH:MM`) of [date] (ISO). The field keeps its legacy name `branch` so the
* state file stays readable across versions.
*/
data class AutoBuildTrigger( data class AutoBuildTrigger(
val branch: String, val branch: String,
val date: String, val date: String,
val slot: String, val slot: String,
) )
/** Auto-build time slot matching (UTC `HH:MM`), like legacy `auto_build_check`. */ /** Scheduled-build time slot matching (UTC `HH:MM`), like legacy `auto_build_check`. */
object AutoBuildSlots { object AutoBuildSlots {
private val log = LoggerFactory.getLogger(AutoBuildSlots::class.java) private val log = LoggerFactory.getLogger(AutoBuildSlots::class.java)
/** The latest valid slot at or before [now], or null when no slot is due yet today. */ /** The latest valid slot at or before [now], or null when no slot is due yet today. */
fun latestDueSlot( fun latestDueSlot(
times: List<AutoBuildSlot>, times: List<String>,
now: LocalTime, now: LocalTime,
): AutoBuildSlot? = ): String? =
times times
.mapNotNull { slot -> .mapNotNull { slot ->
try { try {
LocalTime.parse(slot.time.trim()) to slot LocalTime.parse(slot.trim()) to slot
} catch (_: DateTimeParseException) { } catch (_: DateTimeParseException) {
log.warn("skipping invalid auto-build time slot '{}': expected HH:MM", slot.time) log.warn("skipping invalid scheduled-build time slot '{}': expected HH:MM", slot)
null null
} }
}.filter { (parsed, _) -> !parsed.isAfter(now) } }.filter { (parsed, _) -> !parsed.isAfter(now) }
@@ -7,6 +7,7 @@ import de.hoennig.gittally.build.BuildResultRepository
import de.hoennig.gittally.build.BuildStatus import de.hoennig.gittally.build.BuildStatus
import de.hoennig.gittally.build.GitWorktreeWorkspaces import de.hoennig.gittally.build.GitWorktreeWorkspaces
import de.hoennig.gittally.config.BranchConfig import de.hoennig.gittally.config.BranchConfig
import de.hoennig.gittally.config.BuildDefinition
import de.hoennig.gittally.config.ConfigLoader import de.hoennig.gittally.config.ConfigLoader
import de.hoennig.gittally.config.DurationParser import de.hoennig.gittally.config.DurationParser
import de.hoennig.gittally.config.GitTallyConfig import de.hoennig.gittally.config.GitTallyConfig
@@ -17,6 +18,7 @@ import java.nio.file.Files
import java.nio.file.Path import java.nio.file.Path
import java.nio.file.Paths import java.nio.file.Paths
import java.time.Clock import java.time.Clock
import java.time.Instant
import java.time.LocalDate import java.time.LocalDate
import java.time.LocalTime import java.time.LocalTime
import java.time.ZoneOffset import java.time.ZoneOffset
@@ -48,6 +50,10 @@ class Watcher(
@Volatile @Volatile
private var state = WatcherState() private var state = WatcherState()
/** The branches.*.autoBuild deprecation is logged once per watcher instance, not once per poll. */
@Volatile
private var warnedDeprecatedAutoBuild = false
fun state(): WatcherState = state fun state(): WatcherState = state
/** /**
@@ -109,9 +115,9 @@ class Watcher(
// the executor queue did not survive the restart; the re-enqueued build supersedes the stale entry // the executor queue did not survive the restart; the re-enqueued build supersedes the stale entry
repository.updateByArtifactKey(result.artifactKey) { it.copy(status = BuildStatus.INTERRUPTED) } repository.updateByArtifactKey(result.artifactKey) { it.copy(status = BuildStatus.INTERRUPTED) }
} }
log.info("restarting unfinished build of branch {}", result.branch) log.info("restarting unfinished build {} of branch {}", result.build, result.branch)
// an interrupted auto-slot build re-runs the command its slot dictated, under its name // the re-run resolves its settings from the current config by the recorded build name
buildExecutor.startBuild(result.branch, commit, workingDir, result.buildCommandOverride, result.name) buildExecutor.startBuild(result.branch, commit, workingDir, result.build)
} }
} }
@@ -187,17 +193,30 @@ class Watcher(
) { ) {
// one ls-remote per poll cycle at most, and only when a due branch requires a pull request // one ls-remote per poll cycle at most, and only when a due branch requires a pull request
val pullRequestHeads = lazy { gitService.pullRequestHeads(workingDir) } val pullRequestHeads = lazy { gitService.pullRequestHeads(workingDir) }
// one for-each-ref per cycle at most, and only when a definition filters by activeWithin
val headCommitTimes = lazy { gitService.originBranchCommitTimes(workingDir) }
val definitions = config.effectiveBuildDefinitions()
val changedLocal = val changedLocal =
gitService gitService
.localBranches(workingDir) .localBranches(workingDir)
.filter { it in originBranches && gitService.hasNewCommits(it, workingDir) } .filter { it in originBranches && gitService.hasNewCommits(it, workingDir) }
val newOrigin = val newOrigin =
gitService.newOriginBranches(DurationParser.parse(config.watcher.newBranchMaxAge), workingDir) gitService.newOriginBranches(DurationParser.parse(config.watcher.newBranchMaxAge), workingDir)
for (branch in (changedLocal + newOrigin).distinct()) { val changed = (changedLocal + newOrigin).distinct()
startBuildIfDue(branch, allowSameCommit = false, config, pullRequestHeads, workingDir) for ((buildName, definition) in definitions.filterValues { it.onPush }) {
for (branch in changed.filter { selects(definition, it, headCommitTimes) }) {
startBuildIfDue(branch, allowSameCommit = false, config, pullRequestHeads, workingDir, buildName)
} }
enqueueAutoBuilds(config, originBranches, pullRequestHeads, workingDir)
} }
enqueueScheduledBuilds(definitions, config, originBranches, pullRequestHeads, headCommitTimes, workingDir)
enqueueDeprecatedAutoBuilds(config, originBranches, pullRequestHeads, workingDir)
}
private fun selects(
definition: BuildDefinition,
branch: String,
headCommitTimes: Lazy<Map<String, Instant>>,
): Boolean = definition.selects(branch, { headCommitTimes.value[branch] }, clock.instant())
/** /**
* Enqueues a build of the branch's origin head unless one is already pending or * Enqueues a build of the branch's origin head unless one is already pending or
@@ -217,10 +236,9 @@ class Watcher(
config: GitTallyConfig, config: GitTallyConfig,
pullRequestHeads: Lazy<Set<String>>, pullRequestHeads: Lazy<Set<String>>,
workingDir: Path, workingDir: Path,
buildCommandOverride: String? = null, build: String = BuildDefinition.DEFAULT,
name: String = branch,
): Boolean { ): Boolean {
val latest = repository.latestFor(name) val latest = repository.latestFor(BuildDefinition.poolName(branch, build))
if (latest?.status == BuildStatus.PENDING || latest?.status == BuildStatus.RUNNING) { if (latest?.status == BuildStatus.PENDING || latest?.status == BuildStatus.RUNNING) {
return false return false
} }
@@ -235,8 +253,8 @@ class Watcher(
log.info("not enqueueing branch {}: no pull request has head commit {}", branch, commit) log.info("not enqueueing branch {}: no pull request has head commit {}", branch, commit)
return false return false
} }
log.info("enqueueing build of branch {} at commit {}", branch, commit) log.info("enqueueing build {} of branch {} at commit {}", build, branch, commit)
buildExecutor.startBuild(branch, commit, workingDir, buildCommandOverride, name) buildExecutor.startBuild(branch, commit, workingDir, build)
return true return true
} }
@@ -245,7 +263,47 @@ class Watcher(
branch: String, branch: String,
): BranchConfig = config.branches[branch] ?: config.branches["default"] ?: BranchConfig() ): BranchConfig = config.branches[branch] ?: config.branches["default"] ?: BranchConfig()
private fun enqueueAutoBuilds( /**
* Fires the due `atTimes` slot of every build definition for its selected branches,
* once per day and slot per result pool. Rebuilding the already-built commit is
* the point of a scheduled build.
*/
private fun enqueueScheduledBuilds(
definitions: Map<String, BuildDefinition>,
config: GitTallyConfig,
originBranches: Set<String>,
pullRequestHeads: Lazy<Set<String>>,
headCommitTimes: Lazy<Map<String, Instant>>,
workingDir: Path,
) {
val scheduled = definitions.filterValues { it.atTimes.isNotEmpty() }
if (scheduled.isEmpty()) {
return
}
val autoBuildState = FileAutoBuildState(workingDir.resolve(AUTO_BUILDS_FILE))
val now = clock.instant()
val today = LocalDate.ofInstant(now, ZoneOffset.UTC)
val timeOfDay = LocalTime.ofInstant(now, ZoneOffset.UTC)
for ((buildName, definition) in scheduled) {
val slot = AutoBuildSlots.latestDueSlot(definition.atTimes, timeOfDay) ?: continue
for (branch in originBranches.filter { selects(definition, it, headCommitTimes) }) {
val pool = BuildDefinition.poolName(branch, buildName)
if (autoBuildState.isTriggered(pool, today, slot)) {
continue
}
if (startBuildIfDue(branch, allowSameCommit = true, config, pullRequestHeads, workingDir, buildName)) {
autoBuildState.markTriggered(pool, today, slot)
}
}
}
}
/**
* The pre-ADR-0007 `branches.<name>.autoBuild` schedule, kept for compatibility:
* a daily rebuild of the branch's own pool with its regular command exactly a
* `builds` entry with `atTimes` and a single-branch selector would do.
*/
private fun enqueueDeprecatedAutoBuilds(
config: GitTallyConfig, config: GitTallyConfig,
originBranches: Set<String>, originBranches: Set<String>,
pullRequestHeads: Lazy<Set<String>>, pullRequestHeads: Lazy<Set<String>>,
@@ -258,33 +316,28 @@ class Watcher(
if (autoBuildBranches.isEmpty()) { if (autoBuildBranches.isEmpty()) {
return return
} }
if (!warnedDeprecatedAutoBuild) {
warnedDeprecatedAutoBuild = true
log.warn(
"branches.*.autoBuild is deprecated; define a build with atTimes in the builds section instead (branches: {})",
autoBuildBranches.keys.joinToString(", "),
)
}
val autoBuildState = FileAutoBuildState(workingDir.resolve(AUTO_BUILDS_FILE)) val autoBuildState = FileAutoBuildState(workingDir.resolve(AUTO_BUILDS_FILE))
val now = clock.instant() val now = clock.instant()
val today = LocalDate.ofInstant(now, ZoneOffset.UTC) val today = LocalDate.ofInstant(now, ZoneOffset.UTC)
val timeOfDay = LocalTime.ofInstant(now, ZoneOffset.UTC) val timeOfDay = LocalTime.ofInstant(now, ZoneOffset.UTC)
for ((branch, branchConfig) in autoBuildBranches) { for ((branch, branchConfig) in autoBuildBranches) {
val slot = AutoBuildSlots.latestDueSlot(branchConfig.autoBuild.times, timeOfDay) ?: continue val slot = AutoBuildSlots.latestDueSlot(branchConfig.autoBuild.times, timeOfDay) ?: continue
if (autoBuildState.isTriggered(branch, today, slot.time)) { if (autoBuildState.isTriggered(branch, today, slot)) {
continue continue
} }
if (branch !in originBranches) { if (branch !in originBranches) {
log.warn("skipping auto build of branch {}: branch is not on origin", branch) log.warn("skipping auto build of branch {}: branch is not on origin", branch)
continue continue
} }
// rebuilding the already-built commit is the point of an auto build; a slot if (startBuildIfDue(branch, allowSameCommit = true, config, pullRequestHeads, workingDir)) {
// with its own command dictates it, and a named slot records under its name autoBuildState.markTriggered(branch, today, slot)
val started =
startBuildIfDue(
branch,
allowSameCommit = true,
config,
pullRequestHeads,
workingDir,
slot.buildCommand.ifBlank { null },
slot.name.ifBlank { branch },
)
if (started) {
autoBuildState.markTriggered(branch, today, slot.time)
} }
} }
} }
@@ -219,23 +219,38 @@ class BuildExecutorTest : FunSpec() {
} }
} }
test("a build command override replaces the branch's build command and is recorded in the result") { test("a build definition overrides the branch's build command and records under its own pool") {
val h = harness(buildCommand = "echo regular-\$branch") val h =
Harness(
"""
branches:
default:
buildCommand: "echo regular-${'$'}branch"
cleanCommand: ""
builds:
pitest:
buildCommand: "echo nightly-${'$'}branch"
""".trimIndent(),
)
val nightly = h.executor.startBuild("main", "sha-1", h.workingDir, "echo nightly-\$branch") val nightly = h.executor.startBuild("main", "sha-1", h.workingDir, "pitest")
awaitStatus(h, "main", BuildStatus.SUCCESS) awaitStatus(h, "main@pitest", BuildStatus.SUCCESS)
awaitIdle(h) awaitIdle(h)
val stdoutLog = Files.readString(nightly.stagingDir.resolve("build.stdout.log")) val stdoutLog = Files.readString(nightly.stagingDir.resolve("build.stdout.log"))
stdoutLog shouldContain "nightly-main" stdoutLog shouldContain "nightly-main"
stdoutLog shouldNotContain "regular-main" stdoutLog shouldNotContain "regular-main"
Files.readString(nightly.liveLogFile) shouldContain "triggered by: auto-build slot" Files.readString(nightly.liveLogFile) shouldContain "build: pitest (recorded as main@pitest)"
h.repository val result = h.repository.latestFor("main@pitest").shouldNotBeNull()
.latestFor("main") result.branch shouldBe "main"
.shouldNotBeNull() result.build shouldBe "pitest"
.buildCommandOverride shouldBe "echo nightly-\$branch" result.name shouldBe "main@pitest"
result.artifactKey shouldBe nightly.artifactKey
nightly.artifactKey shouldContain "main_pitest"
// the branch's own pool stays empty — the pitest build does not shadow it
h.repository.latestFor("main") shouldBe null
// the same branch without an override runs the regular command // the same branch under the default build runs the regular command
val regular = h.executor.startBuild("main", "sha-2", h.workingDir) val regular = h.executor.startBuild("main", "sha-2", h.workingDir)
awaitStatus(h, "main", BuildStatus.SUCCESS) awaitStatus(h, "main", BuildStatus.SUCCESS)
awaitIdle(h) awaitIdle(h)
@@ -243,24 +258,17 @@ class BuildExecutorTest : FunSpec() {
h.repository h.repository
.latestFor("main") .latestFor("main")
.shouldNotBeNull() .shouldNotBeNull()
.buildCommandOverride shouldBe null .build shouldBe "default"
} }
test("a named build is recorded under its name, keyed by the sanitized name") { test("a build whose definition was removed from the config falls back to the branch's settings") {
val h = harness(buildCommand = "echo regular-\$branch") val h = harness(buildCommand = "echo regular-\$branch")
val build = h.executor.startBuild("main", "sha-1", h.workingDir, "echo nightly-\$branch", "main@nightly") val build = h.executor.startBuild("main", "sha-1", h.workingDir, "gone-build")
awaitStatus(h, "main@nightly", BuildStatus.SUCCESS) awaitStatus(h, "main@gone-build", BuildStatus.SUCCESS)
awaitIdle(h) awaitIdle(h)
val result = h.repository.latestFor("main@nightly").shouldNotBeNull() Files.readString(build.stagingDir.resolve("build.stdout.log")) shouldContain "regular-main"
result.branch shouldBe "main"
result.name shouldBe "main@nightly"
result.artifactKey shouldBe build.artifactKey
build.artifactKey shouldContain "main_nightly"
Files.readString(build.liveLogFile) shouldContain "build name: main@nightly"
// the branch's own pool stays empty — the named build does not shadow it
h.repository.latestFor("main") shouldBe null
} }
test("startBuild returns the active build of the same branch and commit instead of stacking a duplicate") { test("startBuild returns the active build of the same branch and commit instead of stacking a duplicate") {
@@ -272,8 +280,8 @@ class BuildExecutorTest : FunSpec() {
duplicate.artifactKey shouldBe first.artifactKey duplicate.artifactKey shouldBe first.artifactKey
h.repository.history().map { it.artifactKey } shouldContainExactly listOf(first.artifactKey) h.repository.history().map { it.artifactKey } shouldContainExactly listOf(first.artifactKey)
// a build of the same commit with a command override runs a different command — not a duplicate // another build definition of the same commit is its own pool — not a duplicate
val nightly = h.executor.startBuild("main", "abc123", h.workingDir, "echo full-check") val nightly = h.executor.startBuild("main", "abc123", h.workingDir, "pitest")
nightly.artifactKey shouldNotBe first.artifactKey nightly.artifactKey shouldNotBe first.artifactKey
// another commit of the branch is a distinct build, queued behind the first // another commit of the branch is a distinct build, queued behind the first
@@ -52,15 +52,15 @@ class RetryCommandTest : FunSpec() {
) )
every { gitService.originHeadCommit("main", dir) } returns "head-main" every { gitService.originHeadCommit("main", dir) } returns "head-main"
every { gitService.originHeadCommit("feature/y", dir) } returns "head-y" every { gitService.originHeadCommit("feature/y", dir) } returns "head-y"
every { consoleBuildRunner.buildAndStream(any(), any(), dir, anyNullable(), any()) } returns BuildStatus.SUCCESS every { consoleBuildRunner.buildAndStream(any(), any(), dir, any()) } returns BuildStatus.SUCCESS
var exitCode = -1 var exitCode = -1
captureConsole { exitCode = command().call() } captureConsole { exitCode = command().call() }
exitCode shouldBe 0 exitCode shouldBe 0
verify { consoleBuildRunner.buildAndStream("main", "head-main", dir, null, "main") } verify { consoleBuildRunner.buildAndStream("main", "head-main", dir, "default") }
verify { consoleBuildRunner.buildAndStream("feature/y", "head-y", dir, null, "feature/y") } verify { consoleBuildRunner.buildAndStream("feature/y", "head-y", dir, "default") }
verify(exactly = 0) { consoleBuildRunner.buildAndStream("feature/ok", any(), dir, anyNullable(), any()) } verify(exactly = 0) { consoleBuildRunner.buildAndStream("feature/ok", any(), dir, any()) }
} }
test("exits with code 1 when a retried build fails again") { test("exits with code 1 when a retried build fails again") {
@@ -47,37 +47,73 @@ class ConfigLoaderTest : FunSpec() {
loader.load(dir).builds.maxConcurrent shouldBe 3 loader.load(dir).builds.maxConcurrent shouldBe 3
} }
test("autoBuild.times accepts plain HH:MM entries and slot objects with their own build command, mixed") { test("the builds section splits into the reserved maxConcurrent and named build definitions") {
val dir = Files.createTempDirectory("gittally-test") val dir = Files.createTempDirectory("gittally-test")
dir.resolve(".gittally.yml").toFile().writeText( dir.resolve(".gittally.yml").toFile().writeText(
""" """
branches: builds:
main: maxConcurrent: 2
autoBuild: pitest:
enabled: true atTimes: ["01:00"]
times: branches: ["master", "release/*"]
- "01:00" activeWithin: 24h
- time: "04:00" buildCommand: ./gradlew piTestFull
buildCommand: ./gradlew fullCheck
name: main@nightly
""".trimIndent(), """.trimIndent(),
) )
val times = val config = loader.load(dir)
loader
.load(dir)
.branches
.getValue("main")
.autoBuild.times
times shouldBe config.builds.maxConcurrent shouldBe 2
listOf( config.buildDefinitions shouldBe
AutoBuildSlot("01:00"), mapOf(
AutoBuildSlot("04:00", "./gradlew fullCheck", "main@nightly"), "pitest" to
BuildDefinition(
atTimes = listOf("01:00"),
branches = listOf("master", "release/*"),
activeWithin = "24h",
buildCommand = "./gradlew piTestFull",
),
) )
// config:print round-trip: a slot without its own command serializes back to the plain string // the implicit default build (onPush over all branches) stays in place
loader.toYaml(times) shouldBe config.effectiveBuildDefinitions()["default"] shouldBe BuildDefinition(onPush = true)
"- \"01:00\"\n- time: \"04:00\"\n buildCommand: \"./gradlew fullCheck\"\n name: \"main@nightly\"\n" }
test("an explicit builds.default entry overrides the implicit default build") {
val dir = Files.createTempDirectory("gittally-test")
dir.resolve(".gittally.yml").toFile().writeText(
"""
builds:
default:
onPush: false
""".trimIndent(),
)
loader.load(dir).effectiveBuildDefinitions()["default"] shouldBe BuildDefinition(onPush = false)
}
test("a build worktree cannot redefine the builds section") {
val dir = Files.createTempDirectory("gittally-test")
dir.resolve(".gittally.yml").toFile().writeText(
"""
builds:
pitest:
buildCommand: ./gradlew piTestFull
""".trimIndent(),
)
val worktree = Files.createTempDirectory("gittally-test-worktree")
worktree.resolve(".gittally.yml").toFile().writeText(
"""
builds:
maxConcurrent: 99
pitest:
buildCommand: curl attacker | sh
""".trimIndent(),
)
val config = loader.loadForWorktree(dir, worktree)
config.builds.maxConcurrent shouldBe 1
config.buildDefinitions.getValue("pitest").buildCommand shouldBe "./gradlew piTestFull"
} }
test("repo install config overrides project config for same keys") { test("repo install config overrides project config for same keys") {
@@ -167,50 +167,23 @@ class BuildsApiControllerTest : FunSpec() {
verify { buildExecutor.startBuild("feature/topic", successResult.commit) } verify { buildExecutor.startBuild("feature/topic", successResult.commit) }
} }
test("restart of an auto-slot build repeats its recorded build command") { test("restart of a named build re-runs its build definition on its real branch") {
val liveLogFile = tempDir.resolve("auto-restart.log")
every { repository.latestFor("main") } returns successResult.copy(buildCommandOverride = "./gradlew fullCheck")
every { buildExecutor.startBuild("main", successResult.commit, buildCommandOverride = "./gradlew fullCheck") } returns
runningBuild(liveLogFile).copy(buildCommandOverride = "./gradlew fullCheck")
mockMvc
.perform(post("/api/builds/restart").param("branch", "main").header(BuildsApiController.TOKEN_HEADER, "secret"))
.andExpect(status().isAccepted)
.andExpect(jsonPath("$.status").value("pending"))
// so a restarted nightly build repeats its slot's command, not the regular one
verify { buildExecutor.startBuild("main", successResult.commit, buildCommandOverride = "./gradlew fullCheck") }
}
test("restart of a named slot build re-runs under its name on its real branch") {
val liveLogFile = tempDir.resolve("named-restart.log") val liveLogFile = tempDir.resolve("named-restart.log")
every { repository.latestFor("main@nightly") } returns every { repository.latestFor("main@pitest") } returns
successResult.copy(name = "main@nightly", buildCommandOverride = "./gradlew fullCheck") successResult.copy(build = "pitest", name = "main@pitest")
every { every { buildExecutor.startBuild("main", successResult.commit, build = "pitest") } returns
buildExecutor.startBuild( runningBuild(liveLogFile).copy(build = "pitest", name = "main@pitest")
"main",
successResult.commit,
buildCommandOverride = "./gradlew fullCheck",
name = "main@nightly",
)
} returns runningBuild(liveLogFile).copy(name = "main@nightly")
mockMvc mockMvc
.perform( .perform(
post("/api/builds/restart") post("/api/builds/restart")
.param("branch", "main@nightly") .param("branch", "main@pitest")
.header(BuildsApiController.TOKEN_HEADER, "secret"), .header(BuildsApiController.TOKEN_HEADER, "secret"),
).andExpect(status().isAccepted) ).andExpect(status().isAccepted)
.andExpect(jsonPath("$.name").value("main@nightly")) .andExpect(jsonPath("$.name").value("main@pitest"))
verify { // the re-run resolves its settings from the current config by the build name
buildExecutor.startBuild( verify { buildExecutor.startBuild("main", successResult.commit, build = "pitest") }
"main",
successResult.commit,
buildCommandOverride = "./gradlew fullCheck",
name = "main@nightly",
)
}
} }
test("restart of a never-built branch enqueues its origin head commit") { test("restart of a never-built branch enqueues its origin head commit") {
@@ -1,6 +1,5 @@
package de.hoennig.gittally.watcher package de.hoennig.gittally.watcher
import de.hoennig.gittally.config.AutoBuildSlot
import io.kotest.core.spec.style.FunSpec import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.booleans.shouldBeFalse import io.kotest.matchers.booleans.shouldBeFalse
import io.kotest.matchers.booleans.shouldBeTrue import io.kotest.matchers.booleans.shouldBeTrue
@@ -14,27 +13,19 @@ import java.time.LocalTime
class AutoBuildStateTest : FunSpec() { class AutoBuildStateTest : FunSpec() {
private fun stateFile(): Path = Files.createTempDirectory("gittally-autobuild-test").resolve("auto-builds.json") private fun stateFile(): Path = Files.createTempDirectory("gittally-autobuild-test").resolve("auto-builds.json")
private fun slots(vararg times: String) = times.map { AutoBuildSlot(it) }
init { init {
test("latestDueSlot picks the latest slot at or before now") { test("latestDueSlot picks the latest slot at or before now") {
val times = slots("01:00", "11:00", "13:00") val times = listOf("01:00", "11:00", "13:00")
AutoBuildSlots.latestDueSlot(times, LocalTime.parse("00:59")).shouldBeNull() AutoBuildSlots.latestDueSlot(times, LocalTime.parse("00:59")).shouldBeNull()
AutoBuildSlots.latestDueSlot(times, LocalTime.parse("01:00"))?.time shouldBe "01:00" AutoBuildSlots.latestDueSlot(times, LocalTime.parse("01:00")) shouldBe "01:00"
AutoBuildSlots.latestDueSlot(times, LocalTime.parse("12:00"))?.time shouldBe "11:00" AutoBuildSlots.latestDueSlot(times, LocalTime.parse("12:00")) shouldBe "11:00"
AutoBuildSlots.latestDueSlot(times, LocalTime.parse("23:59"))?.time shouldBe "13:00" AutoBuildSlots.latestDueSlot(times, LocalTime.parse("23:59")) shouldBe "13:00"
}
test("latestDueSlot answers the whole slot including its build command") {
val slot = AutoBuildSlot(time = "01:00", buildCommand = "./gradlew fullCheck")
AutoBuildSlots.latestDueSlot(listOf(slot), LocalTime.parse("02:00")) shouldBe slot
} }
test("latestDueSlot skips invalid slots but keeps the valid ones") { test("latestDueSlot skips invalid slots but keeps the valid ones") {
AutoBuildSlots.latestDueSlot(slots("25:99", "nope", "02:00"), LocalTime.parse("12:00"))?.time shouldBe "02:00" AutoBuildSlots.latestDueSlot(listOf("25:99", "nope", "02:00"), LocalTime.parse("12:00")) shouldBe "02:00"
AutoBuildSlots.latestDueSlot(slots("25:99"), LocalTime.parse("12:00")).shouldBeNull() AutoBuildSlots.latestDueSlot(listOf("25:99"), LocalTime.parse("12:00")).shouldBeNull()
} }
test("latestDueSlot of an empty slot list is null") { test("latestDueSlot of an empty slot list is null") {
@@ -10,8 +10,8 @@ import de.hoennig.gittally.build.GitWorktreeWorkspaces
import de.hoennig.gittally.build.RunningBuild import de.hoennig.gittally.build.RunningBuild
import de.hoennig.gittally.config.ArtifactsConfig import de.hoennig.gittally.config.ArtifactsConfig
import de.hoennig.gittally.config.AutoBuildConfig import de.hoennig.gittally.config.AutoBuildConfig
import de.hoennig.gittally.config.AutoBuildSlot
import de.hoennig.gittally.config.BranchConfig import de.hoennig.gittally.config.BranchConfig
import de.hoennig.gittally.config.BuildDefinition
import de.hoennig.gittally.config.ConfigLoader import de.hoennig.gittally.config.ConfigLoader
import de.hoennig.gittally.config.GitTallyConfig import de.hoennig.gittally.config.GitTallyConfig
import de.hoennig.gittally.config.WatcherConfig import de.hoennig.gittally.config.WatcherConfig
@@ -75,11 +75,12 @@ class WatcherTest : FunSpec() {
every { gitService.newOriginBranches(any(), any()) } returns emptyList() every { gitService.newOriginBranches(any(), any()) } returns emptyList()
every { gitService.hasNewCommits(any(), any()) } returns false every { gitService.hasNewCommits(any(), any()) } returns false
every { gitService.originHeadCommit(any(), any()) } returns null every { gitService.originHeadCommit(any(), any()) } returns null
every { gitService.originBranchCommitTimes(any()) } returns emptyMap()
every { gitService.pullRequestHeads(any()) } returns emptySet() every { gitService.pullRequestHeads(any()) } returns emptySet()
every { gitService.worktreePrune(any()) } returns Unit every { gitService.worktreePrune(any()) } returns Unit
every { gitService.fastForwardLocalBranches(any()) } returns emptyList() every { gitService.fastForwardLocalBranches(any()) } returns emptyList()
every { buildExecutor.currentBuilds() } returns emptyList() every { buildExecutor.currentBuilds() } returns emptyList()
every { buildExecutor.startBuild(any(), any(), any(), anyNullable(), any()) } answers { every { buildExecutor.startBuild(any(), any(), any(), any()) } answers {
val branch = firstArg<String>() val branch = firstArg<String>()
val commit = secondArg<String>() val commit = secondArg<String>()
startedBuilds += branch to commit startedBuilds += branch to commit
@@ -93,19 +94,17 @@ class WatcherTest : FunSpec() {
branch: String, branch: String,
status: BuildStatus, status: BuildStatus,
commit: String = "commit-0", commit: String = "commit-0",
buildCommandOverride: String? = null, build: String = BuildDefinition.DEFAULT,
name: String = branch,
): BuildResult { ): BuildResult {
val startedAt = noon.minusSeconds(3600).plusSeconds(seedCounter++) val startedAt = noon.minusSeconds(3600).plusSeconds(seedCounter++)
val result = val result =
BuildResult( BuildResult(
branch = branch, branch = branch,
name = name, build = build,
commit = commit, commit = commit,
status = status, status = status,
startedAt = startedAt, startedAt = startedAt,
buildCommandOverride = buildCommandOverride, artifactKey = ArtifactKeys.buildKey(BuildDefinition.poolName(branch, build), startedAt),
artifactKey = ArtifactKeys.buildKey(branch, startedAt),
) )
repository.append(result) repository.append(result)
return result return result
@@ -141,7 +140,7 @@ class WatcherTest : FunSpec() {
branches = branches =
mapOf( mapOf(
"default" to BranchConfig(), "default" to BranchConfig(),
"main" to BranchConfig(autoBuild = AutoBuildConfig(enabled = true, times = times.map { AutoBuildSlot(it) })), "main" to BranchConfig(autoBuild = AutoBuildConfig(enabled = true, times = times.toList())),
), ),
) )
@@ -354,7 +353,7 @@ class WatcherTest : FunSpec() {
"main" to "main" to
BranchConfig( BranchConfig(
requirePullRequest = true, requirePullRequest = true,
autoBuild = AutoBuildConfig(enabled = true, times = listOf(AutoBuildSlot("11:00"))), autoBuild = AutoBuildConfig(enabled = true, times = listOf("11:00")),
), ),
), ),
), ),
@@ -379,69 +378,100 @@ class WatcherTest : FunSpec() {
harness.watcher.poll(harness.workingDir) harness.watcher.poll(harness.workingDir)
harness.startedBuilds shouldContainExactly listOf("main" to "commit-abc") harness.startedBuilds shouldContainExactly listOf("main" to "commit-abc")
// a plain HH:MM slot runs the branch's regular buildCommand — no override // the deprecated branch schedule rebuilds the branch's own pool with the default build
verify { harness.buildExecutor.startBuild("main", "commit-abc", any(), null) } verify { harness.buildExecutor.startBuild("main", "commit-abc", any(), BuildDefinition.DEFAULT) }
harness.autoBuildState().isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeTrue() harness.autoBuildState().isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeTrue()
} }
test("an auto-build slot with its own build command dictates that command for the build") { test("a scheduled build definition fires for its selected branches under its own pool") {
val harness = val harness =
Harness( Harness(
GitTallyConfig( GitTallyConfig(
branches = buildDefinitions =
mapOf( mapOf(
"default" to BranchConfig(), "pitest" to
"main" to BuildDefinition(
BranchConfig( atTimes = listOf("11:00"),
autoBuild = branches = listOf("main", "release/*"),
AutoBuildConfig(
enabled = true,
times = listOf(AutoBuildSlot("11:00", "./gradlew fullCheck")),
),
), ),
), ),
), ),
) )
harness.seed("main", BuildStatus.SUCCESS, commit = "commit-abc") // the branch's own pool is busy; the pitest pool is its own and not blocked by it
every { harness.gitService.originBranches(any()) } returns listOf("main")
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-abc"
harness.watcher.poll(harness.workingDir)
verify { harness.buildExecutor.startBuild("main", "commit-abc", any(), "./gradlew fullCheck") }
harness.autoBuildState().isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeTrue()
}
test("a named auto-build slot records under its name, even while the branch's regular build is running") {
val harness =
Harness(
GitTallyConfig(
branches =
mapOf(
"default" to BranchConfig(),
"main" to
BranchConfig(
autoBuild =
AutoBuildConfig(
enabled = true,
times = listOf(AutoBuildSlot("11:00", "./gradlew fullCheck", "main@nightly")),
),
),
),
),
)
// the branch's own pool is busy; the named slot has its own pool and is not blocked by it
harness.seed("main", BuildStatus.RUNNING, commit = "commit-abc") harness.seed("main", BuildStatus.RUNNING, commit = "commit-abc")
every { harness.gitService.originBranches(any()) } returns listOf("main") every { harness.gitService.originBranches(any()) } returns listOf("main", "release/1.x", "feature/x")
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-abc" every { harness.gitService.originHeadCommit("main", any()) } returns "commit-abc"
every { harness.gitService.originHeadCommit("release/1.x", any()) } returns "commit-rel"
harness.watcher.poll(harness.workingDir)
harness.watcher.poll(harness.workingDir)
// glob selector: main and release/1.x fire once, feature/x is not selected
harness.startedBuilds shouldContainExactlyInAnyOrder
listOf("main" to "commit-abc", "release/1.x" to "commit-rel")
verify { harness.buildExecutor.startBuild("main", "commit-abc", any(), "pitest") }
harness.autoBuildState().isTriggered("main@pitest", LocalDate.parse("2026-07-07"), "11:00").shouldBeTrue()
}
test("a scheduled build definition with activeWithin skips branches without recent commits") {
val harness =
Harness(
GitTallyConfig(
buildDefinitions =
mapOf("pitest" to BuildDefinition(atTimes = listOf("11:00"), activeWithin = "24h")),
),
)
every { harness.gitService.originBranches(any()) } returns listOf("active", "dormant")
every { harness.gitService.originHeadCommit("active", any()) } returns "commit-act"
every { harness.gitService.originBranchCommitTimes(any()) } returns
mapOf(
"active" to noon.minusSeconds(3600),
"dormant" to noon.minus(Duration.ofDays(10)),
)
harness.watcher.poll(harness.workingDir) harness.watcher.poll(harness.workingDir)
verify { harness.buildExecutor.startBuild("main", "commit-abc", any(), "./gradlew fullCheck", "main@nightly") } harness.startedBuilds shouldContainExactly listOf("active" to "commit-act")
harness.autoBuildState().isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeTrue() verify { harness.buildExecutor.startBuild("active", "commit-act", any(), "pitest") }
} }
test("a commit-triggered build never carries a build command override") { test("an onPush build definition builds the changed branches it selects") {
val harness =
Harness(
GitTallyConfig(
buildDefinitions =
mapOf("lint" to BuildDefinition(onPush = true, branches = listOf("main"))),
),
)
every { harness.gitService.originBranches(any()) } returns listOf("main", "feature/x")
every { harness.gitService.localBranches(any()) } returns listOf("main", "feature/x")
every { harness.gitService.hasNewCommits(any(), any()) } returns true
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
every { harness.gitService.originHeadCommit("feature/x", any()) } returns "commit-feat"
harness.watcher.poll(harness.workingDir)
// the implicit default build covers both branches; lint only selects main
verify { harness.buildExecutor.startBuild("main", "commit-main", any(), BuildDefinition.DEFAULT) }
verify { harness.buildExecutor.startBuild("feature/x", "commit-feat", any(), BuildDefinition.DEFAULT) }
verify { harness.buildExecutor.startBuild("main", "commit-main", any(), "lint") }
verify(exactly = 0) { harness.buildExecutor.startBuild("feature/x", "commit-feat", any(), "lint") }
}
test("builds.default with onPush false disables the implicit on-push build") {
val harness =
Harness(GitTallyConfig(buildDefinitions = mapOf("default" to BuildDefinition(onPush = false))))
every { harness.gitService.originBranches(any()) } returns listOf("main")
every { harness.gitService.localBranches(any()) } returns listOf("main")
every { harness.gitService.hasNewCommits("main", any()) } returns true
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main"
harness.watcher.poll(harness.workingDir)
harness.startedBuilds.shouldBeEmpty()
}
test("a commit-triggered build belongs to the default build definition") {
val harness = Harness() val harness = Harness()
every { harness.gitService.originBranches(any()) } returns listOf("main") every { harness.gitService.originBranches(any()) } returns listOf("main")
every { harness.gitService.localBranches(any()) } returns listOf("main") every { harness.gitService.localBranches(any()) } returns listOf("main")
@@ -450,7 +480,7 @@ class WatcherTest : FunSpec() {
harness.watcher.poll(harness.workingDir) harness.watcher.poll(harness.workingDir)
verify { harness.buildExecutor.startBuild("main", "commit-main", any(), null) } verify { harness.buildExecutor.startBuild("main", "commit-main", any(), BuildDefinition.DEFAULT) }
} }
test("an auto-build slot stays untriggered while the branch is still building") { test("an auto-build slot stays untriggered while the branch is still building") {
@@ -496,21 +526,15 @@ class WatcherTest : FunSpec() {
harness.startedBuilds shouldContainExactly listOf("main" to "commit-2") harness.startedBuilds shouldContainExactly listOf("main" to "commit-2")
} }
test("startup recovery re-enqueues an interrupted auto-slot build with its recorded command and name") { test("startup recovery re-enqueues an interrupted build under its recorded build definition") {
val harness = Harness() val harness = Harness()
harness.seed( harness.seed("main", BuildStatus.INTERRUPTED, commit = "commit-1", build = "pitest")
"main",
BuildStatus.INTERRUPTED,
commit = "commit-1",
buildCommandOverride = "./gradlew fullCheck",
name = "main@nightly",
)
every { harness.gitService.originHeadCommit("main", any()) } returns "commit-1" every { harness.gitService.originHeadCommit("main", any()) } returns "commit-1"
harness.watcher.recoverOnStartup(harness.workingDir) harness.watcher.recoverOnStartup(harness.workingDir)
// otherwise a restart mid-nightly-build would repeat it with the regular command, in the wrong pool // otherwise a restart mid-nightly-build would repeat it as a regular build in the wrong pool
verify { harness.buildExecutor.startBuild("main", "commit-1", any(), "./gradlew fullCheck", "main@nightly") } verify { harness.buildExecutor.startBuild("main", "commit-1", any(), "pitest") }
} }
test("startup recovery closes out an orphaned PENDING build of a branch gone from origin") { test("startup recovery closes out an orphaned PENDING build of a branch gone from origin") {