From a1db450bdbb4a6976e1af025789fa26c9e4a3b41 Mon Sep 17 00:00:00 2001 From: Michael Hoennig Date: Tue, 7 Jul 2026 10:28:28 +0200 Subject: [PATCH] implemented 06-watcher.md: non-blocking poll cycle enqueueing changed/new/auto-build branches via the async executor, startup recovery, retention/worktree pruning, JSON auto-build slot state, watcher.pollInterval config, and watcher health state Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 6 +- docs/configuration.md | 2 + docs/plan/06-watcher.md | 38 ++ docs/plan/README.md | 2 +- .../hoennig/gittally/commands/InitCommand.kt | 2 + .../hoennig/gittally/config/DurationParser.kt | 15 +- .../hoennig/gittally/config/GitTallyConfig.kt | 2 + .../de/hoennig/gittally/git/GitService.kt | 9 + .../gittally/watcher/AutoBuildState.kt | 98 +++++ .../de/hoennig/gittally/watcher/Watcher.kt | 270 ++++++++++++++ .../gittally/watcher/WatcherConfiguration.kt | 12 + .../hoennig/gittally/watcher/WatcherState.kt | 17 + .../gittally/config/DurationParserTest.kt | 8 + .../de/hoennig/gittally/git/GitServiceTest.kt | 7 + .../gittally/watcher/AutoBuildStateTest.kt | 80 ++++ .../hoennig/gittally/watcher/WatcherTest.kt | 343 ++++++++++++++++++ 16 files changed, 905 insertions(+), 6 deletions(-) create mode 100644 src/main/kotlin/de/hoennig/gittally/watcher/AutoBuildState.kt create mode 100644 src/main/kotlin/de/hoennig/gittally/watcher/Watcher.kt create mode 100644 src/main/kotlin/de/hoennig/gittally/watcher/WatcherConfiguration.kt create mode 100644 src/main/kotlin/de/hoennig/gittally/watcher/WatcherState.kt create mode 100644 src/test/kotlin/de/hoennig/gittally/watcher/AutoBuildStateTest.kt create mode 100644 src/test/kotlin/de/hoennig/gittally/watcher/WatcherTest.kt diff --git a/CLAUDE.md b/CLAUDE.md index ac34e20..c9d4a33 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -62,9 +62,13 @@ Three places must stay in sync when config keys change: the `GitTallyConfig` dat `BuildExecutor` runs builds asynchronously: up to `builds.maxConcurrent` branches concurrently (default 1), but never more than one build per branch at a time. Each branch builds in its own reusable git worktree at `.git/gittally/worktrees/` (`BranchWorkspaces`), checked out detached at the requested commit — the primary checkout is never used for builds. Status transitions are persisted via `BuildResultRepository` (JSON file under `.git/gittally/`), published to Gitea non-fatally, and emitted as `BuildStatusChangedEvent`s. Cancellation addresses a build by artifact key and terminates the whole process tree. Future code (watcher, server, UI) must not assume a single running build. +### Watcher + +`Watcher` replaces the legacy blocking main loop with a non-blocking fixed-delay poll cycle: fetch origin, enqueue due branches (changed local, recent new origin, due auto-build slots) via `BuildExecutor`, then prune results, artifacts, and stale worktrees. Nothing is scheduled until `Watcher.start()` is called explicitly (server/watch mode) — CLI commands and tests never start the loop. "Already built" is tracked via the result repository, not by moving local branch refs. Auto-build slot state lives in `.git/gittally/auto-builds.json`; watcher health is exposed via `Watcher.state()`. + ### Package Structure -All production code lives under `de.hoennig.gittally`, with sub-packages `commands` (picocli subcommands), `config` (YAML config loading and schema), `git` (git CLI access), `gitea` (Gitea commit-status API client), and `build` (build execution, results, workspaces). Tests mirror this structure under `src/test/kotlin`. +All production code lives under `de.hoennig.gittally`, with sub-packages `commands` (picocli subcommands), `config` (YAML config loading and schema), `git` (git CLI access), `gitea` (Gitea commit-status API client), `build` (build execution, results, workspaces), `artifacts` (filesystem artifact store), and `watcher` (branch polling, auto-builds, startup recovery). Tests mirror this structure under `src/test/kotlin`. ## Testing Conventions diff --git a/docs/configuration.md b/docs/configuration.md index 4f14403..3010acf 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -54,6 +54,8 @@ artifacts: # Controls the branch-polling loop. watcher: + # delay between poll cycles; suffixes s (seconds), m (minutes), h (hours), d (days) + pollInterval: 10s # max commit age for new origin branches to be pulled automatically newBranchMaxAge: 5d diff --git a/docs/plan/06-watcher.md b/docs/plan/06-watcher.md index 6e64063..28e5f6c 100644 --- a/docs/plan/06-watcher.md +++ b/docs/plan/06-watcher.md @@ -48,3 +48,41 @@ Update `GitTallyConfig`, `InitCommand` templates, and `docs/configuration.md` to - `./gradlew ktlintFormat` then `./gradlew build` is green. - A test proves a poll cycle completes while a (fake) build is running. + +## Execution Notes (done 2026-07-07) + +Implemented as designed in `de.hoennig.gittally.watcher`; build green, 23 new tests +(`WatcherTest`, `AutoBuildStateTest`, plus new `DurationParserTest` and `GitServiceTest` cases). +Deviations and decisions: + +- The poll loop is a managed single-thread `ScheduledExecutorService` with fixed delay, not Spring `@Scheduled`: + scheduling annotations would run during CLI commands and tests. + Nothing starts automatically; server/watch mode (step 07) must call `Watcher.start(workingDir)` explicitly, + which runs the startup recovery and then polls immediately and on every `watcher.pollInterval`. + `ApplicationContextTest` stays green and fast because constructing the beans schedules nothing. +- "Changed" is detected against the result repository (`latestFor(branch).commit != origin head`), + gated by `hasNewCommits`, instead of resetting local refs like legacy: + builds run detached in per-branch worktrees and never move local branch refs, + so a legacy-style local-ref comparison would re-enqueue the same commit forever. + Consequence: a failed build of a commit is not retried automatically (same as legacy) — step 10 adds the retry command. +- Queue-behind vs. cancel-on-new-commit: DEFERRED, keeping queue-behind semantics at watcher granularity. + A branch whose latest build is PENDING or RUNNING is skipped; the new commit is picked up on a later cycle + once the build finished. Making this per-branch configurable (cancel and rebuild instead) remains open, + see also the step 04 amendment. +- Enqueue precedence per cycle: changed local branches, then recent new origin branches, then due auto-build slots; + each branch at most once (an auto-build slot stays untriggered while its branch is pending/running and fires + on a later cycle instead of being lost). +- Auto-build state lives in `.git/gittally/auto-builds.json` (`FileAutoBuildState`, replaces `auto-builds.tsv`); + entries of past days are dropped on write. Slot matching (`AutoBuildSlots`) picks the latest slot at or before + the current UTC time, like legacy `auto_build_check`. Only branches named in the `branches` config + (other than `default`) can auto-build; `default.autoBuild.enabled` does not extend to unlisted branches. +- Startup recovery re-enqueues branches whose latest result is PENDING or INTERRUPTED and which still exist + on origin, after a best-effort fetch (a failing fetch recovers from the last known origin state). + A stale latest PENDING entry is marked INTERRUPTED before its replacement build is enqueued, + because the executor queue does not survive a restart. +- Worktree cleanup deletes `.git/gittally/worktrees/` directories of branches gone from origin + (never those of queued or running builds) and then calls `git worktree prune`. +- `DurationParser` was extended with `s`/`m` suffixes for `watcher.pollInterval` (it only knew `d`/`h`). +- Watcher health is exposed via `Watcher.state(): WatcherState` (running, last poll time, last fetch/poll error, + queued branches) for the step 07 status endpoints; no events are published for it. +- A UTC `Clock` bean (`WatcherConfiguration`) makes slot matching and poll timestamps testable. diff --git a/docs/plan/README.md b/docs/plan/README.md index de17d24..c661422 100644 --- a/docs/plan/README.md +++ b/docs/plan/README.md @@ -50,7 +50,7 @@ Core engine: - [x] `04-build-executor.md` — async build execution with logs, cancellation, status transitions - [x] `05-artifact-store.md` — artifact persistence, naming, retention -- [ ] `06-watcher.md` — branch watching, scheduling, auto-builds +- [x] `06-watcher.md` — branch watching, scheduling, auto-builds Server and UI: diff --git a/src/main/kotlin/de/hoennig/gittally/commands/InitCommand.kt b/src/main/kotlin/de/hoennig/gittally/commands/InitCommand.kt index 4ba2c43..225d1bd 100644 --- a/src/main/kotlin/de/hoennig/gittally/commands/InitCommand.kt +++ b/src/main/kotlin/de/hoennig/gittally/commands/InitCommand.kt @@ -126,6 +126,8 @@ class InitCommand( # Controls the branch-polling loop. watcher: + # delay between poll cycles (s/m/h/d suffix) + pollInterval: 10s # max commit age for new origin branches to be pulled automatically newBranchMaxAge: 5d diff --git a/src/main/kotlin/de/hoennig/gittally/config/DurationParser.kt b/src/main/kotlin/de/hoennig/gittally/config/DurationParser.kt index d1f565d..fa8c461 100644 --- a/src/main/kotlin/de/hoennig/gittally/config/DurationParser.kt +++ b/src/main/kotlin/de/hoennig/gittally/config/DurationParser.kt @@ -2,18 +2,25 @@ package de.hoennig.gittally.config import java.time.Duration -/** Parses durations in the `watcher.newBranchMaxAge` format: `5d` (days) or `12h` (hours). */ +/** + * Parses durations in the format used by `watcher.newBranchMaxAge` and `watcher.pollInterval`: + * `5d` (days), `12h` (hours), `10m` (minutes), or `30s` (seconds). + */ object DurationParser { - private val pattern = Regex("""(\d+)([dh])""") + private val pattern = Regex("""(\d+)([dhms])""") fun parse(value: String): Duration { val match = pattern.matchEntire(value.trim()) - ?: throw IllegalArgumentException("invalid duration '$value': expected d or h, e.g. 5d or 12h") + ?: throw IllegalArgumentException( + "invalid duration '$value': expected d, h, m, or s, e.g. 5d or 10s", + ) val (amount, unit) = match.destructured return when (unit) { "d" -> Duration.ofDays(amount.toLong()) - else -> Duration.ofHours(amount.toLong()) + "h" -> Duration.ofHours(amount.toLong()) + "m" -> Duration.ofMinutes(amount.toLong()) + else -> Duration.ofSeconds(amount.toLong()) } } } diff --git a/src/main/kotlin/de/hoennig/gittally/config/GitTallyConfig.kt b/src/main/kotlin/de/hoennig/gittally/config/GitTallyConfig.kt index b02ba47..0c54b9e 100644 --- a/src/main/kotlin/de/hoennig/gittally/config/GitTallyConfig.kt +++ b/src/main/kotlin/de/hoennig/gittally/config/GitTallyConfig.kt @@ -41,6 +41,8 @@ data class ArtifactsConfig( ) data class WatcherConfig( + /** Delay between poll cycles, e.g. `10s` or `1m`. */ + val pollInterval: String = "10s", val newBranchMaxAge: String = "5d", ) diff --git a/src/main/kotlin/de/hoennig/gittally/git/GitService.kt b/src/main/kotlin/de/hoennig/gittally/git/GitService.kt index 75cde75..f4d1bf6 100644 --- a/src/main/kotlin/de/hoennig/gittally/git/GitService.kt +++ b/src/main/kotlin/de/hoennig/gittally/git/GitService.kt @@ -151,6 +151,15 @@ class GitService( .trim() .ifEmpty { null } + /** The commit `refs/remotes/origin/[branch]` points at, or null when the branch is not on origin. */ + fun originHeadCommit( + branch: String, + workingDir: Path = Paths.get("."), + ): String? { + val result = runner.run(listOf("git", "rev-parse", "--verify", "refs/remotes/origin/$branch"), workingDir) + return if (result.isSuccess) result.stdout.trim() else null + } + fun headCommit(workingDir: Path = Paths.get(".")): String = runner .runOrThrow(listOf("git", "rev-parse", "HEAD"), workingDir) diff --git a/src/main/kotlin/de/hoennig/gittally/watcher/AutoBuildState.kt b/src/main/kotlin/de/hoennig/gittally/watcher/AutoBuildState.kt new file mode 100644 index 0000000..7308667 --- /dev/null +++ b/src/main/kotlin/de/hoennig/gittally/watcher/AutoBuildState.kt @@ -0,0 +1,98 @@ +package de.hoennig.gittally.watcher + +import com.fasterxml.jackson.databind.DeserializationFeature +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.databind.SerializationFeature +import com.fasterxml.jackson.module.kotlin.readValue +import com.fasterxml.jackson.module.kotlin.registerKotlinModule +import org.slf4j.LoggerFactory +import java.nio.file.Files +import java.nio.file.Path +import java.nio.file.StandardCopyOption +import java.time.LocalDate +import java.time.LocalTime +import java.time.format.DateTimeParseException + +/** One recorded auto-build trigger: [branch] was enqueued for the [slot] (UTC `HH:MM`) of [date] (ISO). */ +data class AutoBuildTrigger( + val branch: String, + val date: String, + val slot: String, +) + +/** Auto-build time slot matching (UTC `HH:MM`), like legacy `auto_build_check`. */ +object AutoBuildSlots { + private val log = LoggerFactory.getLogger(AutoBuildSlots::class.java) + + /** The latest valid slot at or before [now], or null when no slot is due yet today. */ + fun latestDueSlot( + times: List, + now: LocalTime, + ): String? = + times + .mapNotNull { slot -> + try { + LocalTime.parse(slot.trim()) to slot + } catch (_: DateTimeParseException) { + log.warn("skipping invalid auto-build time slot '{}': expected HH:MM", slot) + null + } + }.filter { (parsed, _) -> !parsed.isAfter(now) } + .maxByOrNull { (parsed, _) -> parsed } + ?.second +} + +/** + * Persists which auto-build slots already triggered as a JSON file, + * e.g. `.git/gittally/auto-builds.json` (replaces the legacy `auto-builds.tsv`). + * Entries of past days are dropped on write, so the file never grows unbounded. + */ +class FileAutoBuildState( + private val file: Path, +) { + private val log = LoggerFactory.getLogger(FileAutoBuildState::class.java) + + private val json = + ObjectMapper() + .registerKotlinModule() + .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) + .configure(SerializationFeature.INDENT_OUTPUT, true) + + fun isTriggered( + branch: String, + date: LocalDate, + slot: String, + ): Boolean = AutoBuildTrigger(branch, date.toString(), slot) in load() + + fun markTriggered( + branch: String, + date: LocalDate, + slot: String, + ) { + val current = load().filter { it.date == date.toString() } + save(current + AutoBuildTrigger(branch, date.toString(), slot)) + } + + private fun load(): List { + if (!Files.exists(file)) { + return emptyList() + } + return try { + json.readValue>(file.toFile()) + } catch (e: Exception) { + log.warn("ignoring unreadable auto-builds file {}: {}", file, e.message) + emptyList() + } + } + + private fun save(triggers: List) { + Files.createDirectories(file.parent) + val tempFile = Files.createTempFile(file.parent, file.fileName.toString(), ".tmp") + try { + json.writeValue(tempFile.toFile(), triggers) + Files.move(tempFile, file, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING) + } finally { + Files.deleteIfExists(tempFile) + } + } +} diff --git a/src/main/kotlin/de/hoennig/gittally/watcher/Watcher.kt b/src/main/kotlin/de/hoennig/gittally/watcher/Watcher.kt new file mode 100644 index 0000000..a5cb358 --- /dev/null +++ b/src/main/kotlin/de/hoennig/gittally/watcher/Watcher.kt @@ -0,0 +1,270 @@ +package de.hoennig.gittally.watcher + +import de.hoennig.gittally.build.ArtifactKeys +import de.hoennig.gittally.build.ArtifactStore +import de.hoennig.gittally.build.BuildExecutor +import de.hoennig.gittally.build.BuildResultRepository +import de.hoennig.gittally.build.BuildStatus +import de.hoennig.gittally.build.GitWorktreeWorkspaces +import de.hoennig.gittally.config.ConfigLoader +import de.hoennig.gittally.config.DurationParser +import de.hoennig.gittally.config.GitTallyConfig +import de.hoennig.gittally.git.GitService +import org.slf4j.LoggerFactory +import org.springframework.stereotype.Service +import java.nio.file.Files +import java.nio.file.Path +import java.nio.file.Paths +import java.time.Clock +import java.time.LocalDate +import java.time.LocalTime +import java.time.ZoneOffset +import java.util.concurrent.Executors +import java.util.concurrent.ScheduledExecutorService +import java.util.concurrent.TimeUnit + +/** + * Replaces the legacy blocking main loop: a non-blocking fixed-delay poll cycle that + * fetches origin, enqueues due branches via the async [BuildExecutor], and prunes + * retention — it never waits for a build and never touches the primary checkout. + * The loop only runs after an explicit [start] (server/watch mode, step 07); + * nothing is scheduled during CLI commands or tests. + */ +@Service +class Watcher( + private val gitService: GitService, + private val buildExecutor: BuildExecutor, + private val repository: BuildResultRepository, + private val artifactStore: ArtifactStore, + private val configLoader: ConfigLoader, + private val clock: Clock, +) { + private val log = LoggerFactory.getLogger(Watcher::class.java) + + private var scheduler: ScheduledExecutorService? = null + + @Volatile + private var state = WatcherState() + + fun state(): WatcherState = state + + /** + * Runs the startup recovery and schedules the poll loop with the fixed delay + * `watcher.pollInterval`; the first poll runs immediately. + */ + @Synchronized + fun start(workingDir: Path = Paths.get(".")) { + check(scheduler == null) { "watcher is already running" } + recoverOnStartup(workingDir) + val interval = DurationParser.parse(configLoader.load(workingDir).watcher.pollInterval) + scheduler = + Executors + .newSingleThreadScheduledExecutor { runnable -> + Thread(runnable, "gittally-watcher").apply { isDaemon = true } + }.also { + it.scheduleWithFixedDelay({ pollSafely(workingDir) }, 0, interval.toMillis(), TimeUnit.MILLISECONDS) + } + state = state.copy(running = true) + } + + @Synchronized + fun stop() { + scheduler?.shutdownNow() + scheduler = null + state = state.copy(running = false) + } + + /** + * Port of the legacy startup recovery: best-effort fetch, mark stale RUNNING and + * superseded PENDING builds as INTERRUPTED, then re-enqueue every branch whose + * latest build never finished and which still exists on origin. + */ + fun recoverOnStartup(workingDir: Path = Paths.get(".")) { + try { + gitService.fetchOrigin(workingDir) + } catch (e: Exception) { + log.warn("startup fetch failed; recovering from the last known origin state: {}", e.message) + } + repository.markStaleRunningAsInterrupted().forEach { + log.info("marked stale build of branch {} as interrupted", it.branch) + } + val restartable = + repository + .latestPerBranch() + .filter { it.status == BuildStatus.INTERRUPTED || it.status == BuildStatus.PENDING } + for (result in restartable) { + val commit = gitService.originHeadCommit(result.branch, workingDir) + if (commit == null) { + log.info("not restarting build of branch {}: branch is gone from origin", result.branch) + continue + } + if (result.status == BuildStatus.PENDING) { + // the executor queue did not survive the restart; the re-enqueued build supersedes the stale entry + repository.updateByArtifactKey(result.artifactKey) { it.copy(status = BuildStatus.INTERRUPTED) } + } + log.info("restarting unfinished build of branch {}", result.branch) + buildExecutor.startBuild(result.branch, commit, workingDir) + } + } + + /** + * One poll cycle, never blocking on a build: fetch origin (on failure: log, expose + * in [state], retry next cycle), enqueue due branches — changed local branches + * first, then recent new origin branches, then due auto-build slots — and finally + * prune results, artifacts, and worktrees of branches gone from origin. + */ + fun poll(workingDir: Path = Paths.get(".")) { + val startedAt = clock.instant() + try { + gitService.fetchOrigin(workingDir) + } catch (e: Exception) { + log.warn("fetching origin failed; retrying next cycle: {}", e.message) + state = state.copy(lastPollAt = startedAt, lastFetchError = e.message ?: e.javaClass.simpleName) + return + } + val config = configLoader.load(workingDir) + val originBranches = gitService.originBranches(workingDir) + enqueueDueBranches(config, originBranches.toSet(), workingDir) + prune(config, originBranches, workingDir) + state = + state.copy( + lastPollAt = startedAt, + lastFetchError = null, + lastPollError = null, + queuedBranches = + repository + .latestPerBranch() + .filter { it.status == BuildStatus.PENDING || it.status == BuildStatus.RUNNING } + .map { it.branch }, + ) + } + + private fun pollSafely(workingDir: Path) { + try { + poll(workingDir) + } catch (e: Exception) { + log.error("poll cycle failed", e) + state = state.copy(lastPollAt = clock.instant(), lastPollError = e.message ?: e.javaClass.simpleName) + } + } + + private fun enqueueDueBranches( + config: GitTallyConfig, + originBranches: Set, + workingDir: Path, + ) { + val changedLocal = + gitService + .localBranches(workingDir) + .filter { it in originBranches && gitService.hasNewCommits(it, workingDir) } + val newOrigin = + gitService.newOriginBranches(DurationParser.parse(config.watcher.newBranchMaxAge), workingDir) + for (branch in (changedLocal + newOrigin).distinct()) { + startBuildIfDue(branch, allowSameCommit = false, workingDir = workingDir) + } + enqueueAutoBuilds(config, originBranches, workingDir) + } + + /** + * Enqueues a build of the branch's origin head unless one is already pending or + * running, or that commit was already built. Builds run detached in worktrees and + * never move local branch refs, so "already built" is tracked via the result + * repository, not by resetting the local ref like legacy. A new commit for a + * branch that is still pending/running waits for a later cycle (queue-behind). + */ + private fun startBuildIfDue( + branch: String, + allowSameCommit: Boolean, + workingDir: Path, + ): Boolean { + val latest = repository.latestFor(branch) + if (latest?.status == BuildStatus.PENDING || latest?.status == BuildStatus.RUNNING) { + return false + } + val commit = gitService.originHeadCommit(branch, workingDir) ?: return false + if (!allowSameCommit && latest?.commit == commit) { + return false + } + log.info("enqueueing build of branch {} at commit {}", branch, commit) + buildExecutor.startBuild(branch, commit, workingDir) + return true + } + + private fun enqueueAutoBuilds( + config: GitTallyConfig, + originBranches: Set, + workingDir: Path, + ) { + val autoBuildBranches = + config.branches.filter { (branch, branchConfig) -> + branch != "default" && branchConfig.autoBuild.enabled + } + if (autoBuildBranches.isEmpty()) { + return + } + val autoBuildState = FileAutoBuildState(workingDir.resolve(AUTO_BUILDS_FILE)) + val now = clock.instant() + val today = LocalDate.ofInstant(now, ZoneOffset.UTC) + val timeOfDay = LocalTime.ofInstant(now, ZoneOffset.UTC) + for ((branch, branchConfig) in autoBuildBranches) { + val slot = AutoBuildSlots.latestDueSlot(branchConfig.autoBuild.times, timeOfDay) ?: continue + if (autoBuildState.isTriggered(branch, today, slot)) { + continue + } + if (branch !in originBranches) { + log.warn("skipping auto build of branch {}: branch is not on origin", branch) + continue + } + // rebuilding the already-built commit is the point of an auto build + if (startBuildIfDue(branch, allowSameCommit = true, workingDir = workingDir)) { + autoBuildState.markTriggered(branch, today, slot) + } + } + } + + /** Results first, then artifacts of dropped results, then worktrees of branches gone from origin. */ + private fun prune( + config: GitTallyConfig, + originBranches: List, + workingDir: Path, + ) { + repository.prune(originBranches, config.artifacts.retentionPerBranch) + artifactStore.prune(repository.history()) + pruneWorktrees(originBranches, workingDir) + } + + private fun pruneWorktrees( + originBranches: List, + workingDir: Path, + ) { + val worktreesDir = workingDir.resolve(GitWorktreeWorkspaces.WORKTREES_DIR) + if (!Files.isDirectory(worktreesDir)) { + return + } + val keep = originBranches.map { ArtifactKeys.branchKey(it) }.toMutableSet() + // never delete under a build that is still queued or executing + buildExecutor.currentBuilds().forEach { keep += ArtifactKeys.branchKey(it.branch) } + repository + .latestPerBranch() + .filter { it.status == BuildStatus.PENDING || it.status == BuildStatus.RUNNING } + .forEach { keep += ArtifactKeys.branchKey(it.branch) } + var removed = false + Files.list(worktreesDir).use { entries -> + entries.forEach { entry -> + if (Files.isDirectory(entry) && entry.fileName.toString() !in keep) { + log.info("removing worktree of branch gone from origin: {}", entry.fileName) + entry.toFile().deleteRecursively() + removed = true + } + } + } + if (removed) { + gitService.worktreePrune(workingDir) + } + } + + companion object { + /** Auto-build trigger state next to the build results (replaces legacy `auto-builds.tsv`). */ + const val AUTO_BUILDS_FILE = ".git/gittally/auto-builds.json" + } +} diff --git a/src/main/kotlin/de/hoennig/gittally/watcher/WatcherConfiguration.kt b/src/main/kotlin/de/hoennig/gittally/watcher/WatcherConfiguration.kt new file mode 100644 index 0000000..b2ad7af --- /dev/null +++ b/src/main/kotlin/de/hoennig/gittally/watcher/WatcherConfiguration.kt @@ -0,0 +1,12 @@ +package de.hoennig.gittally.watcher + +import org.springframework.context.annotation.Bean +import org.springframework.context.annotation.Configuration +import java.time.Clock + +@Configuration +class WatcherConfiguration { + /** UTC clock, injectable in tests; auto-build slots are UTC `HH:MM` like legacy. */ + @Bean + fun clock(): Clock = Clock.systemUTC() +} diff --git a/src/main/kotlin/de/hoennig/gittally/watcher/WatcherState.kt b/src/main/kotlin/de/hoennig/gittally/watcher/WatcherState.kt new file mode 100644 index 0000000..28c7b39 --- /dev/null +++ b/src/main/kotlin/de/hoennig/gittally/watcher/WatcherState.kt @@ -0,0 +1,17 @@ +package de.hoennig.gittally.watcher + +import java.time.Instant + +/** Observable watcher health for status endpoints (step 07) and the UI (step 08). */ +data class WatcherState( + /** Whether the poll loop is scheduled. */ + val running: Boolean = false, + /** When the last poll cycle started, successful or not. */ + val lastPollAt: Instant? = null, + /** Why the last `fetchOrigin` failed; null after a successful fetch. */ + val lastFetchError: String? = null, + /** Why the last poll cycle crashed after a successful fetch; null after a clean cycle. */ + val lastPollError: String? = null, + /** Branches whose latest build was PENDING or RUNNING at the end of the last poll. */ + val queuedBranches: List = emptyList(), +) diff --git a/src/test/kotlin/de/hoennig/gittally/config/DurationParserTest.kt b/src/test/kotlin/de/hoennig/gittally/config/DurationParserTest.kt index 2677f64..3645309 100644 --- a/src/test/kotlin/de/hoennig/gittally/config/DurationParserTest.kt +++ b/src/test/kotlin/de/hoennig/gittally/config/DurationParserTest.kt @@ -16,6 +16,14 @@ class DurationParserTest : FunSpec() { DurationParser.parse("12h") shouldBe Duration.ofHours(12) } + test("parses minutes") { + DurationParser.parse("10m") shouldBe Duration.ofMinutes(10) + } + + test("parses seconds") { + DurationParser.parse("30s") shouldBe Duration.ofSeconds(30) + } + test("parses multi-digit amounts") { DurationParser.parse("120h") shouldBe Duration.ofHours(120) } diff --git a/src/test/kotlin/de/hoennig/gittally/git/GitServiceTest.kt b/src/test/kotlin/de/hoennig/gittally/git/GitServiceTest.kt index 40223fc..ca697ee 100644 --- a/src/test/kotlin/de/hoennig/gittally/git/GitServiceTest.kt +++ b/src/test/kotlin/de/hoennig/gittally/git/GitServiceTest.kt @@ -217,6 +217,13 @@ class GitServiceTest : FunSpec() { service.headCommit(fixture.work) shouldMatch Regex("[0-9a-f]{40}") } + test("originHeadCommit returns the origin branch head, or null for an unknown branch") { + val fixture = Fixture() + + service.originHeadCommit("main", fixture.work) shouldBe service.headCommit(fixture.work) + service.originHeadCommit("no-such-branch", fixture.work).shouldBeNull() + } + test("worktreeAdd creates a detached worktree at the commit") { val fixture = Fixture() val head = service.headCommit(fixture.work) diff --git a/src/test/kotlin/de/hoennig/gittally/watcher/AutoBuildStateTest.kt b/src/test/kotlin/de/hoennig/gittally/watcher/AutoBuildStateTest.kt new file mode 100644 index 0000000..59263a4 --- /dev/null +++ b/src/test/kotlin/de/hoennig/gittally/watcher/AutoBuildStateTest.kt @@ -0,0 +1,80 @@ +package de.hoennig.gittally.watcher + +import io.kotest.core.spec.style.FunSpec +import io.kotest.matchers.booleans.shouldBeFalse +import io.kotest.matchers.booleans.shouldBeTrue +import io.kotest.matchers.nulls.shouldBeNull +import io.kotest.matchers.shouldBe +import java.nio.file.Files +import java.nio.file.Path +import java.time.LocalDate +import java.time.LocalTime + +class AutoBuildStateTest : FunSpec() { + private fun stateFile(): Path = Files.createTempDirectory("gittally-autobuild-test").resolve("auto-builds.json") + + init { + test("latestDueSlot picks the latest slot at or before now") { + val times = listOf("01:00", "11:00", "13:00") + + AutoBuildSlots.latestDueSlot(times, LocalTime.parse("00:59")).shouldBeNull() + AutoBuildSlots.latestDueSlot(times, LocalTime.parse("01:00")) shouldBe "01:00" + AutoBuildSlots.latestDueSlot(times, LocalTime.parse("12:00")) shouldBe "11:00" + AutoBuildSlots.latestDueSlot(times, LocalTime.parse("23:59")) shouldBe "13:00" + } + + test("latestDueSlot skips invalid slots but keeps the valid ones") { + AutoBuildSlots.latestDueSlot(listOf("25:99", "nope", "02:00"), LocalTime.parse("12:00")) shouldBe "02:00" + AutoBuildSlots.latestDueSlot(listOf("25:99"), LocalTime.parse("12:00")).shouldBeNull() + } + + test("latestDueSlot of an empty slot list is null") { + AutoBuildSlots.latestDueSlot(emptyList(), LocalTime.parse("12:00")).shouldBeNull() + } + + test("markTriggered records exactly the branch, day, and slot") { + val state = FileAutoBuildState(stateFile()) + val today = LocalDate.parse("2026-07-07") + + state.isTriggered("main", today, "11:00").shouldBeFalse() + state.markTriggered("main", today, "11:00") + + state.isTriggered("main", today, "11:00").shouldBeTrue() + state.isTriggered("main", today, "13:00").shouldBeFalse() + state.isTriggered("main", today.plusDays(1), "11:00").shouldBeFalse() + state.isTriggered("other", today, "11:00").shouldBeFalse() + } + + test("triggers persist across instances") { + val file = stateFile() + val today = LocalDate.parse("2026-07-07") + FileAutoBuildState(file).markTriggered("main", today, "11:00") + + FileAutoBuildState(file).isTriggered("main", today, "11:00").shouldBeTrue() + } + + test("entries of past days are dropped on write") { + val file = stateFile() + val state = FileAutoBuildState(file) + val yesterday = LocalDate.parse("2026-07-06") + val today = LocalDate.parse("2026-07-07") + state.markTriggered("main", yesterday, "11:00") + + state.markTriggered("main", today, "01:00") + + state.isTriggered("main", yesterday, "11:00").shouldBeFalse() + state.isTriggered("main", today, "01:00").shouldBeTrue() + } + + test("an unreadable state file is treated as empty") { + val file = stateFile() + Files.createDirectories(file.parent) + Files.writeString(file, "not json at all {") + val state = FileAutoBuildState(file) + + state.isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeFalse() + state.markTriggered("main", LocalDate.parse("2026-07-07"), "11:00") + state.isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeTrue() + } + } +} diff --git a/src/test/kotlin/de/hoennig/gittally/watcher/WatcherTest.kt b/src/test/kotlin/de/hoennig/gittally/watcher/WatcherTest.kt new file mode 100644 index 0000000..76cd48f --- /dev/null +++ b/src/test/kotlin/de/hoennig/gittally/watcher/WatcherTest.kt @@ -0,0 +1,343 @@ +package de.hoennig.gittally.watcher + +import de.hoennig.gittally.build.ArtifactKeys +import de.hoennig.gittally.build.ArtifactStore +import de.hoennig.gittally.build.BuildExecutor +import de.hoennig.gittally.build.BuildResult +import de.hoennig.gittally.build.BuildStatus +import de.hoennig.gittally.build.FileBuildResultRepository +import de.hoennig.gittally.build.GitWorktreeWorkspaces +import de.hoennig.gittally.build.RunningBuild +import de.hoennig.gittally.config.AutoBuildConfig +import de.hoennig.gittally.config.BranchConfig +import de.hoennig.gittally.config.ConfigLoader +import de.hoennig.gittally.config.GitTallyConfig +import de.hoennig.gittally.config.WatcherConfig +import de.hoennig.gittally.git.GitService +import io.kotest.assertions.throwables.shouldThrow +import io.kotest.core.spec.style.FunSpec +import io.kotest.matchers.booleans.shouldBeFalse +import io.kotest.matchers.booleans.shouldBeTrue +import io.kotest.matchers.collections.shouldBeEmpty +import io.kotest.matchers.collections.shouldContainExactly +import io.kotest.matchers.collections.shouldContainExactlyInAnyOrder +import io.kotest.matchers.nulls.shouldBeNull +import io.kotest.matchers.nulls.shouldNotBeNull +import io.kotest.matchers.shouldBe +import io.kotest.matchers.string.shouldContain +import io.mockk.every +import io.mockk.mockk +import io.mockk.verify +import java.nio.file.Files +import java.nio.file.Path +import java.time.Clock +import java.time.Duration +import java.time.Instant +import java.time.LocalDate +import java.time.ZoneOffset +import java.util.concurrent.CopyOnWriteArrayList +import java.util.concurrent.CountDownLatch +import java.util.concurrent.TimeUnit + +class WatcherTest : FunSpec() { + private val noon = Instant.parse("2026-07-07T12:00:00Z") + + private inner class Harness( + config: GitTallyConfig = GitTallyConfig(), + ) { + val workingDir: Path = Files.createTempDirectory("gittally-watcher-test") + val repository = FileBuildResultRepository(workingDir.resolve(".git/gittally/build-results.json")) + val gitService = mockk() + val buildExecutor = mockk() + val artifactStore = mockk() + val startedBuilds = CopyOnWriteArrayList>() + val configLoader = mockk() + val watcher = + Watcher( + gitService = gitService, + buildExecutor = buildExecutor, + repository = repository, + artifactStore = artifactStore, + configLoader = configLoader, + clock = Clock.fixed(noon, ZoneOffset.UTC), + ) + + private var seedCounter = 0L + + init { + every { configLoader.load(any()) } returns config + every { gitService.fetchOrigin(any()) } returns Unit + every { gitService.localBranches(any()) } returns emptyList() + every { gitService.originBranches(any()) } returns emptyList() + every { gitService.newOriginBranches(any(), any()) } returns emptyList() + every { gitService.hasNewCommits(any(), any()) } returns false + every { gitService.originHeadCommit(any(), any()) } returns null + every { gitService.worktreePrune(any()) } returns Unit + every { buildExecutor.currentBuilds() } returns emptyList() + every { buildExecutor.startBuild(any(), any(), any()) } answers { + val branch = firstArg() + val commit = secondArg() + startedBuilds += branch to commit + runningBuild(branch, commit) + } + every { artifactStore.prune(any()) } returns emptyList() + } + + /** Appends a result; later seeds get later start timestamps. */ + fun seed( + branch: String, + status: BuildStatus, + commit: String = "commit-0", + ): BuildResult { + val startedAt = noon.minusSeconds(3600).plusSeconds(seedCounter++) + val result = + BuildResult( + branch = branch, + commit = commit, + status = status, + startedAt = startedAt, + artifactKey = ArtifactKeys.buildKey(branch, startedAt), + ) + repository.append(result) + return result + } + + fun worktreeDir(branch: String): Path { + val dir = workingDir.resolve(GitWorktreeWorkspaces.WORKTREES_DIR).resolve(ArtifactKeys.branchKey(branch)) + Files.createDirectories(dir) + Files.writeString(dir.resolve("marker.txt"), branch) + return dir + } + + fun autoBuildState() = FileAutoBuildState(workingDir.resolve(Watcher.AUTO_BUILDS_FILE)) + } + + private fun runningBuild( + branch: String, + commit: String, + ): RunningBuild { + val stagingDir = Files.createTempDirectory("gittally-watcher-staging") + return RunningBuild( + branch = branch, + commit = commit, + artifactKey = ArtifactKeys.buildKey(branch, Instant.now()), + startedAt = Instant.now(), + stagingDir = stagingDir, + liveLogFile = stagingDir.resolve("build.log"), + ) + } + + private fun autoBuildConfig(vararg times: String): GitTallyConfig = + GitTallyConfig( + branches = + mapOf( + "default" to BranchConfig(), + "main" to BranchConfig(autoBuild = AutoBuildConfig(enabled = true, times = times.toList())), + ), + ) + + init { + test("a fetch failure is exposed in the state and only retried next cycle") { + val harness = Harness() + every { harness.gitService.fetchOrigin(any()) } throws RuntimeException("origin unreachable") + + harness.watcher.poll(harness.workingDir) + + harness.watcher + .state() + .lastFetchError + .shouldNotBeNull() shouldContain "origin unreachable" + harness.watcher.state().lastPollAt shouldBe noon + harness.startedBuilds.shouldBeEmpty() + verify(exactly = 0) { harness.artifactStore.prune(any()) } + + every { harness.gitService.fetchOrigin(any()) } returns Unit + harness.watcher.poll(harness.workingDir) + + harness.watcher + .state() + .lastFetchError + .shouldBeNull() + } + + test("poll enqueues changed local branches before recent new origin branches") { + val harness = Harness() + every { harness.gitService.originBranches(any()) } returns listOf("main", "feature/new") + every { harness.gitService.localBranches(any()) } returns listOf("main", "untracked-local") + every { harness.gitService.hasNewCommits("main", any()) } returns true + every { harness.gitService.newOriginBranches(any(), any()) } returns listOf("feature/new") + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-main" + every { harness.gitService.originHeadCommit("feature/new", any()) } returns "commit-feature" + + harness.watcher.poll(harness.workingDir) + + harness.startedBuilds shouldContainExactly + listOf("main" to "commit-main", "feature/new" to "commit-feature") + } + + test("poll skips a branch whose build is already pending or running") { + val harness = Harness() + harness.seed("main", BuildStatus.PENDING, commit = "commit-old") + every { harness.gitService.originBranches(any()) } returns listOf("main") + every { harness.gitService.localBranches(any()) } returns listOf("main") + every { harness.gitService.hasNewCommits("main", any()) } returns true + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-new" + + harness.watcher.poll(harness.workingDir) + + harness.startedBuilds.shouldBeEmpty() + } + + test("a poll cycle completes while a build is running and still enqueues other branches") { + val harness = Harness() + harness.seed("main", BuildStatus.RUNNING, commit = "commit-1") + every { harness.buildExecutor.currentBuilds() } returns listOf(runningBuild("main", "commit-1")) + every { harness.gitService.originBranches(any()) } returns listOf("main", "feature/other") + every { harness.gitService.localBranches(any()) } returns listOf("main") + every { harness.gitService.hasNewCommits("main", any()) } returns true + every { harness.gitService.newOriginBranches(any(), any()) } returns listOf("feature/other") + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-2" + every { harness.gitService.originHeadCommit("feature/other", any()) } returns "commit-3" + + harness.watcher.poll(harness.workingDir) + + harness.startedBuilds shouldContainExactly listOf("feature/other" to "commit-3") + harness.watcher.state().queuedBranches shouldContainExactly listOf("main") + } + + test("poll does not re-enqueue a commit that was already built") { + val harness = Harness() + harness.seed("main", BuildStatus.SUCCESS, commit = "commit-abc") + every { harness.gitService.originBranches(any()) } returns listOf("main") + every { harness.gitService.localBranches(any()) } returns listOf("main") + every { harness.gitService.hasNewCommits("main", any()) } returns true + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-abc" + + harness.watcher.poll(harness.workingDir) + harness.startedBuilds.shouldBeEmpty() + + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-def" + harness.watcher.poll(harness.workingDir) + + harness.startedBuilds shouldContainExactly listOf("main" to "commit-def") + } + + test("poll filters new origin branches by the configured newBranchMaxAge") { + val harness = Harness(GitTallyConfig(watcher = WatcherConfig(newBranchMaxAge = "12h"))) + + harness.watcher.poll(harness.workingDir) + + verify { harness.gitService.newOriginBranches(Duration.ofHours(12), any()) } + } + + test("auto builds rebuild the already built commit once per day and slot") { + val harness = Harness(autoBuildConfig("01:00", "11:00", "13:00")) + harness.seed("main", BuildStatus.SUCCESS, commit = "commit-abc") + every { harness.gitService.originBranches(any()) } returns listOf("main") + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-abc" + + harness.watcher.poll(harness.workingDir) + harness.watcher.poll(harness.workingDir) + + harness.startedBuilds shouldContainExactly listOf("main" to "commit-abc") + harness.autoBuildState().isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeTrue() + } + + test("an auto-build slot stays untriggered while the branch is still building") { + val harness = Harness(autoBuildConfig("11:00")) + harness.seed("main", BuildStatus.RUNNING, commit = "commit-abc") + every { harness.gitService.originBranches(any()) } returns listOf("main") + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-abc" + + harness.watcher.poll(harness.workingDir) + + harness.startedBuilds.shouldBeEmpty() + harness.autoBuildState().isTriggered("main", LocalDate.parse("2026-07-07"), "11:00").shouldBeFalse() + } + + test("startup recovery marks stale builds interrupted and re-enqueues them") { + val harness = Harness() + harness.seed("main", BuildStatus.RUNNING, commit = "commit-1") + harness.seed("feature/a", BuildStatus.INTERRUPTED, commit = "commit-2") + harness.seed("queued", BuildStatus.PENDING, commit = "commit-3") + harness.seed("done", BuildStatus.SUCCESS, commit = "commit-4") + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-1" + every { harness.gitService.originHeadCommit("feature/a", any()) } returns "commit-2" + every { harness.gitService.originHeadCommit("queued", any()) } returns "commit-3" + + harness.watcher.recoverOnStartup(harness.workingDir) + + harness.startedBuilds shouldContainExactlyInAnyOrder + listOf("main" to "commit-1", "feature/a" to "commit-2", "queued" to "commit-3") + harness.repository.latestFor("main")!!.status shouldBe BuildStatus.INTERRUPTED + harness.repository.latestFor("queued")!!.status shouldBe BuildStatus.INTERRUPTED + harness.repository.latestFor("done")!!.status shouldBe BuildStatus.SUCCESS + } + + test("startup recovery skips branches gone from origin and survives a failing fetch") { + val harness = Harness() + every { harness.gitService.fetchOrigin(any()) } throws RuntimeException("origin unreachable") + harness.seed("gone", BuildStatus.INTERRUPTED, commit = "commit-1") + harness.seed("main", BuildStatus.INTERRUPTED, commit = "commit-2") + every { harness.gitService.originHeadCommit("main", any()) } returns "commit-2" + + harness.watcher.recoverOnStartup(harness.workingDir) + + harness.startedBuilds shouldContainExactly listOf("main" to "commit-2") + } + + test("poll prunes results, artifacts, and worktrees of branches gone from origin") { + val harness = Harness() + harness.seed("main", BuildStatus.SUCCESS, commit = "commit-1") + harness.seed("gone", BuildStatus.SUCCESS, commit = "commit-2") + val keptWorktree = harness.worktreeDir("main") + val removedWorktree = harness.worktreeDir("gone") + every { harness.gitService.originBranches(any()) } returns listOf("main") + + harness.watcher.poll(harness.workingDir) + + harness.repository.history().map { it.branch } shouldContainExactly listOf("main") + verify { + harness.artifactStore.prune( + match { results -> results.map { it.branch } == listOf("main") }, + ) + } + Files.exists(keptWorktree).shouldBeTrue() + Files.exists(removedWorktree).shouldBeFalse() + verify { harness.gitService.worktreePrune(any()) } + } + + test("worktrees of queued or running builds are never pruned") { + val harness = Harness() + harness.seed("busy", BuildStatus.RUNNING, commit = "commit-1") + val busyWorktree = harness.worktreeDir("busy") + every { harness.gitService.originBranches(any()) } returns listOf("busy") + + harness.watcher.poll(harness.workingDir) + + Files.exists(busyWorktree).shouldBeTrue() + } + + test("start runs recovery plus an immediate first poll; stop halts the loop") { + val harness = Harness() + val fetches = CountDownLatch(2) + every { harness.gitService.fetchOrigin(any()) } answers { fetches.countDown() } + + harness.watcher.start(harness.workingDir) + + fetches.await(5, TimeUnit.SECONDS).shouldBeTrue() + harness.watcher + .state() + .running + .shouldBeTrue() + shouldThrow { harness.watcher.start(harness.workingDir) } + + harness.watcher.stop() + + harness.watcher + .state() + .running + .shouldBeFalse() + } + } +}