implemented 04-build-executor.md incl. concurrency amendment: async builds in per-branch worktrees, builds.maxConcurrent, cancellation, live logs; fix .gitignore build/ rule that silently excluded the de.hoennig.gittally.build package (also recovers the step-01 domain files)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
ae3ae7aa04
commit
b379bc0a6b
@@ -47,3 +47,37 @@ Consider `builds.timeout` only if trivial; otherwise defer.
|
||||
|
||||
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||
- While a test build sleeps, the repository reports `RUNNING` — proven by a test.
|
||||
|
||||
## Execution Notes (done 2026-07-07)
|
||||
|
||||
Implemented as designed in `de.hoennig.gittally.build`; build green, 18 new tests
|
||||
(`BuildExecutorTest`, `ProcessBuildRunnerTest`, `ArtifactKeysTest`, plus two new `FileBuildResultRepositoryTest` cases).
|
||||
Deviations and decisions:
|
||||
|
||||
- One-build-at-a-time uses a single-thread worker executor; a second `startBuild` queues and stays `PENDING` until the first finishes.
|
||||
`PENDING` is persisted synchronously in `startBuild`, so a queued build is immediately visible.
|
||||
- `BuildResultRepository` gained `updateByArtifactKey(...)` (extends step 01) so transitions always hit the exact entry, even when a newer `PENDING` entry of the same branch was queued meanwhile.
|
||||
- `cancel()` sets the flag and destroys the process tree directly (TERM, 2s wait, KILL via `ProcessHandle.descendants()`); no separate monitor thread.
|
||||
The flag is checked before each command and after `waitFor`, so a cancel between clean and build commands still records `CANCELLED`.
|
||||
- Artifact key naming (`ArtifactKeys`) was needed here because `BuildResult` requires a key; it follows the legacy scheme (sanitized name + 12-char SHA-256 prefix + sanitized ISO timestamp + hash) using the UTC `Instant`, not local time.
|
||||
Step 05 should reuse it rather than re-implement.
|
||||
- `ArtifactStore` is an interface in the `build` package with a logging `NoOpArtifactStore` placeholder; step 05 replaces the placeholder and implements the real store in `de.hoennig.gittally.artifacts`.
|
||||
- The combined live log is `build.log` inside the per-build staging directory (a temp dir exposed via `RunningBuild.stagingDir`/`liveLogFile`); output is flushed per read chunk so the log grows while the build runs.
|
||||
- Commands run via `bash -c` with the branch name in the environment as `branch`, like legacy `run_build_command`; a failing `cleanCommand` fails the build without running `buildCommand`.
|
||||
- `BuildResultRepository` is wired as a Spring bean (`BuildConfiguration`) at `.git/gittally/build-results.json` relative to the working directory, matching how `ConfigLoader` resolves the override file; `git rev-parse --git-path` style worktree resolution can come later if needed.
|
||||
- Gitea `target_url` is not published yet; the artifact page URL scheme only exists from step 07 on.
|
||||
- `builds.timeout` was deferred (not trivial alongside cancellation semantics); no config keys were added or changed.
|
||||
|
||||
## Amendment: Concurrent Builds and Per-Branch Worktrees (2026-07-07)
|
||||
|
||||
Refactored on request, superseding parts of the notes above:
|
||||
|
||||
- Builds now run concurrently up to the new config key `builds.maxConcurrent` (default 1), enforced by a global semaphore sized on first use (changing it requires a restart).
|
||||
- At most one build per branch at a time, enforced by one serial worker per branch; a second build of the same branch queues as `PENDING` and runs afterwards ("finish, then next").
|
||||
Whether a new commit should instead cancel the branch's running build is a later, possibly configurable decision (see step 06).
|
||||
- Each branch builds in its own reusable git worktree at `.git/gittally/worktrees/<branchKey>` (`BranchWorkspaces`/`GitWorktreeWorkspaces`), checked out detached at the requested commit — the primary checkout is never touched.
|
||||
Reuse keeps incremental build caches; `cleanCommand` decides how much of them survives.
|
||||
`GitService` gained `worktreeAdd`, `worktreePrune`, and `checkoutDetached` for this.
|
||||
- API change: `currentBuild()` became `currentBuilds(): List<RunningBuild>`, and `cancel()` became `cancel(artifactKey)`; a queued build can be cancelled too and is recorded `CANCELLED` when its worker picks it up.
|
||||
- The Gitea `PENDING` status is now published synchronously in `startBuild`, so queued builds are visible in Gitea while they wait for a slot.
|
||||
- Branch config is still loaded from the primary repository directory, not from the branch's checked-out `.gittally.yml`; honoring the branch's own committed config would be a separate decision.
|
||||
|
||||
@@ -16,8 +16,11 @@ Create package `de.hoennig.gittally.watcher`:
|
||||
1. `fetchOrigin()` (errors: log, publish watcher health state, retry next cycle — no internal retry-sleep loops like legacy `retry_origin_change_check`).
|
||||
2. Determine candidate branches: changed local branches, plus new origin branches within `watcher.newBranchMaxAge` (step 02 operations).
|
||||
3. Check auto-build time slots (below).
|
||||
4. If the executor is idle, dequeue the next branch: checkout, reset to origin, `startBuild` (async).
|
||||
5. Run repository retention pruning and artifact pruning.
|
||||
4. Start builds for due branches via `startBuild(branch, commit)` (async).
|
||||
The executor prepares a per-branch worktree itself (step 04 amendment) — the watcher must never check out or reset the primary worktree.
|
||||
Multiple branches may build concurrently (`builds.maxConcurrent`); the executor already serializes builds of the same branch, so the watcher only has to avoid enqueueing a branch that is already pending or running.
|
||||
5. Run repository retention pruning and artifact pruning; also remove worktrees under `.git/gittally/worktrees/` of branches no longer on origin (`git worktree remove` or delete + `worktreePrune`).
|
||||
- Decide here (or defer with a note): when a new commit arrives for a branch whose build is still running, keep the current queue-behind behavior or cancel the running build and start fresh — this may become a per-branch config option.
|
||||
- Startup sequence (port of legacy recovery): mark stale running builds interrupted, then enqueue restartable branches.
|
||||
- Auto-builds: per-branch `autoBuild.enabled` + `times` (UTC HH:MM) from the merged `branches` config.
|
||||
Persist "already triggered for slot/day" state via a small JSON file next to the build results (replaces `auto-builds.tsv`).
|
||||
|
||||
@@ -22,9 +22,9 @@ JSON API (package `de.hoennig.gittally.server`), replacing the legacy `/control/
|
||||
|
||||
- `GET /api/builds/latest` — latest build per branch.
|
||||
- `GET /api/builds/history` — all builds, newest first.
|
||||
- `GET /api/builds/current` — running build, its live status, and log tail (`?offset=` for incremental log fetch).
|
||||
- `GET /api/builds/current` — the list of running builds (there can be several, one per branch, up to `builds.maxConcurrent`), each with live status and log tail (`?offset=` for incremental log fetch, addressed by artifact key).
|
||||
- `GET /api/status/{commit}` — effective status including Gitea lookup (replaces `/control/status`); must return an explicit error state on Gitea failure, never hang.
|
||||
- `POST /api/builds/{branch}/restart`, `POST /api/builds/current/cancel`, `DELETE /api/builds/{artifactKey}` — guarded by a simple token like the legacy cancel token; wire into executor/watcher/repository.
|
||||
- `POST /api/builds/{branch}/restart`, `POST /api/builds/{artifactKey}/cancel` (cancel takes the artifact key because multiple builds can run concurrently), `DELETE /api/builds/{artifactKey}` — guarded by a simple token like the legacy cancel token; wire into executor/watcher/repository.
|
||||
- `GET /api/watcher` — watcher health (last poll, last error).
|
||||
- Artifact serving: `GET /artifacts/{artifactKey}/**` streaming from the artifact store, with no-cache headers for html/json/log.
|
||||
|
||||
|
||||
@@ -17,7 +17,8 @@ Views (ported from legacy, see analysis for columns and behavior):
|
||||
|
||||
- `/` (Latest): latest build per branch — status badge, branch and commit with Gitea links and copy buttons, times, duration, artifact link, restart/delete actions.
|
||||
- `/history`: all builds.
|
||||
- `/current`: running build with live log view (incremental fetch via `/api/builds/current?offset=`), cancel button, and a clear "no build running" state.
|
||||
- `/current`: running builds with live log view (incremental fetch via the step 07 current-builds API), a cancel button per build, and a clear "no build running" state.
|
||||
Remember there can be several running builds (one per branch, up to `builds.maxConcurrent`); the view must list all of them, not assume a single one.
|
||||
- `/builds/{artifactKey}`: artifact index — build command, logs, links into archived report directories (rendered from the artifact store, not pre-generated HTML).
|
||||
- Shared layout: view toggle nav, footer with version and optional impressum link, `prefers-color-scheme` support (port the legacy CSS look loosely, keep it simple).
|
||||
|
||||
|
||||
+4
-1
@@ -30,6 +30,9 @@ These are proposals baked into the steps.
|
||||
Revisit them in an ADR if a step uncovers problems.
|
||||
|
||||
- Build results are persisted as a JSON file under `.git/gittally/`, behind a `BuildResultRepository` interface (no database, but replaceable).
|
||||
- Builds run concurrently up to `builds.maxConcurrent` (default 1), but never more than one build per branch at a time.
|
||||
Each branch builds in its own reusable git worktree under `.git/gittally/worktrees/`, checked out detached at the requested commit — never in the primary checkout.
|
||||
A later step must decide (possibly per config) whether a new commit on a branch cancels that branch's running build or waits for it; for now new builds queue behind the running one.
|
||||
- Artifacts stay on the filesystem, served by the Spring server.
|
||||
- The web UI is server-rendered HTML plus small JavaScript polling JSON endpoints (no SPA framework).
|
||||
- The watcher is a Spring-managed scheduled component, decoupled from the build executor via the result repository and events.
|
||||
@@ -45,7 +48,7 @@ Foundation:
|
||||
|
||||
Core engine:
|
||||
|
||||
- [ ] `04-build-executor.md` — async build execution with logs, cancellation, status transitions
|
||||
- [x] `04-build-executor.md` — async build execution with logs, cancellation, status transitions
|
||||
- [ ] `05-artifact-store.md` — artifact persistence, naming, retention
|
||||
- [ ] `06-watcher.md` — branch watching, scheduling, auto-builds
|
||||
|
||||
|
||||
Reference in New Issue
Block a user