From 38e080814e30102a4776f5dacb6274914011c239 Mon Sep 17 00:00:00 2001 From: Michael Hoennig Date: Tue, 7 Jul 2026 06:40:27 +0200 Subject: [PATCH] update CLAUDE.md with rewrite plan documentation reference in docs/pland directory --- CLAUDE.md | 1 + docs/plan/00-legacy-analysis.md | 95 ++++++++++++++++++++++++++++ docs/plan/01-build-state-domain.md | 51 +++++++++++++++ docs/plan/02-git-gateway.md | 45 +++++++++++++ docs/plan/03-gitea-client.md | 40 ++++++++++++ docs/plan/04-build-executor.md | 49 ++++++++++++++ docs/plan/05-artifact-store.md | 42 ++++++++++++ docs/plan/06-watcher.md | 47 ++++++++++++++ docs/plan/07-server-mode.md | 45 +++++++++++++ docs/plan/08-web-ui.md | 46 ++++++++++++++ docs/plan/09-system-metrics.md | 35 ++++++++++ docs/plan/10-cli-commands.md | 38 +++++++++++ docs/plan/11-docker-build-runtime.md | 39 ++++++++++++ docs/plan/12-deployment.md | 36 +++++++++++ docs/plan/README.md | 67 ++++++++++++++++++++ 15 files changed, 676 insertions(+) create mode 100644 docs/plan/00-legacy-analysis.md create mode 100644 docs/plan/01-build-state-domain.md create mode 100644 docs/plan/02-git-gateway.md create mode 100644 docs/plan/03-gitea-client.md create mode 100644 docs/plan/04-build-executor.md create mode 100644 docs/plan/05-artifact-store.md create mode 100644 docs/plan/06-watcher.md create mode 100644 docs/plan/07-server-mode.md create mode 100644 docs/plan/08-web-ui.md create mode 100644 docs/plan/09-system-metrics.md create mode 100644 docs/plan/10-cli-commands.md create mode 100644 docs/plan/11-docker-build-runtime.md create mode 100644 docs/plan/12-deployment.md create mode 100644 docs/plan/README.md diff --git a/CLAUDE.md b/CLAUDE.md index 6ceb46c..4b1d1fd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -125,6 +125,7 @@ Keep sentences short. - `docs/GitTally-Konzept.md` — product concept and target architecture (in German): git-centric CI, builds in Docker, one instance per repository, status reported back to Gitea. - `docs/configuration.md` — configuration reference; keep in sync with `GitTallyConfig` and the `init` templates. - `docs/bootstrapping.md` — how `init` prepares a repository. +- `docs/plan/` — the step-by-step rewrite plan; `docs/plan/README.md` explains how to execute a step, `docs/plan/00-legacy-analysis.md` summarizes the legacy bash script. ## Key Architectural Decisions diff --git a/docs/plan/00-legacy-analysis.md b/docs/plan/00-legacy-analysis.md new file mode 100644 index 0000000..6969603 --- /dev/null +++ b/docs/plan/00-legacy-analysis.md @@ -0,0 +1,95 @@ +# Legacy gitTally Analysis + +Condensed analysis of `legacy/gitTally` (bash, ~6000 lines) as input for the rewrite. +Line numbers refer to the legacy script at the time of analysis (version 0.7.8). + +## What the Legacy System Does + +A single bash daemon per repository that: + +1. Polls `origin` for changed local branches and recent new origin branches. +2. Checks out and builds each changed branch (natively or in Docker), one at a time. +3. Records results and publishes commit statuses to Gitea. +4. Archives build logs and report directories as browsable artifacts. +5. Serves a static-HTML web UI via an embedded Python HTTP server. +6. Optionally manages an nginx+certbot Docker container for HTTPS. +7. Installs itself as a systemd user service. + +## Persistent State (formats to replace) + +All state lives in files; there is no database. + +| File | Format | Content | +|---|---|---| +| `.git/git-watch-origin-and-test/build-results.tsv` | TSV | `branch, commit, status, timestamp, duration(MM:SS), artifact_key` | +| `.git/git-watch-origin-and-test/auto-builds.tsv` | TSV | `branch, date, time_slot` — prevents double auto-builds | +| `.git/git-watch-origin-and-test/build.lock` | flock file | build mutex, holder found via `fuser`/`lsof` | +| `.git/git-watch-origin-and-test/cancel-*` | touch/token files | cancellation request/token/accepted handshake | +| `$TMPDIR/git-watch-origin-and-test//` | HTML + files | artifact root: generated pages, per-build artifact dirs | +| `/system.json` + `system_state.dat` | JSON + text | system metrics snapshot and aggregation state | + +Build statuses: `pending`, `running`, `success`, `failed`, `interrupted`, `cancelled` (legacy alias `passed` → `success`). + +Artifact key naming: sanitized branch + 12-char SHA256 prefix, plus sanitized timestamp + hash for per-build dirs. + +## External Interactions + +Git (always via CLI): `rev-parse`, `for-each-ref`, `fetch`, `switch`, `reset --hard`, `show -s --format=%cI`, `rev-list`. +HTTPS git auth uses a generated `GIT_ASKPASS` script feeding username + Gitea token. + +Gitea API: + +- `POST /api/v1/repos/{owner}/{repo}/statuses/{sha}` — publish status; payload `state`, `context`, `description`, `target_url`. +- `GET /api/v1/repos/{owner}/{repo}/commits/{sha}/statuses?sort=recentupdate` — read statuses, filtered by `context`. +- `GET /api/v1/user` — resolve username from token. +- State mapping: success→success; failed/interrupted/cancelled→failure; pending/running→pending. + +Web control endpoints (Python handler): + +- `GET /control/status?commit=&local_status=` — proxy Gitea status for a commit. +- `POST /control/cancel` — cancel running build (CSRF-token protected). +- `POST /control/restart` — append a new pending result row. +- `POST /control/delete` — remove a result row and patch HTML files via regex. + +## Root Causes of the Known Bugs + +Stuck loading animation (UI): + +- Status cells start as `status-loading` and each row fetches `/control/status` with a 15s timeout; failures leave the spinner forever (no error state). +- Page auto-refresh re-fetches the page HTML and diffs `#build-rows`; HTML is regenerated server-side by regex-patching files, which fails silently when structure drifts. +- "Current" and "System" nav views exist, but the page generator only implements `latest`/`branches`/`history`, so some views fall through. + +No status changes observable during a build (control loop): + +- The main loop (lines ~4944–4966) is synchronous: `checkout_and_build` blocks until the build ends; only then is origin re-scanned. +- `retry_origin_change_check()` is an uninterruptible internal retry loop with 10s sleeps. +- The console spinner is cosmetic and runs independently of actual progress. + +## Behavior Worth Preserving + +- Startup recovery: mark stale `running` builds as `interrupted`; restart restartable builds; optional retry of failed builds. +- New-branch age filter (`newBranchMaxAge`) to avoid building stale branches. +- Auto-build time slots (UTC HH:MM) per branch with per-day/slot dedup. +- Retention per branch by count or age, pruning both results and artifact dirs. +- Per-branch build/clean command, artifact dirs, and log file names (now via `branches:` YAML config). +- Cancellation with token handshake; process-tree termination (TERM, wait, KILL). +- Build log capture: stdout/stderr to files plus a live "current build" log. +- Gitea status published on every transition with `target_url` pointing at the artifact page. + +## Not Ported (decided) + +- nginx + certbot/Let's Encrypt container management — replaced by deployment documentation (step 12). +- Self-install (`--install`), self-update script generation — replaced by jar deployment plus systemd docs (step 12). +- Legacy `HSADMIN_NG_*` environment fallbacks and env-file config — replaced by YAML config (done). +- Regex-based in-place HTML patching — replaced by server-rendered pages/JSON endpoints. +- Static HTML artifact index generation with embedded 900-line JavaScript — replaced by templates. +- `.aliases` sourcing (line 731), impressum footer link handling stays optional/simple. + +## Orphaned or Dubious Legacy Config + +Verify need before porting any of these: + +- `GITTALLY_BUILD_DOCKER_PREFLIGHT_COMMAND`, `GITTALLY_BUILD_DOCKER_JAVA_TOOL_OPTIONS` — highly hsadmin-ng-specific defaults. +- `GITTALLY_ARTIFACT_NGINX_*`, `GITTALLY_ARTIFACT_LETSENCRYPT_EMAIL` — dropped with nginx management. +- `GITTALLY_IMPRESSUM_URL` — keep as optional simple footer link if wanted. +- `GITTALLY_INSTALL_DIR` — dropped with self-install. diff --git a/docs/plan/01-build-state-domain.md b/docs/plan/01-build-state-domain.md new file mode 100644 index 0000000..9b91d09 --- /dev/null +++ b/docs/plan/01-build-state-domain.md @@ -0,0 +1,51 @@ +# Step 01: Build State Domain and Repository + +Prerequisites: none. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +A tested domain model for build results plus a persistent repository, replacing the legacy `build-results.tsv`. + +## Design + +Create package `de.hoennig.gittally.build`: + +- `BuildStatus` enum: `PENDING`, `RUNNING`, `SUCCESS`, `FAILED`, `INTERRUPTED`, `CANCELLED`. + Add `isTerminal`, `isRestartable` (pending/running/interrupted) properties. +- `BuildResult` data class: branch, commit SHA, status, startedAt, duration, artifactKey. + Use `java.time.Instant`/`Duration`; format only at the edges. +- `BuildResultRepository` interface: append, update status of latest entry for a branch, query latest per branch, query history, delete entry, prune. +- `FileBuildResultRepository`: JSON file at `.git/gittally/build-results.json`. + Write atomically (write temp file, then `Files.move` with `ATOMIC_MOVE`). + Reuse the Jackson YAML/JSON setup style from `ConfigLoader`. + +Business logic to include (port from legacy, see analysis): + +- `markStaleRunningAsInterrupted()` — called at startup; running → interrupted, superseded pending → interrupted. +- Retention pruning: keep N builds per branch (`artifacts.retentionPerBranch`); drop entries for branches no longer on origin (branch list passed in as a parameter, no git dependency here). + +## Out of Scope + +- No git access, no Gitea, no execution — pure domain and file I/O. +- Artifact directory pruning (step 05 consumes the pruning result). +- Age-based retention (legacy supported `h`/`d` suffixes); count-based only, extend later if needed. + +## Config + +Uses existing `artifacts.retentionPerBranch`. +No new keys expected. + +## Tests + +Kotest `FunSpec`, no Spring context needed. + +- Round-trip persistence, atomicity (temp file cleaned up). +- Status transition helpers and `markStaleRunningAsInterrupted` edge cases. +- Retention pruning: per-branch count, removed branches, ordering by timestamp. +- Corrupt/missing file → empty repository, no crash (legacy failed silently; we log a warning). + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- New code has no dependency on picocli or web classes. diff --git a/docs/plan/02-git-gateway.md b/docs/plan/02-git-gateway.md new file mode 100644 index 0000000..88d46ba --- /dev/null +++ b/docs/plan/02-git-gateway.md @@ -0,0 +1,45 @@ +# Step 02: Git Gateway + +Prerequisites: none. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +Extend `GitService` into a complete, tested gateway for all git operations the watcher and builder need. + +## Design + +Keep the existing approach: shell out to the `git` CLI via `ProcessBuilder`. +Extract a small `GitCommandRunner` (command + workingDir → exit code, stdout, stderr) so `GitService` becomes testable and readable. + +Operations to add (legacy references in parentheses): + +- `fetchOrigin()` and `fetchBranch(branch)` with authentication (legacy `git_with_gitea_token`, askpass). + Use the `GIT_ASKPASS` environment technique: write a temp script returning `git.account` / `git.token` from config; `GIT_TERMINAL_PROMPT=0`. + Only needed for HTTPS origins; skip auth setup for SSH origins. +- `localBranches()`, `originBranches()` (legacy `branch_candidates`). +- `hasNewCommits(branch)` — compare local head with upstream/origin (legacy `branch_has_new_commits`, `has_new_commits` via `rev-list`). +- `newOriginBranches(maxAge)` — origin branches without local counterpart whose latest commit is younger than `watcher.newBranchMaxAge` (legacy `recent_new_origin_branches`). +- `checkout(branch)` — switch, or create tracking branch from origin (legacy `switch_to_branch`). +- `resetHardToOrigin(branch)` (legacy `pull_branch_if_possible`). +- `commitTimestamp(sha)`, `currentBranch()`, `headCommit()`. + +Parse the `newBranchMaxAge` duration format (`5d`, `12h`) in a small dedicated parser with tests. + +## Out of Scope + +- No polling loop (step 06), no build triggering. +- No Gitea API calls (step 03); only the askpass credential bridge is set up here. + +## Tests + +- `GitCommandRunner` unit tests. +- Integration tests against local fixture repositories: create a bare "origin" repo and a clone in a temp dir via the runner itself, then exercise fetch/branch/commit operations. + No network access needed. +- Duration parser edge cases. +- Askpass script content test (do not test against a real remote). + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- No temp askpass files leak (created per invocation or cleaned via try/finally; legacy leaked them). diff --git a/docs/plan/03-gitea-client.md b/docs/plan/03-gitea-client.md new file mode 100644 index 0000000..081f512 --- /dev/null +++ b/docs/plan/03-gitea-client.md @@ -0,0 +1,40 @@ +# Step 03: Gitea Client + +Prerequisites: step 01 (for `BuildStatus`). +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +A tested client for the Gitea commit-status API. + +## Design + +Create package `de.hoennig.gittally.gitea`: + +- `GiteaClient` using Spring's `RestClient`. +- `publishStatus(sha, state, description, targetUrl)` → `POST /api/v1/repos/{owner}/{repo}/statuses/{sha}` with header `Authorization: token `; body fields `state`, `context`, `description`, `target_url`. +- `readStatus(sha)` → `GET /api/v1/repos/{owner}/{repo}/commits/{sha}/statuses?sort=recentupdate`; pick the newest entry matching `gitea.statusContext`. +- `resolveUsername()` → `GET /api/v1/user` (used as fallback for `git.account`). +- State mapping in both directions (`BuildStatus` ↔ Gitea `success|failure|pending|error`), as in the legacy analysis. +- `isEnabled()` — true only when `gitea.baseUrl`, `gitea.owner`, `gitea.repo`, and `git.token` are configured. + All callers must treat a disabled or failing client as non-fatal (log and continue); the legacy behaved the same but failed silently. + +Configuration comes from `GitTallyConfig` (`gitea.*`, `git.token`). + +## Out of Scope + +- No callers yet; the build executor (step 04) wires status publishing. +- No webhook receiving; GitTally remains poll-based. + +## Tests + +WireMock (already a test dependency, see `WireMockSmokeTest`): + +- Publish: correct URL, auth header, JSON body per status. +- Read: filtering by context, newest-first, empty result, malformed JSON → error status, HTTP 4xx/5xx → non-fatal error result. +- State-mapping unit tests. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- No call path throws when Gitea is unconfigured or down. diff --git a/docs/plan/04-build-executor.md b/docs/plan/04-build-executor.md new file mode 100644 index 0000000..9fbe449 --- /dev/null +++ b/docs/plan/04-build-executor.md @@ -0,0 +1,49 @@ +# Step 04: Build Executor + +Prerequisites: steps 01, 02, 03. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +Asynchronous build execution with log capture, cancellation, and immediately visible status transitions. +This step fixes the legacy defect that nothing could observe status changes while a build ran. + +## Design + +Create package `de.hoennig.gittally.build` (extends step 01): + +- `BuildExecutor` service; one build at a time (a `ReentrantLock` or single-thread executor replaces the legacy flock file). +- `startBuild(branch, commit)` runs asynchronously and returns immediately; expose `currentBuild(): RunningBuild?`. +- Execution sequence per build: + 1. Record `PENDING`, then `RUNNING` via `BuildResultRepository`; publish each transition to Gitea (non-fatal on failure). + 2. Run the branch's `cleanCommand`, then `buildCommand` (from the merged `branches` config) via `ProcessBuilder` with the branch name in the environment as `branch`. + 3. Stream stdout/stderr to the configured log files in a working/staging directory, plus a combined live log file. + 4. On exit: record `SUCCESS`/`FAILED`, publish status, hand the staging directory to the artifact store (step 05 interface; use a stub interface now if 05 is not done). +- Cancellation: `cancel()` flag checked by a monitor; destroy the process tree (`ProcessHandle.descendants()`, TERM-wait-KILL like legacy `terminate_process_tree`); record `CANCELLED`. +- Status transitions must be readable at any time via the repository — no in-memory-only state. + +Emit a Spring `ApplicationEvent` on every status transition so the UI (step 08) can later push updates without polling internals. + +## Out of Scope + +- Docker execution (step 11); native `ProcessBuilder` only, but keep a `BuildRunner` interface so Docker can plug in. +- Scheduling and branch selection (step 06). +- Artifact index HTML (step 05/08). + +## Config + +Uses existing `branches..buildCommand/cleanCommand/stdoutLog/stderrLog`. +Consider `builds.timeout` only if trivial; otherwise defer. + +## Tests + +- Fake commands (`sh -c 'echo ok'`, failing command, sleeping command) in temp dirs. +- Status sequence assertions: pending → running → success/failed/cancelled, each persisted before/after execution. +- Cancellation kills a sleeping process tree and records `CANCELLED`. +- Log files contain captured output; live log grows during the build (poll in test). +- Gitea publishing mocked with MockK; a Gitea failure must not fail the build. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- While a test build sleeps, the repository reports `RUNNING` — proven by a test. diff --git a/docs/plan/05-artifact-store.md b/docs/plan/05-artifact-store.md new file mode 100644 index 0000000..3d34301 --- /dev/null +++ b/docs/plan/05-artifact-store.md @@ -0,0 +1,42 @@ +# Step 05: Artifact Store + +Prerequisites: steps 01, 04. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +Persist build artifacts (logs plus configured report directories) with stable naming and retention pruning. + +## Design + +Create package `de.hoennig.gittally.artifacts`: + +- `ArtifactStore` service implementing the interface stubbed in step 04. +- Artifact root: a configurable directory (new key `artifacts.rootDir`), defaulting to `${XDG_STATE_HOME:-~/.local/state}/gittally/artifacts/`. + Do NOT default to `/tmp` like legacy — artifacts vanished on reboot. +- Repo key: sanitized absolute repo path (legacy `repository_key`): non `[A-Za-z0-9._-]` → `_`. +- Artifact key per build: sanitized branch name + 12-char SHA-256 prefix, plus sanitized start timestamp + hash (legacy `build_artifact_key`); keep this scheme so URLs stay predictable. +- `persist(build, stagingDir)`: copy configured `artifactDirs` and the log files into staging, then atomically move staging → `/branches//`. +- `prune(keptResults)`: delete artifact directories whose keys are no longer in the result repository (call after repository retention pruning from step 01). +- Provide `artifactDir(artifactKey)` lookups for the server (step 07). + +## Out of Scope + +- HTTP serving (step 07). +- HTML index pages (step 08 renders from data instead of generated files). + +## Config + +New key `artifacts.rootDir` (empty = platform default above). +Update `GitTallyConfig`, `InitCommand` templates, and `docs/configuration.md` together. + +## Tests + +- Key naming: sanitization, hash stability, collision of similar branch names. +- Persist: copies configured dirs, missing artifact dirs are skipped with a log line, atomic move (no partial dirs on failure). +- Prune: deletes exactly the unreferenced dirs, never anything outside the artifact root. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- Step 04's executor persists artifacts through the real store (integration test). diff --git a/docs/plan/06-watcher.md b/docs/plan/06-watcher.md new file mode 100644 index 0000000..cd7b31b --- /dev/null +++ b/docs/plan/06-watcher.md @@ -0,0 +1,47 @@ +# Step 06: Watcher and Scheduling + +Prerequisites: steps 01, 02, 04. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +Replace the legacy blocking main loop with a non-blocking, observable scheduler. + +## Design + +Create package `de.hoennig.gittally.watcher`: + +- `Watcher` component with a fixed-delay poll cycle (Spring `@Scheduled` or a managed executor; enabled only in server/watch mode, not during CLI commands or tests). +- One poll cycle, never blocking on a build: + 1. `fetchOrigin()` (errors: log, publish watcher health state, retry next cycle — no internal retry-sleep loops like legacy `retry_origin_change_check`). + 2. Determine candidate branches: changed local branches, plus new origin branches within `watcher.newBranchMaxAge` (step 02 operations). + 3. Check auto-build time slots (below). + 4. If the executor is idle, dequeue the next branch: checkout, reset to origin, `startBuild` (async). + 5. Run repository retention pruning and artifact pruning. +- Startup sequence (port of legacy recovery): mark stale running builds interrupted, then enqueue restartable branches. +- Auto-builds: per-branch `autoBuild.enabled` + `times` (UTC HH:MM) from the merged `branches` config. + Persist "already triggered for slot/day" state via a small JSON file next to the build results (replaces `auto-builds.tsv`). +- Expose watcher state (last poll time, last fetch error, queue) for the UI/status endpoints. + +## Out of Scope + +- HTTP endpoints (step 07); expose state via a service bean only. +- Retry-failed-builds command (step 10 triggers it through the same queue). + +## Config + +New key `watcher.pollInterval` (e.g. `10s`, default matching legacy cadence). +Update `GitTallyConfig`, `InitCommand` templates, and `docs/configuration.md` together. + +## Tests + +- Poll cycle unit tests with MockK: branch selection precedence, skip-when-building, fetch failure resilience. +- Auto-build slot logic: slot matching, per-day dedup, state persistence. +- New-branch age filtering. +- Startup recovery: interrupted marking and re-enqueue (integration with steps 01/04 fakes). +- No test may sleep for real poll intervals; trigger cycles directly. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- A test proves a poll cycle completes while a (fake) build is running. diff --git a/docs/plan/07-server-mode.md b/docs/plan/07-server-mode.md new file mode 100644 index 0000000..ee746f4 --- /dev/null +++ b/docs/plan/07-server-mode.md @@ -0,0 +1,45 @@ +# Step 07: Server Mode and HTTP API + +Prerequisites: steps 04, 05, 06. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +Implement the `server` subcommand: a persistent web server exposing build state as JSON and serving artifacts. + +## Design + +Bootstrapping: + +- `CLAUDE.md` notes the context starts with web type `none`; the `server` subcommand must run a web context. + Preferred approach: `ServerCommand` launches a second `SpringApplication` with `WebApplicationType.SERVLET` and a `server` profile, then blocks until shutdown. + Document the chosen mechanism in the code and, if it deviates, in an ADR. +- Add `spring-boot-starter-web` dependency. +- The watcher (step 06) is active only in the `server` profile. +- New config keys `server.port` and `server.bindAddress` (defaults 18080 / 0.0.0.0, as legacy). + +JSON API (package `de.hoennig.gittally.server`), replacing the legacy `/control/*` endpoints: + +- `GET /api/builds/latest` — latest build per branch. +- `GET /api/builds/history` — all builds, newest first. +- `GET /api/builds/current` — running build, its live status, and log tail (`?offset=` for incremental log fetch). +- `GET /api/status/{commit}` — effective status including Gitea lookup (replaces `/control/status`); must return an explicit error state on Gitea failure, never hang. +- `POST /api/builds/{branch}/restart`, `POST /api/builds/current/cancel`, `DELETE /api/builds/{artifactKey}` — guarded by a simple token like the legacy cancel token; wire into executor/watcher/repository. +- `GET /api/watcher` — watcher health (last poll, last error). +- Artifact serving: `GET /artifacts/{artifactKey}/**` streaming from the artifact store, with no-cache headers for html/json/log. + +## Out of Scope + +- HTML pages (step 08); JSON plus artifact files only. +- TLS/reverse proxy (documented in step 12). + +## Tests + +- `@WebMvcTest` slices with `@MockkBean` per controller (see `CLAUDE.md` conventions). +- Contract tests: JSON shapes, error states (Gitea down → explicit `unknown` status, HTTP 200), cancel token rejection. +- One `@SpringBootTest` on the server profile proving the context boots with watcher and web enabled. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- `java -jar ... server` starts, `GET /api/builds/latest` answers, Ctrl-C shuts down cleanly (manual smoke test; document result in this file). diff --git a/docs/plan/08-web-ui.md b/docs/plan/08-web-ui.md new file mode 100644 index 0000000..74d0076 --- /dev/null +++ b/docs/plan/08-web-ui.md @@ -0,0 +1,46 @@ +# Step 08: Web UI + +Prerequisites: step 07. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +The browsable UI: build overview, history, current build with live log, per-build artifact index. +This step fixes the legacy stuck-loading-animation defect. + +## Design + +Server-rendered Thymeleaf templates plus a small, hand-written JavaScript file polling the step 07 JSON API. +No SPA framework, no build pipeline for the frontend. + +Views (ported from legacy, see analysis for columns and behavior): + +- `/` (Latest): latest build per branch — status badge, branch and commit with Gitea links and copy buttons, times, duration, artifact link, restart/delete actions. +- `/history`: all builds. +- `/current`: running build with live log view (incremental fetch via `/api/builds/current?offset=`), cancel button, and a clear "no build running" state. +- `/builds/{artifactKey}`: artifact index — build command, logs, links into archived report directories (rendered from the artifact store, not pre-generated HTML). +- Shared layout: view toggle nav, footer with version and optional impressum link, `prefers-color-scheme` support (port the legacy CSS look loosely, keep it simple). + +Robust live updates (the actual bug fix): + +- Poll JSON endpoints on an interval (10–15s) and re-render table bodies from data; never re-fetch and diff whole HTML pages. +- Every async fetch has a timeout and renders an explicit error/"unknown" badge on failure — no permanent spinners by construction. +- Pause polling when the tab is hidden (`visibilitychange`), resume and refresh immediately when visible. +- Running durations tick client-side from a `data-started-at` attribute. + +## Out of Scope + +- System metrics page (step 09). +- WebSocket/SSE push; plain polling first, extend later if wanted. + +## Tests + +- MockMvc view tests: templates render for empty state, running build, mixed history. +- Escaping test: branch names with HTML/JS metacharacters render safely (legacy had injection risks). +- JavaScript logic that is non-trivial (duration formatting, poll scheduling) should live in small pure functions; test via MockMvc-rendered attributes or keep trivially simple. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- Manual smoke test with a real build: status flips pending → running → success in the open browser tab without a manual reload, and a killed server results in error badges, not spinners. + Document the result in this file. diff --git a/docs/plan/09-system-metrics.md b/docs/plan/09-system-metrics.md new file mode 100644 index 0000000..b35c4f1 --- /dev/null +++ b/docs/plan/09-system-metrics.md @@ -0,0 +1,35 @@ +# Step 09: System Metrics + +Prerequisites: step 07 (API), step 08 (layout). +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +Port the legacy system page: CPU, RAM, disk, and repository size with min/max/avg aggregation. + +## Design + +Create package `de.hoennig.gittally.metrics`: + +- `SystemMetricsCollector` sampling every 60s (server profile only): + CPU used/idle from `/proc/stat` deltas, RAM from `/proc/meminfo`, disk from `java.nio.file.FileStore`, repo size via periodic `du -sk` (or a file walk) — throttle repo-size sampling (legacy ran `du` every cycle, which was expensive). +- Keep running min/max/avg per metric since server start; persist aggregation state in the artifact root so restarts continue the series (legacy `system_state.dat`, but as JSON). +- `GET /api/system` returning the current snapshot plus aggregates (legacy `system.json` fields are the reference). +- `/system` HTML view in the step 08 layout, polling `/api/system` every 60s with the same error-badge rules. + +## Out of Scope + +- Alerting, historical time series, external monitoring integration. +- Windows/macOS support beyond graceful degradation (missing `/proc` → metric shows "n/a"). + +## Tests + +- Collector unit tests with fake `/proc` file content (read paths injectable). +- Aggregation math: min/max/avg over samples, persistence round-trip. +- Controller slice test for `/api/system`. +- Graceful degradation when a source is unreadable. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- `/system` renders live values on Linux (manual smoke test; document in this file). diff --git a/docs/plan/10-cli-commands.md b/docs/plan/10-cli-commands.md new file mode 100644 index 0000000..84ece60 --- /dev/null +++ b/docs/plan/10-cli-commands.md @@ -0,0 +1,38 @@ +# Step 10: CLI Commands + +Prerequisites: steps 04, 05, 06. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +CLI parity for interactive use without the server. + +## Design + +New picocli subcommands (Spring components in `commands/`, wired like the existing ones): + +- `status` — print the latest build per branch as a table (branch, status, commit, time, duration); `--history` for all builds. + Reads `BuildResultRepository` directly; works while a server instance runs (file-based store, read-only). +- `build []` — one-shot: fetch, checkout, build the given branch (default: current), print the log to the console, exit with the build's exit code. + Replaces legacy `--stay` and explicit branch arguments. + Support the legacy partial-name matching: resolve unique branch-name fragments, list candidates on ambiguity (legacy `resolve_branch_name`). +- `retry` — re-enqueue/build branches whose latest build failed (legacy `--retry`). +- Update the root command's subcommand list and `--help` texts. + +Exit codes: 0 on success, 1 on build failure, 2 on usage/config errors (align with `CliRunner`'s `ExitCodeGenerator` contract). + +## Out of Scope + +- Talking to a running server over HTTP; direct file/executor access is sufficient for now. +- Shell completion. + +## Tests + +- Command unit tests with MockK-ed services (pattern of `InitCommandTest`). +- Branch fragment resolution: unique match, ambiguity handling, no match. +- Exit code assertions. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- `java -jar ... status` and `java -jar ... build` work in this repository (manual smoke test; document in this file). diff --git a/docs/plan/11-docker-build-runtime.md b/docs/plan/11-docker-build-runtime.md new file mode 100644 index 0000000..de2a25f --- /dev/null +++ b/docs/plan/11-docker-build-runtime.md @@ -0,0 +1,39 @@ +# Step 11: Docker Build Runtime (optional) + +Prerequisites: step 04. +Read `README.md` and `00-legacy-analysis.md` first. +This step is optional; skip it until builds actually need container isolation. + +## Goal + +Run build commands inside a Docker container, as the legacy `--docker` mode did. + +## Design + +Implement a `DockerBuildRunner` for the `BuildRunner` interface from step 04, shelling out to the `docker` CLI (consistent with the git gateway approach; no Docker Java SDK dependency). + +Port from legacy (see analysis, lines ~4400+): + +- Ensure image: build from configured Dockerfile/context when missing or stale; track staleness via an image label holding the SHA-256 of Dockerfile + context (legacy `org.gittally.build-inputs-sha256`). +- Gradle cache volume per repository (`gittally-gradle-`), mounted and chowned to the host UID/GID. +- Run the build container: workspace mount, branch env var, configured extra env, network mode, docker socket mount for Testcontainers-based builds. +- Post-build ownership repair of the workspace (legacy `repair_docker_workspace_ownership`). +- Label all containers (`org.hoennig.gittally=true`, repository, role) and clean up stale ones on startup. + +Decide during implementation whether the hsadmin-ng-specific legacy options (preflight command, `JAVA_TOOL_OPTIONS` injection) are needed; default to NOT porting them (see orphaned-config list in the analysis). + +## Config + +New `branches..docker` section: `enabled`, `image`, `dockerfile`, `context`, `network`, `env`. +Update `GitTallyConfig`, `InitCommand` templates, and `docs/configuration.md` together. + +## Tests + +- Unit tests for command assembly (assert the exact `docker run`/`docker build` argv) with a mocked command runner. +- Checksum staleness logic. +- Optional: one Testcontainers-gated integration test that runs a trivial build in a stock image; skip when Docker is unavailable. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green with Docker absent. +- Native execution path (step 04) is unchanged and remains the default. diff --git a/docs/plan/12-deployment.md b/docs/plan/12-deployment.md new file mode 100644 index 0000000..4a687e6 --- /dev/null +++ b/docs/plan/12-deployment.md @@ -0,0 +1,36 @@ +# Step 12: Deployment and Legacy Migration + +Prerequisites: steps 07, 08, 10. +Read `README.md` and `00-legacy-analysis.md` first. + +## Goal + +Make the new GitTally deployable as a service and retire the legacy script. + +## Design + +Deployment (documentation plus a small generator, no self-install): + +- Extend `init` (or add `init --systemd`) to generate a systemd user unit running `java -jar gittally.jar server` with `WorkingDirectory` set to the repo, `Restart=always`, and an `EnvironmentFile` for overrides — port the shape of the legacy unit, drop the self-copy/update machinery. +- Write `docs/deployment.md`: JRE requirement, jar location convention, systemd enable/start/log commands, and reverse-proxy guidance (example nginx `server` block proxying to `server.port`; TLS via the host's existing certbot — replaces the legacy managed nginx container). + +Migration: + +- Write `docs/migration-from-legacy.md`: mapping table legacy env vars → YAML keys (source: `00-legacy-analysis.md` and legacy `--env` output), what is intentionally not ported, and the manual steps (stop legacy service, run `init`, fill in token, install new service). +- Decide and document: no automatic import of `build-results.tsv` (history starts fresh) unless trivially cheap. + +Housekeeping: + +- Mark `legacy/gitTally` as deprecated in its header comment and in `README.md`. +- Review `docs/GitTally-Konzept.md` against what was actually built; update or note deviations. +- Add an ADR summarizing the architecture decisions that emerged during the rewrite (persistence choice, polling UI, no nginx management). + +## Tests + +- Unit test for the systemd unit generator (content assertions, no systemd interaction). +- Docs have no test, but verify every command in them by running it once. + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- A fresh clone can follow `docs/deployment.md` to a running service (manual walkthrough; document the result in this file). diff --git a/docs/plan/README.md b/docs/plan/README.md new file mode 100644 index 0000000..11c336c --- /dev/null +++ b/docs/plan/README.md @@ -0,0 +1,67 @@ +# GitTally Rewrite Plan + +This directory contains the step-by-step plan for rewriting `legacy/gitTally` (bash) as the Kotlin/Spring application in this repository. +Each step file is self-contained and sized for one focused Claude Code session. + +## How to Execute a Step + +Start a fresh Claude Code session and prompt, for example: "Execute docs/plan/01-build-state-domain.md". +The executing session should: + +1. Read this file, `00-legacy-analysis.md`, and the step file. +2. Read the referenced parts of `legacy/gitTally` only if the step file says so. +3. Implement with tests, following `CLAUDE.md` conventions. +4. Run `./gradlew ktlintFormat` and then `./gradlew build` until green. +5. Update the step's checkbox below and note deviations inside the step file. + +## Guiding Principles + +- The legacy script defines intended behavior, but it is buggy — treat it as a reference, not a spec. +- Fix the two known legacy defects by design, not by patching: + - Build status must be observable while a build runs (event-driven status transitions, async build execution). + - The web UI must never get stuck loading (JSON status endpoints with explicit error states instead of regex-rewritten HTML). +- Do not port orphaned or half-implemented legacy config options (see `00-legacy-analysis.md`). +- Every step leaves the build green and the application runnable. +- Config keys added by a step must be updated in three places: `GitTallyConfig`, the `InitCommand` templates, and `docs/configuration.md`. + +## Proposed Architecture Decisions + +These are proposals baked into the steps. +Revisit them in an ADR if a step uncovers problems. + +- Build results are persisted as a JSON file under `.git/gittally/`, behind a `BuildResultRepository` interface (no database, but replaceable). +- Artifacts stay on the filesystem, served by the Spring server. +- The web UI is server-rendered HTML plus small JavaScript polling JSON endpoints (no SPA framework). +- The watcher is a Spring-managed scheduled component, decoupled from the build executor via the result repository and events. +- nginx/Let's Encrypt container management is NOT ported; deployment behind an existing reverse proxy is documented instead. + +## Steps + +Foundation: + +- [ ] `01-build-state-domain.md` — build result domain model and persistent repository +- [ ] `02-git-gateway.md` — full git access layer (fetch, branches, commits, checkout) +- [ ] `03-gitea-client.md` — Gitea API client for commit statuses + +Core engine: + +- [ ] `04-build-executor.md` — async build execution with logs, cancellation, status transitions +- [ ] `05-artifact-store.md` — artifact persistence, naming, retention +- [ ] `06-watcher.md` — branch watching, scheduling, auto-builds + +Server and UI: + +- [ ] `07-server-mode.md` — `server` subcommand, REST/JSON endpoints, artifact serving +- [ ] `08-web-ui.md` — HTML views with robust live updates +- [ ] `09-system-metrics.md` — system resource monitoring page + +Completion: + +- [ ] `10-cli-commands.md` — CLI build/status commands +- [ ] `11-docker-build-runtime.md` — optional Docker build execution +- [ ] `12-deployment.md` — systemd service, migration from legacy, docs + +Steps 01–03 are independent of each other. +Steps 04–06 depend on 01–03. +Steps 07–09 depend on 04–06. +Steps 11 and 12 are optional/deferrable; 10 only needs 04–06.