update CLAUDE.md with rewrite plan documentation reference in docs/pland directory
This commit is contained in:
@@ -125,6 +125,7 @@ Keep sentences short.
|
|||||||
- `docs/GitTally-Konzept.md` — product concept and target architecture (in German): git-centric CI, builds in Docker, one instance per repository, status reported back to Gitea.
|
- `docs/GitTally-Konzept.md` — product concept and target architecture (in German): git-centric CI, builds in Docker, one instance per repository, status reported back to Gitea.
|
||||||
- `docs/configuration.md` — configuration reference; keep in sync with `GitTallyConfig` and the `init` templates.
|
- `docs/configuration.md` — configuration reference; keep in sync with `GitTallyConfig` and the `init` templates.
|
||||||
- `docs/bootstrapping.md` — how `init` prepares a repository.
|
- `docs/bootstrapping.md` — how `init` prepares a repository.
|
||||||
|
- `docs/plan/` — the step-by-step rewrite plan; `docs/plan/README.md` explains how to execute a step, `docs/plan/00-legacy-analysis.md` summarizes the legacy bash script.
|
||||||
|
|
||||||
## Key Architectural Decisions
|
## Key Architectural Decisions
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,95 @@
|
|||||||
|
# Legacy gitTally Analysis
|
||||||
|
|
||||||
|
Condensed analysis of `legacy/gitTally` (bash, ~6000 lines) as input for the rewrite.
|
||||||
|
Line numbers refer to the legacy script at the time of analysis (version 0.7.8).
|
||||||
|
|
||||||
|
## What the Legacy System Does
|
||||||
|
|
||||||
|
A single bash daemon per repository that:
|
||||||
|
|
||||||
|
1. Polls `origin` for changed local branches and recent new origin branches.
|
||||||
|
2. Checks out and builds each changed branch (natively or in Docker), one at a time.
|
||||||
|
3. Records results and publishes commit statuses to Gitea.
|
||||||
|
4. Archives build logs and report directories as browsable artifacts.
|
||||||
|
5. Serves a static-HTML web UI via an embedded Python HTTP server.
|
||||||
|
6. Optionally manages an nginx+certbot Docker container for HTTPS.
|
||||||
|
7. Installs itself as a systemd user service.
|
||||||
|
|
||||||
|
## Persistent State (formats to replace)
|
||||||
|
|
||||||
|
All state lives in files; there is no database.
|
||||||
|
|
||||||
|
| File | Format | Content |
|
||||||
|
|---|---|---|
|
||||||
|
| `.git/git-watch-origin-and-test/build-results.tsv` | TSV | `branch, commit, status, timestamp, duration(MM:SS), artifact_key` |
|
||||||
|
| `.git/git-watch-origin-and-test/auto-builds.tsv` | TSV | `branch, date, time_slot` — prevents double auto-builds |
|
||||||
|
| `.git/git-watch-origin-and-test/build.lock` | flock file | build mutex, holder found via `fuser`/`lsof` |
|
||||||
|
| `.git/git-watch-origin-and-test/cancel-*` | touch/token files | cancellation request/token/accepted handshake |
|
||||||
|
| `$TMPDIR/git-watch-origin-and-test/<repo_key>/` | HTML + files | artifact root: generated pages, per-build artifact dirs |
|
||||||
|
| `<artifact_root>/system.json` + `system_state.dat` | JSON + text | system metrics snapshot and aggregation state |
|
||||||
|
|
||||||
|
Build statuses: `pending`, `running`, `success`, `failed`, `interrupted`, `cancelled` (legacy alias `passed` → `success`).
|
||||||
|
|
||||||
|
Artifact key naming: sanitized branch + 12-char SHA256 prefix, plus sanitized timestamp + hash for per-build dirs.
|
||||||
|
|
||||||
|
## External Interactions
|
||||||
|
|
||||||
|
Git (always via CLI): `rev-parse`, `for-each-ref`, `fetch`, `switch`, `reset --hard`, `show -s --format=%cI`, `rev-list`.
|
||||||
|
HTTPS git auth uses a generated `GIT_ASKPASS` script feeding username + Gitea token.
|
||||||
|
|
||||||
|
Gitea API:
|
||||||
|
|
||||||
|
- `POST /api/v1/repos/{owner}/{repo}/statuses/{sha}` — publish status; payload `state`, `context`, `description`, `target_url`.
|
||||||
|
- `GET /api/v1/repos/{owner}/{repo}/commits/{sha}/statuses?sort=recentupdate` — read statuses, filtered by `context`.
|
||||||
|
- `GET /api/v1/user` — resolve username from token.
|
||||||
|
- State mapping: success→success; failed/interrupted/cancelled→failure; pending/running→pending.
|
||||||
|
|
||||||
|
Web control endpoints (Python handler):
|
||||||
|
|
||||||
|
- `GET /control/status?commit=<sha>&local_status=<s>` — proxy Gitea status for a commit.
|
||||||
|
- `POST /control/cancel` — cancel running build (CSRF-token protected).
|
||||||
|
- `POST /control/restart` — append a new pending result row.
|
||||||
|
- `POST /control/delete` — remove a result row and patch HTML files via regex.
|
||||||
|
|
||||||
|
## Root Causes of the Known Bugs
|
||||||
|
|
||||||
|
Stuck loading animation (UI):
|
||||||
|
|
||||||
|
- Status cells start as `status-loading` and each row fetches `/control/status` with a 15s timeout; failures leave the spinner forever (no error state).
|
||||||
|
- Page auto-refresh re-fetches the page HTML and diffs `#build-rows`; HTML is regenerated server-side by regex-patching files, which fails silently when structure drifts.
|
||||||
|
- "Current" and "System" nav views exist, but the page generator only implements `latest`/`branches`/`history`, so some views fall through.
|
||||||
|
|
||||||
|
No status changes observable during a build (control loop):
|
||||||
|
|
||||||
|
- The main loop (lines ~4944–4966) is synchronous: `checkout_and_build` blocks until the build ends; only then is origin re-scanned.
|
||||||
|
- `retry_origin_change_check()` is an uninterruptible internal retry loop with 10s sleeps.
|
||||||
|
- The console spinner is cosmetic and runs independently of actual progress.
|
||||||
|
|
||||||
|
## Behavior Worth Preserving
|
||||||
|
|
||||||
|
- Startup recovery: mark stale `running` builds as `interrupted`; restart restartable builds; optional retry of failed builds.
|
||||||
|
- New-branch age filter (`newBranchMaxAge`) to avoid building stale branches.
|
||||||
|
- Auto-build time slots (UTC HH:MM) per branch with per-day/slot dedup.
|
||||||
|
- Retention per branch by count or age, pruning both results and artifact dirs.
|
||||||
|
- Per-branch build/clean command, artifact dirs, and log file names (now via `branches:` YAML config).
|
||||||
|
- Cancellation with token handshake; process-tree termination (TERM, wait, KILL).
|
||||||
|
- Build log capture: stdout/stderr to files plus a live "current build" log.
|
||||||
|
- Gitea status published on every transition with `target_url` pointing at the artifact page.
|
||||||
|
|
||||||
|
## Not Ported (decided)
|
||||||
|
|
||||||
|
- nginx + certbot/Let's Encrypt container management — replaced by deployment documentation (step 12).
|
||||||
|
- Self-install (`--install`), self-update script generation — replaced by jar deployment plus systemd docs (step 12).
|
||||||
|
- Legacy `HSADMIN_NG_*` environment fallbacks and env-file config — replaced by YAML config (done).
|
||||||
|
- Regex-based in-place HTML patching — replaced by server-rendered pages/JSON endpoints.
|
||||||
|
- Static HTML artifact index generation with embedded 900-line JavaScript — replaced by templates.
|
||||||
|
- `.aliases` sourcing (line 731), impressum footer link handling stays optional/simple.
|
||||||
|
|
||||||
|
## Orphaned or Dubious Legacy Config
|
||||||
|
|
||||||
|
Verify need before porting any of these:
|
||||||
|
|
||||||
|
- `GITTALLY_BUILD_DOCKER_PREFLIGHT_COMMAND`, `GITTALLY_BUILD_DOCKER_JAVA_TOOL_OPTIONS` — highly hsadmin-ng-specific defaults.
|
||||||
|
- `GITTALLY_ARTIFACT_NGINX_*`, `GITTALLY_ARTIFACT_LETSENCRYPT_EMAIL` — dropped with nginx management.
|
||||||
|
- `GITTALLY_IMPRESSUM_URL` — keep as optional simple footer link if wanted.
|
||||||
|
- `GITTALLY_INSTALL_DIR` — dropped with self-install.
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
# Step 01: Build State Domain and Repository
|
||||||
|
|
||||||
|
Prerequisites: none.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
A tested domain model for build results plus a persistent repository, replacing the legacy `build-results.tsv`.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Create package `de.hoennig.gittally.build`:
|
||||||
|
|
||||||
|
- `BuildStatus` enum: `PENDING`, `RUNNING`, `SUCCESS`, `FAILED`, `INTERRUPTED`, `CANCELLED`.
|
||||||
|
Add `isTerminal`, `isRestartable` (pending/running/interrupted) properties.
|
||||||
|
- `BuildResult` data class: branch, commit SHA, status, startedAt, duration, artifactKey.
|
||||||
|
Use `java.time.Instant`/`Duration`; format only at the edges.
|
||||||
|
- `BuildResultRepository` interface: append, update status of latest entry for a branch, query latest per branch, query history, delete entry, prune.
|
||||||
|
- `FileBuildResultRepository`: JSON file at `.git/gittally/build-results.json`.
|
||||||
|
Write atomically (write temp file, then `Files.move` with `ATOMIC_MOVE`).
|
||||||
|
Reuse the Jackson YAML/JSON setup style from `ConfigLoader`.
|
||||||
|
|
||||||
|
Business logic to include (port from legacy, see analysis):
|
||||||
|
|
||||||
|
- `markStaleRunningAsInterrupted()` — called at startup; running → interrupted, superseded pending → interrupted.
|
||||||
|
- Retention pruning: keep N builds per branch (`artifacts.retentionPerBranch`); drop entries for branches no longer on origin (branch list passed in as a parameter, no git dependency here).
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- No git access, no Gitea, no execution — pure domain and file I/O.
|
||||||
|
- Artifact directory pruning (step 05 consumes the pruning result).
|
||||||
|
- Age-based retention (legacy supported `h`/`d` suffixes); count-based only, extend later if needed.
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
Uses existing `artifacts.retentionPerBranch`.
|
||||||
|
No new keys expected.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
Kotest `FunSpec`, no Spring context needed.
|
||||||
|
|
||||||
|
- Round-trip persistence, atomicity (temp file cleaned up).
|
||||||
|
- Status transition helpers and `markStaleRunningAsInterrupted` edge cases.
|
||||||
|
- Retention pruning: per-branch count, removed branches, ordering by timestamp.
|
||||||
|
- Corrupt/missing file → empty repository, no crash (legacy failed silently; we log a warning).
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- New code has no dependency on picocli or web classes.
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# Step 02: Git Gateway
|
||||||
|
|
||||||
|
Prerequisites: none.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Extend `GitService` into a complete, tested gateway for all git operations the watcher and builder need.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Keep the existing approach: shell out to the `git` CLI via `ProcessBuilder`.
|
||||||
|
Extract a small `GitCommandRunner` (command + workingDir → exit code, stdout, stderr) so `GitService` becomes testable and readable.
|
||||||
|
|
||||||
|
Operations to add (legacy references in parentheses):
|
||||||
|
|
||||||
|
- `fetchOrigin()` and `fetchBranch(branch)` with authentication (legacy `git_with_gitea_token`, askpass).
|
||||||
|
Use the `GIT_ASKPASS` environment technique: write a temp script returning `git.account` / `git.token` from config; `GIT_TERMINAL_PROMPT=0`.
|
||||||
|
Only needed for HTTPS origins; skip auth setup for SSH origins.
|
||||||
|
- `localBranches()`, `originBranches()` (legacy `branch_candidates`).
|
||||||
|
- `hasNewCommits(branch)` — compare local head with upstream/origin (legacy `branch_has_new_commits`, `has_new_commits` via `rev-list`).
|
||||||
|
- `newOriginBranches(maxAge)` — origin branches without local counterpart whose latest commit is younger than `watcher.newBranchMaxAge` (legacy `recent_new_origin_branches`).
|
||||||
|
- `checkout(branch)` — switch, or create tracking branch from origin (legacy `switch_to_branch`).
|
||||||
|
- `resetHardToOrigin(branch)` (legacy `pull_branch_if_possible`).
|
||||||
|
- `commitTimestamp(sha)`, `currentBranch()`, `headCommit()`.
|
||||||
|
|
||||||
|
Parse the `newBranchMaxAge` duration format (`5d`, `12h`) in a small dedicated parser with tests.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- No polling loop (step 06), no build triggering.
|
||||||
|
- No Gitea API calls (step 03); only the askpass credential bridge is set up here.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- `GitCommandRunner` unit tests.
|
||||||
|
- Integration tests against local fixture repositories: create a bare "origin" repo and a clone in a temp dir via the runner itself, then exercise fetch/branch/commit operations.
|
||||||
|
No network access needed.
|
||||||
|
- Duration parser edge cases.
|
||||||
|
- Askpass script content test (do not test against a real remote).
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- No temp askpass files leak (created per invocation or cleaned via try/finally; legacy leaked them).
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
# Step 03: Gitea Client
|
||||||
|
|
||||||
|
Prerequisites: step 01 (for `BuildStatus`).
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
A tested client for the Gitea commit-status API.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Create package `de.hoennig.gittally.gitea`:
|
||||||
|
|
||||||
|
- `GiteaClient` using Spring's `RestClient`.
|
||||||
|
- `publishStatus(sha, state, description, targetUrl)` → `POST /api/v1/repos/{owner}/{repo}/statuses/{sha}` with header `Authorization: token <git.token>`; body fields `state`, `context`, `description`, `target_url`.
|
||||||
|
- `readStatus(sha)` → `GET /api/v1/repos/{owner}/{repo}/commits/{sha}/statuses?sort=recentupdate`; pick the newest entry matching `gitea.statusContext`.
|
||||||
|
- `resolveUsername()` → `GET /api/v1/user` (used as fallback for `git.account`).
|
||||||
|
- State mapping in both directions (`BuildStatus` ↔ Gitea `success|failure|pending|error`), as in the legacy analysis.
|
||||||
|
- `isEnabled()` — true only when `gitea.baseUrl`, `gitea.owner`, `gitea.repo`, and `git.token` are configured.
|
||||||
|
All callers must treat a disabled or failing client as non-fatal (log and continue); the legacy behaved the same but failed silently.
|
||||||
|
|
||||||
|
Configuration comes from `GitTallyConfig` (`gitea.*`, `git.token`).
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- No callers yet; the build executor (step 04) wires status publishing.
|
||||||
|
- No webhook receiving; GitTally remains poll-based.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
WireMock (already a test dependency, see `WireMockSmokeTest`):
|
||||||
|
|
||||||
|
- Publish: correct URL, auth header, JSON body per status.
|
||||||
|
- Read: filtering by context, newest-first, empty result, malformed JSON → error status, HTTP 4xx/5xx → non-fatal error result.
|
||||||
|
- State-mapping unit tests.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- No call path throws when Gitea is unconfigured or down.
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
# Step 04: Build Executor
|
||||||
|
|
||||||
|
Prerequisites: steps 01, 02, 03.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Asynchronous build execution with log capture, cancellation, and immediately visible status transitions.
|
||||||
|
This step fixes the legacy defect that nothing could observe status changes while a build ran.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Create package `de.hoennig.gittally.build` (extends step 01):
|
||||||
|
|
||||||
|
- `BuildExecutor` service; one build at a time (a `ReentrantLock` or single-thread executor replaces the legacy flock file).
|
||||||
|
- `startBuild(branch, commit)` runs asynchronously and returns immediately; expose `currentBuild(): RunningBuild?`.
|
||||||
|
- Execution sequence per build:
|
||||||
|
1. Record `PENDING`, then `RUNNING` via `BuildResultRepository`; publish each transition to Gitea (non-fatal on failure).
|
||||||
|
2. Run the branch's `cleanCommand`, then `buildCommand` (from the merged `branches` config) via `ProcessBuilder` with the branch name in the environment as `branch`.
|
||||||
|
3. Stream stdout/stderr to the configured log files in a working/staging directory, plus a combined live log file.
|
||||||
|
4. On exit: record `SUCCESS`/`FAILED`, publish status, hand the staging directory to the artifact store (step 05 interface; use a stub interface now if 05 is not done).
|
||||||
|
- Cancellation: `cancel()` flag checked by a monitor; destroy the process tree (`ProcessHandle.descendants()`, TERM-wait-KILL like legacy `terminate_process_tree`); record `CANCELLED`.
|
||||||
|
- Status transitions must be readable at any time via the repository — no in-memory-only state.
|
||||||
|
|
||||||
|
Emit a Spring `ApplicationEvent` on every status transition so the UI (step 08) can later push updates without polling internals.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- Docker execution (step 11); native `ProcessBuilder` only, but keep a `BuildRunner` interface so Docker can plug in.
|
||||||
|
- Scheduling and branch selection (step 06).
|
||||||
|
- Artifact index HTML (step 05/08).
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
Uses existing `branches.<name>.buildCommand/cleanCommand/stdoutLog/stderrLog`.
|
||||||
|
Consider `builds.timeout` only if trivial; otherwise defer.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Fake commands (`sh -c 'echo ok'`, failing command, sleeping command) in temp dirs.
|
||||||
|
- Status sequence assertions: pending → running → success/failed/cancelled, each persisted before/after execution.
|
||||||
|
- Cancellation kills a sleeping process tree and records `CANCELLED`.
|
||||||
|
- Log files contain captured output; live log grows during the build (poll in test).
|
||||||
|
- Gitea publishing mocked with MockK; a Gitea failure must not fail the build.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- While a test build sleeps, the repository reports `RUNNING` — proven by a test.
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# Step 05: Artifact Store
|
||||||
|
|
||||||
|
Prerequisites: steps 01, 04.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Persist build artifacts (logs plus configured report directories) with stable naming and retention pruning.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Create package `de.hoennig.gittally.artifacts`:
|
||||||
|
|
||||||
|
- `ArtifactStore` service implementing the interface stubbed in step 04.
|
||||||
|
- Artifact root: a configurable directory (new key `artifacts.rootDir`), defaulting to `${XDG_STATE_HOME:-~/.local/state}/gittally/artifacts/<repo-key>`.
|
||||||
|
Do NOT default to `/tmp` like legacy — artifacts vanished on reboot.
|
||||||
|
- Repo key: sanitized absolute repo path (legacy `repository_key`): non `[A-Za-z0-9._-]` → `_`.
|
||||||
|
- Artifact key per build: sanitized branch name + 12-char SHA-256 prefix, plus sanitized start timestamp + hash (legacy `build_artifact_key`); keep this scheme so URLs stay predictable.
|
||||||
|
- `persist(build, stagingDir)`: copy configured `artifactDirs` and the log files into staging, then atomically move staging → `<root>/branches/<artifactKey>/`.
|
||||||
|
- `prune(keptResults)`: delete artifact directories whose keys are no longer in the result repository (call after repository retention pruning from step 01).
|
||||||
|
- Provide `artifactDir(artifactKey)` lookups for the server (step 07).
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- HTTP serving (step 07).
|
||||||
|
- HTML index pages (step 08 renders from data instead of generated files).
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
New key `artifacts.rootDir` (empty = platform default above).
|
||||||
|
Update `GitTallyConfig`, `InitCommand` templates, and `docs/configuration.md` together.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Key naming: sanitization, hash stability, collision of similar branch names.
|
||||||
|
- Persist: copies configured dirs, missing artifact dirs are skipped with a log line, atomic move (no partial dirs on failure).
|
||||||
|
- Prune: deletes exactly the unreferenced dirs, never anything outside the artifact root.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- Step 04's executor persists artifacts through the real store (integration test).
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# Step 06: Watcher and Scheduling
|
||||||
|
|
||||||
|
Prerequisites: steps 01, 02, 04.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Replace the legacy blocking main loop with a non-blocking, observable scheduler.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Create package `de.hoennig.gittally.watcher`:
|
||||||
|
|
||||||
|
- `Watcher` component with a fixed-delay poll cycle (Spring `@Scheduled` or a managed executor; enabled only in server/watch mode, not during CLI commands or tests).
|
||||||
|
- One poll cycle, never blocking on a build:
|
||||||
|
1. `fetchOrigin()` (errors: log, publish watcher health state, retry next cycle — no internal retry-sleep loops like legacy `retry_origin_change_check`).
|
||||||
|
2. Determine candidate branches: changed local branches, plus new origin branches within `watcher.newBranchMaxAge` (step 02 operations).
|
||||||
|
3. Check auto-build time slots (below).
|
||||||
|
4. If the executor is idle, dequeue the next branch: checkout, reset to origin, `startBuild` (async).
|
||||||
|
5. Run repository retention pruning and artifact pruning.
|
||||||
|
- Startup sequence (port of legacy recovery): mark stale running builds interrupted, then enqueue restartable branches.
|
||||||
|
- Auto-builds: per-branch `autoBuild.enabled` + `times` (UTC HH:MM) from the merged `branches` config.
|
||||||
|
Persist "already triggered for slot/day" state via a small JSON file next to the build results (replaces `auto-builds.tsv`).
|
||||||
|
- Expose watcher state (last poll time, last fetch error, queue) for the UI/status endpoints.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- HTTP endpoints (step 07); expose state via a service bean only.
|
||||||
|
- Retry-failed-builds command (step 10 triggers it through the same queue).
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
New key `watcher.pollInterval` (e.g. `10s`, default matching legacy cadence).
|
||||||
|
Update `GitTallyConfig`, `InitCommand` templates, and `docs/configuration.md` together.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Poll cycle unit tests with MockK: branch selection precedence, skip-when-building, fetch failure resilience.
|
||||||
|
- Auto-build slot logic: slot matching, per-day dedup, state persistence.
|
||||||
|
- New-branch age filtering.
|
||||||
|
- Startup recovery: interrupted marking and re-enqueue (integration with steps 01/04 fakes).
|
||||||
|
- No test may sleep for real poll intervals; trigger cycles directly.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- A test proves a poll cycle completes while a (fake) build is running.
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# Step 07: Server Mode and HTTP API
|
||||||
|
|
||||||
|
Prerequisites: steps 04, 05, 06.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Implement the `server` subcommand: a persistent web server exposing build state as JSON and serving artifacts.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Bootstrapping:
|
||||||
|
|
||||||
|
- `CLAUDE.md` notes the context starts with web type `none`; the `server` subcommand must run a web context.
|
||||||
|
Preferred approach: `ServerCommand` launches a second `SpringApplication` with `WebApplicationType.SERVLET` and a `server` profile, then blocks until shutdown.
|
||||||
|
Document the chosen mechanism in the code and, if it deviates, in an ADR.
|
||||||
|
- Add `spring-boot-starter-web` dependency.
|
||||||
|
- The watcher (step 06) is active only in the `server` profile.
|
||||||
|
- New config keys `server.port` and `server.bindAddress` (defaults 18080 / 0.0.0.0, as legacy).
|
||||||
|
|
||||||
|
JSON API (package `de.hoennig.gittally.server`), replacing the legacy `/control/*` endpoints:
|
||||||
|
|
||||||
|
- `GET /api/builds/latest` — latest build per branch.
|
||||||
|
- `GET /api/builds/history` — all builds, newest first.
|
||||||
|
- `GET /api/builds/current` — running build, its live status, and log tail (`?offset=` for incremental log fetch).
|
||||||
|
- `GET /api/status/{commit}` — effective status including Gitea lookup (replaces `/control/status`); must return an explicit error state on Gitea failure, never hang.
|
||||||
|
- `POST /api/builds/{branch}/restart`, `POST /api/builds/current/cancel`, `DELETE /api/builds/{artifactKey}` — guarded by a simple token like the legacy cancel token; wire into executor/watcher/repository.
|
||||||
|
- `GET /api/watcher` — watcher health (last poll, last error).
|
||||||
|
- Artifact serving: `GET /artifacts/{artifactKey}/**` streaming from the artifact store, with no-cache headers for html/json/log.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- HTML pages (step 08); JSON plus artifact files only.
|
||||||
|
- TLS/reverse proxy (documented in step 12).
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- `@WebMvcTest` slices with `@MockkBean` per controller (see `CLAUDE.md` conventions).
|
||||||
|
- Contract tests: JSON shapes, error states (Gitea down → explicit `unknown` status, HTTP 200), cancel token rejection.
|
||||||
|
- One `@SpringBootTest` on the server profile proving the context boots with watcher and web enabled.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- `java -jar ... server` starts, `GET /api/builds/latest` answers, Ctrl-C shuts down cleanly (manual smoke test; document result in this file).
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
# Step 08: Web UI
|
||||||
|
|
||||||
|
Prerequisites: step 07.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
The browsable UI: build overview, history, current build with live log, per-build artifact index.
|
||||||
|
This step fixes the legacy stuck-loading-animation defect.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Server-rendered Thymeleaf templates plus a small, hand-written JavaScript file polling the step 07 JSON API.
|
||||||
|
No SPA framework, no build pipeline for the frontend.
|
||||||
|
|
||||||
|
Views (ported from legacy, see analysis for columns and behavior):
|
||||||
|
|
||||||
|
- `/` (Latest): latest build per branch — status badge, branch and commit with Gitea links and copy buttons, times, duration, artifact link, restart/delete actions.
|
||||||
|
- `/history`: all builds.
|
||||||
|
- `/current`: running build with live log view (incremental fetch via `/api/builds/current?offset=`), cancel button, and a clear "no build running" state.
|
||||||
|
- `/builds/{artifactKey}`: artifact index — build command, logs, links into archived report directories (rendered from the artifact store, not pre-generated HTML).
|
||||||
|
- Shared layout: view toggle nav, footer with version and optional impressum link, `prefers-color-scheme` support (port the legacy CSS look loosely, keep it simple).
|
||||||
|
|
||||||
|
Robust live updates (the actual bug fix):
|
||||||
|
|
||||||
|
- Poll JSON endpoints on an interval (10–15s) and re-render table bodies from data; never re-fetch and diff whole HTML pages.
|
||||||
|
- Every async fetch has a timeout and renders an explicit error/"unknown" badge on failure — no permanent spinners by construction.
|
||||||
|
- Pause polling when the tab is hidden (`visibilitychange`), resume and refresh immediately when visible.
|
||||||
|
- Running durations tick client-side from a `data-started-at` attribute.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- System metrics page (step 09).
|
||||||
|
- WebSocket/SSE push; plain polling first, extend later if wanted.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- MockMvc view tests: templates render for empty state, running build, mixed history.
|
||||||
|
- Escaping test: branch names with HTML/JS metacharacters render safely (legacy had injection risks).
|
||||||
|
- JavaScript logic that is non-trivial (duration formatting, poll scheduling) should live in small pure functions; test via MockMvc-rendered attributes or keep trivially simple.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- Manual smoke test with a real build: status flips pending → running → success in the open browser tab without a manual reload, and a killed server results in error badges, not spinners.
|
||||||
|
Document the result in this file.
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
# Step 09: System Metrics
|
||||||
|
|
||||||
|
Prerequisites: step 07 (API), step 08 (layout).
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Port the legacy system page: CPU, RAM, disk, and repository size with min/max/avg aggregation.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Create package `de.hoennig.gittally.metrics`:
|
||||||
|
|
||||||
|
- `SystemMetricsCollector` sampling every 60s (server profile only):
|
||||||
|
CPU used/idle from `/proc/stat` deltas, RAM from `/proc/meminfo`, disk from `java.nio.file.FileStore`, repo size via periodic `du -sk` (or a file walk) — throttle repo-size sampling (legacy ran `du` every cycle, which was expensive).
|
||||||
|
- Keep running min/max/avg per metric since server start; persist aggregation state in the artifact root so restarts continue the series (legacy `system_state.dat`, but as JSON).
|
||||||
|
- `GET /api/system` returning the current snapshot plus aggregates (legacy `system.json` fields are the reference).
|
||||||
|
- `/system` HTML view in the step 08 layout, polling `/api/system` every 60s with the same error-badge rules.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- Alerting, historical time series, external monitoring integration.
|
||||||
|
- Windows/macOS support beyond graceful degradation (missing `/proc` → metric shows "n/a").
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Collector unit tests with fake `/proc` file content (read paths injectable).
|
||||||
|
- Aggregation math: min/max/avg over samples, persistence round-trip.
|
||||||
|
- Controller slice test for `/api/system`.
|
||||||
|
- Graceful degradation when a source is unreadable.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- `/system` renders live values on Linux (manual smoke test; document in this file).
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
# Step 10: CLI Commands
|
||||||
|
|
||||||
|
Prerequisites: steps 04, 05, 06.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
CLI parity for interactive use without the server.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
New picocli subcommands (Spring components in `commands/`, wired like the existing ones):
|
||||||
|
|
||||||
|
- `status` — print the latest build per branch as a table (branch, status, commit, time, duration); `--history` for all builds.
|
||||||
|
Reads `BuildResultRepository` directly; works while a server instance runs (file-based store, read-only).
|
||||||
|
- `build [<branch>]` — one-shot: fetch, checkout, build the given branch (default: current), print the log to the console, exit with the build's exit code.
|
||||||
|
Replaces legacy `--stay` and explicit branch arguments.
|
||||||
|
Support the legacy partial-name matching: resolve unique branch-name fragments, list candidates on ambiguity (legacy `resolve_branch_name`).
|
||||||
|
- `retry` — re-enqueue/build branches whose latest build failed (legacy `--retry`).
|
||||||
|
- Update the root command's subcommand list and `--help` texts.
|
||||||
|
|
||||||
|
Exit codes: 0 on success, 1 on build failure, 2 on usage/config errors (align with `CliRunner`'s `ExitCodeGenerator` contract).
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- Talking to a running server over HTTP; direct file/executor access is sufficient for now.
|
||||||
|
- Shell completion.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Command unit tests with MockK-ed services (pattern of `InitCommandTest`).
|
||||||
|
- Branch fragment resolution: unique match, ambiguity handling, no match.
|
||||||
|
- Exit code assertions.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- `java -jar ... status` and `java -jar ... build` work in this repository (manual smoke test; document in this file).
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# Step 11: Docker Build Runtime (optional)
|
||||||
|
|
||||||
|
Prerequisites: step 04.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
This step is optional; skip it until builds actually need container isolation.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Run build commands inside a Docker container, as the legacy `--docker` mode did.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Implement a `DockerBuildRunner` for the `BuildRunner` interface from step 04, shelling out to the `docker` CLI (consistent with the git gateway approach; no Docker Java SDK dependency).
|
||||||
|
|
||||||
|
Port from legacy (see analysis, lines ~4400+):
|
||||||
|
|
||||||
|
- Ensure image: build from configured Dockerfile/context when missing or stale; track staleness via an image label holding the SHA-256 of Dockerfile + context (legacy `org.gittally.build-inputs-sha256`).
|
||||||
|
- Gradle cache volume per repository (`gittally-gradle-<repo-key>`), mounted and chowned to the host UID/GID.
|
||||||
|
- Run the build container: workspace mount, branch env var, configured extra env, network mode, docker socket mount for Testcontainers-based builds.
|
||||||
|
- Post-build ownership repair of the workspace (legacy `repair_docker_workspace_ownership`).
|
||||||
|
- Label all containers (`org.hoennig.gittally=true`, repository, role) and clean up stale ones on startup.
|
||||||
|
|
||||||
|
Decide during implementation whether the hsadmin-ng-specific legacy options (preflight command, `JAVA_TOOL_OPTIONS` injection) are needed; default to NOT porting them (see orphaned-config list in the analysis).
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
New `branches.<name>.docker` section: `enabled`, `image`, `dockerfile`, `context`, `network`, `env`.
|
||||||
|
Update `GitTallyConfig`, `InitCommand` templates, and `docs/configuration.md` together.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Unit tests for command assembly (assert the exact `docker run`/`docker build` argv) with a mocked command runner.
|
||||||
|
- Checksum staleness logic.
|
||||||
|
- Optional: one Testcontainers-gated integration test that runs a trivial build in a stock image; skip when Docker is unavailable.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green with Docker absent.
|
||||||
|
- Native execution path (step 04) is unchanged and remains the default.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Step 12: Deployment and Legacy Migration
|
||||||
|
|
||||||
|
Prerequisites: steps 07, 08, 10.
|
||||||
|
Read `README.md` and `00-legacy-analysis.md` first.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Make the new GitTally deployable as a service and retire the legacy script.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
Deployment (documentation plus a small generator, no self-install):
|
||||||
|
|
||||||
|
- Extend `init` (or add `init --systemd`) to generate a systemd user unit running `java -jar gittally.jar server` with `WorkingDirectory` set to the repo, `Restart=always`, and an `EnvironmentFile` for overrides — port the shape of the legacy unit, drop the self-copy/update machinery.
|
||||||
|
- Write `docs/deployment.md`: JRE requirement, jar location convention, systemd enable/start/log commands, and reverse-proxy guidance (example nginx `server` block proxying to `server.port`; TLS via the host's existing certbot — replaces the legacy managed nginx container).
|
||||||
|
|
||||||
|
Migration:
|
||||||
|
|
||||||
|
- Write `docs/migration-from-legacy.md`: mapping table legacy env vars → YAML keys (source: `00-legacy-analysis.md` and legacy `--env` output), what is intentionally not ported, and the manual steps (stop legacy service, run `init`, fill in token, install new service).
|
||||||
|
- Decide and document: no automatic import of `build-results.tsv` (history starts fresh) unless trivially cheap.
|
||||||
|
|
||||||
|
Housekeeping:
|
||||||
|
|
||||||
|
- Mark `legacy/gitTally` as deprecated in its header comment and in `README.md`.
|
||||||
|
- Review `docs/GitTally-Konzept.md` against what was actually built; update or note deviations.
|
||||||
|
- Add an ADR summarizing the architecture decisions that emerged during the rewrite (persistence choice, polling UI, no nginx management).
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Unit test for the systemd unit generator (content assertions, no systemd interaction).
|
||||||
|
- Docs have no test, but verify every command in them by running it once.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `./gradlew ktlintFormat` then `./gradlew build` is green.
|
||||||
|
- A fresh clone can follow `docs/deployment.md` to a running service (manual walkthrough; document the result in this file).
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
# GitTally Rewrite Plan
|
||||||
|
|
||||||
|
This directory contains the step-by-step plan for rewriting `legacy/gitTally` (bash) as the Kotlin/Spring application in this repository.
|
||||||
|
Each step file is self-contained and sized for one focused Claude Code session.
|
||||||
|
|
||||||
|
## How to Execute a Step
|
||||||
|
|
||||||
|
Start a fresh Claude Code session and prompt, for example: "Execute docs/plan/01-build-state-domain.md".
|
||||||
|
The executing session should:
|
||||||
|
|
||||||
|
1. Read this file, `00-legacy-analysis.md`, and the step file.
|
||||||
|
2. Read the referenced parts of `legacy/gitTally` only if the step file says so.
|
||||||
|
3. Implement with tests, following `CLAUDE.md` conventions.
|
||||||
|
4. Run `./gradlew ktlintFormat` and then `./gradlew build` until green.
|
||||||
|
5. Update the step's checkbox below and note deviations inside the step file.
|
||||||
|
|
||||||
|
## Guiding Principles
|
||||||
|
|
||||||
|
- The legacy script defines intended behavior, but it is buggy — treat it as a reference, not a spec.
|
||||||
|
- Fix the two known legacy defects by design, not by patching:
|
||||||
|
- Build status must be observable while a build runs (event-driven status transitions, async build execution).
|
||||||
|
- The web UI must never get stuck loading (JSON status endpoints with explicit error states instead of regex-rewritten HTML).
|
||||||
|
- Do not port orphaned or half-implemented legacy config options (see `00-legacy-analysis.md`).
|
||||||
|
- Every step leaves the build green and the application runnable.
|
||||||
|
- Config keys added by a step must be updated in three places: `GitTallyConfig`, the `InitCommand` templates, and `docs/configuration.md`.
|
||||||
|
|
||||||
|
## Proposed Architecture Decisions
|
||||||
|
|
||||||
|
These are proposals baked into the steps.
|
||||||
|
Revisit them in an ADR if a step uncovers problems.
|
||||||
|
|
||||||
|
- Build results are persisted as a JSON file under `.git/gittally/`, behind a `BuildResultRepository` interface (no database, but replaceable).
|
||||||
|
- Artifacts stay on the filesystem, served by the Spring server.
|
||||||
|
- The web UI is server-rendered HTML plus small JavaScript polling JSON endpoints (no SPA framework).
|
||||||
|
- The watcher is a Spring-managed scheduled component, decoupled from the build executor via the result repository and events.
|
||||||
|
- nginx/Let's Encrypt container management is NOT ported; deployment behind an existing reverse proxy is documented instead.
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
Foundation:
|
||||||
|
|
||||||
|
- [ ] `01-build-state-domain.md` — build result domain model and persistent repository
|
||||||
|
- [ ] `02-git-gateway.md` — full git access layer (fetch, branches, commits, checkout)
|
||||||
|
- [ ] `03-gitea-client.md` — Gitea API client for commit statuses
|
||||||
|
|
||||||
|
Core engine:
|
||||||
|
|
||||||
|
- [ ] `04-build-executor.md` — async build execution with logs, cancellation, status transitions
|
||||||
|
- [ ] `05-artifact-store.md` — artifact persistence, naming, retention
|
||||||
|
- [ ] `06-watcher.md` — branch watching, scheduling, auto-builds
|
||||||
|
|
||||||
|
Server and UI:
|
||||||
|
|
||||||
|
- [ ] `07-server-mode.md` — `server` subcommand, REST/JSON endpoints, artifact serving
|
||||||
|
- [ ] `08-web-ui.md` — HTML views with robust live updates
|
||||||
|
- [ ] `09-system-metrics.md` — system resource monitoring page
|
||||||
|
|
||||||
|
Completion:
|
||||||
|
|
||||||
|
- [ ] `10-cli-commands.md` — CLI build/status commands
|
||||||
|
- [ ] `11-docker-build-runtime.md` — optional Docker build execution
|
||||||
|
- [ ] `12-deployment.md` — systemd service, migration from legacy, docs
|
||||||
|
|
||||||
|
Steps 01–03 are independent of each other.
|
||||||
|
Steps 04–06 depend on 01–03.
|
||||||
|
Steps 07–09 depend on 04–06.
|
||||||
|
Steps 11 and 12 are optional/deferrable; 10 only needs 04–06.
|
||||||
Reference in New Issue
Block a user