implemented 12-deployment.md: added systemd service generation (init --systemd) and migration guide from legacy script; introduced JSON-file persistence, server-rendered UI with polling, and reverse-proxy-based deployment; updated documentation

This commit is contained in:
Michael Hoennig
2026-07-07 14:18:41 +02:00
parent e869b46cbf
commit 60ff595a9d
14 changed files with 578 additions and 5 deletions
+6 -2
View File
@@ -9,7 +9,7 @@ Ziel ist es, die Komplexität klassischer CI-Systeme wie Jenkins erheblich zu re
## Grundprinzipien
- Git wird immer verwendet.
- Builds laufen in Docker.
- Builds laufen nativ oder optional in Docker (pro Branch konfigurierbar).
- Die Konfiguration erfolgt primär über YAML-Dateien.
- Eine Instanz verwaltet zunächst genau ein Repository.
- Build-Status werden an das Git-System zurückgemeldet.
@@ -20,6 +20,7 @@ Ziel ist es, die Komplexität klassischer CI-Systeme wie Jenkins erheblich zu re
- Interaktive Nutzung
- Status anzeigen
- Builds starten und wiederholen
- Konfiguration anzeigen
- Initialisierung durchführen
@@ -110,6 +111,7 @@ flowchart LR
### CLIFrontend
- Status anzeigen
- Builds starten und wiederholen
- Konfiguration anzeigen
- Initialisierung durchführen
@@ -168,6 +170,8 @@ java -jar build/libs/gittally-0.1.0-SNAPSHOT.jar init
java -jar build/libs/gittally-0.1.0-SNAPSHOT.jar server
```
Für den Dauerbetrieb als systemd-User-Service siehe [deployment.md](deployment.md) (`init --systemd`).
### Konfigurationsanzeige
```bash
@@ -177,7 +181,7 @@ java -jar build/libs/gittally-0.1.0-SNAPSHOT.jar config:print --full
## Erweiterungen
- Deployment / CD
- Continuous Delivery (CD; das Deployment von GitTally selbst ist in [deployment.md](deployment.md) beschrieben)
- SQLite statt Dateisystem
- Mehrere BuildWorker
- Multi-Repository-Verwaltung
@@ -0,0 +1,77 @@
# Rewrite Architecture Decisions
**Status:**
- proposed: 2026-07-07
- accepted: 2026-07-07
- rejected: -
- superseded: -
**Decision [accepted]:** JSON-file persistence behind a repository interface, server-rendered UI with JSON polling, no managed nginx/TLS — deployment via systemd user unit behind the host's reverse proxy.
## Context and Problem Statement
The rewrite of `legacy/gitTally` (bash) as a Kotlin/Spring application (see `docs/plan/`) required several cross-cutting architecture decisions.
They were proposed in `docs/plan/README.md`, validated step by step during implementation, and are summarized here as one record.
### Technical Background
The legacy script kept all state in TSV/HTML files, patched its web UI with regex rewrites, and managed its own nginx+certbot Docker container.
Its two structural defects — build status not observable during a build, and a web UI that could get stuck loading forever — had to be fixed by design, not by patching.
## Considered Options
* Database persistence (SQLite) vs. JSON files behind a repository interface
* SPA frontend or server push (WebSocket/SSE) vs. server-rendered HTML with JSON polling
* Managed nginx/Let's Encrypt container vs. documented deployment behind an existing reverse proxy
### Persistence: JSON files behind `BuildResultRepository`
Build results are persisted as a JSON file under `.git/gittally/`, accessed only through the `BuildResultRepository` interface.
#### Advantages
- No database dependency, no schema migrations; state stays inspectable with a text editor, like legacy.
- The volume is tiny (retention prunes per branch), so file rewrites are cheap.
- The interface keeps a later switch to SQLite possible without touching callers.
#### Disadvantages
- No queries or transactions; concurrent writers must be serialized in-process.
### Web UI: server-rendered Thymeleaf plus JSON polling
Pages render the full state server-side; one hand-written JavaScript file polls JSON endpoints and re-renders table bodies.
#### Advantages
- No SPA framework and no frontend build pipeline.
- Fixes the legacy stuck-spinner defect by design: every fetch has a timeout and an explicit error badge, and status transitions are event-driven and observable while a build runs.
- Pages stay useful without JavaScript (initial render is complete).
#### Disadvantages
- Updates are only as fresh as the polling interval; no server push.
### Deployment: no managed nginx, systemd user unit instead
nginx/Let's Encrypt container management was not ported; `init --systemd` generates a user unit running `java -jar gittally.jar server`, and `docs/deployment.md` documents the reverse-proxy setup with the host's certbot.
#### Advantages
- Removes the largest and most brittle legacy subsystem (container lifecycle, certificate renewal, config templating).
- Hosts usually already run a web server with TLS; one `server` block suffices.
- The generated unit replaces the legacy self-copy/self-update machinery with plain jar deployment.
#### Disadvantages
- HTTPS setup is a manual, host-specific step outside GitTally's control.
## Decision Outcome
All three proposals from the plan were confirmed during implementation and are in force:
- JSON-file persistence behind `BuildResultRepository` (steps 01, 05, 06).
- Server-rendered HTML with polling JSON endpoints and explicit error states (steps 0709).
- No nginx management; systemd user unit plus reverse-proxy documentation (step 12).
Related, previously decided in the same spirit: external systems are accessed by shelling out to the `git` and `docker` CLIs instead of SDK dependencies (steps 02, 11).
+3 -1
View File
@@ -1,7 +1,8 @@
# GitTally Bootstrapping
Bootstrapping prepares a git repository for use with GitTally.
It creates the config files described in [configuration.md](configuration.md) and optionally installs GitTally as a system service.
It creates the config files described in [configuration.md](configuration.md).
With `init --systemd` it also generates a systemd user unit for running the server permanently, see [deployment.md](deployment.md).
Run `init` once per repository, from within a checked-out working tree.
@@ -125,6 +126,7 @@ Until then, a Java runtime must be available on the host.
```bash
java -jar build/libs/gittally-0.1.0-SNAPSHOT.jar server
```
5. For permanent operation, install the systemd user service described in [deployment.md](deployment.md).
## Example: Self-Hosting GitTally
+124
View File
@@ -0,0 +1,124 @@
# GitTally Deployment
This document describes how to run GitTally as a permanent service.
The recommended setup is a systemd user service behind an existing reverse proxy.
GitTally does not manage nginx or TLS certificates itself (unlike the legacy script); it relies on the host's existing web server and certbot.
## Prerequisites
- Linux with systemd.
- Java runtime (JRE 21).
- `git` CLI on the `PATH`.
- `docker` CLI on the `PATH`, only if any branch uses `docker.enabled` (see [configuration.md](configuration.md)).
- A checked-out working tree of the repository to watch, with a remote named `origin`.
## Jar Location Convention
Build the executable jar once:
```bash
./gradlew build
ls build/libs/gittally-*-SNAPSHOT.jar
```
Copy the jar to a stable path outside any watched repository, by convention `~/bin/gittally.jar`:
```bash
mkdir -p ~/bin
cp build/libs/gittally-0.1.0-SNAPSHOT.jar ~/bin/gittally.jar
```
The systemd unit generated below points at the jar that was used to run `init --systemd`.
So always run it via the stable path, not via `build/libs/`.
## Install the Service
Initialize GitTally in the repository to watch (see [bootstrapping.md](bootstrapping.md) for details):
```bash
cd /path/to/repo
java -jar ~/bin/gittally.jar init
# fill in git.account and git.token in .git/gittally/.gittally.yml
# review .gittally.yml
```
Generate the systemd user unit:
```bash
java -jar ~/bin/gittally.jar init --systemd
```
This writes `.git/gittally/gittally-<repo-name>.service` and `.git/gittally/gittally.env` and prints the install commands:
```bash
ln -sf /path/to/repo/.git/gittally/gittally-<repo-name>.service ~/.config/systemd/user/gittally-<repo-name>.service
systemctl --user daemon-reload
systemctl --user enable --now gittally-<repo-name>.service
```
The unit runs `java -jar ~/bin/gittally.jar server` with the repository as working directory and `Restart=always`.
The unit name contains the repository name, so several repositories can be served by one host, each with its own service and port.
User services stop at logout unless lingering is enabled once per user:
```bash
loginctl enable-linger "$USER"
```
## Operating the Service
```bash
systemctl --user status gittally-<repo-name>.service # state and last log lines
journalctl --user -u gittally-<repo-name>.service -f # follow the log
systemctl --user restart gittally-<repo-name>.service # restart (e.g. after config changes)
systemctl --user stop gittally-<repo-name>.service # stop
```
To update GitTally, replace `~/bin/gittally.jar` and restart the service.
## Environment File
`.git/gittally/gittally.env` is loaded by the unit as `EnvironmentFile`.
It only tunes the JVM process, e.g. `JAVA_OPTS=-Xmx256m`.
All GitTally configuration lives in the YAML files described in [configuration.md](configuration.md), not in environment variables.
`init --systemd` never overwrites an existing environment file.
## Reverse Proxy (nginx)
Bind GitTally to localhost and set the public URL in `.gittally.yml`:
```yaml
server:
bindAddress: 127.0.0.1
port: 18080
publicBaseUrl: "https://ci.example.org/"
```
`publicBaseUrl` is used for all links posted to Gitea, so it must be the externally reachable URL.
Add a `server` block to the host's nginx:
```nginx
server {
listen 443 ssl;
server_name ci.example.org;
ssl_certificate /etc/letsencrypt/live/ci.example.org/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ci.example.org/privkey.pem;
location / {
proxy_pass http://127.0.0.1:18080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
```
Obtain and renew the certificate with the host's existing certbot, e.g.:
```bash
sudo certbot --nginx -d ci.example.org
```
This replaces the legacy script's managed nginx/Let's Encrypt Docker container, which was intentionally not ported (see [migration-from-legacy.md](migration-from-legacy.md)).
+74
View File
@@ -0,0 +1,74 @@
# Migration from the Legacy Script
The bash script `legacy/gitTally` is deprecated and replaced by this application.
This document maps the legacy environment-variable configuration to the YAML configuration and lists the manual migration steps.
See [configuration.md](configuration.md) for the full configuration reference and [deployment.md](deployment.md) for the new service setup.
## Configuration Mapping
Legacy configuration came from environment variables (`gitTally --env` template, sourced env files).
The new configuration lives in two YAML files: `.gittally.yml` (committed) and `.git/gittally/.gittally.yml` (machine-specific, secrets).
Branch-level keys below live under `branches.<name>`; use `branches.default` for what used to be the global value.
| Legacy environment variable | New YAML key |
|---|---|
| `GITTALLY_BUILD_COMMAND` | `branches.<name>.buildCommand` |
| `GITTALLY_BUILD_CLEAN_COMMAND` | `branches.<name>.cleanCommand` |
| `GITTALLY_BUILD_ARTEFACT_DIRS` | `branches.<name>.artifactDirs` — YAML list instead of `;`-separated |
| `GITTALLY_BUILD_STDOUT_LOG` | `branches.<name>.stdoutLog` |
| `GITTALLY_BUILD_STDERR_LOG` | `branches.<name>.stderrLog` |
| `GITTALLY_NEW_BRANCH_COMMIT_MAX_AGE` | `watcher.newBranchMaxAge` |
| `GITTALLY_BUILD_DOCKER_IMAGE` | `branches.<name>.docker.image` — also set `docker.enabled: true` (replaces the `--docker` flag) |
| `GITTALLY_BUILD_DOCKERFILE` | `branches.<name>.docker.dockerfile` |
| `GITTALLY_BUILD_DOCKER_CONTEXT` | `branches.<name>.docker.context` |
| `GITTALLY_BUILD_DOCKER_NETWORK` | `branches.<name>.docker.network` — default is now Docker's default network, not `host` |
| `GITTALLY_BUILD_DOCKER_ENV` | `branches.<name>.docker.env` — YAML map instead of space-separated assignments |
| `GITTALLY_ARTIFACT_SERVER_PORT` | `server.port` |
| `GITTALLY_ARTIFACT_SERVER_BIND_ADDRESS` | `server.bindAddress` |
| `GITTALLY_ARTIFACT_PUBLIC_BASE_URL` | `server.publicBaseUrl` |
| `GITTALLY_ARTIFACT_BUILD_RETENTION_PER_BRANCH` | `artifacts.retentionPerBranch` — build count only; the legacy age suffix (`h`/`d`) is not supported |
| `GITTALLY_IMPRESSUM_URL` | `server.impressumUrl` |
| `GITTALLY_AUTO_BUILD_BRANCHES` | `branches.<name>.autoBuild.enabled: true` per branch instead of a branch list |
| `GITTALLY_AUTO_BUILD_TIMES` | `branches.<name>.autoBuild.times` — YAML list, per branch |
| `GITTALLY_GITEA_BASE_URL` | `gitea.baseUrl` |
| `GITTALLY_GITEA_OWNER` | `gitea.owner` |
| `GITTALLY_GITEA_REPO` | `gitea.repo` |
| `GITTALLY_GITEA_STATUS_CONTEXT` | `gitea.statusContext` |
| `GITTALLY_GITEA_GIT_USERNAME` | `git.account` — in `.git/gittally/.gittally.yml` |
| `GITTALLY_GITEA_TOKEN` | `git.token` — in `.git/gittally/.gittally.yml`, never committed |
New keys without a legacy counterpart: `builds.maxConcurrent`, `artifacts.rootDir`, and `watcher.pollInterval`.
## Intentionally Not Ported
- Managed nginx/Let's Encrypt container (`GITTALLY_ARTIFACT_NGINX_*`, `GITTALLY_ARTIFACT_LETSENCRYPT_EMAIL`) — use the host's reverse proxy, see [deployment.md](deployment.md).
- Self-install and self-update (`--install`, `--pull`, `GITTALLY_INSTALL_DIR`) — replaced by jar deployment plus `init --systemd`.
- `GITTALLY_BUILD_DOCKER_PREFLIGHT_COMMAND` and `GITTALLY_BUILD_DOCKER_JAVA_TOOL_OPTIONS` — hsadmin-ng-specific; use `branches.<name>.docker.env` if needed.
- `HSADMIN_NG_*` environment-variable fallbacks.
- Env-file configuration itself — the systemd `EnvironmentFile` now only tunes the JVM (`JAVA_OPTS`).
- `GITTALLY_GITEA_DELETED_STATUS_DESCRIPTION`, `GITTALLY_BIN_FORWARD`, `GITTALLY_CONFIG_*` — internal legacy mechanics without a counterpart.
## Build History
Legacy build history (`.git/git-watch-origin-and-test/build-results.tsv`) is **not** imported; history starts fresh.
The formats differ substantially (TSV vs. JSON with commit metadata and artifact keys), and retention would prune imported rows quickly anyway.
Old artifacts under the legacy artifact root remain readable on disk until you delete them.
## Manual Migration Steps
1. Stop and remove the legacy service:
```bash
systemctl --user disable --now gitTally.service
rm -f ~/.config/systemd/user/gitTally.service
systemctl --user daemon-reload
```
2. Build and place the jar as described in [deployment.md](deployment.md).
3. In the repository, run `java -jar ~/bin/gittally.jar init`.
4. Transfer your settings from the legacy env file into `.gittally.yml` using the table above.
5. Put `git.account` and `git.token` into `.git/gittally/.gittally.yml`.
6. Verify the effective configuration: `java -jar ~/bin/gittally.jar config:print --full`.
7. Install and start the new service: `init --systemd` plus the printed commands, see [deployment.md](deployment.md).
8. Optionally clean up legacy state: `.git/git-watch-origin-and-test/` and the legacy artifact root.
+25
View File
@@ -34,3 +34,28 @@ Housekeeping:
- `./gradlew ktlintFormat` then `./gradlew build` is green.
- A fresh clone can follow `docs/deployment.md` to a running service (manual walkthrough; document the result in this file).
## Implementation Notes (2026-07-07)
Implemented as designed: `init --systemd` (an option on `init`, not a separate subcommand) generates the unit and its `EnvironmentFile` under `.git/gittally/`, prints the install commands, and never touches `~/.config/systemd` itself (no self-install).
`SystemdServiceFiles` builds the file contents and is unit-tested by content assertions, including the legacy `%` escaping and `ExecStart` quoting.
`docs/deployment.md` and `docs/migration-from-legacy.md` were written; `README.md`, `docs/bootstrapping.md`, `docs/GitTally-Konzept.md`, and `CLAUDE.md` were updated to reference them.
Deviations and decisions:
- The unit is named per repository (`gittally-<repo-name>.service`) instead of the global legacy `gitTally.service`, because one instance serves one repository and several repositories can share a host.
- `ExecStart` uses the `java` binary and the jar path of the JVM that ran `init --systemd`, so the unit points at the jar in place (legacy copied the script to an install dir); systemd expands `$JAVA_OPTS` from the `EnvironmentFile` into the command line.
When not started via `java -jar` (e.g. from Gradle), `init --systemd` prints an error instead of generating a broken unit.
- The `EnvironmentFile` only tunes the JVM (`JAVA_OPTS`); the legacy env file carried username/token, which now live in `.git/gittally/.gittally.yml`.
An existing `gittally.env` is kept; the unit file is regenerated on every run (same as legacy).
- The legacy `--nginx --docker` `ExecStart` flags were dropped (runtime selection is per-branch config now); the `After=… docker.service` ordering was kept.
- Legacy build history (`build-results.tsv`) is not imported — decided and documented in `docs/migration-from-legacy.md` (formats differ substantially; retention would prune imported rows quickly).
- ADR 0004 records the rewrite architecture decisions (JSON-file persistence behind a repository interface, polling UI, no managed nginx).
- `docs/GitTally-Konzept.md` review: only one real deviation found — "Builds laufen in Docker" became "nativ oder optional in Docker (pro Branch konfigurierbar)"; CLI capability lists gained build/retry; deployment links added.
Manual walkthrough (2026-07-07, fresh clone under `~/.cache`):
- Followed `docs/deployment.md` end to end: built the jar, copied it to a stable path, cloned the repository freshly, ran `init` and `init --systemd`, linked the generated unit, `daemon-reload`, started the service.
- The clone already contained the committed `.gittally.yml`, so only the machine config was created; `server.port` was overridden to a free port via `.git/gittally/.gittally.yml` to avoid clashing with a locally running instance.
- Result: unit `active (running)`, `GET /` returned 200, `/api/watcher` showed a successful poll, `journalctl --user -u …` showed the startup log, `restart` and `stop` worked; the unit link and the scratch clone were removed afterwards.
- Not machine-verified: `systemctl --user enable` and `loginctl enable-linger` (the walkthrough used a transient `start` to leave no persistent service behind) and the nginx/certbot section (no public host available); those commands were reviewed against the systemd/certbot documentation instead.
+1 -1
View File
@@ -62,7 +62,7 @@ Completion:
- [x] `10-cli-commands.md` — CLI build/status commands
- [x] `11-docker-build-runtime.md` — optional Docker build execution
- [ ] `12-deployment.md` — systemd service, migration from legacy, docs
- [x] `12-deployment.md` — systemd service, migration from legacy, docs
Steps 0103 are independent of each other.
Steps 0406 depend on 0103.