From 8b71d0db8e794e2efaaabe70bed31e9c25cf8e84 Mon Sep 17 00:00:00 2001 From: mhoennig Date: Mon, 10 Aug 2026 15:34:39 +0200 Subject: [PATCH] Fix Docker builds under rootless daemons and expose git metadata to build containers Two fixes from the vm4006 rollout (docs/plan/16-git-in-docker-builds.md): Rootless daemons map the host user to container root, so running the build container as --user put it into the subuid range and it could not even create .gradle in a fresh worktree (legacy only worked because its ownership-repair chown had accidentally moved build/ and .gradle/ into subuid ownership in its reused primary checkout). The container now always runs as --user 0: the unprivileged host user under rootless, real root under rootful where the ownership repair still applies; under rootless it degenerates to 0:0. Git now works inside build containers: the primary .git is mounted read-only with .git/gittally/ masked by an empty tmpfs (git.token and the control token stay unreachable, the workspace bind resurfaces only the build's own worktree) and the worktree admin dir mounted read-write for index-refreshing commands. Verified on vm4006: git log/status succeed, the machine config is invisible, ref writes fail on the read-only mount. Co-Authored-By: Claude Fable 5 --- .claude/skills/architecture/SKILL.md | 2 +- docs/configuration.md | 2 + docs/plan/16-git-in-docker-builds.md | 44 +++++++++++ .../gittally/build/DockerBuildRunner.kt | 74 +++++++++++++++++-- .../gittally/build/DockerSocketLocator.kt | 2 +- .../gittally/build/DockerBuildRunnerTest.kt | 38 +++++++++- 6 files changed, 151 insertions(+), 11 deletions(-) create mode 100644 docs/plan/16-git-in-docker-builds.md diff --git a/.claude/skills/architecture/SKILL.md b/.claude/skills/architecture/SKILL.md index 04a383f..943ef9d 100644 --- a/.claude/skills/architecture/SKILL.md +++ b/.claude/skills/architecture/SKILL.md @@ -56,7 +56,7 @@ Three places must stay in sync when config keys change: the `GitTallyConfig` dat `BuildExecutor` runs builds asynchronously: up to `builds.maxConcurrent` branches concurrently (default 1), but never more than one build per branch at a time. Each branch builds in its own reusable git worktree at `.git/gittally/worktrees/` (`BranchWorkspaces`), checked out detached at the requested commit — the primary checkout is never used for builds. Status transitions are persisted via `BuildResultRepository` (JSON file under `.git/gittally/`), published to Gitea non-fatally, and emitted as `BuildStatusChangedEvent`s. Cancellation addresses a build by artifact key and terminates the whole process tree. Future code (watcher, server, UI) must not assume a single running build. -The runtime is selected per branch behind the `BuildRunner` interface: `DispatchingBuildRunner` (`@Primary`) routes to native `ProcessBuildRunner` (the default) or to `DockerBuildRunner` when `branches..docker.enabled`. The Docker runner shells out to the `docker` CLI (no SDK): it (re)builds the configured image when the Dockerfile inputs changed (tracked via the `org.gittally.build-inputs-sha256` image label), maintains a per-repo Gradle cache volume, mounts the worktree and the Docker socket into a labelled (`org.hoennig.gittally`) `--rm --init` container, and repairs workspace ownership in-container after each command. The returned `Process` is the attached `docker run` client, so log streaming and termination work exactly like native builds. +The runtime is selected per branch behind the `BuildRunner` interface: `DispatchingBuildRunner` (`@Primary`) routes to native `ProcessBuildRunner` (the default) or to `DockerBuildRunner` when `branches..docker.enabled`. The Docker runner shells out to the `docker` CLI (no SDK): it (re)builds the configured image when the Dockerfile inputs changed (tracked via the `org.gittally.build-inputs-sha256` image label), maintains a per-repo Gradle cache volume, mounts the worktree and the Docker socket into a labelled (`org.hoennig.gittally`) `--rm --init` container, and repairs workspace ownership in-container after each command (under a rootless daemon the container runs as root, which is the host user, and the repair degenerates to `0:0`). Git works inside the container: the primary `.git` is mounted read-only with `.git/gittally/` masked by an empty tmpfs (credential isolation) and the worktree's admin dir mounted read-write (`gitMetadataMounts`). The returned `Process` is the attached `docker run` client, so log streaming and termination work exactly like native builds. ## Watcher diff --git a/docs/configuration.md b/docs/configuration.md index a618928..91dc56e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -216,6 +216,8 @@ When `dockerfile` is set, the image is (re)built whenever the Dockerfile content Staleness is tracked via the image label `org.gittally.build-inputs-sha256`. A Gradle cache volume `gittally-gradle-` is created per repository and mounted as `GRADLE_USER_HOME`. The build worktree is bind-mounted into the container; after each command the ownership of `build/` and `.gradle/` is repaired to the host user. +Git works inside the container: the primary repository's `.git` is mounted read-only (so build steps can run read-only git commands like `git log` or `git describe`), with `.git/gittally/` masked by an empty tmpfs so the build can never read the machine config (`git.token`) or the control token. +Note that the rest of `.git` — including `.git/config` — is visible to builds; GitTally never stores credentials there, and neither should you. The Docker socket is mounted into the container and `DOCKER_HOST`/`TESTCONTAINERS_*` variables are set, so Testcontainers-based builds work inside the container. All GitTally containers carry `org.hoennig.gittally` labels; stale build containers of the repository are removed before the first Docker build after a restart. diff --git a/docs/plan/16-git-in-docker-builds.md b/docs/plan/16-git-in-docker-builds.md new file mode 100644 index 0000000..00da2e9 --- /dev/null +++ b/docs/plan/16-git-in-docker-builds.md @@ -0,0 +1,44 @@ +# Step 16: Git Access Inside Docker Build Containers + +Prerequisites: steps 11, 15. +Read `README.md` first. +Motivated by the vm4006 rollout (step 15, fourth finding): hs.hsadmin.ng's build calls git in several places; `:prQuickCheck` failed hard with "fatal: not a git repository", other call sites swallowed the failure silently. + +## Problem + +Builds run in git worktrees under `.git/gittally/worktrees/`. +A worktree's `.git` is a pointer file into the primary repository's `.git/worktrees/`, and `DockerBuildRunner` bind-mounts only the worktree — so every git call inside the build container fails. +The legacy script did not have this problem because it built in the primary checkout with the real `.git` present (read-write, including all secrets stored next to it — full exposure). +Hard invariant to preserve: a branch build must never be able to reach credentials; `.git/gittally/.gittally.yml` (`git.token`) and the control token live under `.git`. + +## Considered Options + +- **Read-only `.git` mount with `.git/gittally/` masked (chosen)** — three layered mounts, no config key, no workspace mutation; strictly less privileged than legacy. +- Copy minimal git metadata into the workspace (admin dir plus `objects/info/alternates`) — mutates the workspace, still needs the object database mounted, more moving parts. +- Document the limitation and require git-free build commands — pushes the problem onto every watched project; hsadmin-ng shows real builds do call git. + +## Design + +`DockerBuildRunner.gitMetadataMounts(workspace, repoDir)` adds three mounts when (and only when) the workspace is a worktree of `repoDir` (detected via the `gitdir:` pointer file, which must resolve into `repoDir/.git`): + +1. `repoDir/.git` → same path, **read-only**: objects, refs, and the worktree admin metadata become resolvable; object and ref writes stay impossible. +2. An empty **tmpfs over `repoDir/.git/gittally`**: masks the machine config (`git.token`), the control token, and all GitTally state; the workspace bind (deeper path, Docker nests mounts by target depth) resurfaces only this build's own worktree inside the masked directory. +3. `repoDir/.git/worktrees/` → same path, **read-write**: the worktree's admin dir (HEAD, index), so index-refreshing commands like `git status` work. + +No configuration key: the exposure is strictly smaller than the legacy baseline, and a knob would join the pinned sandbox-policy set without a known use case. +Remaining, documented exposure: the rest of `.git` — including `.git/config` — is readable by builds; GitTally never stores credentials there (fetch auth uses a secret-free `GIT_ASKPASS` with env-passed credentials). + +## Tests + +- `DockerBuildRunnerTest`: worktree workspace → the three mounts with `:ro` and `--tmpfs`; non-worktree workspace → no git metadata mounts (also keeps the exact-argv test valid). + +## Acceptance Criteria + +- `./gradlew ktlintFormat` then `./gradlew build` is green. +- In a real Docker build worktree: `git log`/`git status` succeed inside the container, `.git/gittally/.gittally.yml` and `control-token` are not readable, and a `git push`/ref write fails. +- `docs/configuration.md` (docker notes) and the architecture skill describe the mounts. + +## Result (2026-08-10) + +Implemented as designed; verified on vm4006 (see below) and in unit tests. +`sh -c 'git log -1 && git status --short && cat .../.git/gittally/.gittally.yml'` inside a build container of the hs.hsadmin.ng worktree: git commands succeed, the machine config read fails with "No such file or directory", `git update-ref` fails on the read-only filesystem. diff --git a/src/main/kotlin/de/hoennig/gittally/build/DockerBuildRunner.kt b/src/main/kotlin/de/hoennig/gittally/build/DockerBuildRunner.kt index 0ccc3e1..fd8c4e9 100644 --- a/src/main/kotlin/de/hoennig/gittally/build/DockerBuildRunner.kt +++ b/src/main/kotlin/de/hoennig/gittally/build/DockerBuildRunner.kt @@ -5,6 +5,7 @@ import de.hoennig.gittally.config.DockerConfig import de.hoennig.gittally.git.GitCommandRunner import org.slf4j.LoggerFactory import org.springframework.stereotype.Component +import java.nio.file.Files import java.nio.file.Path /** @@ -17,8 +18,12 @@ import java.nio.file.Path * tracked via the [DockerImageInputs.INPUTS_LABEL] image label), and the per-repo * Gradle cache volume. The returned [Process] is the attached `docker run` client * (`--init`, so termination signals reach the build inside the container); the - * executor streams and terminates it like a native build. Ownership of `build/` - * and `.gradle/` is repaired to the host user inside the same container run. + * executor streams and terminates it like a native build. On rootful daemons the + * ownership of `build/` and `.gradle/` is repaired to the host user inside the + * same container run; under a rootless daemon the container runs as root, which + * already is the host user, so the repair chown degenerates to `0:0`. + * Git works inside the container: the primary `.git` is mounted read-only with + * `.git/gittally/` masked, see [gitMetadataMounts]. */ @Component class DockerBuildRunner( @@ -51,8 +56,15 @@ class DockerBuildRunner( ensureImage(docker, repoDir) val uid = id("-u", repoDir) val gid = id("-g", repoDir) + val socket = socketLocator.locate(uid) + // Under a rootless daemon, container root IS the unprivileged host user (identity mapping), + // so files in the bind-mounted worktree stay host-owned and no chown is needed; chowning to + // the host ids there would push the files into the subuid range and lock the host user out. + val rootless = socket?.rootless == true + val ownershipUid = if (rootless) "0" else uid + val ownershipGid = if (rootless) "0" else gid val volume = gradleVolumeName(repoKey) - prepareGradleVolume(volume, docker.image, uid, gid, repoDir) + prepareGradleVolume(volume, docker.image, ownershipUid, ownershipGid, repoDir) val containerName = containerName(repoKey, environment["branch"]) removeContainer(containerName, repoDir) val runCommand = @@ -61,11 +73,13 @@ class DockerBuildRunner( workspace = workingDir.toAbsolutePath().normalize(), environment = environment, docker = docker, + repoDir = repoDir, repoKey = repoKey, containerName = containerName, volume = volume, - uid = uid, - gid = gid, + socket = socket, + uid = ownershipUid, + gid = ownershipGid, ) return processStarter(runCommand, repoDir) } @@ -207,13 +221,14 @@ class DockerBuildRunner( workspace: Path, environment: Map, docker: DockerConfig, + repoDir: Path, repoKey: String, containerName: String, volume: String, + socket: DockerSocket?, uid: String, gid: String, ): List { - val socket = socketLocator.locate(uid) val args = mutableListOf("docker", "run", "--rm", "--init", "--name", containerName) args += listOf( @@ -225,6 +240,7 @@ class DockerBuildRunner( "$GITTALLY_LABEL.role=build", ) args += listOf("--workdir", "$workspace", "--volume", "$workspace:$workspace") + args += gitMetadataMounts(workspace, repoDir) args += listOf("--volume", "$volume:/gradle-user-home") args += listOf("--env", "HOME=/tmp/docker-home", "--env", "GRADLE_USER_HOME=/gradle-user-home") for ((key, value) in environment) { @@ -244,7 +260,9 @@ class DockerBuildRunner( args += listOf("--group-add", socket.gid.toString()) } } - args += listOf("--user", if (socket?.rootless == true) uid else "0") + // root in the container: the real host user under a rootless daemon (identity mapping), + // or actual root under a rootful daemon, where the ownership-repair chown fixes the files. + args += listOf("--user", "0") val hostOverride = if (docker.network == "host") "localhost" else "host.docker.internal" args += listOf("--env", "TESTCONTAINERS_HOST_OVERRIDE=$hostOverride") if (docker.network != "host") { @@ -255,6 +273,48 @@ class DockerBuildRunner( return args } + /** + * Makes git work inside the build container without exposing GitTally's secrets. + * + * The workspace is a git worktree whose `.git` file points into the primary + * repository's `.git`, which is not part of the workspace mount — so any git call + * in the build would fail. Three layered mounts fix that (Docker nests mounts by + * target path): the primary `.git` read-only, an empty tmpfs masking `.git/gittally/` + * (machine config with `git.token`, control token, build state — the workspace bind + * resurfaces only this build's own worktree inside it), and this worktree's admin + * directory read-write, so index-refreshing commands like `git status` keep working. + * Object and ref writes stay blocked by the read-only `.git` mount. + * No mounts are added when the workspace is not a worktree of [repoDir]. + */ + private fun gitMetadataMounts( + workspace: Path, + repoDir: Path, + ): List { + val gitDir = repoDir.toAbsolutePath().normalize().resolve(".git") + val workspaceGitFile = workspace.resolve(".git") + if (!Files.isDirectory(gitDir) || !Files.isRegularFile(workspaceGitFile)) { + return emptyList() + } + val adminDir = + Files + .readString(workspaceGitFile) + .substringAfter("gitdir:", "") + .trim() + .takeIf { it.isNotEmpty() } + ?.let { workspace.resolve(it).normalize() } + ?: return emptyList() + if (!adminDir.startsWith(gitDir) || !Files.isDirectory(adminDir)) { + return emptyList() + } + val args = mutableListOf("--volume", "$gitDir:$gitDir:ro") + val gittallyDir = gitDir.resolve("gittally") + if (Files.isDirectory(gittallyDir)) { + args += listOf("--tmpfs", "$gittallyDir") + } + args += listOf("--volume", "$adminDir:$adminDir") + return args + } + private fun id( flag: String, repoDir: Path, diff --git a/src/main/kotlin/de/hoennig/gittally/build/DockerSocketLocator.kt b/src/main/kotlin/de/hoennig/gittally/build/DockerSocketLocator.kt index 788a7d7..a32bdee 100644 --- a/src/main/kotlin/de/hoennig/gittally/build/DockerSocketLocator.kt +++ b/src/main/kotlin/de/hoennig/gittally/build/DockerSocketLocator.kt @@ -8,7 +8,7 @@ import java.nio.file.Paths /** The host-side Docker socket a build container should talk to. */ data class DockerSocket( val path: Path, - /** True for a rootless daemon socket (`/run/user//docker.sock`); the container then runs as the host user. */ + /** True for a rootless daemon socket (`/run/user//docker.sock`); container root then maps to the host user. */ val rootless: Boolean, /** Group id owning the socket, for `--group-add` on rootful daemons; null when unavailable. */ val gid: Long?, diff --git a/src/test/kotlin/de/hoennig/gittally/build/DockerBuildRunnerTest.kt b/src/test/kotlin/de/hoennig/gittally/build/DockerBuildRunnerTest.kt index a38d6d9..d222819 100644 --- a/src/test/kotlin/de/hoennig/gittally/build/DockerBuildRunnerTest.kt +++ b/src/test/kotlin/de/hoennig/gittally/build/DockerBuildRunnerTest.kt @@ -126,7 +126,7 @@ class DockerBuildRunnerTest : FunSpec() { script shouldContain "exit \$build_exit" } - test("a rootless socket runs the container as the host user without group-add") { + test("a rootless socket runs the container as root (the host user) without group-add or host-id chown") { every { socketLocator.locate("1000") } returns DockerSocket(Paths.get("/run/user/1000/docker.sock"), rootless = true, gid = 998L) @@ -134,8 +134,42 @@ class DockerBuildRunnerTest : FunSpec() { val args = captured.single() args shouldContain "/run/user/1000/docker.sock:/var/run/docker.sock" - args[args.indexOf("--user") + 1] shouldBe "1000" + args[args.indexOf("--user") + 1] shouldBe "0" args shouldNotContain "--group-add" + // container root already is the host user — the repair chown must not target the host ids, + // which would push the worktree files into the subuid range + args[args.size - 3] shouldBe "0" + args[args.size - 2] shouldBe "0" + verify { + commandRunner.runOrThrow( + match { it.take(2) == listOf("docker", "run") && it.takeLast(2) == listOf("0", "0") }, + repoDir, + ) + } + } + + test("exposes git metadata read-only with the gittally dir masked for a worktree workspace") { + val gitDir = repoDir.resolve(".git") + val adminDir = gitDir.resolve("worktrees/workspace") + Files.createDirectories(adminDir) + Files.createDirectories(gitDir.resolve("gittally")) + Files.createDirectories(workspace) + Files.writeString(workspace.resolve(".git"), "gitdir: $adminDir\n") + + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, dockerBranchConfig()) + + val args = captured.single() + args shouldContain "$gitDir:$gitDir:ro" + args[args.indexOf("--tmpfs") + 1] shouldBe "${gitDir.resolve("gittally")}" + args shouldContain "$adminDir:$adminDir" + } + + test("mounts no git metadata when the workspace is not a worktree") { + runner.start("./gradlew test", workspace, mapOf("branch" to "main"), repoDir, dockerBranchConfig()) + + val args = captured.single() + args shouldNotContain "--tmpfs" + args.none { it.endsWith(":ro") } shouldBe true } test("host network keeps Testcontainers on localhost without add-host") {