werkator build succeeded after 00:06
RFC 0004 had ruled `build` out as "not planned: needs writable rootfs snapshots per step and `pull` for FROM". Both halves expired: `pull` arrived in 0.3.0, and per-step snapshots are what a build *cache* needs, not what a build needs — one writable tree over the base is enough. ## RFC 0006 [docs/rfcs/0006-dockerfile-builds.md](docs/rfcs/0006-dockerfile-builds.md) decides the build model, cuts the one version-dependent interface, and argues the "no layers" question from measurements rather than from the README's shorthand. Measured on a `debian:trixie` rootfs with bwrap 0.11.1 and a `0 <uid> 1` map — every number in the RFC was taken, not reasoned about: - A writable root bind (`--bind` instead of `--ro-bind`) works unprivileged. - `CapEff` is `0000000000000000`: `mount -t tmpfs` and `mknod` are both EPERM, so the sandbox can mount nothing itself and every mount must stand in bwrap's argv. **That** is what "no layers" was ever about. - Plain `apt-get update` dies on apt's `_apt` user (`seteuid 42 failed`, `Method http has died unexpectedly`); `APT::Sandbox::User root` fixes it. - `apt-get install nginx` still fails (`chown: changing ownership of '/var/log/nginx': Invalid argument`, dpkg exit 100). `fakeroot` **alone does not help** — libfakeroot is demonstrably active and its `chown` wrapper still reports the real syscall's EINVAL. `fakeroot` plus `FAKEROOTDONTTRYCHOWN=1` does. - Overlay works unprivileged on bwrap 0.11, `--overlay-src` stacks, and the upper directory is already a layer diff: changed files as themselves, deletions as character devices 0:0 — what OCI writes as `.wh.NAME`. Two snags: the workdir's `work/` is left at `d---------`, and bwrap refuses overlay when installed setuid. - With a writable root, bwrap creates a missing bind mountpoint itself and it stays in the tree — so a build binds only `/etc/resolv.conf`, which base images already have. ## The interface (the point of the RFC) `build.Backend` / `build.Tree` / `build.Step`. The boundary is exactly "how does a writable tree over a base come into being, and how is its result harvested"; no caller ever sets a `RunSpec`'s root fields. Everything else — parser, instructions, COPY, multi-stage, CLI, uid levers, commit — is identical in both worlds. - **Copy backend** (implemented): a real full copy of the base, root bound read-write, commit is the `rename`, no cache. It stays after the Debian 13 upgrade, as the fallback `doctor` picks when bwrap is too old or setuid, and for single-shot builds. - **Overlay backend** (deliberately *not* implemented): designed, with the measurements above as the evidence that it fits. The per-step cache waits for it, because without a union mount a cached step costs a whole tree copy. The warning that belongs with this, in the RFC and as a comment in the code: `store.copyTree` hardlinks so a second tag costs no data — there both copies are read-only. For a build tree it is unusable, because a `RUN` step writing in place through such a hardlink would modify the base image. ## Stage 1, implemented `werkdock build [-f Dockerfile] -t IMAGE CONTEXT` with docker's flags. `FROM`, `RUN`, `COPY` plus the metadata instructions `import -c` already knew, through the same `store.ApplyChange`. Staging and commit reuse the mechanics `import` has had since 0.11.0 (`<image>.tmp`, the per-image `flock`, `rename`), now exported as `store.Staging`. Werkdock sets both uid levers itself so the Dockerfile stays portable; the base image needs `fakeroot` installed, and `build` says so up front when it has not. `engine.RunSpec` gained `WritableRoot`, and a writable-root sandbox gets **no** implicit tmpfs over `/tmp`, `/root`, `/run` — with one, what a `RUN` step writes there would vanish in the next step while docker keeps it in the image. Refused loudly rather than ignored: `EXPOSE`, `HEALTHCHECK`, `STOPSIGNAL`, `ONBUILD`, `MAINTAINER` at parse time; `ARG`, `SHELL`, `ADD` and multi-stage named as stages 2 and 3; BuildKit's `RUN --mount` with the CapEff measurement as the reason; and the dozen `docker build` flags werkdock cannot keep the promise of. ## Verified - `go test ./...`, `go vet ./...`, `gofmt -l .` — green and empty. - A real sandbox integration test builds `FROM`/`WORKDIR`/`COPY`/`RUN`/`CMD` and then *runs* the result; it skips without bwrap/userns like the existing ones. - The apt path, which the levers exist for, is a second integration test pointed at a Debian base (`WERKDOCK_TEST_BASE_ROOTFS`). Run here: `nginx-common` configures, `dpkg-query` in the built image reports `install ok installed`, the apt fragment is gone from the result. - End to end through the CLI: `import` a 143 MB `debian:trixie` base with fakeroot, `build -t probe:1` with `apt-get install -y nginx jq` in the Dockerfile — **6.8 s** — then `run --rm probe:1` prints what its `RUN` steps and its `COPY` produced. Follow-ups the RFC names without deciding: store dedup between image versions, keeping layer blobs as a transport and cache format (the one with the highest running cost today), `doctor IMAGE`, and `ADD` from a URL. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: claude-code <claude-code@javagil.de> Co-authored-by: mhoennig <michael@hoennig.de> Reviewed-on: #14
261 lines
8.0 KiB
Go
261 lines
8.0 KiB
Go
package build
|
|
|
|
import (
|
|
"bytes"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
|
|
"werkdock/internal/engine"
|
|
"werkdock/internal/store"
|
|
)
|
|
|
|
// requireSandbox skips where bwrap or unprivileged user namespaces are
|
|
// not usable, exactly as the engine's integration tests do.
|
|
func requireSandbox(t *testing.T) {
|
|
t.Helper()
|
|
if _, err := exec.LookPath("bwrap"); err != nil {
|
|
t.Skip("bwrap not installed")
|
|
}
|
|
if err := exec.Command("bwrap", "--unshare-user", "--uid", "0", "--ro-bind", "/", "/", "true").Run(); err != nil {
|
|
t.Skipf("unprivileged user namespaces not usable here: %v", err)
|
|
}
|
|
}
|
|
|
|
// TestBuildAndRunInARealSandbox builds a small image from a Dockerfile
|
|
// in a real bwrap sandbox and then runs the result, the way `werkdock
|
|
// run` would. The base image is assembled from the host's own shell
|
|
// and its shared libraries, so the test needs no network and no
|
|
// registry. The apt path — the one the uid levers exist for — is in
|
|
// TestBuildInstallsWithApt below.
|
|
func TestBuildAndRunInARealSandbox(t *testing.T) {
|
|
requireSandbox(t)
|
|
base := hostToolsImage(t, "/bin/sh", "/bin/cat")
|
|
|
|
dir := t.TempDir()
|
|
st := store.Store{Root: filepath.Join(dir, "store")}
|
|
context := filepath.Join(dir, "context")
|
|
if err := os.MkdirAll(context, 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := os.WriteFile(filepath.Join(context, "greeting"), []byte("from the context\n"), 0o644); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
dockerfile := filepath.Join(dir, "Dockerfile")
|
|
if err := os.WriteFile(dockerfile, []byte(`
|
|
FROM base:probe
|
|
WORKDIR /srv
|
|
COPY greeting /srv/greeting
|
|
RUN echo "from a RUN step" > /srv/built \
|
|
&& cat /srv/greeting >> /srv/built
|
|
CMD ["/bin/cat", "/srv/built"]
|
|
`), 0o644); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
log := &bytes.Buffer{}
|
|
result, err := Run(st, Options{
|
|
Dockerfile: dockerfile,
|
|
Context: context,
|
|
Tag: "probe-built:1",
|
|
Backend: CopyBackend{},
|
|
Resolve: staticBase(base),
|
|
Stdout: log,
|
|
Stderr: log,
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("build failed: %v\n%s", err, log.String())
|
|
}
|
|
if !strings.Contains(log.String(), "Successfully tagged probe-built:1") {
|
|
t.Errorf("build log:\n%s", log.String())
|
|
}
|
|
|
|
// Now run the built image the way `werkdock run` does: its CMD,
|
|
// the rootfs bound read-only.
|
|
rootfs, err := st.RootFS(result.Reference)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
meta, err := st.Meta(result.Reference)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
var stdout, stderr bytes.Buffer
|
|
code, err := (&engine.Bwrap{Stdout: &stdout, Stderr: &stderr}).Run(engine.RunSpec{
|
|
RootFS: rootfs,
|
|
Workdir: meta.Config.WorkingDir,
|
|
Command: meta.Config.Cmd,
|
|
})
|
|
if err != nil || code != 0 {
|
|
t.Fatalf("running the built image: code %d, err %v, stderr %q", code, err, stderr.String())
|
|
}
|
|
if want := "from a RUN step\nfrom the context\n"; stdout.String() != want {
|
|
t.Errorf("the built image printed %q, want %q", stdout.String(), want)
|
|
}
|
|
if meta.Config.WorkingDir != "/srv" {
|
|
t.Errorf("workdir %q", meta.Config.WorkingDir)
|
|
}
|
|
}
|
|
|
|
// TestBuildInstallsWithApt is the test the two uid levers exist for:
|
|
// without `APT::Sandbox::User root` apt's methods die on seteuid, and
|
|
// without fakeroot plus FAKEROOTDONTTRYCHOWN a package whose postinst
|
|
// chowns (nginx-common) fails in dpkg — both measured (RFC 0006).
|
|
//
|
|
// It needs a Debian-shaped base image with apt and fakeroot, and
|
|
// network access to a mirror, so it is pointed at one explicitly:
|
|
//
|
|
// WERKDOCK_TEST_BASE_ROOTFS=/path/to/unpacked/rootfs go test ./internal/build/
|
|
//
|
|
// The rootfs is only read; the build works in a copy of it.
|
|
func TestBuildInstallsWithApt(t *testing.T) {
|
|
requireSandbox(t)
|
|
base := os.Getenv("WERKDOCK_TEST_BASE_ROOTFS")
|
|
if base == "" {
|
|
t.Skip("set WERKDOCK_TEST_BASE_ROOTFS to an unpacked Debian rootfs with apt and fakeroot")
|
|
}
|
|
if fi, err := os.Stat(filepath.Join(base, "usr/bin/fakeroot")); err != nil || fi.IsDir() {
|
|
t.Skipf("%s has no fakeroot; the apt path needs one in the base image", base)
|
|
}
|
|
|
|
dir := t.TempDir()
|
|
st := store.Store{Root: filepath.Join(dir, "store")}
|
|
context := filepath.Join(dir, "context")
|
|
if err := os.MkdirAll(context, 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
dockerfile := filepath.Join(dir, "Dockerfile")
|
|
if err := os.WriteFile(dockerfile, []byte(`
|
|
FROM base:trixie
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends nginx jq \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
CMD ["/usr/sbin/nginx", "-v"]
|
|
`), 0o644); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
log := &bytes.Buffer{}
|
|
result, err := Run(st, Options{
|
|
Dockerfile: dockerfile,
|
|
Context: context,
|
|
Tag: "apt-built:1",
|
|
Backend: CopyBackend{},
|
|
Resolve: staticBase(base),
|
|
Stdout: log,
|
|
Stderr: log,
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("build failed: %v\n%s", err, log.String())
|
|
}
|
|
|
|
rootfs, err := st.RootFS(result.Reference)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
// nginx-common's postinst is where a build without the levers
|
|
// stops; dpkg records the package as installed only if it ran.
|
|
var stdout, stderr bytes.Buffer
|
|
code, err := (&engine.Bwrap{Stdout: &stdout, Stderr: &stderr}).Run(engine.RunSpec{
|
|
RootFS: rootfs,
|
|
Command: []string{"/usr/bin/dpkg-query", "-W", "-f", "${Status}", "nginx"},
|
|
})
|
|
if err != nil || code != 0 {
|
|
t.Fatalf("dpkg-query in the built image: code %d, err %v, stderr %q", code, err, stderr.String())
|
|
}
|
|
if !strings.Contains(stdout.String(), "install ok installed") {
|
|
t.Errorf("nginx is %q in the built image, want it configured", stdout.String())
|
|
}
|
|
if _, err := os.Stat(filepath.Join(rootfs, "etc/apt/apt.conf.d/00werkdock-sandbox")); !os.IsNotExist(err) {
|
|
t.Errorf("the apt fragment stayed in the built image: %v", err)
|
|
}
|
|
}
|
|
|
|
func staticBase(rootfs string) func(string) (string, store.ImageMeta, error) {
|
|
return func(string) (string, store.ImageMeta, error) {
|
|
return rootfs, store.ImageMeta{}, nil
|
|
}
|
|
}
|
|
|
|
// hostToolsImage assembles a base rootfs from host binaries and the
|
|
// shared libraries ldd reports for them — a real, runnable image
|
|
// without a registry. The test skips when a tool or ldd is missing.
|
|
func hostToolsImage(t *testing.T, binaries ...string) string {
|
|
t.Helper()
|
|
if _, err := exec.LookPath("ldd"); err != nil {
|
|
t.Skip("ldd not available to assemble a base rootfs")
|
|
}
|
|
rootfs := filepath.Join(t.TempDir(), "base")
|
|
for _, binary := range binaries {
|
|
for _, file := range append([]string{binary}, sharedLibraries(t, binary)...) {
|
|
copyHostFile(t, rootfs, file)
|
|
}
|
|
}
|
|
// A base image needs the mountpoints a sandbox brings in.
|
|
for _, dir := range []string{"proc", "dev", "tmp", "root", "run", "etc"} {
|
|
if err := os.MkdirAll(filepath.Join(rootfs, dir), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
if err := os.WriteFile(filepath.Join(rootfs, "etc/resolv.conf"), nil, 0o644); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
return rootfs
|
|
}
|
|
|
|
func sharedLibraries(t *testing.T, binary string) []string {
|
|
t.Helper()
|
|
out, err := exec.Command("ldd", binary).Output()
|
|
if err != nil {
|
|
t.Skipf("ldd %s: %v", binary, err)
|
|
}
|
|
var libraries []string
|
|
for _, line := range strings.Split(string(out), "\n") {
|
|
for _, field := range strings.Fields(line) {
|
|
if strings.HasPrefix(field, "/") && strings.Contains(field, ".so") {
|
|
libraries = append(libraries, field)
|
|
}
|
|
}
|
|
}
|
|
return libraries
|
|
}
|
|
|
|
// copyHostFile places one host file in the rootfs at its own path,
|
|
// following a symlink to its target and keeping the link as well, so
|
|
// that /bin/sh -> dash works inside the sandbox.
|
|
func copyHostFile(t *testing.T, rootfs, file string) {
|
|
t.Helper()
|
|
target := filepath.Join(rootfs, file)
|
|
if _, err := os.Lstat(target); err == nil {
|
|
return
|
|
}
|
|
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
fi, err := os.Lstat(file)
|
|
if err != nil {
|
|
t.Skipf("%s is not on this host: %v", file, err)
|
|
}
|
|
if fi.Mode()&os.ModeSymlink != 0 {
|
|
link, err := os.Readlink(file)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
resolved := link
|
|
if !filepath.IsAbs(resolved) {
|
|
resolved = filepath.Join(filepath.Dir(file), link)
|
|
}
|
|
copyHostFile(t, rootfs, resolved)
|
|
if err := os.Symlink(link, target); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
return
|
|
}
|
|
if err := copyFile(file, target, 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|