Files
7504b97235
werkator build succeeded after 00:06
A Dockerfile builds on the target, and two backends for one interface (0.12.0) (#14)
RFC 0004 had ruled `build` out as "not planned: needs writable rootfs snapshots per step and `pull` for FROM". Both halves expired: `pull` arrived in 0.3.0, and per-step snapshots are what a build *cache* needs, not what a build needs — one writable tree over the base is enough.

## RFC 0006

[docs/rfcs/0006-dockerfile-builds.md](docs/rfcs/0006-dockerfile-builds.md) decides the build model, cuts the one version-dependent interface, and argues the "no layers" question from measurements rather than from the README's shorthand.

Measured on a `debian:trixie` rootfs with bwrap 0.11.1 and a `0 <uid> 1` map — every number in the RFC was taken, not reasoned about:

- A writable root bind (`--bind` instead of `--ro-bind`) works unprivileged.
- `CapEff` is `0000000000000000`: `mount -t tmpfs` and `mknod` are both EPERM, so the sandbox can mount nothing itself and every mount must stand in bwrap's argv. **That** is what "no layers" was ever about.
- Plain `apt-get update` dies on apt's `_apt` user (`seteuid 42 failed`, `Method http has died unexpectedly`); `APT::Sandbox::User root` fixes it.
- `apt-get install nginx` still fails (`chown: changing ownership of '/var/log/nginx': Invalid argument`, dpkg exit 100). `fakeroot` **alone does not help** — libfakeroot is demonstrably active and its `chown` wrapper still reports the real syscall's EINVAL. `fakeroot` plus `FAKEROOTDONTTRYCHOWN=1` does.
- Overlay works unprivileged on bwrap 0.11, `--overlay-src` stacks, and the upper directory is already a layer diff: changed files as themselves, deletions as character devices 0:0 — what OCI writes as `.wh.NAME`. Two snags: the workdir's `work/` is left at `d---------`, and bwrap refuses overlay when installed setuid.
- With a writable root, bwrap creates a missing bind mountpoint itself and it stays in the tree — so a build binds only `/etc/resolv.conf`, which base images already have.

## The interface (the point of the RFC)

`build.Backend` / `build.Tree` / `build.Step`. The boundary is exactly "how does a writable tree over a base come into being, and how is its result harvested"; no caller ever sets a `RunSpec`'s root fields. Everything else — parser, instructions, COPY, multi-stage, CLI, uid levers, commit — is identical in both worlds.

- **Copy backend** (implemented): a real full copy of the base, root bound read-write, commit is the `rename`, no cache. It stays after the Debian 13 upgrade, as the fallback `doctor` picks when bwrap is too old or setuid, and for single-shot builds.
- **Overlay backend** (deliberately *not* implemented): designed, with the measurements above as the evidence that it fits. The per-step cache waits for it, because without a union mount a cached step costs a whole tree copy.

The warning that belongs with this, in the RFC and as a comment in the code: `store.copyTree` hardlinks so a second tag costs no data — there both copies are read-only. For a build tree it is unusable, because a `RUN` step writing in place through such a hardlink would modify the base image.

## Stage 1, implemented

`werkdock build [-f Dockerfile] -t IMAGE CONTEXT` with docker's flags. `FROM`, `RUN`, `COPY` plus the metadata instructions `import -c` already knew, through the same `store.ApplyChange`. Staging and commit reuse the mechanics `import` has had since 0.11.0 (`<image>.tmp`, the per-image `flock`, `rename`), now exported as `store.Staging`.

Werkdock sets both uid levers itself so the Dockerfile stays portable; the base image needs `fakeroot` installed, and `build` says so up front when it has not. `engine.RunSpec` gained `WritableRoot`, and a writable-root sandbox gets **no** implicit tmpfs over `/tmp`, `/root`, `/run` — with one, what a `RUN` step writes there would vanish in the next step while docker keeps it in the image.

Refused loudly rather than ignored: `EXPOSE`, `HEALTHCHECK`, `STOPSIGNAL`, `ONBUILD`, `MAINTAINER` at parse time; `ARG`, `SHELL`, `ADD` and multi-stage named as stages 2 and 3; BuildKit's `RUN --mount` with the CapEff measurement as the reason; and the dozen `docker build` flags werkdock cannot keep the promise of.

## Verified

- `go test ./...`, `go vet ./...`, `gofmt -l .` — green and empty.
- A real sandbox integration test builds `FROM`/`WORKDIR`/`COPY`/`RUN`/`CMD` and then *runs* the result; it skips without bwrap/userns like the existing ones.
- The apt path, which the levers exist for, is a second integration test pointed at a Debian base (`WERKDOCK_TEST_BASE_ROOTFS`). Run here: `nginx-common` configures, `dpkg-query` in the built image reports `install ok installed`, the apt fragment is gone from the result.
- End to end through the CLI: `import` a 143 MB `debian:trixie` base with fakeroot, `build -t probe:1` with `apt-get install -y nginx jq` in the Dockerfile — **6.8 s** — then `run --rm probe:1` prints what its `RUN` steps and its `COPY` produced.

Follow-ups the RFC names without deciding: store dedup between image versions, keeping layer blobs as a transport and cache format (the one with the highest running cost today), `doctor IMAGE`, and `ADD` from a URL.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: claude-code <claude-code@javagil.de>
Co-authored-by: mhoennig <michael@hoennig.de>
Reviewed-on: #14
2026-10-20 15:41:48 +02:00

163 lines
4.8 KiB
Go

package build
import (
"fmt"
"io"
"os"
"path/filepath"
"syscall"
"werkdock/internal/engine"
)
// CopyBackend builds in a full copy of the base rootfs, bound
// read-write into every RUN step (RFC 0006). It needs nothing from
// bwrap that 0.8 does not have: a writable root bind works in an
// unprivileged user namespace (measured), and no mount is made inside
// the sandbox, which could not make one anyway (CapEff is 0).
type CopyBackend struct{}
func (CopyBackend) Name() string { return "copy" }
// CachesSteps is false: without a union mount a cached step's result is
// a whole tree, so caching four steps of a 2 GiB image would keep four
// 2 GiB copies. The per-step cache waits for the overlay backend.
func (CopyBackend) CachesSteps() bool { return false }
// Open copies base to dir/rootfs.
func (CopyBackend) Open(base, dir string) (Tree, error) {
rootfs := filepath.Join(dir, "rootfs")
if err := copyTree(base, rootfs); err != nil {
return nil, fmt.Errorf("copying the base image: %w", err)
}
return &copyTreeHandle{rootfs: rootfs}, nil
}
type copyTreeHandle struct {
rootfs string
}
// Step hands out the same writable tree for every step: the copy
// backend has one tree, and a step's writes are simply in it.
func (t *copyTreeHandle) Step(int) (Step, error) {
if t.rootfs == "" {
return nil, fmt.Errorf("the work tree has been committed")
}
return copyStep{rootfs: t.rootfs}, nil
}
// Commit hands the tree over: it is already the flat rootfs the store
// wants, so the commit the caller then performs is a rename.
func (t *copyTreeHandle) Commit() (string, error) {
if t.rootfs == "" {
return "", fmt.Errorf("the work tree has already been committed")
}
rootfs := t.rootfs
t.rootfs = ""
return rootfs, nil
}
// Close has nothing to release: the tree lives in the caller's staging
// directory, which the caller removes when the build failed.
func (t *copyTreeHandle) Close() error { return nil }
type copyStep struct {
rootfs string
}
func (s copyStep) Root(spec *engine.RunSpec) {
spec.RootFS = s.rootfs
spec.WritableRoot = true
}
func (s copyStep) Writable() string { return s.rootfs }
// copyTree duplicates a directory tree with real copies of the files.
//
// It is deliberately NOT store.copyTree, which hardlinks so that a
// second tag of one image costs no data — there both copies are
// read-only. Here a RUN step writes into the tree in place, and
// through a hardlink that write would land in the base image as well.
// A build needs a real copy; that copy is the cost of the copy backend
// (RFC 0006, Costs).
//
// Hardlinks *inside* the base are reproduced inside the copy (busybox
// links hundreds of names to one binary), which costs nothing and
// keeps the tree's own structure: those links point at the copy, never
// at the base.
func copyTree(src, dst string) error {
linked := map[uint64]string{}
return filepath.Walk(src, func(p string, fi os.FileInfo, err error) error {
if err != nil {
return err
}
rel, relErr := filepath.Rel(src, p)
if relErr != nil {
return relErr
}
target := filepath.Join(dst, rel)
switch {
case fi.IsDir():
if err := os.MkdirAll(target, 0o700); err != nil {
return err
}
// Owner-writable, whatever the image says: the steps have
// to be able to add to every directory, and the finished
// tree is bound read-only into every run anyway — as
// store.applyLayer does for the same reason.
return os.Chmod(target, fi.Mode().Perm()|0o700)
case fi.Mode()&os.ModeSymlink != 0:
link, linkErr := os.Readlink(p)
if linkErr != nil {
return linkErr
}
return os.Symlink(link, target)
case fi.Mode().IsRegular():
if ino, ok := hardlinkKey(fi); ok {
if first, seen := linked[ino]; seen {
return os.Link(first, target)
}
linked[ino] = target
}
return copyFile(p, target, fi.Mode().Perm())
default:
// Sockets, fifos, device nodes: not image content. /dev
// comes from the host (RFC 0002) and store.applyLayer
// skips them when unpacking, so a base image holds none.
return nil
}
})
}
// hardlinkKey is the inode of a file that has more than one name, so
// that the second name becomes a link inside the copy.
func hardlinkKey(fi os.FileInfo) (uint64, bool) {
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok || st.Nlink < 2 {
return 0, false
}
return st.Ino, true
}
func copyFile(src, dst string, mode os.FileMode) error {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.OpenFile(dst, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, mode)
if err != nil {
return err
}
if _, err := io.Copy(out, in); err != nil {
out.Close()
return err
}
if err := out.Close(); err != nil {
return err
}
// O_CREATE applies the umask to mode, so set it explicitly, as the
// layer unpacking does.
return os.Chmod(dst, mode)
}