werkator build succeeded after 00:06
RFC 0004 had ruled `build` out as "not planned: needs writable rootfs snapshots per step and `pull` for FROM". Both halves expired: `pull` arrived in 0.3.0, and per-step snapshots are what a build *cache* needs, not what a build needs — one writable tree over the base is enough. ## RFC 0006 [docs/rfcs/0006-dockerfile-builds.md](docs/rfcs/0006-dockerfile-builds.md) decides the build model, cuts the one version-dependent interface, and argues the "no layers" question from measurements rather than from the README's shorthand. Measured on a `debian:trixie` rootfs with bwrap 0.11.1 and a `0 <uid> 1` map — every number in the RFC was taken, not reasoned about: - A writable root bind (`--bind` instead of `--ro-bind`) works unprivileged. - `CapEff` is `0000000000000000`: `mount -t tmpfs` and `mknod` are both EPERM, so the sandbox can mount nothing itself and every mount must stand in bwrap's argv. **That** is what "no layers" was ever about. - Plain `apt-get update` dies on apt's `_apt` user (`seteuid 42 failed`, `Method http has died unexpectedly`); `APT::Sandbox::User root` fixes it. - `apt-get install nginx` still fails (`chown: changing ownership of '/var/log/nginx': Invalid argument`, dpkg exit 100). `fakeroot` **alone does not help** — libfakeroot is demonstrably active and its `chown` wrapper still reports the real syscall's EINVAL. `fakeroot` plus `FAKEROOTDONTTRYCHOWN=1` does. - Overlay works unprivileged on bwrap 0.11, `--overlay-src` stacks, and the upper directory is already a layer diff: changed files as themselves, deletions as character devices 0:0 — what OCI writes as `.wh.NAME`. Two snags: the workdir's `work/` is left at `d---------`, and bwrap refuses overlay when installed setuid. - With a writable root, bwrap creates a missing bind mountpoint itself and it stays in the tree — so a build binds only `/etc/resolv.conf`, which base images already have. ## The interface (the point of the RFC) `build.Backend` / `build.Tree` / `build.Step`. The boundary is exactly "how does a writable tree over a base come into being, and how is its result harvested"; no caller ever sets a `RunSpec`'s root fields. Everything else — parser, instructions, COPY, multi-stage, CLI, uid levers, commit — is identical in both worlds. - **Copy backend** (implemented): a real full copy of the base, root bound read-write, commit is the `rename`, no cache. It stays after the Debian 13 upgrade, as the fallback `doctor` picks when bwrap is too old or setuid, and for single-shot builds. - **Overlay backend** (deliberately *not* implemented): designed, with the measurements above as the evidence that it fits. The per-step cache waits for it, because without a union mount a cached step costs a whole tree copy. The warning that belongs with this, in the RFC and as a comment in the code: `store.copyTree` hardlinks so a second tag costs no data — there both copies are read-only. For a build tree it is unusable, because a `RUN` step writing in place through such a hardlink would modify the base image. ## Stage 1, implemented `werkdock build [-f Dockerfile] -t IMAGE CONTEXT` with docker's flags. `FROM`, `RUN`, `COPY` plus the metadata instructions `import -c` already knew, through the same `store.ApplyChange`. Staging and commit reuse the mechanics `import` has had since 0.11.0 (`<image>.tmp`, the per-image `flock`, `rename`), now exported as `store.Staging`. Werkdock sets both uid levers itself so the Dockerfile stays portable; the base image needs `fakeroot` installed, and `build` says so up front when it has not. `engine.RunSpec` gained `WritableRoot`, and a writable-root sandbox gets **no** implicit tmpfs over `/tmp`, `/root`, `/run` — with one, what a `RUN` step writes there would vanish in the next step while docker keeps it in the image. Refused loudly rather than ignored: `EXPOSE`, `HEALTHCHECK`, `STOPSIGNAL`, `ONBUILD`, `MAINTAINER` at parse time; `ARG`, `SHELL`, `ADD` and multi-stage named as stages 2 and 3; BuildKit's `RUN --mount` with the CapEff measurement as the reason; and the dozen `docker build` flags werkdock cannot keep the promise of. ## Verified - `go test ./...`, `go vet ./...`, `gofmt -l .` — green and empty. - A real sandbox integration test builds `FROM`/`WORKDIR`/`COPY`/`RUN`/`CMD` and then *runs* the result; it skips without bwrap/userns like the existing ones. - The apt path, which the levers exist for, is a second integration test pointed at a Debian base (`WERKDOCK_TEST_BASE_ROOTFS`). Run here: `nginx-common` configures, `dpkg-query` in the built image reports `install ok installed`, the apt fragment is gone from the result. - End to end through the CLI: `import` a 143 MB `debian:trixie` base with fakeroot, `build -t probe:1` with `apt-get install -y nginx jq` in the Dockerfile — **6.8 s** — then `run --rm probe:1` prints what its `RUN` steps and its `COPY` produced. Follow-ups the RFC names without deciding: store dedup between image versions, keeping layer blobs as a transport and cache format (the one with the highest running cost today), `doctor IMAGE`, and `ADD` from a URL. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: claude-code <claude-code@javagil.de> Co-authored-by: mhoennig <michael@hoennig.de> Reviewed-on: #14
170 lines
5.7 KiB
Go
170 lines
5.7 KiB
Go
package build
|
|
|
|
import (
|
|
"bufio"
|
|
"fmt"
|
|
"io"
|
|
"strings"
|
|
)
|
|
|
|
// Instruction is one logical Dockerfile line: the keyword uppercased
|
|
// (docker's own convention is upper case, but the syntax is
|
|
// case-insensitive) and the rest of the line with continuations
|
|
// joined. Line is the physical line the instruction started on, so an
|
|
// error names the place the author sees.
|
|
type Instruction struct {
|
|
Keyword string
|
|
Args string
|
|
Line int
|
|
}
|
|
|
|
// support says what the builder does with an instruction.
|
|
type support int
|
|
|
|
const (
|
|
// supportTree changes the work tree: FROM, RUN, COPY.
|
|
supportTree support = iota
|
|
// supportMeta goes into image.json through store.ApplyChange.
|
|
supportMeta
|
|
// supportLater is a real Dockerfile instruction that a later stage
|
|
// of RFC 0006 implements.
|
|
supportLater
|
|
// supportRefused will not be implemented, and says why.
|
|
supportRefused
|
|
)
|
|
|
|
// instructions is the whole instruction surface in one table: what is
|
|
// supported now, what a later stage brings, and what is refused with
|
|
// its reason. A keyword outside it is not a Dockerfile instruction.
|
|
var instructions = map[string]struct {
|
|
support support
|
|
note string
|
|
}{
|
|
"FROM": {supportTree, ""},
|
|
"RUN": {supportTree, ""},
|
|
"COPY": {supportTree, ""},
|
|
|
|
"ENTRYPOINT": {supportMeta, ""},
|
|
"CMD": {supportMeta, ""},
|
|
"ENV": {supportMeta, ""},
|
|
"WORKDIR": {supportMeta, ""},
|
|
"USER": {supportMeta, ""},
|
|
"VOLUME": {supportMeta, ""},
|
|
"LABEL": {supportMeta, ""},
|
|
|
|
"ARG": {supportLater, "ARG and ${} expansion are stage 2 of RFC 0006"},
|
|
"SHELL": {supportLater, "SHELL is stage 2 of RFC 0006; RUN uses /bin/sh -c"},
|
|
"ADD": {supportLater, "ADD is stage 2 of RFC 0006; COPY handles local files"},
|
|
|
|
"EXPOSE": {supportRefused, "the sandbox shares the host's network by contract, so there is no port to publish (RFC 0002)"},
|
|
"HEALTHCHECK": {supportRefused, "there is no daemon to poll a health check; werkdock runs no background probe"},
|
|
"STOPSIGNAL": {supportRefused, "stop sends SIGTERM and then SIGKILL after the timeout (RFC 0005); no per-image signal is read"},
|
|
"ONBUILD": {supportRefused, "an ONBUILD trigger needs a builder that re-reads it from the image config on a later FROM"},
|
|
"MAINTAINER": {supportRefused, "deprecated by docker itself; use LABEL maintainer=\"...\""},
|
|
}
|
|
|
|
// Parse reads a Dockerfile into its instructions: `#` comment lines
|
|
// and blank lines are dropped, a line ending in `\` continues on the
|
|
// next one (comment lines inside a continuation are dropped too, as
|
|
// docker does). Instructions a later stage brings and instructions
|
|
// werkdock refuses both fail here, with the reason and the line — a
|
|
// build that cannot be finished must not start by copying a 2 GiB
|
|
// tree.
|
|
func Parse(r io.Reader) ([]Instruction, error) {
|
|
var result []Instruction
|
|
var pending strings.Builder
|
|
startLine := 0
|
|
flush := func() error {
|
|
if startLine == 0 {
|
|
return nil
|
|
}
|
|
instruction, err := parseLine(pending.String(), startLine)
|
|
pending.Reset()
|
|
startLine = 0
|
|
if err != nil {
|
|
return err
|
|
}
|
|
result = append(result, instruction)
|
|
return nil
|
|
}
|
|
scanner := bufio.NewScanner(r)
|
|
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
|
|
for line := 1; scanner.Scan(); line++ {
|
|
text := scanner.Text()
|
|
trimmed := strings.TrimSpace(text)
|
|
if strings.HasPrefix(trimmed, "#") {
|
|
continue // a comment, inside a continuation as well
|
|
}
|
|
if startLine == 0 && trimmed == "" {
|
|
continue
|
|
}
|
|
continues := strings.HasSuffix(trimmed, "\\")
|
|
trimmed = strings.TrimSpace(strings.TrimSuffix(trimmed, "\\"))
|
|
if startLine == 0 {
|
|
startLine = line
|
|
} else if trimmed != "" {
|
|
pending.WriteString(" ")
|
|
}
|
|
pending.WriteString(trimmed)
|
|
if !continues {
|
|
if err := flush(); err != nil {
|
|
return nil, err
|
|
}
|
|
}
|
|
}
|
|
if err := scanner.Err(); err != nil {
|
|
return nil, err
|
|
}
|
|
// A file whose last line ends in a backslash: take what there is,
|
|
// as docker does.
|
|
if err := flush(); err != nil {
|
|
return nil, err
|
|
}
|
|
if len(result) == 0 {
|
|
return nil, fmt.Errorf("the Dockerfile holds no instruction")
|
|
}
|
|
if result[0].Keyword != "FROM" {
|
|
return nil, fmt.Errorf("line %d: a Dockerfile begins with FROM, not %s", result[0].Line, result[0].Keyword)
|
|
}
|
|
return result, nil
|
|
}
|
|
|
|
func parseLine(text string, line int) (Instruction, error) {
|
|
keyword, args, _ := strings.Cut(text, " ")
|
|
keyword = strings.ToUpper(strings.TrimSpace(keyword))
|
|
args = strings.TrimSpace(args)
|
|
known, ok := instructions[keyword]
|
|
if !ok {
|
|
return Instruction{}, fmt.Errorf("line %d: unknown instruction %s", line, keyword)
|
|
}
|
|
switch known.support {
|
|
case supportLater:
|
|
return Instruction{}, fmt.Errorf("line %d: %s is not supported yet: %s", line, keyword, known.note)
|
|
case supportRefused:
|
|
return Instruction{}, fmt.Errorf("line %d: %s is not supported: %s", line, keyword, known.note)
|
|
}
|
|
if args == "" {
|
|
return Instruction{}, fmt.Errorf("line %d: %s needs an argument", line, keyword)
|
|
}
|
|
if keyword == "RUN" && strings.HasPrefix(args, "--") {
|
|
// BuildKit's flags; as a shell command they would fail with a message that says nothing
|
|
flag, _, _ := strings.Cut(strings.Fields(args)[0], "=")
|
|
return Instruction{}, fmt.Errorf("line %d: RUN %s is not supported: %s", line, flag, runFlagNote(flag))
|
|
}
|
|
return Instruction{Keyword: keyword, Args: args, Line: line}, nil
|
|
}
|
|
|
|
// runFlagNote is why a RUN flag of BuildKit is refused.
|
|
func runFlagNote(flag string) string {
|
|
if flag == "--mount" {
|
|
return "a RUN step cannot mount anything (CapEff is 0 in the sandbox, RFC 0006); every mount stands in bwrap's argv"
|
|
}
|
|
return "it is BuildKit's, and werkdock build is no BuildKit (RFC 0006)"
|
|
}
|
|
|
|
// changesTree reports whether an instruction is one of the three that
|
|
// touch the work tree; the others are metadata for image.json.
|
|
func (i Instruction) changesTree() bool {
|
|
return instructions[i.Keyword].support == supportTree
|
|
}
|