werkdock/docs/rfcs/0005-services-and-testcontainers.md

RFC 0005: Service Images, Instance Lifecycle, and the Road to Testcontainers

Status:

• proposed: 2026-09-10
• accepted: 2026-09-10
• rejected: -

Proposal: Werkdock grows from a build sandbox into a runner for service images (PostgreSQL first), in four milestones that each pay for themselves: (1) a non-root uid inside the sandbox and the image's VOLUMEs, (2) an instance lifecycle (create, start, stop, rm, ps, logs, exec) as thin verbs over one internal service with state on disk, (3) that same service on a unix socket speaking the Docker Engine API subset Testcontainers uses, (4) Testcontainers validated against it with PostgreSQLContainer. Where Docker and Podman differ, werkdock follows Podman.

Context and Problem Statement

The goal (2026-09-10): Testcontainers — or a lightly adapted Testcontainers — shall work against werkdock, so that integration tests in the sandboxed builds get a real PostgreSQL. RFC 0002 designed for this as "level 3" and deferred it; pull (0.3.0) closed level 2, so the image side is done: werkdock pull postgres:16 lands with entrypoint, env, and volumes intact.

What is missing is everything a service needs and a build does not. Checked against what the official postgres image does at start:

1. A non-root uid. initdb and postgres refuse geteuid() == 0, real or namespace-internal. Werkdock maps the caller to uid 0 by contract (RFC 0002). The image's entrypoint is written for "runs as any non-root uid" and then skips its chown/gosu dance — so one non-zero uid suffices, which is all an unprivileged user namespace can map anyway.
2. Writable places the image declares. /var/lib/postgresql/data and /var/run/postgresql are VOLUMEs in the image; docker and podman mount anonymous volumes there. Werkdock's rootfs is read-only; today the caller must know to add -v and --tmpfs.
3. A long-lived process. Only run --rm in the foreground exists. A server needs -d, and with it instances that outlive the CLI call, a way to stop them, and their output on disk for logs.
4. The API. Testcontainers never invokes a CLI; it speaks the Engine API over DOCKER_HOST. Podman reaches Testcontainers exactly this way (podman system service), with a socket that is optional and may exit when idle — a good model for a daemonless tool.

The network stays the host's (RFC 0002): a service binds a host port directly, and two instances wanting the same port collide. That limit is accepted here and worked around per image where it hurts (see Ports below); lifting it needs a network namespace and its own RFC.

Principles

• Podman before Docker. Werkdock is rootless and daemonless, as Podman is. Where the two differ in flags, defaults, or API details, werkdock takes Podman's side and says so in the CHANGELOG: --userns=keep-id for "run as myself", fully qualified image names preferred over silent Docker Hub defaults, --mount type=image in Podman's spelling, the API server as an optional frontend that may exit. Where Podman serves the Docker Engine API unchanged, that API is the target — it is what Testcontainers speaks.
• One uid. An unprivileged user namespace maps the caller to exactly one uid inside. --user chooses which; there is no second user, no setuid, no chown to another owner. Images that need two users (gosu-style entrypoints as root) are out of scope; the official service images support the single-uid mode.
• Verbs are frontends. Lifecycle logic lives in one internal service with instance state on disk (RFC 0002). The CLI and the API server call it; neither owns it. This is what keeps milestone 3 small.
• Every milestone stands alone. Non-root plus volumes lets a build start PostgreSQL in the foreground beside its tests, without any lifecycle. The lifecycle serves long-running builds and manual use without any API. The API serves Testcontainers.
• Filesystem-only stays. Nothing here adds network, resource, or device isolation; refusals stay refusals.

Milestones

1. Non-root uid and image volumes (this branch, first)

• --user UID[:GID] (numeric, or a name resolved in the image's /etc/passwd) maps the caller to that uid/gid inside; --userns=keep-id (Podman) maps the caller to the caller's own uid/gid. Default stays uid 0, as it is for Podman rootless. An image User (OCI config) applies when no flag is given, as podman does.
• HOME follows the image's passwd entry for the uid when there is one, else /root. /root stays a tmpfs, so a nameless uid still has a writable home.
• /run joins /tmp and /root as an implicit tmpfs: runtime state by FHS definition, /var/run's target in practically every image, and where services write without declaring a VOLUME (postgres fails on /var/run/postgresql otherwise). A writable instance layer — docker's container layer — is not this milestone's answer: without overlayfs it would be a rootfs copy per run.
• VOLUME paths from the image config get an anonymous tmpfs unless a -v/--mount covers them — docker's anonymous volume, ephemeral like everything under --rm. import -c 'VOLUME /data' and USER join the accepted instructions; inspect shows both.
• Proof: werkdock pull postgres:16-alpine, then one run --rm --user postgres postgres:16-alpine sh -c 'initdb … && pg_ctl start && psql -c "select 1" && pg_ctl stop', locally and on the webspace. Done 2026-09-10 (0.4.0).
• Finding: --userns=keep-id is not enough for PostgreSQL — initdb looks its uid up in /etc/passwd and the image has no entry for the caller's uid (could not look up effective user ID 1000). Podman has the same limit; --user postgres is the answer for images that ship the user, an /etc/passwd bind for the rest.

2. Instance lifecycle

• An instance is a directory under $WERKDOCK_HOME/instances/<id>/ with config.json (the RunSpec plus name, image, created time, --rm; version: 1), state.json (status, supervisor pid, the command's host pid, started, finished, exit code, error), and stdout/stderr files. Naming follows docker: 64-hex id, --name, else the image's basename plus six characters of the id; verbs take an id, a unique prefix, or a name.
• run -d and create/start fork bwrap under a small supervisor process (werkdock supervise ID, this binary in a session of its own) that writes state, records the exit code, and removes an --rm instance at the end; ps, logs [-f], wait, rm, inspect read the files. stop is SIGTERM to the command, SIGKILL after -t seconds; kill any signal. Neither talks to the supervisor: the command's host pid is in state.json, and a state that says running while the supervisor is gone (reboot, kill -9) is repaired on the next read as exited 255 — the same 255 docker reports.
• The command is pid 1 of the sandbox (bwrap --as-pid-1), as in docker: a signal from outside reaches it if it handles the signal, and bwrap reports its host pid on --info-fd. bwrap's own init would swallow SIGTERM (measured: SIGTERM to bwrap's pid-1 init is ignored; to the command under --as-pid-1 it arrives). Consequence, also docker's: an entrypoint without a SIGTERM handler (sh -c) stops only by the SIGKILL after the timeout, and orphaned children are the command's to reap.
• /run and the anonymous VOLUMEs are directories of the instance (run/, volumes/<path>), bound read-write, not tmpfs: so a socket in /var/run/postgresql and the data in /var/lib/postgresql/data survive stop/start (docker's anonymous volumes live as long as the container) and are visible to exec. /tmp and /root stay tmpfs per bwrap invocation.
• exec is a second bwrap over the same instance: rootfs, mounts (including /run and the volumes), uid, and environment, with -e/-w/-u overrides — not the same pid namespace, so psql works over the socket or TCP, kill and ps do not see the service. A detached exec is refused: it would have no instance to record it.
• --rm keeps working as today for foreground runs (the instance exists while the command runs, so ps and stop see it, and no logs are recorded); a foreground run without --rm records its output and stays for ps -a, logs, start. The first SIGINT/SIGTERM/SIGHUP a foreground run receives is forwarded to the command, a second one or ten seconds without exit kills it; bwrap runs in its own process group so a terminal's Ctrl-C reaches werkdock, not bwrap.
• Proof: run -d --name pg --user postgres -e PGPORT=… postgres:16-alpine, exec pg psql -U postgres over the unix socket, stop, start, data still there, rm. Done 2026-09-10 (0.5.0), locally and on the webspace.

3. API socket

• werkdock system service [--time D] [unix://PATH] (Podman's verb) serves, with net/http, on $XDG_RUNTIME_DIR/werkdock.sock. The endpoints are those a real Testcontainers run sends, measured rather than taken from the API documentation and written down in docs/testcontainers-api.md: /info, /version, /_ping, images list/inspect/create, containers create/start/json/logs/kill/stop/wait/delete/list, exec create/start/inspect, and the three prune calls. Unimplemented endpoints answer 501 with the reason, never 404.
• Findings from the measurement, each of which removed work or risk:
  • POST /exec/{id}/start is a plain request with a framed response body, not an HTTP upgrade. No connection hijacking is needed.
  • logs and exec use docker's frame format: an 8-byte header per chunk, the stream number first, the length big-endian in the last four bytes.
  • A container is created with an empty HostPort, meaning "choose a free one", and the answer is read back from NetworkSettings.Ports.
  • Testcontainers refuses a server below Docker 1.6.0, so /version must report the Engine version whose API is served, not werkdock's 0.x. Werkdock's own version goes into Components and /info's OperatingSystem.
  • The session cleanup lists containers filtered by label, so instances carry labels and the list filters on them.
• Ryuk stays disabled (TESTCONTAINERS_RYUK_DISABLED=true): it wants the API socket bind-mounted into a container of its own, which werkdock does not offer. Werkdock's --rm and rm cover what Ryuk would.
• Done 2026-09-10 (0.6.0).

4. Testcontainers

• Validated with PostgreSQLContainer (Testcontainers 1.19.3) under DOCKER_HOST=unix://…/werkdock.sock and TESTCONTAINERS_RYUK_DISABLED=true: the container starts, the log wait strategy passes, JDBC connects and queries, execInContainer returns output and an exit code, and the session cleanup finds and removes what it made. The "adapted" Testcontainers turned out to be configuration only — one environment variable — as hoped.
• Done 2026-09-10 (0.6.0), on the developer machine and on the Hostsharing managed webspace, the environment that had to work.

Ports

Testcontainers asks for the host port mapped to a container port; without a network namespace werkdock can map nothing. Measured on the target, the identity answer is not merely imprecise but wrong: the webspace's own PostgreSQL holds 5432, so a container told to publish it cannot bind at all. The API server therefore picks a free port and passes it to the service, through the environment variable the image honours (PGPORT, MARIADB_TCP_PORT, MYSQL_TCP_PORT, REDIS_PORT), and reports that port. This per-image knowledge lives in a small table (package api, quirks), which also carries the user an image needs when its entrypoint expects to drop privileges itself, and is the deliberate deviation from docker. An image outside the table keeps its own port, which is correct as long as one instance of it runs at a time; a request to move such a port fails with the reason rather than reporting a port nothing listens on.

Consequences

• RFC 0002's --user refusal ends; the "always uid 0" wording in README and usage becomes "uid 0 by default".
• USER and VOLUME are no longer refused by import -c; existing images are unaffected (zero values).
• The instance directory format (milestone 2) is the contract between CLI and API server and is versioned in config.json.
• No change to the store, to pull/load, or to Werkator's build path.

Considered Alternatives

• Adapting Testcontainers to call the werkdock CLI: rejected — Testcontainers core is welded to docker-java and the Engine API; the API subset is smaller than a fork.
• PostgreSQL only, without a lifecycle (start it inside the build's own run): kept as what milestone 1 already enables, not as the end state — Testcontainers needs the API regardless.
• Network namespaces now (pasta/slirp4netns, like rootless Podman): the honest fix for port collisions, but a second isolation dimension with host prerequisites (/dev/net/tun, a userspace stack binary) that the webspaces do not offer; own RFC when the identity mapping proves insufficient.

Decision Outcome

Accepted — all four milestones are implemented (0.4.0, 0.5.0, 0.6.0) and Testcontainers runs against werkdock on the target environment. What remains is breadth, not structure: the socket inside a build sandbox for Werkator's own integration tests.

Since then, jdbc:tc URLs — Testcontainers as the JDBC driver, the form most projects use — were validated too, with Testcontainers 1.21.4 and a Spring Boot application in Werkprobe. They needed no endpoint beyond the subset, and a JDBC init script (TC_INITSCRIPT) travels over the connection rather than through the archive endpoint, so that gap was narrower than it was written down as. It closed in 0.7.0: PUT /containers/{id}/archive unpacks into the instance and binds the result in, rather than writing into the shared read-only image, and withCopyFileToContainer works.

The quirk table was checked against the real images in 0.8.0, and three of its four entries were wrong: only postgres reads an environment variable for its port. redis, mariadb, and mysql take a command-line flag, which the table can now express — and until then werkdock reported ports that nothing listened on. The lesson is the RFC's own: a deviation from docker has to be measured against the thing it deviates for, or it is a guess wearing a table's clothes.