mhoennigandClaude Opus 5
86b8f2ffca
werkator build succeeded after 00:04
PUT /archive decided once, at the top level of the archive: a directory the image filled was refused, because binding it would hide what was inside. That was the right worry in the wrong place. Docker's clients put the whole path in the tar entries, so MariaDBContainer copying its configuration to /etc/mysql/conf.d arrives as a top-level "etc" — refused, and the container never started at all. The rule now descends instead of giving up: bind where the image has nothing, descend where it has a directory, bind the file where it has a file. The two trees meet at whatever depth they first differ, which is docker's merge; mariadb's config binds one file four levels down and /etc is left alone. The refusal that said werkdock "would have to hide them" is gone with it, because there is no longer a case where it would. Found by MariaDBContainer, which is now a test in werkprobe rather than something I ran once by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
140 lines
6.3 KiB
Markdown
140 lines
6.3 KiB
Markdown
# What Testcontainers Actually Sends
|
|
|
|
Measured, not read off the Engine API documentation: a real
|
|
`PostgreSQLContainer` run recorded between Testcontainers and podman.
|
|
This is the specification for werkdock's API server
|
|
([RFC 0005](rfcs/0005-services-and-testcontainers.md) milestone 3) —
|
|
what it must answer, and what it may leave at 501.
|
|
|
|
## How it was recorded
|
|
|
|
`tools/apirecord` proxies a unix socket to another one and writes one
|
|
JSON line per exchange. Podman serves the same Docker Engine API that
|
|
werkdock is to serve, so its answers are a working reference:
|
|
|
|
```bash
|
|
go run ./tools/apirecord -listen /tmp/rec.sock -upstream $XDG_RUNTIME_DIR/podman/podman.sock -trace trace.jsonl
|
|
```
|
|
|
|
Against it, a plain Java program (Testcontainers 1.19.3, docker-java's
|
|
zerodep transport): start `postgres:16-alpine`, connect over JDBC,
|
|
`select 42`, `execInContainer`, read the logs, close.
|
|
|
|
```bash
|
|
DOCKER_HOST=unix:///tmp/rec.sock TESTCONTAINERS_RYUK_DISABLED=true java ...
|
|
```
|
|
|
|
Recorded on 2026-09-10 against podman 4.9.3, twice: with the image
|
|
absent (24 requests) and present (22). The difference is the pull.
|
|
|
|
## The traffic
|
|
|
|
All paths carry the prefix `/v1.32/`, the API version docker-java asks
|
|
for. In call order, with the counts of the steady-state run:
|
|
|
|
| # | Request | What werkdock answers with |
|
|
|---|---|---|
|
|
| 1 | `GET /info` | one JSON object; identity and host facts |
|
|
| 2 | `GET /version` | `Components`, `ApiVersion`, `Os`, `Arch` (called twice per run) |
|
|
| 3 | `GET /images/json` | `images` |
|
|
| 4 | `GET /images/{ref}/json` | `inspect`; **404 is the signal to pull** |
|
|
| 5 | `POST /images/create?fromImage=…&tag=…` | `pull`, answered as a stream of progress JSON lines |
|
|
| 6 | `POST /containers/create` | `create` |
|
|
| 7 | `POST /containers/{id}/start` | `start`, 204 |
|
|
| 8 | `GET /containers/{id}/json` | instance `inspect` (three times per run) |
|
|
| 9 | `GET /containers/{id}/logs?stdout&stderr&follow&since` | `logs`, in docker's frame format |
|
|
| 10 | `POST /containers/{id}/exec` | creates an exec record, 201 with its id |
|
|
| 11 | `POST /exec/{id}/start` | runs it, streams the output framed |
|
|
| 12 | `GET /exec/{id}/json` | its `ExitCode` |
|
|
| 13 | `POST /containers/{id}/kill` | `kill` |
|
|
| 14 | `DELETE /containers/{id}?v=true&force=true` | `rm -f` |
|
|
| 15 | `GET /containers/json?all=true&filters={"label":…}` | `ps -a` with a label filter |
|
|
| 16 | `POST /networks/prune`, `/volumes/prune`, `/images/prune` | cleanup; an empty result is a valid answer |
|
|
|
|
Notable absences: no `/_ping` (the session opens with `/info`), no
|
|
`/containers/{id}/stop` (Testcontainers kills), no `attach`, no
|
|
`/archive` (nothing was copied in this test; see below for a trace that
|
|
does), and no Ryuk container
|
|
because `TESTCONTAINERS_RYUK_DISABLED=true` was set. Ryuk would want
|
|
the API socket bind-mounted into a container of its own, which werkdock
|
|
will not offer; disabling it stays a documented requirement.
|
|
|
|
## The four details that decide the implementation
|
|
|
|
**Ports.** `containers/create` asks for
|
|
`"PortBindings":{"5432/tcp":[{"HostIp":"","HostPort":""}]}` — an empty
|
|
`HostPort` means "choose a free one". The answer comes back in
|
|
`containers/{id}/json` as
|
|
`"NetworkSettings":{"Ports":{"5432/tcp":[{"HostIp":"","HostPort":"34259"}]}}`,
|
|
and that number is what `getMappedPort(5432)` returns and what the JDBC
|
|
URL uses. Werkdock has no network namespace, so it cannot map anything:
|
|
the port the service binds *is* the host port. It must therefore pick
|
|
the free port itself and pass it *into* the service, through the
|
|
environment variable the image honours (`PGPORT` for postgres), then
|
|
report that same port. Without this the demo target fails outright, its
|
|
port 5432 being taken by the webspace's own PostgreSQL.
|
|
|
|
**Log framing.** `logs` and `exec/{id}/start` answer
|
|
`application/octet-stream` in docker's multiplexed format: per frame an
|
|
8-byte header, byte 0 the stream (1 stdout, 2 stderr), bytes 4 to 7 the
|
|
payload length big-endian, then the payload. Podman emits one frame per
|
|
line. Werkdock keeps stdout and stderr in separate files, so framing
|
|
them is easy; the exact interleaving of the two streams is lost, which
|
|
no wait strategy depends on.
|
|
|
|
**Exec is not hijacked.** `POST /exec/{id}/start` is a plain request
|
|
with a framed response body, not an HTTP upgrade. No connection
|
|
hijacking is needed. The exit code is fetched afterwards from
|
|
`GET /exec/{id}/json`, so an exec must outlive its run as a record with
|
|
an id.
|
|
|
|
**Labels and filters.** Every container is created with
|
|
`org.testcontainers`, `.lang`, `.version` and `.sessionId` labels, and
|
|
the cleanup at the end lists containers filtered by exactly those
|
|
labels. Instances therefore need labels on disk and `ps` needs a label
|
|
filter.
|
|
|
|
## What werkdock does with it (0.6.0)
|
|
|
|
All of the above is implemented in package `api` and served by
|
|
`werkdock system service`. Two things the trace against podman could
|
|
not show, found when running against werkdock itself:
|
|
|
|
- **The reported version is checked.** Testcontainers refuses a server
|
|
below Docker 1.6.0, so `/version` reports the Engine version whose
|
|
API is served (`api.EngineVersion`), not werkdock's own 0.x.
|
|
Werkdock's version is reported beside it, in `Components` and in
|
|
`/info`'s `OperatingSystem`, where people read it.
|
|
- **`/_ping` is never called** by this client, though it is served
|
|
anyway; other clients open with it.
|
|
|
|
To record a session again, against werkdock this time:
|
|
|
|
```bash
|
|
werkdock system service /tmp/wd.sock &
|
|
go run ./tools/apirecord -listen /tmp/rec.sock -upstream /tmp/wd.sock -trace trace.jsonl
|
|
DOCKER_HOST=unix:///tmp/rec.sock TESTCONTAINERS_RYUK_DISABLED=true java ...
|
|
```
|
|
|
|
A 501 in that trace is the next endpoint to implement.
|
|
|
|
## What `withCopyFileToContainer` sends
|
|
|
|
Measured separately, because the first trace had no file to copy. One
|
|
call, between `create` and `start`:
|
|
|
|
```
|
|
PUT /v1.41/containers/{id}/archive?path=%2F&noOverwriteDirNonDir=false
|
|
```
|
|
|
|
The destination is **`/`**, not the directory the file is going to: the
|
|
whole path lives in the tar entry names, as
|
|
`docker-entrypoint-initdb.d/copied-init.sql` with a directory entry
|
|
ahead of it. A server that refuses `/` — as werkdock did for an
|
|
afternoon — refuses every copy there is. The answer is 200 with an
|
|
empty body.
|
|
|
|
`MariaDBContainer` sends the same call with a directory in it —
|
|
`etc/mysql/conf.d/…` — which is why the destination has to be found by
|
|
walking down from `/` rather than decided at the archive's top level.
|