Answer to the review of 8e89649:
- tests lost or disabled in
tests/test_codex_eval.py: no test is lost. The four tests of PR#87 moved with their code totests/test_codex.py(`werkrevis…
Fixed in 8e89649: both bullets now name the critical PR and the finding outside the diff, as gate_state does.
Fixed in 16af857: the comparison uses the first 250 characters, as Gitea keeps them; the test covers a description of 300 characters.
Fixed in 16af857: the runner reviews only a head that werkrevisor rated GREEN, YELLOW or ORANGE; the test covers a RED head.
16af857e26
The runner skips RED heads and compares the shortened status text; scenario 88.07 without two actions
45743f10d2
A Codex answer needs every field of the schema; tests of tools/codex-eval; scenario 88.07 without an earlier run
Fixed in 45743f1: the Given now names only the state before the run, a PR whose only unresolved line comment comes from Codex, as the test sets it up.
Fixed in 45743f1: valid_answer also requires critical_reason, title and explanation as strings; the test covers a missing title and a missing explanation.
Fixed in 98a07b9: the test now adds a real Codex review with a line comment before werkrevisor runs, instead of rewriting a writing-rule comment.
Fixed in 98a07b9, with one correction: the sandbox of Codex already blocks the network of its commands; a probe inside the runner setup ended curl https://example.org with exit 6. The real gap…
Fixed in 98a07b9: values of the login from 8 characters on are secrets now; the test covers a value of 10 characters, and it fails with the old limit of 16. Shorter values of auth.json are…
98a07b945c
Codex runs in bwrap without the secrets of the workstation; short login values are secrets too
Fixed in af6613e: a WIP PR gets no status at all; only a PR without "WIP: " and without a result of werkrevisor gets pending in gate mode, as run_pr does.