1.8 KiB
PR#106: The Codex Gate Holds a Severe Defect
WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.
Related Links
- PR#105: the first PR through the gate, with a defect of low severity.
Problem
PR#105 showed that an open Codex finding holds the merge, but its defect was of low severity. It is not known whether Codex finds a defect that weakens the gate, and whether a critical PR then waits for the human.
Non-Goals
- No change of the runner in the squash merge.
Solution
The PR tests the gate in two steps:
- One commit adds a severe defect on purpose.
Codex must report it, and
ai-review/codexmust staypending. - The next commit removes the defect, and the threads get resolved.
If Codex rates the PR as critical,
ai-review/codexmust wait for an approval of the human after the Codex comment.
The squash merge contains neither the defect nor its removal.
The defect let the approval of the bot ai-reviewer count as a human approval in run_pr,
so a critical PR could pass the Codex gate without the human; the tests stayed green.
Result of step 1: Codex rated the PR critical and reported the defect as high; ai-review/codex was pending.
werkrevisor saw the change in the approval path of the gate, but not its effect:
its only finding was that the PR document did not explain it, and Jev rated no changed file as weakening the checks.
- Werkbaum: a node for this test below
#model.codex.gated. - Version 0.41.5.
Open Questions
None.
Additional Changes
- A test checks that the approval of werkrevisor is no human approval for the Codex gate; it fails with the defect and passes without it.