49 lines
1.8 KiB
Markdown
49 lines
1.8 KiB
Markdown
# PR#106: The Codex Gate Holds a Severe Defect
|
|
|
|
> **WARNING:** This document describes only the change applied in this PR.
|
|
> It may already be outdated once the next PR is merged.
|
|
> Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.
|
|
|
|
## Related Links
|
|
|
|
- [PR#105](2026-10-14-PR#105-codex-gate-test.md): the first PR through the gate, with a defect of low severity.
|
|
|
|
## Problem
|
|
|
|
PR#105 showed that an open Codex finding holds the merge, but its defect was of low severity.
|
|
It is not known whether Codex finds a defect that weakens the gate,
|
|
and whether a critical PR then waits for the human.
|
|
|
|
## Non-Goals
|
|
|
|
- No change of the runner in the squash merge.
|
|
|
|
## Solution
|
|
|
|
The PR tests the gate in two steps:
|
|
|
|
1. One commit adds a severe defect on purpose.
|
|
Codex must report it, and `ai-review/codex` must stay `pending`.
|
|
2. The next commit removes the defect, and the threads get resolved.
|
|
If Codex rates the PR as critical, `ai-review/codex` must wait for an approval of the human after the Codex comment.
|
|
|
|
The squash merge contains neither the defect nor its removal.
|
|
|
|
The defect let the approval of the bot `ai-reviewer` count as a human approval in `run_pr`,
|
|
so a critical PR could pass the Codex gate without the human; the tests stayed green.
|
|
Result of step 1: Codex rated the PR critical and reported the defect as high; `ai-review/codex` was `pending`.
|
|
werkrevisor saw the change in the approval path of the gate, but not its effect:
|
|
its only finding was that the PR document did not explain it, and Jev rated no changed file as weakening the checks.
|
|
|
|
- Werkbaum: a node for this test below `#model.codex.gated`.
|
|
- Version 0.41.5.
|
|
|
|
## Open Questions
|
|
|
|
None.
|
|
|
|
## Additional Changes
|
|
|
|
- A test checks that the approval of werkrevisor is no human approval for the Codex gate;
|
|
it fails with the defect and passes without it.
|