werkrevisor 0.77.0: the Codex gate keeps a human approval when the change since then is not critical #144
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mi/werkrevisor#144
Reference in New Issue
Block a user
The Codex gate carries a human approval over to a later head when Codex rates the commits since then as not critical, or when a rebase kept the same blocks.
See
docs/prs/for the details.🤖 Generated with Claude Code
1301d7febatocca713b961🟠 werkrevisor ORANGE for
cca713b, superseded by the result fore1070e9🟠 werkrevisor: ORANGE / FINDINGS
Gate: the approval of mi came before this result; review requested again, please approve again after reading it; merge scheduled for when the checks succeed.
Change kind:
mixed.Reasons
Findings
werkrevisor/codexrunner.py:433-435, ORANGE, as a line commentClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 46152, calls 1, tokens in 44341 / out 27742 (reasoning included in out), duration 244.9 s, cost 0.0466 USD (reported), classifier `jev-1.14.0` calls 10, tokens in 8341, cost 0.0004 USD, finding judge `jev-1.14.0` calls 1, tokens in 1807, cost 0.0001 USD, PR document check `jev-1.14.0` calls 9, tokens in 28725, cost 0.0012 USD, build file judge `jev-1.14.0` calls 6, tokens in 8912, cost 0.0004 USD, phases: path_judge 0.3 s, classifier 0.4 s, doc_check 0.5 s, context 0.3 s, model_review 242.7 s, finding_judge 0.1 s, config: defaults + host + repository (.werkrevisor.toml at main3 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `cca713b961f14a726802a701eaca289ed28a4e56` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
Diagrams
Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.
werkrevisor: ORANGE / FINDINGS for
cca713b961f14a726802a701eaca289ed28a4e56.The result comment of werkrevisor holds the details.
@@ -382,2 +431,3 @@unattached = [i for i in range(len(findings)) if i not in attached]body = result_comment(head, record, set(attached), unattached, mode)answer = record["answer"]carried = None[high] Delta ignored (werkrevisor)
ORANGE; Jev: real 0.81, severity 3.0 of 3
The new delta rating is used only to decide whether to create a carried marker; it does not determine whether the gate requires a human.
result_commentstill derives the critical marker solely from the overall answer. For example, if an approved head adds a critical protection and the new head removes it, the net PR can be ratedcritical: falsewhile the delta is correctly ratedsince_approval_critical: true. No carried marker is created in that case, andgate_statesees a non-critical marker and passes without a current-head approval. Use the delta rating to require human review, or reject such answers, and add this net-revert scenario to the runner tests.Fixed in
e1070e9:codex.needs_humanis true when Codex rates the change since the approval as critical, also in a PR that is not critical as a whole; the result comment names the rating. New testtest_a_critical_change_since_the_approval_needs_a_human_also_in_an_uncritical_pr, red without the fix.Codex review of
cca713b961f14a726802a701eaca289ed28a4e56@@ -85,1 +93,4 @@SINCE_APPROVAL = """4. A human approved the earlier head {approved} after its review.The commits since then are `git log -p --cc {approved}..HEAD ^{base}`:[high] Combined diffs hide security-relevant merge resolutions (Codex review)
git log --cc omits resolutions that select one parent's content. If main adds a security check and a merge retains the PR parent's version without that check, this command shows no patch for the removal; main's commits are also excluded. The incremental assessment can consequently report no critical change and carry the approval even when the full review correctly marks the removal as critical.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
e1070e9: Codex now compares the change of the PR then and now,git diff <merge base of approved and base>..<approved>againstgit diff <base>..HEAD, also in binary files, modes and renames. A merge that keeps the PR side and drops a check ofmainshows as a removal in the second diff.@@ -79,0 +81,4 @@"""After a rebase: werkrevisor found the same changed blocks for both heads."""found = [carryover.parse_marker((report.find_existing(comments, sha, login)[0] or {}).get("body"))for sha in (approved_head, head)]return None not in found and found[0] == found[1][high] Block equality overlooks new binary changes (Codex review)
The markers contain only text-hunk fingerprints; binary changes, pure renames and mode-only changes contribute no blocks. After a rebase, retaining the approved text edits while adding an executable binary therefore satisfies this comparison. I reproduced the resulting Codex gate success despite critical=true and no current-head approval.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
e1070e9: after a rebase the runner no longer compares the block fingerprints itself. It needs werkrevisor's own carry-over of the same human approval, "the changed blocks are those of the approved head"; werkrevisor counts a file without text hunks, e.g. a binary file, a mode change or a rename, as a new file and carries nothing over then. The test with equal fingerprints but no such carry-over is red with the old comparison.Codex review: critical
This changes the human-approval requirement of a merge gate. Mistakes can authorize critical changes that no human reviewed.
Security effect: weakens; fails closed: no; a test of the closed case: no.
Findings
werkrevisor/codexrunner.py:84, as a line commentwerkrevisor/codex.py:96, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 375904 (cached 318848) / out 3555, 102.3 s, 8 commands; reviewed head `cca713b961f14a726802a701eaca289ed28a4e56`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟠 werkrevisor ORANGE for
e1070e9, superseded by the result for55ec69c🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
Findings
[high] Marker Injection at
werkrevisor/codexrunner.py:98-100, ORANGE, as a line comment[high] Prose Authorization at
werkrevisor/codexrunner.py:80-87, ORANGE, as a line comment[medium] Missing Diagram at
docs/werkrevisor.werkbaum:167-168hint; Jev: real 0.78, severity 2.1 of 3
This adds a new Codex carry-over decision, but the PR does not update the review diagrams under
docs/.AGENTS.mdexplicitly requires diagram updates whenever review steps or decisions change; leaving the diagrams unchanged documents the old rule that an earlier approval never unlocksai-review/codex. Update the relevant diagrams in this PR.Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 47899, calls 1, tokens in 46032 / out 37758 (reasoning included in out), duration 325.7 s, cost 0.0591 USD (reported), classifier `jev-1.14.0` calls 10, tokens in 8545, cost 0.0004 USD, finding judge `jev-1.14.0` calls 3, tokens in 5254, cost 0.0002 USD, PR document check `jev-1.14.0` calls 9, tokens in 31183, cost 0.0013 USD, build file judge `jev-1.14.0` calls 6, tokens in 9540, cost 0.0004 USD, phases: path_judge 0.3 s, classifier 0.4 s, doc_check 0.5 s, context 0.3 s, model_review 323.3 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main2 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `e1070e9f47e3307bcadef297c0c5b417f4d04676` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
Diagrams
Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.
werkrevisor: ORANGE / FINDINGS for
e1070e9f47e3307bcadef297c0c5b417f4d04676.The result comment of werkrevisor holds the details.
@@ -76,10 +77,29 @@ def load_config(path):int(data.get("max_reviews_per_run", 3)), servers)def _same_blocks(comments, approval, head, login):[high] Prose Authorization (werkrevisor)
ORANGE; Jev: real 0.89, severity 3.0 of 3
_same_blocks()treats an English sentence anywhere in the werkrevisor result as proof that a rebase preserved approved blocks. That result also contains model-generated reasons, so if the model follows a crafted instruction, an author can make it emit the exact sentence and SHA; the runner then skips the Codex comparison and carries the old approval even for a critical rebased change. Represent this fact with a dedicated machine-readable marker in a fixed section.Fixed in
55ec69c: the runner no longer reads the werkrevisor result for the rebase case. It compares the change itself in git,codex.change_signatureof<base then>..<approved>and<base>..HEAD.@@ -83,0 +95,4 @@return f"<!-- werkrevisor-codex-carried:{sha} -->"def carried_head(body):[high] Marker Injection (werkrevisor)
ORANGE; Jev: real 0.79, severity 3.0 of 3
carried_head()searches the entire result comment for a carry marker, butresult_comment()places model-produced fields such ascritical_reasonandsince_approval_reasonin that same comment. A crafted PR description can ask Codex to echo a marker containing the known approved SHA; if the model complies,update_gate()treats it as a genuine carried approval andgate_state()suppresses the critical rating. Parse only a fixed bot-generated marker line, never model-produced text.Fixed in
55ec69c: the runner reads its markers only fromheader, the marker lines at the start of its own comment; Codex text comes after the heading. This also covers the older markerswerkrevisor-codex-unattachedand the head marker. Testtest_text_of_codex_never_passes_for_a_markerforges all three in the reasons; it is red with the old substring search.Codex review of
e1070e9f47e3307bcadef297c0c5b417f4d04676@@ -344,1 +392,3 @@ok = record["exit_code"] == 0 and codex.valid_answer(record["answer"])approval, comments = self.earlier_approval(server, gitea, number, head, login)since = approval.head_sha if approval and codex.is_ancestor(work, approval.head_sha) else Nonesame_blocks = bool(approval) and not since and _same_blocks(comments, approval, head, login)[high] Rebase carry-over overlooks mode changes alongside approved text hunks (Codex review)
The werkrevisor carry-over message does not prove that file metadata stayed unchanged: its block comparison ignores modes on files containing approved text hunks. A rebased head can therefore add an executable-bit change to an already edited file and still satisfy _same_blocks. This bypasses the since-approval assessment and carries the approval even when Codex marks the current change critical. An in-memory reproduction with an additional 100644→100755 change produced a successful Codex gate. Validate metadata changes before accepting this carry-over.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
55ec69c:codex.change_signaturehashesgit diff --binary -Mwithout line numbers, blob ids, similarity indexes and context lines, and keeps modes, renames and binary patches. A real-git test shows that a rebase keeps the signature and that a mode change, a binary file and a rename each change it; with the mode lines dropped, the test is red.Codex review: critical
This changes the human-approval merge gate. A mistake can allow unreviewed security-sensitive changes to merge, so human review is required.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codexrunner.py:394, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 544024 (cached 475520) / out 3486, 105.6 s, 10 commands; reviewed head `e1070e9f47e3307bcadef297c0c5b417f4d04676`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.Answer to the review of
e1070e9:The diagrams show no decision of the Codex review:
docs/werkrevisor-pr-workflow.svghas one box "Codex review after werkrevisor, optional; in gate mode it holds the merge", andreview_decisions.pyandreview_inside.pydo not name Codex.55ec69cwith the line comments above.🟢 werkrevisor GREEN for
55ec69c, superseded by the result for25ef3e2🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 52853, duration 724.9 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 8738, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 35933, cost 0.0015 USD, build file judge `jev-1.14.0` calls 6, tokens in 11555, cost 0.0005 USD, phases: path_judge 0.3 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 722.6 s, config: defaults + host + repository (.werkrevisor.toml at main2 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `55ec69c797eea9179fe2e1b4f7200cac6b6708ed` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
55ec69c797eea9179fe2e1b4f7200cac6b6708ed.The result comment of werkrevisor holds the details.
Codex review of
55ec69c797eea9179fe2e1b4f7200cac6b6708ed@@ -195,0 +246,4 @@if line.startswith(b"index "):# "index <old>..<new> <mode>": the mode of an unchanged mode stays.kept.append(b"index " + b" ".join(line.split(b" ")[2:]))elif line.startswith(b"@@"):[high] Change signatures treat edits to different functions as identical (Codex review)
Removing hunk locations and all context makes identical replacements elsewhere in the same file produce equal signatures. For example, restoring an approved
return False→return Trueedit in a public-view permission function and applying it instead in an account-deletion permission function preserves the signature._reviewthen skips the since-approval assessment and carries the old approval;gate_statepasses even when Codex marks the current head critical. This allows a new authorization change to merge without human approval.Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
25ef3e2: the signature keeps the context lines and the function heading of each hunk; only the line numbers, blob ids and similarity indexes are dropped. The real-git test now applies the samereturn False→return Truein another function and expects another signature; it is red with the old signature. A rebase that changes the context of an edit now needs a new approval.Codex review: critical
This changes the merge gate’s authorization rules by allowing an earlier human approval to authorize a later head. A mistake can bypass required human review of security-sensitive changes.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codex.py:249, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 432911 (cached 374528) / out 3376, 102.6 s, 14 commands; reviewed head `55ec69c797eea9179fe2e1b4f7200cac6b6708ed`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
25ef3e2, superseded by the result for10b62fa🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
55ec69cnot carried over: 4 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 53514, duration 724.8 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 8799, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 36701, cost 0.0015 USD, build file judge `jev-1.14.0` calls 6, tokens in 11768, cost 0.0005 USD, phases: path_judge 0.4 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 722.5 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `25ef3e22af2dbe1acebefcd887ea227f0d668998` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
25ef3e22af2dbe1acebefcd887ea227f0d668998.The result comment of werkrevisor holds the details.
Codex review of
25ef3e22af2dbe1acebefcd887ea227f0d668998@@ -195,0 +250,4 @@kept.append(b"index " + b" ".join(line.split(b" ")[2:]))elif line.startswith(b"@@"):# "@@ -1,2 +3,4 @@ heading": the heading names the function of the hunk.kept.append(b"@@" + line.split(b"@@", 2)[-1])[high] Repeated blocks let different permission changes share an approval (Codex review)
Removing hunk positions and blob IDs makes distinct edits identical when their surrounding context repeats. Reproduced with JSON permission entries: changing
allowedfor one resource produces the same signature as changing it for another resource whose name lies outside the three context lines. After approval, a commit can restore the first permission and change the second._reviewthen treats these as the same change, skips the since-approval assessment, and carries the approval even when Codex marks the current change critical.Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
e2582ae: the signature is nowgit diff --raw --full-index -M, i.e. path, modes and blob ids of each changed file before and after, so it compares the files exactly, without positions to lose. A rebase keeps the approval only if the base branch changed none of the PR's files; otherwise a human approves again. The real-git test has your case: JSON entries with repeated context,allowedchanged for another resource; it is red with the old signature.Codex review: critical
The change relaxes the merge gate by carrying human approvals across commits. An incorrect comparison can authorize critical changes that the human never approved.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codex.py:253, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 437111 (cached 384640) / out 4330, 115.6 s, 12 commands; reviewed head `25ef3e22af2dbe1acebefcd887ea227f0d668998`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
10b62fa, superseded by the result for5eb5a92🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
25ef3e2not carried over: 4 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 53369, duration 725.0 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 8811, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 36738, cost 0.0015 USD, build file judge `jev-1.14.0` calls 6, tokens in 11688, cost 0.0005 USD, phases: path_judge 0.3 s, classifier 0.4 s, doc_check 0.5 s, context 0.2 s, model_review 722.6 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `10b62fa6875c9a825b7e923d2c73b3accf7996ea` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
10b62fa6875c9a825b7e923d2c73b3accf7996ea.The result comment of werkrevisor holds the details.
Codex review of
10b62fa6875c9a825b7e923d2c73b3accf7996ea@@ -195,0 +239,4 @@e.g. after a rebase onto a base branch that changed other files.A comparison without positions failed twice: an edit elsewhere with the same lines passed as the same change."""raw = _git("diff", "--raw", "--full-index", "--no-abbrev", "-M", "-z", base, head, cwd=work)[high] Ignored submodule changes can reuse an old approval (Codex review)
The signature command inherits submodule ignore settings instead of specifying --ignore-submodules=none. If a submodule has ignore=all in .gitmodules or Git configuration, changing its recorded commit can leave the signature unchanged. The runner then carries the old approval through same_change, skips the incremental criticality assessment, and can pass the gate even when the overall review remains critical. Include ignored gitlink changes in the signature.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
5eb5a92:--ignore-submodules=none. The real-git test adds a submodule withignore = allin.gitmodulesand moves its commit; the signature changes, and without the flag the test is red.Codex review: critical
This changes the merge authorization gate by allowing an earlier human approval to authorize a later head. A mistake can permit unapproved security-sensitive changes to merge.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codex.py:242, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 448107 (cached 389888) / out 3142, 94.1 s, 8 commands; reviewed head `10b62fa6875c9a825b7e923d2c73b3accf7996ea`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
5eb5a92, superseded by the result forbc85826🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
10b62fanot carried over: 3 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 53894, duration 725.0 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 8855, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 37365, cost 0.0016 USD, build file judge `jev-1.14.0` calls 6, tokens in 11919, cost 0.0005 USD, phases: path_judge 0.3 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 722.6 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `5eb5a92d557843cbd0acf2b20b0723cf8c7e6ce2` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
5eb5a92d557843cbd0acf2b20b0723cf8c7e6ce2.The result comment of werkrevisor holds the details.
Codex review of
5eb5a92d557843cbd0acf2b20b0723cf8c7e6ce2@@ -195,0 +240,4 @@A comparison without positions failed twice: an edit elsewhere with the same lines passed as the same change."""# A submodule with "ignore = all" in the .gitmodules of the PR must not hide a new commit of it.raw = _git("diff", "--raw", "--full-index", "--no-abbrev", "-M", "-z", "--ignore-submodules=none", base, head,[high] Text decoding makes distinct paths share an approval signature (Codex review)
change_signature passes Git's NUL-delimited raw output through _git, whose text=True converts carriage returns and CRLF to LF. Thus edits with identical modes and before/after blobs in distinct valid paths such as "policy\r.json" and "policy\n.json" produce identical signatures; this was reproduced without modifying files. Moving the approved edit between those paths makes same_change true, skips the since-approval assessment and carries the approval even when the current review rates the PR critical. Hash the raw subprocess bytes without newline conversion.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
bc85826: the signature hashes the raw bytes ofgit diff --raw -z, without text mode. The real-git test commitsp\r.jsonandp\n.jsonand expects different signatures; it is red in text mode.Codex review: critical
The change relaxes a merge gate by allowing earlier human approvals to authorize later heads. A mistake can permit critical changes to merge without human review.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codex.py:243, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 388134 (cached 331392) / out 2876, 73.7 s, 15 commands; reviewed head `5eb5a92d557843cbd0acf2b20b0723cf8c7e6ce2`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
bc85826, superseded by the result foreb07159🟢 werkrevisor: GREEN / PASS
Gate: approved by werkrevisor; merge waits for 1 open hint(s): reply to each line comment and resolve it, then restart the review or push a commit.
Change kind:
mixed.Reasons
5eb5a92not carried over: 3 approved block(s) are missingFindings
werkrevisor/codex.py:261-262, hint, as a line commentClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 54200, calls 1, tokens in 51974 / out 46591 (reasoning included in out), duration 344.7 s, cost 0.0715 USD (reported), classifier `jev-1.14.0` calls 10, tokens in 8889, cost 0.0004 USD, finding judge `jev-1.14.0` calls 1, tokens in 1876, cost 0.0001 USD, PR document check `jev-1.14.0` calls 9, tokens in 37792, cost 0.0016 USD, build file judge `jev-1.14.0` calls 6, tokens in 12040, cost 0.0005 USD, phases: path_judge 0.3 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 341.9 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `bc858266f6d6a4e1601249fab213694141b6db6b` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
bc858266f6d6a4e1601249fab213694141b6db6b.The result comment of werkrevisor holds the details.
@@ -204,6 +258,8 @@ def checkout(base_url, owner, repo, number, head, base, token, root):_git("clone", "--bare", "--quiet", url, mirror, env=git_env)_git("fetch", "--quiet", url, f"+refs/pull/{number}/head:refs/pull/{number}/head", head, base,cwd=mirror, env=git_env)# Keeps a reviewed head after a force push, so that a rebase can be compared with it.[medium] Missing Test (werkrevisor)
hint; Jev: real 0.70, severity 2.1 of 3
The new
refs/reviewed/<number>/<head>ref, which keeps an earlier head available after a force push, has no test. The rebase carry-over depends on it: if theupdate-refcall is dropped or misspelled, the old head disappears from the mirror and the comparison against it fails silently while the test suite stays green.tests/test_codex.py::test_a_token_reaches_only_its_own_serveralready records every_gitcall, so asserting that the ref is created there would cover the behavior.Fixed in
eb07159:test_a_token_reaches_only_its_own_servernow checks the refsrefs/reviewed/<n>/<head>andrefs/reviewed/<n>/<head>-base.Codex review of
bc858266f6d6a4e1601249fab213694141b6db6b@@ -79,0 +83,4 @@if approval is None or not codex.has_commit(work, approval.head_sha):return None, Falsetry:approved_base = codex.merge_base(work, approval.head_sha, base)[high] Compare against the base used for the approved review (Codex review)
The approved base is reconstructed from the current base rather than preserved from the approved review. Suppose A was approved against T, then the PR is retargeted to an ancestor B of T and receives an empty commit H. Both signatures now compare against B and match, although B..T was absent from the reviewed diff T..A. The same_change path therefore carries the approval and unlocks even a critical result for H. Preserve and validate the original review base before carrying an approval.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
eb07159: each Codex result comment records its base in the header markerwerkrevisor-codex-base; the carry-over takes the base of the approved review from there, for the rebase comparison and for the comparison by Codex, and a comment without it carries nothing over.codex.checkoutkeeps a ref of that base too. Testtest_the_approved_change_is_taken_from_the_base_of_its_reviewis red with the current base.Codex review: critical
This changes merge authorization by allowing earlier human approvals to unlock later heads. An incorrect comparison can permit critical changes that the human never reviewed.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codexrunner.py:86, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 462167 (cached 400128) / out 3220, 114.8 s, 5 commands; reviewed head `bc858266f6d6a4e1601249fab213694141b6db6b`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟠 werkrevisor ORANGE for
eb07159, superseded by the result for992fbb3🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
bc85826not carried over: 17 approved block(s) are missingFindings
[high] Approval Bypass at
werkrevisor/codexrunner.py:428-430, ORANGE, as a line comment[medium] Ref Growth at
werkrevisor/codex.py:257-259, ORANGE, as a line comment[medium] Stale Diagram at
docs/werkrevisor.werkbaum:167-168hint; Jev: real 0.67, severity 2.0 of 3
This records a new Codex gate carry-over decision, but no corresponding review-flow diagram under
docs/is updated. Repository guidance requires diagrams to change with review decisions; leaving them unchanged will continue to depict the former no-carry-over behavior and can mislead operators.Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 55828, calls 1, tokens in 53536 / out 42947 (reasoning included in out), duration 338.7 s, cost 0.0676 USD (reported), classifier `jev-1.14.0` calls 10, tokens in 8976, cost 0.0004 USD, finding judge `jev-1.14.0` calls 3, tokens in 5815, cost 0.0002 USD, PR document check `jev-1.14.0` calls 9, tokens in 39239, cost 0.0016 USD, build file judge `jev-1.14.0` calls 6, tokens in 12687, cost 0.0005 USD, phases: path_judge 0.3 s, classifier 0.4 s, doc_check 0.5 s, context 0.2 s, model_review 336.0 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `eb071591938c74e73bb4112178e12f20b98437d5` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
Diagrams
Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.
werkrevisor: ORANGE / FINDINGS for
eb071591938c74e73bb4112178e12f20b98437d5.The result comment of werkrevisor holds the details.
@@ -204,6 +254,9 @@ def checkout(base_url, owner, repo, number, head, base, token, root):_git("clone", "--bare", "--quiet", url, mirror, env=git_env)_git("fetch", "--quiet", url, f"+refs/pull/{number}/head:refs/pull/{number}/head", head, base,cwd=mirror, env=git_env)# Keeps a reviewed head and its base after a force push or a new target, so that a later head can be compared.[medium] Ref Growth (werkrevisor)
ORANGE; Jev: real 0.85, severity 2.8 of 3
Every checkout creates permanent
refs/reviewed/...refs for the head and base, and no code removes them. Because these refs keep old commits and blobs reachable, repeatedly reviewed or pushed heads cause the runner's bare mirror to grow indefinitely; a contributor can amplify this by repeatedly pushing heads. Retain only refs needed by live approvals or add deterministic pruning.Fixed in
992fbb3:codex.checkoutkeeps the refs of at mostMAX_REVIEWED_HEADS= 20 reviewed heads per PR and deletes older ones with their base; an approval of an older head carries nothing over. The real-git test checks the pruning, and that another PR keeps its refs; it is red without the pruning.@@ -345,0 +425,4 @@approval, approved_review_base = self.earlier_approval(server, gitea, number, head, login)approved_base, same_change = _compare(work, approval, approved_review_base, base, head)# After a rebase only the same change keeps the approval; Codex judges a difference only on top of it.since = approval.head_sha if approved_base and not same_change and codex.is_ancestor([high] Approval Bypass (werkrevisor)
ORANGE; Jev: real 0.76, severity 3.0 of 3
A rebased head with a changed diff does not necessarily wait for a human. The
sincerating is only enabled when the approved head remains an ancestor; after a rebase with a different signature, no carried marker is emitted, so the gate passes whenever Codex rates the whole PR as noncritical. This contradicts the documented rule that a rebase with any other change waits for a human. The new test does not expose this because it always uses a critical whole-PR rating.Does not hold, no change. After a rebase with another change the approval does not carry over: no carried marker, so the gate treats the head as one without an earlier approval. A head that Codex rates as not critical as a whole needs no human in the Codex gate, before this PR and after it; an earlier approval cannot make it stricter than a PR without one. Scenario#144.03 is about a critical PR ("the same PR"), and there the gate waits:
test_a_rebase_with_the_same_blocks_keeps_the_approval, caseother.Codex review: critical
The change relaxes the merge gate by allowing approval of an older head to authorize a newer one. A mistake could permit critical changes to merge without human approval.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 238791 (cached 175232) / out 1787, 47.6 s, 10 commands; reviewed head `eb071591938c74e73bb4112178e12f20b98437d5`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.Answer to the review of
eb07159:992fbb3.e1070e9.🟢 werkrevisor GREEN for
992fbb3, superseded by the result for9567fe5🟢 werkrevisor: GREEN / PASS
Gate: approved by werkrevisor; merge waits for 3 open hint(s): reply to each line comment and resolve it; answer the 2 hint(s) that stand only in the result comment in a PR comment, then restart the review or push a commit.
Change kind:
mixed.Reasons
Findings
[low] Incorrect bound at
docs/prs/2026-10-25-PR#144-codex-carry-over.mdhint
The document states that an approval of a head older than the 20 newest reviewed heads carries nothing over. The code only deletes the refs of the pruned heads (
_prune_reviewed); the commits themselves stay reachable in the mirror as long as that head is an ancestor of the current head, sohas_commitstill succeeds and the approval still carries over. Only after a force push or a rebase, where the old head is no longer reachable, does the approval really stop carrying over. Either the documented bound must be corrected or the code must check the refs instead of the objects.[low] Wrong git mechanism at
docs/prs/2026-10-25-PR#144-codex-carry-over.mdhint
The document claims that
ignore = allin the.gitmodulesof the PR can hide a new submodule commit from the change signature and that--ignore-submodules=noneprevents it. In Git, theignoreattribute of.gitmodulesonly affectsgit submodulecommands and nevergit diff; onlydiff.ignoreSubmodulesandsubmodule.<name>.ignoreof the repository configuration do, and that configuration belongs to the runner's mirror, not to the PR. The flag is harmless, but the documented scenario does not exist and the new test does not cover what it claims.Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 56928, calls 1, tokens in 54563 / out 56011 (reasoning included in out), duration 397.0 s, cost 0.0836 USD (reported), classifier `jev-1.14.0` calls 10, tokens in 9025, cost 0.0004 USD, finding judge `none` calls 0, tokens in 0, cost 0.0000 USD, PR document check `jev-1.14.0` calls 9, tokens in 40152, cost 0.0017 USD, build file judge `jev-1.14.0` calls 6, tokens in 13159, cost 0.0006 USD, phases: path_judge 0.3 s, classifier 0.4 s, doc_check 0.5 s, context 0.3 s, model_review 394.5 s, finding_judge 0.0 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `992fbb3d7335f188198d721466c4dab38ce0d777` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
992fbb3d7335f188198d721466c4dab38ce0d777.The result comment of werkrevisor holds the details.
Codex review: critical
This changes a merge gate to accept earlier human approvals for later commits. A mistake could let critical changes merge without the required human review.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 351336 (cached 291200) / out 2549, 65.6 s, 16 commands; reviewed head `992fbb3d7335f188198d721466c4dab38ce0d777`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.Answer to the review of
992fbb3:9567fe5. The PR document now says that an older head stays in the mirror while it is an ancestor of the PR head, and that the bound takes effect after a force push.git difffollowssubmodule.<name>.ignorefrom.gitmodules(git-config: "the corresponding setting in .gitmodules" is used when the repository config sets none). The real-git test shows it: the subtest "ignored submodule" is red without--ignore-submodules=noneand green with it.New commits pushed, approval review dismissed automatically according to repository settings
🟠 werkrevisor ORANGE for
9567fe5, superseded by the result for153a20f🟠 werkrevisor: ORANGE / FINDINGS
Change kind:
mixed.Reasons
992fbb3not carried over: 1 approved block(s) are missingFindings
[high] Stale Base at
werkrevisor/codexrunner.py:394-400, ORANGE, as a line comment[medium] Missing Diagrams at
docs/werkrevisor.werkbaum:167-168hint; Jev: real 0.68, severity 1.9 of 3
The new carry-over decision changes the Codex review flow, but no diagram under
docs/is updated. Repository guidance requires diagrams to change whenever review steps or decisions change; leaving them unchanged makes the published flow inconsistent with the new gate behavior.[low] Retention Claim at
README.md:1071-1072hint
The README says a ref of every reviewed head and base is kept so a force push cannot remove it. The implementation retains only 20 selected heads and deletes both refs for the rest. Qualify the statement with the retention limit so users do not expect carry-over to survive for older approvals.
Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 56958, calls 1, tokens in 54603 / out 37555 (reasoning included in out), duration 339.5 s, cost 0.0614 USD (reported), classifier `jev-1.14.0` calls 10, tokens in 9051, cost 0.0004 USD, finding judge `jev-1.14.0` calls 2, tokens in 3999, cost 0.0002 USD, PR document check `jev-1.14.0` calls 9, tokens in 40386, cost 0.0017 USD, build file judge `jev-1.14.0` calls 6, tokens in 13159, cost 0.0006 USD, phases: path_judge 0.3 s, classifier 0.5 s, doc_check 0.5 s, context 0.3 s, model_review 336.3 s, finding_judge 0.2 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `9567fe5e4d56eaa7fcb3d033c5c43262358a1701` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
Diagrams
Drawn by a model from the diff with PR Lens: a view of what the PR touches, not a statement of what it does.
werkrevisor: ORANGE / FINDINGS for
9567fe5e4d56eaa7fcb3d033c5c43262358a1701.The result comment of werkrevisor holds the details.
@@ -328,3 +393,2 @@number, head = pr["number"], pr["head"]["sha"]approved = human_approval(gitea.list_reviews(number), head, server.human_reviewers,since=comment.get("created_at")) is not Nonereviews = gitea.list_reviews(number)[high] Stale Base (werkrevisor)
ORANGE; Jev: real 0.75, severity 3.0 of 3
Existing result comments are reused for an unchanged head. This new path revalidates only the human review, not the base used for the comparison: the reviewed base marker is never compared with the PR's current merge base. Retarget the PR to an older or unrelated base without changing the head and the old carried marker still passes, although the new diff was never reviewed. Invalidate and rerun the result when its recorded base differs from the current base.
Fixed in
153a20f:find_codex_commenttakes the current base; a result whose header names another base does not count, so a retargeted head gets a new review. This also closes the gap for a result without a carried approval, which existed before this PR. The newest result of a head counts, so an approval before a new review carries nothing over. Testtest_a_result_for_another_base_does_not_countis red without the base.Codex review: critical
The change relaxes a merge gate by carrying human approvals to later commits. An error could authorize critical changes that the human never reviewed, so human review is required.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 397769 (cached 353280) / out 2083, 54.0 s, 10 commands; reviewed head `9567fe5e4d56eaa7fcb3d033c5c43262358a1701`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.Answer to the review of
9567fe5:153a20f, see its thread.153a20f: the README names the 20 newest heads.e1070e9.🟢 werkrevisor GREEN for
153a20f, superseded by the result forec8469b🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 58138, duration 725.8 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9114, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 41503, cost 0.0017 USD, build file judge `jev-1.14.0` calls 6, tokens in 13670, cost 0.0006 USD, phases: path_judge 0.3 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 722.9 s, config: defaults + host + repository (.werkrevisor.toml at main2 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `153a20fc25aa01504eab3a0db47d7d1d50a1be1d` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
153a20fc25aa01504eab3a0db47d7d1d50a1be1d.The result comment of werkrevisor holds the details.
Codex review of
153a20fc25aa01504eab3a0db47d7d1d50a1be1d@@ -111,0 +163,4 @@lines = header(c.get("body"))recorded = reviewed_base(c.get("body"))if (_own(c, login) and (head_marker(head) in lines or error_marker(head) in lines)and not (base and recorded and recorded != base)):[high] Failed reviews bypass the new base check (Codex review)
The base check accepts comments without a recorded base, but _review still creates every failed-review comment without base_marker. After a human approves such a failure, retargeting the same head reuses that comment and approval: collect queues no review and update_gate reports success. I reproduced this path. Failure comments must also identify their base so an approval for the previous target cannot unlock the changed PR.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
ec8469b: the comment of a failed review names its base in the header too, so a retargeted head gets a new review and the approval of the failure no longer counts. New testtest_a_failed_review_names_its_base; red without the fix.@@ -300,3 +370,3 @@number, head = pr["number"], pr["head"]["sha"]comments = gitea.list_issue_comments(number)comment, failed = find_codex_comment(comments, head, login)comment, failed = find_codex_comment(comments, head, login, pr.get("merge_base") or pr["base"]["sha"])[high] Invalidate the successful status before queuing a replacement review (Codex review)
When retargeting invalidates a cached review, collect returns a Candidate without resetting the existing successful status. run_repository then calls release before serve sets the status to pending. I reproduced the actual schedule_merge call with the old success still active, allowing a held merge to proceed before the new base is reviewed.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
ec8469b: in gate modecollectsetsqueued ⏳before it returns a candidate, so the release in the same run seespending, not the success for the old base.test_a_result_for_another_base_does_not_countchecks it; red without the fix.Codex review: critical
This changes the merge gate’s authorization rules by allowing earlier human approvals to unlock later heads. A mistake can permit unreviewed critical changes to merge.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codexrunner.py:372, as a line commentwerkrevisor/codexrunner.py:166, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 554031 (cached 488320) / out 4075, 102.5 s, 14 commands; reviewed head `153a20fc25aa01504eab3a0db47d7d1d50a1be1d`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
ec8469b, superseded by the result for0070059🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
153a20fnot carried over: 3 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 60178, duration 731.6 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9170, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 42794, cost 0.0018 USD, build file judge `jev-1.14.0` calls 6, tokens in 14457, cost 0.0006 USD, phases: path_judge 0.4 s, classifier 0.6 s, doc_check 0.5 s, context 0.4 s, model_review 727.6 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `ec8469b9fbca3fbddffe595268539e30adfa7098` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
ec8469b9fbca3fbddffe595268539e30adfa7098.The result comment of werkrevisor holds the details.
Codex review of
ec8469b9fbca3fbddffe595268539e30adfa7098@@ -111,0 +162,4 @@for c in reversed(list(comments or ())):lines = header(c.get("body"))recorded = reviewed_base(c.get("body"))if (_own(c, login) and (head_marker(head) in lines or error_marker(head) in lines)[high] Legacy reviews bypass the retargeting check (Codex review)
The base comparison accepts comments without a recorded base. For an open PR reviewed before this upgrade, retargeting to an older base can introduce critical changes into the diff while preserving the head SHA. Its legacy noncritical result still passes the gate without another review. Reproduced through collect(): it sets success and queues no review. Require a fresh review when the cached result cannot establish its base.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
0070059: a result without a base no longer counts for the current base, so a head reviewed before 0.74.0 gets a new review; after the deployment each ready PR of agateoradviserepository gets one more Codex review. New testtest_a_result_without_its_base_gets_a_new_review; red without the fix.Codex review: critical
This changes the merge gate’s authorization rules by allowing earlier human approvals to unlock later commits. A mistake can permit critical changes to merge without human review.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codexrunner.py:165, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 427869 (cached 364672) / out 3174, 93.1 s, 11 commands; reviewed head `ec8469b9fbca3fbddffe595268539e30adfa7098`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
0070059, superseded by the result for7f6a237🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
ec8469bnot carried over: 4 approved block(s) are missingFindings
[low] Contradictory force-push statement at
docs/prs/2026-10-25-PR#144-codex-carry-over.md:129-131hint
The solution states that after a force push an approval of the older head carries nothing over. This contradicts Scenario#144.03, the README and the code: a rebase is normally pushed with force, and there the approval still carries over whenever
codex.change_signatureof the approved change and of the current change is equal, while the rating of the change since the approval only requires the approved head to be an ancestor. Please say that only a force push which changes the change itself drops the approval. The same bullet also refers to "review round 9", which does not exist in the document (the list jumps from round 8 to round 10).Classifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM done, tokens counted before the call 60638, calls 1, tokens in 58106 / out 53922 (reasoning included in out), duration 422.4 s, cost 0.0821 USD (reported), classifier `jev-1.14.0` calls 10, tokens in 9228, cost 0.0004 USD, finding judge `none` calls 0, tokens in 0, cost 0.0000 USD, PR document check `jev-1.14.0` calls 9, tokens in 43510, cost 0.0018 USD, build file judge `jev-1.14.0` calls 6, tokens in 14645, cost 0.0006 USD, phases: path_judge 0.6 s, classifier 0.5 s, doc_check 0.6 s, context 1.4 s, model_review 418.0 s, finding_judge 0.0 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `0070059abfa379ef7d90749ac218c6d9cbaeeabe` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
0070059abfa379ef7d90749ac218c6d9cbaeeabe.The result comment of werkrevisor holds the details.
Codex review of
0070059abfa379ef7d90749ac218c6d9cbaeeabe@@ -331,1 +406,4 @@still = sha and carryover.human_approval(reviews, lambda: gitea.list_timeline(number), head,server.human_reviewers)carried = bool(still) and still.head_sha == shaself.set_status(gitea, head, *gate_state(comment, failed, open_codex_line_comments(gitea, number, login),[high] Retargeting during review bypasses the base check (Codex review)
If the PR is retargeted while Codex runs, serve() passes the original PR snapshot to update_gate(), which publishes success without checking the current base. An in-memory reproduction confirmed success despite different reviewed and current bases. Because retargeting preserves the head SHA, that status can unlock a scheduled merge of the unreviewed change before the next poll. Re-fetch the PR and verify its base before publishing success.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
7f6a237: after a review,servereads the PR again; if its base or head changed during the review, the status staysqueued ⏳and the next run reviews the head for the new base. New testtest_a_retarget_during_the_review_sets_no_success; red without the fix.Codex review: critical
This relaxes a merge gate by allowing earlier human approvals to authorize later heads. Incorrect carry-over can permit critical changes to merge without human review.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codexrunner.py:409, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 491955 (cached 425216) / out 3376, 104.5 s, 12 commands; reviewed head `0070059abfa379ef7d90749ac218c6d9cbaeeabe`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.Answer to the review of
0070059:7f6a237. The PR document says now that only a force push that removes the head from the PR can drop its pruned commit, and no longer refers to a round 9.7f6a237, see its thread.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
7f6a237, superseded by the result for48b7482🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
0070059not carried over: 3 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 61683, duration 725.9 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9287, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 44562, cost 0.0019 USD, build file judge `jev-1.14.0` calls 6, tokens in 15166, cost 0.0006 USD, phases: path_judge 0.4 s, classifier 0.4 s, doc_check 0.5 s, context 0.3 s, model_review 722.7 s, config: defaults + host + repository (.werkrevisor.toml at main3 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `7f6a23797ff7a4c1c8ef8333db38527565772c7b` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
7f6a23797ff7a4c1c8ef8333db38527565772c7b.The result comment of werkrevisor holds the details.
Codex review of
7f6a23797ff7a4c1c8ef8333db38527565772c7b@@ -300,3 +374,3 @@number, head = pr["number"], pr["head"]["sha"]comments = gitea.list_issue_comments(number)comment, failed = find_codex_comment(comments, head, login)comment, failed = find_codex_comment(comments, head, login, pr.get("merge_base") or pr["base"]["sha"])[high] Revalidate the current target before accepting a cached review (Codex review)
If a PR is retargeted after list_open_prs() returns, collect() matches its cached review against the old snapshot and calls update_gate() without checking the current target. The unchanged_target() check only runs after a new review in serve(). I reproduced collect() publishing success and queuing no review while unchanged_target() returned false. Because the head SHA stays unchanged, this can unlock a merge against an unreviewed base. Apply the target check to cached results before publishing success.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
48b7482: the check moved fromserveintoupdate_gate, so everysuccessfirst reads the PR again, also for a result from before; with another base or head the status staysqueued ⏳. New testtest_a_retarget_after_the_list_of_prs_sets_no_success; red without the fix. Between that read and setting the status, a retarget within milliseconds stays possible; Gitea offers no conditional status, and the next run corrects it.Codex review: critical
This changes a merge gate to accept earlier human approvals for later heads. A mistake can permit critical changes to merge without human approval.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codexrunner.py:376, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 603582 (cached 551040) / out 3435, 111.3 s, 11 commands; reviewed head `7f6a23797ff7a4c1c8ef8333db38527565772c7b`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
48b7482, superseded by the result for368bb7d🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
7f6a237not carried over: 5 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 62032, duration 725.5 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9329, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 45070, cost 0.0019 USD, build file judge `jev-1.14.0` calls 6, tokens in 15296, cost 0.0006 USD, phases: path_judge 0.3 s, classifier 0.4 s, doc_check 0.5 s, context 0.3 s, model_review 722.6 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
568ae2b).Reviewed head `48b7482c7f311dfd3b4aad2b1d705a54165809ad` against base `568ae2b309698457383611a4db0f6b9502340425`, werkrevisor 0.72.0.
werkrevisor: GREEN / PASS for
48b7482c7f311dfd3b4aad2b1d705a54165809ad.The result comment of werkrevisor holds the details.
Codex review: critical
The change relaxes the merge gate by allowing an earlier human approval to authorize a later head. An error in approval reuse could permit critical changes to merge without human review.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 476464 (cached 431104) / out 2747, 72.5 s, 9 commands; reviewed head `48b7482c7f311dfd3b4aad2b1d705a54165809ad`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.werkrevisor 0.74.0: the Codex gate keeps a human approval when the change since then is not criticalto werkrevisor 0.76.0: the Codex gate keeps a human approval when the change since then is not critical48b7482c7fto368bb7d00dNew commits pushed, approval review dismissed automatically according to repository settings
New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
368bb7d, superseded by the result for1c964b0🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
48b7482not carried over: 4 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 62036, duration 725.7 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9341, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 45178, cost 0.0019 USD, build file judge `jev-1.14.0` calls 6, tokens in 15296, cost 0.0006 USD, phases: path_judge 0.3 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 722.7 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
4f9319c).Reviewed head `368bb7d00d04182639d60dd776ee9fe050242c89` against base `4f9319cefe72e806fa2a843515be7dfd97bd792f`, werkrevisor 0.74.1.
werkrevisor: GREEN / PASS for
368bb7d00d04182639d60dd776ee9fe050242c89.The result comment of werkrevisor holds the details.
Codex review of
368bb7d00d04182639d60dd776ee9fe050242c89@@ -111,0 +167,4 @@lines = header(c.get("body"))recorded = reviewed_base(c.get("body"))if (_own(c, login) and (head_marker(head) in lines or error_marker(head) in lines)and not (base and recorded != base)):[high] Retargeting back reuses an approval for a different base (Codex review)
The search skips newer results for other bases and accepts an older matching result. If head H is reviewed against B1, then reviewed and human-approved against B2, retargeting back to B1 selects the original result. update_gate accepts the B2 approval because its head matches and its timestamp follows that older result. I reproduced a success status for this sequence: critical changes present only against B1 can pass without approval of that change. A newer review for another base must invalidate reuse of the older result.
Fix it, or reply why it does not hold, and resolve this comment.
Fixed in
1c964b0: only the newest result of a head counts; when it names another base,find_codex_commentreturns none and the head gets a new review, so a retarget back to B1 does not revive the older B1 result. New testtest_a_retarget_back_needs_a_new_reviewwith your sequence; red when an older result for the base still counts.Codex review: critical
This changes the authorization enforced by a merge gate, allowing older human approvals to unlock later heads. A mistake could permit critical changes to merge without human review.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
Findings
werkrevisor/codexrunner.py:170, as a line commentThe merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 469709 (cached 419968) / out 3352, 162.7 s, 11 commands; reviewed head `368bb7d00d04182639d60dd776ee9fe050242c89`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
1c964b0, superseded by the result forea14417🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
368bb7dnot carried over: 3 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 62822, duration 725.5 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9388, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 45933, cost 0.0019 USD, build file judge `jev-1.14.0` calls 6, tokens in 15628, cost 0.0007 USD, phases: path_judge 0.4 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 722.5 s, config: defaults + host + repository (.werkrevisor.toml at main5 answers ignored because of a confidence below [classifier] min_confidence.
4f9319c).Reviewed head `1c964b0ca53f956d53ea9d6f2b798488a1a05302` against base `4f9319cefe72e806fa2a843515be7dfd97bd792f`, werkrevisor 0.74.1.
werkrevisor: GREEN / PASS for
1c964b0ca53f956d53ea9d6f2b798488a1a05302.The result comment of werkrevisor holds the details.
Codex review: critical
The change relaxes a merge gate by allowing earlier human approvals to authorize later heads. An incorrect carry-over decision could permit unapproved security-sensitive changes to merge, which the tests cannot fully rule out.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 419493 (cached 351360) / out 3613, 159.6 s, 10 commands; reviewed head `1c964b0ca53f956d53ea9d6f2b798488a1a05302`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.1c964b0ca5toea1441774aNew commits pushed, approval review dismissed automatically according to repository settings
New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor GREEN for
ea14417, superseded by the result forfaa3318🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
1c964b0not carried over: 3 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 62758, duration 727.8 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9380, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 45861, cost 0.0019 USD, build file judge `jev-1.14.0` calls 6, tokens in 15628, cost 0.0007 USD, phases: path_judge 0.4 s, classifier 0.5 s, doc_check 0.5 s, context 0.3 s, model_review 722.9 s, config: defaults + host + repository (.werkrevisor.toml at main4 answers ignored because of a confidence below [classifier] min_confidence.
9131a08).Reviewed head `ea1441774afebef566325f1b2c42c3dde20b4e9b` against base `9131a08453ece0bf5bb1e78fa26852d1c8620410`, werkrevisor 0.75.0.
werkrevisor: GREEN / PASS for
ea1441774afebef566325f1b2c42c3dde20b4e9b.The result comment of werkrevisor holds the details.
Codex review: critical
This changes the merge gate’s authorization logic by allowing an earlier human approval to unlock a later head. A mistake could permit critical, unapproved changes to merge despite passing tests.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 355026 (cached 292992) / out 3306, 157.4 s, 9 commands; reviewed head `ea1441774afebef566325f1b2c42c3dde20b4e9b`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.werkrevisor 0.76.0: the Codex gate keeps a human approval when the change since then is not criticalto werkrevisor 0.77.0: the Codex gate keeps a human approval when the change since then is not criticalea1441774atofaa3318743New commits pushed, approval review dismissed automatically according to repository settings
🟢 werkrevisor: GREEN / PASS
Change kind:
mixed.Reasons
ea14417not carried over: 4 approved block(s) are missingClassifier
10 of 10 units passed all questions (1 PR document(s)).
Usage: model `openrouter/minimax/minimax-m3`, LLM failed (TimeoutError), tokens counted before the call 62705, duration 726.0 s, cost unknown, classifier `jev-1.14.0` calls 10, tokens in 9380, cost 0.0004 USD, PR document check `jev-1.14.0` calls 9, tokens in 45861, cost 0.0019 USD, build file judge `jev-1.14.0` calls 6, tokens in 15628, cost 0.0007 USD, phases: path_judge 0.4 s, classifier 0.5 s, doc_check 0.5 s, context 0.2 s, model_review 722.5 s, config: defaults + host + repository (.werkrevisor.toml at main3 answers ignored because of a confidence below [classifier] min_confidence.
4230f62).Reviewed head `faa3318743422fa2808fd44121bd3e5ea6c82c0b` against base `4230f6262da5f7c2964b6158a9db1368c2ff25ef`, werkrevisor 0.76.1.
werkrevisor: GREEN / PASS for
faa3318743422fa2808fd44121bd3e5ea6c82c0b.The result comment of werkrevisor holds the details.
Codex review: critical
This changes the merge gate to accept an earlier human approval for a later head. A mistake could let a critical change merge without fresh human review, and tests cannot establish the correctness of every model judgment.
Security effect: mixed; fails closed: no; a test of the closed case: yes.
No findings.
The merge waits for the status
Model `gpt-6-astra`, effort high, tokens in 621955 (cached 544768) / out 3264, 126.1 s, 13 commands; reviewed head `faa3318743422fa2808fd44121bd3e5ea6c82c0b`.ai-review/codex: every line comment resolved, and the approval of a human for a critical PR or for a finding that is only here.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.