2.5 KiB
PR#71: An Answer "No" Without Quotes Is No Security Concern
WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.
Related Links
- werkcrappit PR#52, head 76d4ee2: YELLOW with the single reason "model reports a security concern: False".
The Problem
PR-Agent answers security_concerns with "No" or with a text that describes the concern.
When the model writes security_concerns: No without a block scalar or quotes,
the YAML loader of PR-Agent reads the value as the bool False.
werkrevisor made the text "False" of it and compared only with "no".
So a review without a security concern became YELLOW, with the reason "model reports a security concern: False".
Non-Goals
- No change to the verdict of a real concern: it stays YELLOW and quoted.
- No interpretation of free text such as "No security concerns found": it stays a concern, as before.
The Scenarios
Feature: an answer without a security concern gives no reason
Scenario#71.01: every spelling of "no" gives no reason
So that the bool of the YAML loader and the case of the answer do not decide the verdict.
- Given the model answers
security_concernswith the boolFalse, "False", "false", "no", "No.", "None" or nothing - When werkrevisor assesses the answer
- Then the verdict is GREEN
- And no reason
model_securityis given
Verified by
Scenario#71.02: the unquoted YAML answer "No" of a real PR-Agent run is no concern
So that the fix covers the value as the YAML loader of PR-Agent 0.47.0 gives it.
- Given the model answer holds the line
security_concerns: No - When PR-Agent loads the answer and werkrevisor assesses it
- Then the assessment holds no security concern
Verified by
The Solution
parse_model_reviewmaps the boolFalse,Noneand the texts "no", "false" and "none", in any case and with an optional final ".", to an empty concern.assess_modelgives the reasonmodel_securityonly for a non-empty concern.- The bool
Truestays the concern "True": YELLOW, as before. - The README says so; the Werkbaum adds the item as done.
- Version 0.31.5.