2.7 KiB
PR#93: The Error Handling Category Asks for Lost or Hidden Failures, Not for Changed Error Handling
WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR documentation is not maintained along with new PRs; treat it as a snapshot.
Problem
The category error_handling of jvm-spring-backend asks whether a change alters error handling, retries,
or what happens after a failed operation.
That holds for almost every change of error handling, also a correct one.
Jev rated the bounded retry of a commit status in werkator PR#155 at 0.98, so the PR became YELLOW;
the Codex review rated it as not critical and found nothing.
Non-Goals
- No change of the other categories.
- A retry that is not safe to run twice stays with the category
concurrency(PR#60).
Solution
- The new question asks whether the change lets a failure go unnoticed: an exception caught and ignored or only logged at a low level, a failure reported as a success, an error result no longer checked, or a partial update no longer rolled back.
- From 0.85 on, the category gives YELLOW; from 0.99 on, ORANGE, as before.
- docs/werkrevisor.werkbaum: the measurement.
- werkrevisor 0.37.0.
Measurement
On 2026-10-12 Jev (jev-latest) answered the old question and two new wordings
for each main-code file of 30 merged werkator PRs (#125 to #154), of hs.hsadmin.ng #281 to #316,
werkator PR#155 and seven constructed diffs.
27 of the 36 hs.hsadmin.ng PRs were merged; the other 9 were closed without a merge and are left out.
32 of the 57 merged PRs change main code; together they have 179 main-code files.
The table shows the chosen wording.
| case | old question | new question |
|---|---|---|
| defect: an exception around a save is caught and ignored | 0.99 | 0.98 |
| defect: a failed create answers 200 OK, logged at debug | 0.98 | 0.95 |
| defect: a failed credit after a debit is only logged, no rollback | 0.97 | 0.87 |
| defect: the result of publishing a final status is ignored | 0.96 | 0.95 |
| correct: a new exception handler answers 404 | 0.93 | 0.16 |
| correct: a narrower exception type with a better message | 0.97 | 0.38 |
| correct: a changed log text | 0.05 | 0.07 |
| werkator PR#155: one retry of a commit status after an I/O error | 0.98 | 0.23 |
| 57 merged PRs, highest value | 0.95 | 0.69 |
From 0.85 on, the new question finds the four constructed defects and none of the merged PRs. The other wording, which asks for a failure hidden from the caller, the user or the operator, rated the missing rollback at 0.92 but the narrower exception type at 0.63. The defects are constructed and plain, so a real one may score lower.
Open Questions
None.