MPF-L-001 skips media probes instead of warning about them - #27
Merged
Conversation
The rule warned on every media target, and the "perceivable text" it inspected was a string markproof wrote itself. The media probe records `"3 asset(s): images-0, images-1"` as the turn's response so a finding can name what it looked at; that summary is non-empty, so the "nothing to read" branch never fired, and it can never contain a label, so the outcome was decided before the check began. A warning that appears for every target carries no information and teaches its reader to skip it — the same defect as a guessed PASS with the sign flipped, and the same one the applicability work removed for static pages. The duty is real. Guidelines para 117 attaches it to what is perceivable *where the content is displayed*, and an images API is not that place. So the check reports honestly that it saw no perceivable surface, and the finding says what to do instead: point a `ui` probe at the page where the image appears. Accepting a label found in an API's JSON body would be worse than the false warning — it would let a perceivability duty be discharged by words no reader ever sees, which is exactly what para 117 rules out when it refuses to let machine-readable marking stand in for a perceivable one. `_perceivable_text` already carried a docstring saying this is only a faithful reading for probes that capture rendered output. The code now agrees with it. The label tests used `ProbeKind.MEDIA` as an arbitrary stand-in while exercising pattern matching. That arbitrariness is now meaningful, so they moved to `ui` — the probe kind where a label is actually perceived — and three new tests pin the media behaviour, including that a real label in the response body still does not count. Two golden diffs, both reviewed rather than refreshed: the media cases move from WARN to SKIP, and in `multi-probe` the UI probe's warning correctly stays while the media probe's disappears. A third diff is the rulepack digest from #25 — editing the rationale changed the file, and every report now says so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fifth step of the audit remediation. Closes the honesty axis's highest finding.
The defect
MPF-L-001warned on every media target. The "perceivable text" it inspectedwas a string markproof composed itself:
Non-empty, so the "nothing to read" branch never fired. Incapable of containing a
label, so the outcome was fixed before the check started. Every media run, the
same warning, about a string the tool wrote.
A warning that fires for every target carries no information and teaches its
reader to skip it — the same defect as a guessed PASS with the sign flipped, and
the same one the applicability work removed for static pages.
The fix, and why not the other one
The duty is real. Guidelines para 117 attaches it to what is perceivable where
the content is displayed, and an images API is not that place. So a media probe
now reports no perceivable surface, and the finding says what to do instead:
The tempting alternative — match labels against the response body anyway — would
be worse than the false warning. It would let a perceivability duty be
discharged by words in a JSON payload no reader ever sees, which is precisely what
para 117 refuses when it rules out machine-readable marking standing in for a
perceivable label. There is a test for that: a real, correct German label placed
in the response body still yields
NO_PERCEIVABLE_TEXT._perceivable_textalready had a docstring saying this is only a faithful readingfor probes that capture rendered output. The code now agrees with its own comment.
Tests
The label suite used
ProbeKind.MEDIAas an arbitrary stand-in while exercisingpattern matching. That arbitrariness is now load-bearing, so those cases moved to
ui, and three new ones pin the media behaviour end to end through the engine.Three golden diffs, all reviewed
media-*MPF-L-001WARN → SKIPmulti-probesha256— editing the rationale changed the fileThat last one is #25 doing its job: a change to the rules is visible in every
report produced under them. The
multi-probediff is the one that shows the fixis narrow rather than a blanket silencing.
432 tests, ruff + mypy strict clean.
🤖 Generated with Claude Code