Skip to content

MPF-L-001 skips media probes instead of warning about them - #27

Merged
Tippel-AI merged 1 commit into
mainfrom
fix/label-scope
Aug 31, 2026
Merged

MPF-L-001 skips media probes instead of warning about them#27
Tippel-AI merged 1 commit into
mainfrom
fix/label-scope

Conversation

@Tippel-AI

Copy link
Copy Markdown
Owner

Fifth step of the audit remediation. Closes the honesty axis's highest finding.

The defect

MPF-L-001 warned on every media target. The "perceivable text" it inspected
was a string markproof composed itself:

summary = f"{len(artifacts)} asset(s): " + ", ".join(a.id for a in artifacts)

Non-empty, so the "nothing to read" branch never fired. Incapable of containing a
label, so the outcome was fixed before the check started. Every media run, the
same warning, about a string the tool wrote.

A warning that fires for every target carries no information and teaches its
reader to skip it — the same defect as a guessed PASS with the sign flipped, and
the same one the applicability work removed for static pages.

The fix, and why not the other one

The duty is real. Guidelines para 117 attaches it to what is perceivable where
the content is displayed
, and an images API is not that place. So a media probe
now reports no perceivable surface, and the finding says what to do instead:

MPF-L-001  SKIP  no perceivable text to inspect for a label — a media endpoint
                 returns an API payload, not the page where the content is shown.
                 Point a 'ui' probe at that page to check this obligation.

The tempting alternative — match labels against the response body anyway — would
be worse than the false warning. It would let a perceivability duty be
discharged by words in a JSON payload no reader ever sees, which is precisely what
para 117 refuses when it rules out machine-readable marking standing in for a
perceivable label. There is a test for that: a real, correct German label placed
in the response body still yields NO_PERCEIVABLE_TEXT.

_perceivable_text already had a docstring saying this is only a faithful reading
for probes that capture rendered output. The code now agrees with its own comment.

Tests

The label suite used ProbeKind.MEDIA as an arbitrary stand-in while exercising
pattern matching. That arbitrariness is now load-bearing, so those cases moved to
ui, and three new ones pin the media behaviour end to end through the engine.

Three golden diffs, all reviewed

Case Change
media-* MPF-L-001 WARN → SKIP
multi-probe the media probe's warning goes, the UI probe's stays
all rulepack sha256 — editing the rationale changed the file

That last one is #25 doing its job: a change to the rules is visible in every
report produced under them. The multi-probe diff is the one that shows the fix
is narrow rather than a blanket silencing.

432 tests, ruff + mypy strict clean.

🤖 Generated with Claude Code

The rule warned on every media target, and the "perceivable text" it inspected was
a string markproof wrote itself. The media probe records
`"3 asset(s): images-0, images-1"` as the turn's response so a finding can name
what it looked at; that summary is non-empty, so the "nothing to read" branch
never fired, and it can never contain a label, so the outcome was decided before
the check began.

A warning that appears for every target carries no information and teaches its
reader to skip it — the same defect as a guessed PASS with the sign flipped, and
the same one the applicability work removed for static pages.

The duty is real. Guidelines para 117 attaches it to what is perceivable *where
the content is displayed*, and an images API is not that place. So the check
reports honestly that it saw no perceivable surface, and the finding says what to
do instead: point a `ui` probe at the page where the image appears. Accepting a
label found in an API's JSON body would be worse than the false warning — it would
let a perceivability duty be discharged by words no reader ever sees, which is
exactly what para 117 rules out when it refuses to let machine-readable marking
stand in for a perceivable one.

`_perceivable_text` already carried a docstring saying this is only a faithful
reading for probes that capture rendered output. The code now agrees with it.

The label tests used `ProbeKind.MEDIA` as an arbitrary stand-in while exercising
pattern matching. That arbitrariness is now meaningful, so they moved to `ui` —
the probe kind where a label is actually perceived — and three new tests pin the
media behaviour, including that a real label in the response body still does not
count.

Two golden diffs, both reviewed rather than refreshed: the media cases move from
WARN to SKIP, and in `multi-probe` the UI probe's warning correctly stays while
the media probe's disappears. A third diff is the rulepack digest from #25 —
editing the rationale changed the file, and every report now says so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Tippel-AI
Tippel-AI merged commit edce70e into main Aug 31, 2026
8 checks passed
@Tippel-AI
Tippel-AI deleted the fix/label-scope branch August 31, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant