Skip to content

review/review-fixtures have no regression guard in this repo — 9 tests could not travel #3

Description

@schickling-assistant

The integration suite that guards review and review-fixtures did not move into this repository, and
cannot until intent can drive a coding-agent runner it owns.

What is covered today

crates/intent/tests/vrs_check.rs holds 10 integration tests, ported from the suite that originally
exercised this checker before it was extracted. They cover check (both profiles) and graph, plus the
two review cases that refuse before the runner executes.

What is not

Nine tests did not travel. All nine drive review / review-fixtures through a coding-agent runner and
assert on that runner's own result envelope — its result schema, the context files it reports, and its
effective permissions. Those assertions are about the runner's contract, not this checker's.

The runner is a separate binary owned by the upstream tool this checker was extracted from. Bringing it
here would invert the dependency the extraction deliberately established: this repository holds the corpus
and the checker that gates it, and depends on nothing that depends on it.

A stub is not a substitute. To make those nine pass, a stub would have to reimplement the collaborator
under test — which produces a green suite that guards nothing, and costs more than leaving them out
honestly. The individual tests are named in the module doc of crates/intent/tests/vrs_check.rs, with the
reason.

Why this needs fixing rather than noting

Those nine remain green in the upstream tool's own suite, but only for as long as that tool keeps
consuming this crate. The day it stops, review and review-fixtures — two of the four subcommands — have
no regression guard here at all, and nothing fails to announce it. A note in a document does not catch the
first regression; only a test does.

What would resolve it

A runner boundary this repository can drive itself: a seam the review path can be tested against without
depending on the upstream binary, so the nine can be ported without either inverting the dependency or
faking the collaborator.

This is not a plan or an estimate. It is a real gap, currently unowned, recorded so that whoever removes
the upstream-side checks — or wonders why review has thinner coverage than check — finds the reason
rather than rediscovering it.

Posted on behalf of @schickling
field value
agent_name unknown
agent_tool Claude Code
agent_tool_version 2.1.220
agent_runtime Claude Code 2.1.220
agent_session_lookup unavailable
tooling_profile dotfiles@unknown-dirty

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions