Skip to content

feat(intent): standalone checker crate and the CI that gates the corpus - #1

Merged
schickling merged 6 commits into
mainfrom
schickling-assistant/2026-08-08-intent-cli
Aug 8, 2026
Merged

feat(intent): standalone checker crate and the CI that gates the corpus#1
schickling merged 6 commits into
mainfrom
schickling-assistant/2026-08-08-intent-cli

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Lifts the VRS checker out of schickling/dotfiles into a standalone crate in this
repo, and wires the two checks that gate the corpus onto this repo's own CI.

Packages it as a flake too, which is the only supported distribution.

What is here

Path What
crates/intent/ Standalone package: intent binary + library exposing run
.github/workflows/ci.yml materialises, crate, corpus-strict, semantic-review-fixtures
flake.nix / flake.lock Packages the CLI; checks for fmt, clippy, tests, help, corpus
.github/workflows/nix.yml The nix lane — a separate workflow, deliberately
crates/intent/README.md Command surface + the enforcement-asset contract
.gitignore target/, result, and the JSON reports CI writes at the root

axe vrs embeds the library and calls intent::run directly, so main.rs is a
thin shell over the same entry point — anything living only in the binary would be
behavior the embedded caller silently does not get.

The path bug this fixes, and the proof it was real

The staged corpus-strict job built with --manifest-path crates/intent/Cargo.toml
and then invoked ./target/release/intent. crates/intent is a standalone
package
— its own Cargo.lock, no workspace root above it — so cargo writes to
crates/intent/target/, not ./target/. Both the strict step and the graph step
would have died on "no such file" before either gate ran, which is a green-looking
job that never checked anything.

Reproduced from a clean checkout root with no pre-existing target/:

$ cargo build --locked --manifest-path crates/intent/Cargo.toml   # the staged command
$ ls ./target/debug/intent                     # ABSENT  <- the step dies here
$ ls crates/intent/target/debug/intent         # PRESENT <- where cargo actually wrote it

Fixed by passing --target-dir target explicitly, rather than by rewriting the two
consumer steps to the nested path. --target-dir pins the output location
independent of workspace detection, so if a workspace root ever appears above the
crate the binary does not silently relocate again.

Path audit, closed: the cargo output directory was the only cwd-dependent path
in the file. Every other path — intent, intent/15-evaluation/semantic-review,
intent/16-enforcement/review-result.schema.json, report.json, graph.json — is
relative to the checkout root, and each was verified against the real tree.

Why the checks look paranoid

intent check exits 0 on an empty directory and on a directory holding no VRS
artifacts, so "the check passed" cannot by itself distinguish a healthy corpus from
a missing one. Porting the assertion as-is would reproduce a known silent-pass class.

So corpus-strict asserts the corpus exists before checking, and afterwards asserts
the derived graph is non-empty. The graph is what discriminates: it is empty for both
an empty directory and a wrong path, and non-empty only when artifacts were genuinely
read. semantic-review-fixtures fails when it finds zero fixtures for the same
reason — a loop that is green over nothing is the same silent pass.

The jobs are deliberately kept separate and separately named rather than
collapsed behind a single gate, so a run shows which one concluded and how.

Behavior change: enforcement assets resolve under the corpus

review resolves 16-enforcement/review-prompt.md and
16-enforcement/review-result.schema.json relative to the corpus root, not the
enclosing repository. A corpus without its own 16-enforcement/ now fails rather
than quietly borrowing the repository's copies — silently falling back produced a
review graded against a rubric the corpus never declared.

Exit 2, naming both the missing asset and the corpus it was missing from:

$ intent review ./some-corpus
axe vrs review: missing review asset 16-enforcement/review-prompt.md under corpus /abs/path/to/some-corpus

The asset alone would not say which corpus was searched; the root alone would not say
what it was expected to contain. Documented in crates/intent/README.md.

Verified locally before pushing

From a clean checkout root, against this repo's actual corpus:

Check Result
cargo metadata --locked lock satisfies --locked
cargo test --locked 8 passed, 0 failed
check intent --profile strict --json exit 0, 0 diagnostics
graph intent --json 246 nodes / 215 edges (non-vacuous)
5 fixtures vs enforcement schema all 5 validate, each with ≥1 finding
actionlint on both workflows clean
cargo clippy --all-targets -D warnings clean
cargo fmt --check clean
nix flake check all 5 checks pass
checks.reads-the-corpus packaged binary reports 246 nodes

The Nix build's test phase was checked in the derivation log rather than assumed: it
runs all 8 tests, so doCheck is not silently vacuous here either.

check-jsonschema runs fully offline — every schema is a local file, so none is ever
fetched.

Packaging, and why the nix lane is its own workflow

Two attributes are needed to package a crate that is not at the repository root, and
they are not the same knob — this is the part that is easy to get wrong:

  • buildAndTestSubdir moves only the build and test phases.
  • cargoSetupPostPatchHook still reconciles the vendored lockfile against
    $sourceRoot/Cargo.lock — the repository root — and fails with "Missing Cargo.lock
    from src". cargoRoot is what points that reconciliation at the crate.

src stays the whole repository on purpose, so the corpus at intent/ remains visible
to a check aimed at it from the same source tree. That is what checks.reads-the-corpus
uses to prove the packaged binary reads real artifacts, not merely that it compiles.

The lane is a separate workflow rather than a job inside ci.yml. Running the corpus
gates through nix flake check would collapse corpus-strict and
semantic-review-fixtures into a single check named check, and a run would no longer
show which gate concluded and how. Folding a multi-minute Nix build in beside the fast
corpus gates would couple them for no benefit. This lane packages the CLI and re-gates
nothing.

Two deliberate changes to the lifted source

Both are called out rather than buried, because this is a lift whose acceptance bar is
that axe vrs behaves identically:

  • Comments. The block comments carried hand-aligned continuation lines that rustfmt
    de-indents to column 0 — which reads worse than what it replaced. Rather than commit
    that, or drop the fmt gate, they became line comments, which rustfmt leaves alone.
    Wording is unchanged.
  • Clippy. automated_context_indicator becomes the Iterator::find it was already
    written as by hand. require_section keeps its eight arguments under an explicit
    #[allow(clippy::too_many_arguments)] with the reason inline: regrouping them is a
    refactor motivated only by a style lint, and doing it inside a lift would make any
    resulting regression unattributable. Worth doing later, on its own.

The integration suite that stayed behind — and what did not travel

The checker was lifted into this crate but its integration suite was not: 19 tests in
flakes/axe/tests/vrs_check.rs stayed in schickling/dotfiles and drive the axe
binary. They are the differential oracle for the extraction, so a tool whose real test
suite lives in its own consumer is backwards — and invisible until the day that
consumer stops depending on it.

10 of the 19 travelled. The other 9 did not, and crates/intent/tests/vrs_check.rs
names each one and why, rather than presenting a suite that looks complete:

Ported 8 check/graph tests + the 2 review tests that refuse before CAIC runs
Omitted 9 tests that drive review / review-fixtures through the CAIC runner

The omitted 9 assert on the CAIC envelope itself — coding_agent.result.v1,
run.context_files, run.permission.effective. CAIC is a separate binary
(coding-agent, a shim over axe::caic) owned by axe. This crate builds no such
binary, and lifting caic here would invert the settled dependency: axe consumes
intent, not the reverse. A stub could only make them pass by reimplementing the
collaborator under test — the "green because the hard cases were dropped" failure in a
more expensive disguise.

They stay green in axe's own suite, so the behaviour is still covered — but only
while axe keeps consuming this crate.
Closing that properly needs a CAIC boundary
this crate can drive on its own.

The corpus-relative asset resolution the lift introduced was checked rather than
assumed: the harness helpers pass repo/context/vrs — a corpus root, not a repository
root — and build 16-enforcement/ underneath it, which is what resolution now requires.
--coding-agent is handed a deliberately non-existent path, so if a test ever needs it
to execute, that test does not belong in this file.

Totals: 8 unit + 10 integration, passing natively and inside nix flake check,
where the sandbox runs the exec'd fake providers without complaint.

About the CI runs

concurrency.cancel-in-progress: true means each push cancels the run in flight, and a
cancelled run demonstrates nothing — the citable run is the last one on this branch.

The materialises job answers a question that is not a given: does this repository
materialise jobs at all? A run object can exist with zero jobs, and that is how a
sibling repo's CI sat "queued" for a month unnoticed. It does — 4 jobs in the ci run
and 1 in the nix run, all concluding.

A README-only commit from another agent landed on this branch as expected. It was kept
and rebased onto, not reverted.

Not in scope here

  • Merging. This repo is private and stays private; the merge is not the author's.
  • Renaming axe.vrs.check.v1 or the AXE.VRS-R* ids — a separate coordinated pass,
    and renaming now would cost the differential oracle these checks depend on.
  • crates.io publication or a release workflow. Distribution is the flake only.
Posted on behalf of @schickling
field value
agent_name unknown
agent_tool Claude Code
agent_tool_version 2.1.220
agent_runtime Claude Code 2.1.220
agent_session_lookup unavailable
tooling_profile dotfiles@unknown-dirty

schickling-assistant and others added 6 commits August 8, 2026 13:07
…the corpus

Lifts `flakes/axe/src/vrs.rs` out of schickling/dotfiles into a standalone
package at `crates/intent/`, exposing both the `intent` binary and a library
whose `run` entry point `axe vrs` calls directly.

Wires the two checks that previously lived in dotfiles onto this repo's own CI,
kept as separately-named jobs rather than collapsed behind a single gate, so a
run shows which one concluded and how.

Both checks are written against the way this class of gate fails silently.
`intent check` exits 0 on an empty directory and on a directory holding no VRS
artifacts, so "the check passed" cannot on its own distinguish a healthy corpus
from a missing one. `corpus-strict` therefore asserts the corpus exists before
checking, and afterwards asserts the derived graph is non-empty — the graph is
what discriminates, being empty for both an empty directory and a wrong path.
`semantic-review-fixtures` likewise fails when it finds zero fixtures, since a
loop that is green over nothing is the same silent pass.

`crates/intent` is a standalone package with no workspace root above it, so
cargo writes to `crates/intent/target/` and not to `./target/`. The corpus
check invokes the built binary by path from the checkout root, so it passes
`--target-dir` explicitly instead of assuming either location.

Enforcement assets now resolve under the corpus root rather than the enclosing
repository, so a corpus without its own `16-enforcement/` fails instead of
quietly borrowing the repository's copies and being graded against a rubric it
never declared. The error names both the missing asset and the corpus root;
`crates/intent/README.md` documents the behavior.

Verified locally against this corpus from a clean checkout: strict check reports
0 diagnostics, the graph is 246 nodes / 215 edges, all 5 fixtures validate
against the enforcement schema, and the crate's 8 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
agent-tool: Claude Code
agent-tool-version: 2.1.220
agent-runtime: Claude Code 2.1.220
agent-session-lookup: unavailable
tooling-profile: dotfiles@unknown-dirty
…page

The paragraph asserted that the conventions travel, and the only evidence it
offered was a second project using them. Removing that sentence alone would
leave the assertion standing on support that is no longer on the page — the
unfalsifiable prose this corpus exists to catch.

Rewritten so every claim left in it is carried by something a reader can see:
the artifacts are plain Markdown, and the corpus is written in the conventions
it specifies. The narrower claim is the one the closing paragraph actually
needs — "conventions that need a bespoke toolchain do not get adopted" pairs
with no-toolchain, not with adoption elsewhere.

No outside project is substituted as evidence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
agent-tool: Claude Code
agent-tool-version: 2.1.220
agent-runtime: Claude Code 2.1.220
agent-session-lookup: unavailable
tooling-profile: dotfiles@unknown-dirty
Distribution is the flake only — there is no crates.io release — so this is how
the CLI is consumed. `nix build .#intent` packages it; `nix flake check` gates
fmt, clippy, the crate's test suite, a `--help` smoke test, and a proof that the
packaged binary reads a real corpus.

Two attributes are needed to package a crate that is not at the repository root,
and they are not the same knob. `buildAndTestSubdir` moves only the build and
test phases; `cargoSetupPostPatchHook` still reconciles the vendored lockfile
against the repository root and fails with "Missing Cargo.lock from src".
`cargoRoot` is what points that reconciliation at the crate. `src` stays the
whole repository on purpose, so the corpus at `intent/` remains visible to a
check aimed at it from the same source tree.

The nix lane is a separate workflow rather than a job inside `ci.yml`. Folding a
multi-minute Nix build in beside the fast corpus gates would couple them, and
running the corpus gates through `nix flake check` would collapse `corpus-strict`
and `semantic-review-fixtures` into a single check named `check` — a run would
no longer show which gate concluded and how. This lane packages the CLI and
re-gates nothing.

Adopting the formatter and the lint required two changes to the lifted source:

- The block comments carried hand-aligned continuation lines that rustfmt
  de-indents to column 0, which reads worse than what it replaced. Converted to
  line comments, which rustfmt leaves alone, so the wording is unchanged and the
  gate is real rather than permanently red or absent.
- `automated_context_indicator` becomes the `Iterator::find` it was written as by
  hand. `require_section` keeps its eight arguments under an explicit `allow`
  with the reason inline: this crate is a lift whose acceptance bar is behaving
  identically, and grouping those arguments is a refactor motivated only by a
  style lint, better done on its own where a regression would be attributable.

Verified locally: `nix flake check` passes all five checks, the packaged binary
reports 246 graph nodes against the real corpus, and after these source edits the
strict check still reports 0 diagnostics with the graph unchanged at 246/215.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
agent-tool: Claude Code
agent-tool-version: 2.1.220
agent-runtime: Claude Code 2.1.220
agent-session-lookup: unavailable
tooling-profile: dotfiles@unknown-dirty
…fiers

Two sections authored by the seat that built the original extraction, taken as a
patch rather than a push.

The first exists because the CI here encodes a defence the README did not explain:
`check` and `graph` both exit 0 against a path holding nothing, so a reader copying
`corpus-strict` could keep the first assertion and drop the node-count one — which
is the half that actually discriminates a clean corpus from an absent one.

The second is load-bearing. `AXE.VRS-R*`, `axe.vrs.check.v1` and the `axe vrs …`
message prefixes survive inside a binary called `intent` on purpose: they are the
differential oracle proving the lift was faithful, and the schema version is a wire
contract with a live consumer. There are 29 `axe vrs` occurrences in `lib.rs`, so
this is the whole user-visible surface rather than a stray literal — a partial
rename would be worse than either leaving it or doing it all at once, and one
coordinated pass is scheduled. Without this written down, the next reader files a
bug or, worse, "fixes" it.

Also sharpens the toolchain note: rustup resolves `rust-toolchain.toml` from the
working directory upward rather than from `--manifest-path`, so a command run at
the repository root — every CI job here — never sees it, and neither does the Nix
build. It applies when working inside `crates/intent/` with rustup, and nowhere
else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
agent-tool: Claude Code
agent-tool-version: 2.1.220
agent-runtime: Claude Code 2.1.220
agent-session-lookup: unavailable
tooling-profile: dotfiles@unknown-dirty
The checker was lifted into this crate but its integration suite was not: 19 tests
in `flakes/axe/tests/vrs_check.rs` stayed in `schickling/dotfiles` and drive the
`axe` binary. They are the differential oracle for the extraction, so a tool whose
real test suite lives in its own consumer is backwards — and invisible until the day
that consumer stops depending on it. `cargo test` here reported 8 passing unit tests,
which is a much weaker suite than the "19/19" figure the extraction has been quoted
as having.

Ported: the harness now drives `CARGO_BIN_EXE_intent` with top-level subcommands
instead of the `axe vrs …` layer. The corpus-relative asset resolution the lift
introduced needed checking rather than assuming, and it holds: the helpers pass
`repo/context/vrs` — a corpus root, not a repository root — and the harness builds
`16-enforcement/` underneath it, which is exactly what resolution now requires.

10 of the 19 travelled. The other 9 did not, and the file names each one and why.
They exercise `review` / `review-fixtures` through the CAIC runner, a separate
binary owned by `axe`, and they assert on the CAIC envelope itself
(`coding_agent.result.v1`, `run.context_files`, `run.permission.effective`). This
crate builds no such binary, and lifting `caic` here would invert the settled
dependency — `axe` consumes `intent`, not the reverse. A stub could only make them
pass by reimplementing the collaborator under test, which is the "green because the
hard cases were dropped" failure in a more expensive disguise.

Those 9 stay green in `axe`'s own suite, so the behaviour is still covered — but only
while `axe` keeps consuming this crate. Closing the gap needs a CAIC boundary this
crate can drive on its own, and that is not this commit.

The two `review_*` tests that did travel are the ones that refuse before CAIC is ever
executed, so no runner is needed to reach their assertions. `--coding-agent` is handed
a deliberately non-existent path to keep that property honest: if a test ever needs it
to run, that test does not belong in this file.

Verified: 8 unit + 10 integration pass natively and inside `nix flake check`, where
the sandbox runs the exec'd fake providers without complaint; fmt and
`clippy -D warnings` clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
agent-tool: Claude Code
agent-tool-version: 2.1.220
agent-runtime: Claude Code 2.1.220
agent-session-lookup: unavailable
tooling-profile: dotfiles@unknown-dirty
…not one thing

The section as written said "Rule ids are `AXE.VRS-R01..R19`" alongside the
`schema_version` and the message prefixes, which reads as though all three are part
of what this binary emits. They are not, and measuring says so plainly: this binary
emits **zero** `AXE.VRS-R*` ids. Those are requirement ids in the VRS documents. The
`rule` field of a diagnostic carries a different vocabulary entirely —
`VRS.ENF.link.local-target`, `VRS.ENF.delta-shape` and four others.

The distinction is load-bearing for the coordinated rename the same section points
at. Only `schema_version` and the message prefixes are parsed by a consumer, so only
those are a wire contract; the requirement ids are documentation and can move on
their own schedule. Conflating them would either over-constrain the rename or, worse,
let someone treat the wire string as prose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
agent-tool: Claude Code
agent-tool-version: 2.1.220
agent-runtime: Claude Code 2.1.220
agent-session-lookup: unavailable
tooling-profile: dotfiles@unknown-dirty
@schickling
schickling marked this pull request as ready for review August 8, 2026 20:58
@schickling
schickling merged commit 7cb9102 into main Aug 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants