feat(intent): standalone checker crate and the CI that gates the corpus - #1
Merged
Merged
Conversation
…the corpus Lifts `flakes/axe/src/vrs.rs` out of schickling/dotfiles into a standalone package at `crates/intent/`, exposing both the `intent` binary and a library whose `run` entry point `axe vrs` calls directly. Wires the two checks that previously lived in dotfiles onto this repo's own CI, kept as separately-named jobs rather than collapsed behind a single gate, so a run shows which one concluded and how. Both checks are written against the way this class of gate fails silently. `intent check` exits 0 on an empty directory and on a directory holding no VRS artifacts, so "the check passed" cannot on its own distinguish a healthy corpus from a missing one. `corpus-strict` therefore asserts the corpus exists before checking, and afterwards asserts the derived graph is non-empty — the graph is what discriminates, being empty for both an empty directory and a wrong path. `semantic-review-fixtures` likewise fails when it finds zero fixtures, since a loop that is green over nothing is the same silent pass. `crates/intent` is a standalone package with no workspace root above it, so cargo writes to `crates/intent/target/` and not to `./target/`. The corpus check invokes the built binary by path from the checkout root, so it passes `--target-dir` explicitly instead of assuming either location. Enforcement assets now resolve under the corpus root rather than the enclosing repository, so a corpus without its own `16-enforcement/` fails instead of quietly borrowing the repository's copies and being graded against a rubric it never declared. The error names both the missing asset and the corpus root; `crates/intent/README.md` documents the behavior. Verified locally against this corpus from a clean checkout: strict check reports 0 diagnostics, the graph is 246 nodes / 215 edges, all 5 fixtures validate against the enforcement schema, and the crate's 8 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> agent-tool: Claude Code agent-tool-version: 2.1.220 agent-runtime: Claude Code 2.1.220 agent-session-lookup: unavailable tooling-profile: dotfiles@unknown-dirty
…page The paragraph asserted that the conventions travel, and the only evidence it offered was a second project using them. Removing that sentence alone would leave the assertion standing on support that is no longer on the page — the unfalsifiable prose this corpus exists to catch. Rewritten so every claim left in it is carried by something a reader can see: the artifacts are plain Markdown, and the corpus is written in the conventions it specifies. The narrower claim is the one the closing paragraph actually needs — "conventions that need a bespoke toolchain do not get adopted" pairs with no-toolchain, not with adoption elsewhere. No outside project is substituted as evidence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> agent-tool: Claude Code agent-tool-version: 2.1.220 agent-runtime: Claude Code 2.1.220 agent-session-lookup: unavailable tooling-profile: dotfiles@unknown-dirty
Distribution is the flake only — there is no crates.io release — so this is how the CLI is consumed. `nix build .#intent` packages it; `nix flake check` gates fmt, clippy, the crate's test suite, a `--help` smoke test, and a proof that the packaged binary reads a real corpus. Two attributes are needed to package a crate that is not at the repository root, and they are not the same knob. `buildAndTestSubdir` moves only the build and test phases; `cargoSetupPostPatchHook` still reconciles the vendored lockfile against the repository root and fails with "Missing Cargo.lock from src". `cargoRoot` is what points that reconciliation at the crate. `src` stays the whole repository on purpose, so the corpus at `intent/` remains visible to a check aimed at it from the same source tree. The nix lane is a separate workflow rather than a job inside `ci.yml`. Folding a multi-minute Nix build in beside the fast corpus gates would couple them, and running the corpus gates through `nix flake check` would collapse `corpus-strict` and `semantic-review-fixtures` into a single check named `check` — a run would no longer show which gate concluded and how. This lane packages the CLI and re-gates nothing. Adopting the formatter and the lint required two changes to the lifted source: - The block comments carried hand-aligned continuation lines that rustfmt de-indents to column 0, which reads worse than what it replaced. Converted to line comments, which rustfmt leaves alone, so the wording is unchanged and the gate is real rather than permanently red or absent. - `automated_context_indicator` becomes the `Iterator::find` it was written as by hand. `require_section` keeps its eight arguments under an explicit `allow` with the reason inline: this crate is a lift whose acceptance bar is behaving identically, and grouping those arguments is a refactor motivated only by a style lint, better done on its own where a regression would be attributable. Verified locally: `nix flake check` passes all five checks, the packaged binary reports 246 graph nodes against the real corpus, and after these source edits the strict check still reports 0 diagnostics with the graph unchanged at 246/215. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> agent-tool: Claude Code agent-tool-version: 2.1.220 agent-runtime: Claude Code 2.1.220 agent-session-lookup: unavailable tooling-profile: dotfiles@unknown-dirty
…fiers Two sections authored by the seat that built the original extraction, taken as a patch rather than a push. The first exists because the CI here encodes a defence the README did not explain: `check` and `graph` both exit 0 against a path holding nothing, so a reader copying `corpus-strict` could keep the first assertion and drop the node-count one — which is the half that actually discriminates a clean corpus from an absent one. The second is load-bearing. `AXE.VRS-R*`, `axe.vrs.check.v1` and the `axe vrs …` message prefixes survive inside a binary called `intent` on purpose: they are the differential oracle proving the lift was faithful, and the schema version is a wire contract with a live consumer. There are 29 `axe vrs` occurrences in `lib.rs`, so this is the whole user-visible surface rather than a stray literal — a partial rename would be worse than either leaving it or doing it all at once, and one coordinated pass is scheduled. Without this written down, the next reader files a bug or, worse, "fixes" it. Also sharpens the toolchain note: rustup resolves `rust-toolchain.toml` from the working directory upward rather than from `--manifest-path`, so a command run at the repository root — every CI job here — never sees it, and neither does the Nix build. It applies when working inside `crates/intent/` with rustup, and nowhere else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> agent-tool: Claude Code agent-tool-version: 2.1.220 agent-runtime: Claude Code 2.1.220 agent-session-lookup: unavailable tooling-profile: dotfiles@unknown-dirty
The checker was lifted into this crate but its integration suite was not: 19 tests in `flakes/axe/tests/vrs_check.rs` stayed in `schickling/dotfiles` and drive the `axe` binary. They are the differential oracle for the extraction, so a tool whose real test suite lives in its own consumer is backwards — and invisible until the day that consumer stops depending on it. `cargo test` here reported 8 passing unit tests, which is a much weaker suite than the "19/19" figure the extraction has been quoted as having. Ported: the harness now drives `CARGO_BIN_EXE_intent` with top-level subcommands instead of the `axe vrs …` layer. The corpus-relative asset resolution the lift introduced needed checking rather than assuming, and it holds: the helpers pass `repo/context/vrs` — a corpus root, not a repository root — and the harness builds `16-enforcement/` underneath it, which is exactly what resolution now requires. 10 of the 19 travelled. The other 9 did not, and the file names each one and why. They exercise `review` / `review-fixtures` through the CAIC runner, a separate binary owned by `axe`, and they assert on the CAIC envelope itself (`coding_agent.result.v1`, `run.context_files`, `run.permission.effective`). This crate builds no such binary, and lifting `caic` here would invert the settled dependency — `axe` consumes `intent`, not the reverse. A stub could only make them pass by reimplementing the collaborator under test, which is the "green because the hard cases were dropped" failure in a more expensive disguise. Those 9 stay green in `axe`'s own suite, so the behaviour is still covered — but only while `axe` keeps consuming this crate. Closing the gap needs a CAIC boundary this crate can drive on its own, and that is not this commit. The two `review_*` tests that did travel are the ones that refuse before CAIC is ever executed, so no runner is needed to reach their assertions. `--coding-agent` is handed a deliberately non-existent path to keep that property honest: if a test ever needs it to run, that test does not belong in this file. Verified: 8 unit + 10 integration pass natively and inside `nix flake check`, where the sandbox runs the exec'd fake providers without complaint; fmt and `clippy -D warnings` clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> agent-tool: Claude Code agent-tool-version: 2.1.220 agent-runtime: Claude Code 2.1.220 agent-session-lookup: unavailable tooling-profile: dotfiles@unknown-dirty
…not one thing The section as written said "Rule ids are `AXE.VRS-R01..R19`" alongside the `schema_version` and the message prefixes, which reads as though all three are part of what this binary emits. They are not, and measuring says so plainly: this binary emits **zero** `AXE.VRS-R*` ids. Those are requirement ids in the VRS documents. The `rule` field of a diagnostic carries a different vocabulary entirely — `VRS.ENF.link.local-target`, `VRS.ENF.delta-shape` and four others. The distinction is load-bearing for the coordinated rename the same section points at. Only `schema_version` and the message prefixes are parsed by a consumer, so only those are a wire contract; the requirement ids are documentation and can move on their own schedule. Conflating them would either over-constrain the rename or, worse, let someone treat the wire string as prose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> agent-tool: Claude Code agent-tool-version: 2.1.220 agent-runtime: Claude Code 2.1.220 agent-session-lookup: unavailable tooling-profile: dotfiles@unknown-dirty
schickling
marked this pull request as ready for review
August 8, 2026 20:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lifts the VRS checker out of
schickling/dotfilesinto a standalone crate in thisrepo, and wires the two checks that gate the corpus onto this repo's own CI.
Packages it as a flake too, which is the only supported distribution.
What is here
crates/intent/intentbinary + library exposingrun.github/workflows/ci.ymlmaterialises,crate,corpus-strict,semantic-review-fixturesflake.nix/flake.lockchecksfor fmt, clippy, tests, help, corpus.github/workflows/nix.ymlnixlane — a separate workflow, deliberatelycrates/intent/README.md.gitignoretarget/,result, and the JSON reports CI writes at the rootaxe vrsembeds the library and callsintent::rundirectly, somain.rsis athin shell over the same entry point — anything living only in the binary would be
behavior the embedded caller silently does not get.
The path bug this fixes, and the proof it was real
The staged
corpus-strictjob built with--manifest-path crates/intent/Cargo.tomland then invoked
./target/release/intent.crates/intentis a standalonepackage — its own
Cargo.lock, no workspace root above it — so cargo writes tocrates/intent/target/, not./target/. Both the strict step and the graph stepwould have died on "no such file" before either gate ran, which is a green-looking
job that never checked anything.
Reproduced from a clean checkout root with no pre-existing
target/:Fixed by passing
--target-dir targetexplicitly, rather than by rewriting the twoconsumer steps to the nested path.
--target-dirpins the output locationindependent of workspace detection, so if a workspace root ever appears above the
crate the binary does not silently relocate again.
Path audit, closed: the cargo output directory was the only cwd-dependent path
in the file. Every other path —
intent,intent/15-evaluation/semantic-review,intent/16-enforcement/review-result.schema.json,report.json,graph.json— isrelative to the checkout root, and each was verified against the real tree.
Why the checks look paranoid
intent checkexits 0 on an empty directory and on a directory holding no VRSartifacts, so "the check passed" cannot by itself distinguish a healthy corpus from
a missing one. Porting the assertion as-is would reproduce a known silent-pass class.
So
corpus-strictasserts the corpus exists before checking, and afterwards assertsthe derived graph is non-empty. The graph is what discriminates: it is empty for both
an empty directory and a wrong path, and non-empty only when artifacts were genuinely
read.
semantic-review-fixturesfails when it finds zero fixtures for the samereason — a loop that is green over nothing is the same silent pass.
The jobs are deliberately kept separate and separately named rather than
collapsed behind a single gate, so a run shows which one concluded and how.
Behavior change: enforcement assets resolve under the corpus
reviewresolves16-enforcement/review-prompt.mdand16-enforcement/review-result.schema.jsonrelative to the corpus root, not theenclosing repository. A corpus without its own
16-enforcement/now fails ratherthan quietly borrowing the repository's copies — silently falling back produced a
review graded against a rubric the corpus never declared.
Exit 2, naming both the missing asset and the corpus it was missing from:
The asset alone would not say which corpus was searched; the root alone would not say
what it was expected to contain. Documented in
crates/intent/README.md.Verified locally before pushing
From a clean checkout root, against this repo's actual corpus:
cargo metadata --locked--lockedcargo test --lockedcheck intent --profile strict --jsongraph intent --jsonactionlinton both workflowscargo clippy --all-targets -D warningscargo fmt --checknix flake checkchecks.reads-the-corpusThe Nix build's test phase was checked in the derivation log rather than assumed: it
runs all 8 tests, so
doCheckis not silently vacuous here either.check-jsonschemaruns fully offline — every schema is a local file, so none is everfetched.
Packaging, and why the nix lane is its own workflow
Two attributes are needed to package a crate that is not at the repository root, and
they are not the same knob — this is the part that is easy to get wrong:
buildAndTestSubdirmoves only the build and test phases.cargoSetupPostPatchHookstill reconciles the vendored lockfile against$sourceRoot/Cargo.lock— the repository root — and fails with "Missing Cargo.lockfrom src".
cargoRootis what points that reconciliation at the crate.srcstays the whole repository on purpose, so the corpus atintent/remains visibleto a check aimed at it from the same source tree. That is what
checks.reads-the-corpususes to prove the packaged binary reads real artifacts, not merely that it compiles.
The lane is a separate workflow rather than a job inside
ci.yml. Running the corpusgates through
nix flake checkwould collapsecorpus-strictandsemantic-review-fixturesinto a single check namedcheck, and a run would no longershow which gate concluded and how. Folding a multi-minute Nix build in beside the fast
corpus gates would couple them for no benefit. This lane packages the CLI and re-gates
nothing.
Two deliberate changes to the lifted source
Both are called out rather than buried, because this is a lift whose acceptance bar is
that
axe vrsbehaves identically:de-indents to column 0 — which reads worse than what it replaced. Rather than commit
that, or drop the fmt gate, they became line comments, which rustfmt leaves alone.
Wording is unchanged.
automated_context_indicatorbecomes theIterator::findit was alreadywritten as by hand.
require_sectionkeeps its eight arguments under an explicit#[allow(clippy::too_many_arguments)]with the reason inline: regrouping them is arefactor motivated only by a style lint, and doing it inside a lift would make any
resulting regression unattributable. Worth doing later, on its own.
The integration suite that stayed behind — and what did not travel
The checker was lifted into this crate but its integration suite was not: 19 tests in
flakes/axe/tests/vrs_check.rsstayed inschickling/dotfilesand drive theaxebinary. They are the differential oracle for the extraction, so a tool whose real test
suite lives in its own consumer is backwards — and invisible until the day that
consumer stops depending on it.
10 of the 19 travelled. The other 9 did not, and
crates/intent/tests/vrs_check.rsnames each one and why, rather than presenting a suite that looks complete:
check/graphtests + the 2reviewtests that refuse before CAIC runsreview/review-fixturesthrough the CAIC runnerThe omitted 9 assert on the CAIC envelope itself —
coding_agent.result.v1,run.context_files,run.permission.effective. CAIC is a separate binary(
coding-agent, a shim overaxe::caic) owned byaxe. This crate builds no suchbinary, and lifting
caichere would invert the settled dependency:axeconsumesintent, not the reverse. A stub could only make them pass by reimplementing thecollaborator under test — the "green because the hard cases were dropped" failure in a
more expensive disguise.
They stay green in
axe's own suite, so the behaviour is still covered — but onlywhile
axekeeps consuming this crate. Closing that properly needs a CAIC boundarythis crate can drive on its own.
The corpus-relative asset resolution the lift introduced was checked rather than
assumed: the harness helpers pass
repo/context/vrs— a corpus root, not a repositoryroot — and build
16-enforcement/underneath it, which is what resolution now requires.--coding-agentis handed a deliberately non-existent path, so if a test ever needs itto execute, that test does not belong in this file.
Totals: 8 unit + 10 integration, passing natively and inside
nix flake check,where the sandbox runs the exec'd fake providers without complaint.
About the CI runs
concurrency.cancel-in-progress: truemeans each push cancels the run in flight, and acancelled run demonstrates nothing — the citable run is the last one on this branch.
The
materialisesjob answers a question that is not a given: does this repositorymaterialise jobs at all? A run object can exist with zero jobs, and that is how a
sibling repo's CI sat "queued" for a month unnoticed. It does — 4 jobs in the
cirunand 1 in the
nixrun, all concluding.A README-only commit from another agent landed on this branch as expected. It was kept
and rebased onto, not reverted.
Not in scope here
axe.vrs.check.v1or theAXE.VRS-R*ids — a separate coordinated pass,and renaming now would cost the differential oracle these checks depend on.
Posted on behalf of @schickling
agent_nameagent_toolagent_tool_versionagent_runtimeagent_session_lookuptooling_profile