Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,34 @@ jobs:
- run: cargo build --locked --manifest-path crates/intent/Cargo.toml
- run: cargo test --locked --manifest-path crates/intent/Cargo.toml

# Port of dotfiles' `axe-vrs-context-strict`.
# Strict validation of the repository's Intent corpus.
corpus-strict:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# `crates/intent` is a standalone package with no workspace root, so cargo

- name: retired name is absent
run: |
set -euo pipefail
retired="$(printf '\166\162\163')"
if git grep -I -n -i "$retired" -- .; then
echo "::error::retired name found in tracked content"
exit 1
fi
if git ls-files | grep -i "$retired"; then
echo "::error::retired name found in a tracked path"
exit 1
fi

# `crates/intent` is a standalone package with no workspace root, so cargo
# writes to `crates/intent/target/` and the steps below would not find the
# binary at `./target/`. `--target-dir` pins the output next to the checkout
# root regardless of whether a workspace root ever appears above the crate.
- run: cargo build --locked --release --manifest-path crates/intent/Cargo.toml --target-dir target

- name: corpus is present
# `intent check` exits 0 on an empty directory and on a directory holding no
# VRS artifacts, so "the check passed" cannot by itself distinguish a healthy
# Intent artifacts, so "the check passed" cannot by itself distinguish a healthy
# corpus from a missing one. Fail on absence explicitly, before checking.
run: |
set -euo pipefail
Expand All @@ -55,7 +69,7 @@ jobs:
cat report.json >&2; exit 1;
}
jq -e '
.schema_version == "axe.vrs.check.v1"
.schema_version == "axe.intent.check.v1"
and .profile == "strict"
and (.diagnostics | length) == 0
' report.json > /dev/null
Expand All @@ -73,7 +87,7 @@ jobs:
jq -e '(.nodes | length) > 0' graph.json > /dev/null \
|| { echo "::error::strict check examined 0 artifacts — wrong path or empty corpus"; exit 1; }

# Port of dotfiles' `vrs-semantic-review-fixtures`.
# Validate the semantic-review fixture corpus and enforcement schema.
semantic-review-fixtures:
runs-on: ubuntu-latest
steps:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
# otherwise, since the crate is a standalone package with no workspace root.
target/

# Local scratch artifacts and render intermediates.
/tmp/

# Artifacts the CI steps write into the checkout root while running.
/report.json
/graph.json
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ intent tree with its own `requirements.md` and `spec.md`.
| `11-roadmap` | `roadmap.md` — directions under consideration, not a plan |
| `12-common-subsystems` | recurring subsystem shapes worth naming once |
| `13-review-smells` | what a reviewer should look for in an intent tree |
| `14-grill-vrs` | the procedure for interrogating a tree until it holds up |
| `14-grill-intent` | the procedure for interrogating a tree until it holds up |
| `15-evaluation` | isolated evaluation runs and their fixtures |
| `16-enforcement` | the rules a checker can enforce mechanically |

Expand Down
14 changes: 7 additions & 7 deletions context/cli/.decisions/0001-use-per-backend-review-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,19 @@ Status: accepted

## Context

`axe vrs review` can run through multiple stock CAIC backends. Manual
`intent review` can run through multiple stock CAIC backends. Manual
real-provider evals showed uneven provider state: `codex` produced a
schema-valid VRS review report for `context/vrs/16-enforcement`, while `claude`
schema-valid Intent review report for `context/intent/16-enforcement`, while `claude`
still needs an authenticated isolated real-provider report after fixing its
wrapper-level command mapping.

The design question was whether one provider gap should keep the whole `axe vrs
The design question was whether one provider gap should keep the whole `intent
review` surface marked not production-ready.

## Evidence and Argument

CAIC already exposes explicit backend ids and reports the effective backend in
result metadata. `axe vrs review` now preflights the selected backend through
result metadata. `intent review` now preflights the selected backend through
`$CODING_AGENT capabilities --json` before provider invocation, so readiness can
be evaluated against the same contract for each backend.

Expand All @@ -35,14 +35,14 @@ backend on an unrelated provider authentication/eval gap.

## Decision

`axe vrs review` production readiness is backend-scoped.
`intent review` production readiness is backend-scoped.

A backend is ready when:

- its CAIC capability preflight satisfies the review contract;
- token-free fake-provider tests cover its provider-specific command mapping;
- at least one bounded manual real-provider run against a small existing VRS
subsystem writes a schema-valid `axe.vrs.review.v1` report.
- at least one bounded manual real-provider run against a small existing Intent
subsystem writes a schema-valid `axe.intent.review.v1` report.

## Consequences

Expand Down
26 changes: 13 additions & 13 deletions context/cli/.delta/DELTA-001-claude-real-provider-review-pending.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ Status: open

## Divergence

`axe vrs review` is implemented and covered by fake-provider tests. Production
`intent review` is implemented and covered by fake-provider tests. Production
readiness is tracked per backend: the stock `codex` path has produced a
semantic review report for a small VRS subsystem, while the stock `claude` path
semantic review report for a small Intent subsystem, while the stock `claude` path
still lacks an authenticated isolated real-provider report.

## VRS
## Intent

- [spec.md](../spec.md) defines `axe vrs review` backend readiness as
- [spec.md](../spec.md) defines `intent review` backend readiness as
backend-scoped.
- [0001-use-per-backend-review-readiness.md](../.decisions/0001-use-per-backend-review-readiness.md)
records the readiness policy.
Expand All @@ -29,12 +29,12 @@ still lacks an authenticated isolated real-provider report.
## Evidence

- Improved CAIC provider-failure diagnostics exposed Codex native schema
compatibility issues in the baked `axe.vrs.review.v1` schema.
compatibility issues in the baked `axe.intent.review.v1` schema.
- After adding explicit string types and requiring `gate`,
`axe vrs review context/vrs/16-enforcement --backend codex
`intent review context/intent/16-enforcement --backend codex
--timeout-seconds 180 --report ...` succeeded and wrote an
`axe.vrs.review.v1` report.
- `axe vrs review context/vrs/16-enforcement --backend claude
`axe.intent.review.v1` report.
- `intent review context/intent/16-enforcement --backend claude
--timeout-seconds 180 --report ...` failed with CAIC `timeout` before writing
a report.
- Follow-up wrapper inspection found that the Claude command path passed
Expand All @@ -52,15 +52,15 @@ still lacks an authenticated isolated real-provider report.
`ANTHROPIC_API_KEY` is present. The wrapper timeout class is covered by fake
provider tests; the full real Claude report still requires API-key-backed
isolated auth.
- The VRS/Axe decision is to treat review production readiness per backend
- The Intent/CLI decision is to treat review production readiness per backend
rather than requiring every stock backend to pass before a proven backend can
be used.

## Required Reconciliation

- `axe vrs review --backend claude` should complete a bounded manual
real-provider eval against a small existing VRS subsystem using isolated auth
and write an `axe.vrs.review.v1` report.
- `intent review --backend claude` should complete a bounded manual
real-provider eval against a small existing Intent subsystem using isolated auth
and write an `axe.intent.review.v1` report.
- The successful Claude eval evidence should be captured under the relevant
`.experiments/` directory before this delta is removed.
- This delta does not block `codex` readiness. It blocks claiming Claude
Expand All @@ -73,5 +73,5 @@ update implementation
## Resolution Signal

Delete this delta when a bounded manual `claude` run writes a schema-valid
`axe.vrs.review.v1` report for a small existing VRS subsystem, and that report
`axe.intent.review.v1` report for a small existing Intent subsystem, and that report
is captured under the owning `.experiments/` directory.
62 changes: 31 additions & 31 deletions context/cli/.experiments/caic-semantic-review-prototype.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,25 @@

Date: 2026-06-18

Current status: superseded by the implemented `axe vrs review` command, the
baked assets in `context/vrs/16-enforcement/`, and the real-provider eval in
`context/vrs/16-enforcement/.experiments/2026-06-19-real-provider-review-eval.md`.
Current status: superseded by the implemented `intent review` command, the
baked assets in `context/intent/16-enforcement/`, and the real-provider eval in
`context/intent/16-enforcement/.experiments/2026-06-19-real-provider-review-eval.md`.
Keep this document as historical prototype evidence, not as current remaining
work.

## Question

Can `axe vrs review` use the Coding Agent Invocation Contract (CAIC) cleanly for
semantic review while keeping deterministic `axe vrs` primitives as the first
Can `intent review` use the Coding Agent Invocation Contract (CAIC) cleanly for
semantic review while keeping deterministic `intent` primitives as the first
production milestone?

## Method

The prototype used the implemented `coding-agent` binary with a fake Codex
provider. The temporary scenario contained:

- a small VRS node with `requirements.md` and `spec.md`;
- a generated `axe vrs check` diagnostics artifact;
- a small Intent node with `requirements.md` and `spec.md`;
- a generated `intent check` diagnostics artifact;
- a candidate baked review prompt;
- a task-specific JSON Schema for the review result.

Expand All @@ -29,24 +29,24 @@ The command shape was:
```text
$CODING_AGENT run \
--cwd <repo-root> \
--prompt-file <baked-vrs-review-prompt> \
--context-file normative:<vrs-requirements> \
--context-file normative:<vrs-spec> \
--context-file generated-diagnostics:<axe-vrs-check-json> \
--prompt-file <baked-intent-review-prompt> \
--context-file normative:<intent-requirements> \
--context-file normative:<intent-spec> \
--context-file generated-diagnostics:<intent-check-json> \
--mode review \
--permission read-only \
--approval never \
--config-policy isolated \
--network-policy disabled \
--output-format json \
--output-schema <vrs-review-result-schema> \
--output-schema <intent-review-result-schema> \
--backend codex
```

Candidate prompt shape:

```text
You are running `axe vrs review`. Review only the supplied VRS artifacts and
You are running `intent review`. Review only the supplied Intent artifacts and
diagnostics. Return JSON matching the schema. Do not propose file edits.
```

Expand All @@ -58,7 +58,7 @@ Candidate schema shape:
"type": "object",
"required": ["schema_version", "summary", "findings"],
"properties": {
"schema_version": { "const": "axe.vrs.review.v1" },
"schema_version": { "const": "axe.intent.review.v1" },
"summary": { "type": "string" },
"findings": {
"type": "array",
Expand Down Expand Up @@ -90,7 +90,7 @@ Candidate schema shape:

The CAIC route worked with the fake provider. The adapter:

- accepted workspace VRS files as `normative` context;
- accepted workspace Intent files as `normative` context;
- accepted an outside diagnostics file only when its JSON had a generated
artifact marker (`schema_version` or `producer`);
- invoked the backend in `review`, `read-only`, `approval never`,
Expand All @@ -103,13 +103,13 @@ The successful inner result was:

```json
{
"schema_version": "axe.vrs.review.v1",
"schema_version": "axe.intent.review.v1",
"summary": "1 semantic concern found",
"findings": [
{
"rule": "VRS.REVIEW.spec.actionability",
"rule": "INTENT.REVIEW.spec.actionability",
"severity": "warning",
"artifact": "context/vrs/spec.md",
"artifact": "context/intent/spec.md",
"evidence": "Spec uses vague language: should be good and flexible.",
"suggested_fix": "Replace vague qualities with observable behavior and constraints."
}
Expand All @@ -135,36 +135,36 @@ machine-readable generated marker, CAIC failed before provider execution:
}
```

This is desirable for `axe vrs review`: the checker output must be a generated
This is desirable for `intent review`: the checker output must be a generated
artifact, not an arbitrary host file smuggled into review context.

## Tradeoffs

| Option | Result |
| --- | --- |
| Implement `axe vrs review` now | CAIC is ready enough, but the deterministic checker, baked prompt file, and review schema are not yet production-shaped. This would couple review quality to unstable inputs. |
| Implement `intent review` now | CAIC is ready enough, but the deterministic checker, baked prompt file, and review schema are not yet production-shaped. This would couple review quality to unstable inputs. |
| Keep review experimental until primitives exist | Preserves the accepted implementation order and gives review a bounded diagnostics packet later. Delays semantic smells, but avoids noisy or ungrounded review output. |
| Add only hidden/prototype review command | Useful for local tuning, but risks an untracked command contract unless it is clearly excluded from public docs and CI. |

## Conclusion

Keep production `axe vrs` focused on deterministic primitives first. In
parallel, create the baked semantic-review assets under VRS ownership:
Keep production `intent` focused on deterministic primitives first. In
parallel, create the baked semantic-review assets under Intent ownership:

- `context/vrs/16-enforcement/review-prompt.md` or a sibling prompt artifact
- `context/intent/16-enforcement/review-prompt.md` or a sibling prompt artifact
with the review rubric;
- `context/vrs/16-enforcement/review-result.schema.json` for
`axe.vrs.review.v1`;
- evaluation fixtures under `context/vrs/15-evaluation/` that tune the prompt
against known good and bad VRS examples.
- `context/intent/16-enforcement/review-result.schema.json` for
`axe.intent.review.v1`;
- evaluation fixtures under `context/intent/15-evaluation/` that tune the prompt
against known good and bad Intent examples.

Once `axe vrs check --json` emits stable diagnostics, `axe vrs review` should
Once `intent check --json` emits stable diagnostics, `intent review` should
compose those diagnostics with the prompt/schema and call CAIC exactly through
the command route above.

## VRS Impact
## Intent Impact

This prototype informed the current Axe VRS review command, the baked
semantic-review prompt and schema owned by `context/vrs/16-enforcement`, and the
This prototype informed the current Intent CLI review command, the baked
semantic-review prompt and schema owned by `context/intent/16-enforcement`, and the
manual-only real-provider eval policy. It is historical evidence only; current
readiness state lives in the spec, decisions, deltas, and real-provider evals.
Loading
Loading