Differential conformance harness: Harper's REST parser vs. the RQL 2.0 reference parser

[kriszyp]

Builds the differential conformance harness for RQL 2.0: a corpus of query strings is parsed by both Harper's REST query parser and the reference parser in src/, Harper's output is mapped into the canonical model (§6) by an adapter, the two are diffed, and every difference is classified. Nothing in specification/ or src/ is touched — the harness lives in new conformance/, scripts/ and test/conformance/ directories.

Spec Appendix D deliberately carries no vendor rows; it links to each implementation's own public ledger, which for Harper is HarperFast/harper#2440. This turns that ledger from a hand-written list into an exhaustive, re-runnable one.

What the first run found

349 corpus cases: 190 agree, 159 diverge, 0 unclassified.

	Cases
Existing ledger row	33	rows 1, 2, 6, 7, 13, 14 confirmed by witnesses; 8 and 9 confirmed in deferred-error mode
NEW divergence	117	across 22 proposed ledger rows, each with the spec clause and a suggested action
Reference-parser bug	9	6 defects in src/parser.ts / src/comparators.ts, not in Harper

The full classification, the worked examples and a row-per-case appendix are in conformance/conformance-report.md. The largest single divergence is that Harper's typedDecoding never applies §5.2.2 to a bare token, so a==3 parses as the string "3" and a==true as "true" — 70 cases carry it. The most consequential are the ones that silently change which records match: ?a== drops the condition entirely and returns everything; ()/[]/prop[] become an empty AND group that matches everything; a duplicate limit(...) silently wins on last-write; not_lt/not_ge/not_<unknown> are not negations at all.

The 6 reference-parser bugs are the reason to read this before the harness is trusted as a Harper scorecard: the reference parser applies the * wildcard to != (§5.1.2 restricts it to ==), lets a URIError escape uncaught where §6.1 requires a 400, refuses to chain onto a between, accepts a leading |, and is missing the notEqual alias. They are described in the report; this PR does not fix them — src/ is out of scope here.

How it runs

npm run conformance          # replay + regenerate the report (no Harper checkout needed)
npm run conformance:check    # CI: fail if the committed report is out of date
npm test                     # 111 parser tests + 89 conformance tests
HARPER_PATH=../harper npm run conformance:record   # re-record against a built Harper checkout

Record/replay, per the ruling on the planning gate. --record gives every Harper parse its own short-lived process — Harper's parser holds module-global state (lastIndex/currentQuery/queryString in resources/search.ts), so two parses must never interleave, and a separate process makes that structural instead of a convention. Ordinary runs replay the committed fixture, so CI needs no cross-repo dependency and --check asserts the report is reproduced byte for byte. Replay uses one persistent reference worker (the reference parser holds no global state), killed and replaced on a timeout.

conformance/ledger.json is a provenance-stamped cache of harper#2440 with a refresh command; the issue stays canonical and the report prints the snapshot's age.

For the human reviewer

Where to look. The diff is ~29k lines, but ~25k of that is two generated artifacts —
conformance/fixtures/harper-parse.json (the recorded raw Harper output) and
conformance/conformance-report.md (the result). Everything else is ~4.1k lines, roughly half of
it tests, and the part that carries judgement is conformance/harperAdapter.ts and
conformance/classify.ts.

• Adapter fidelity is the judgement call — conformance/harperAdapter.ts states five rules at the top, and the load-bearing one is that values pass through untouched: re-interpreting "3" as 3 there would erase the single largest divergence the run found. The second is that alias resolution follows Harper's vocabulary, not the spec's — includes maps to in because that is what Harper means by it, so the disagreement with Appendix B is reported rather than absorbed. Both are worth disagreeing with if you read them differently.
• Where a rule pins witness queries rather than matching a shape, that is deliberate but it is the part most likely to rot: a future corpus case with the same root cause becomes unclassified and fails the run until someone widens the rule. test/conformance/replay.test.ts also fails on a rule that matches nothing, so a rule made obsolete by a Harper fix has to be deleted with it.
• Verdicts name the side that has to change. Several cases have both parsers deviating from §4 in different ways ((), a===1===2, a bare b in a=1&b); the verdict goes to Harper and the rationale records the reference parser's own gap. If you would rather see those split into two findings, say so — the shape of classify.ts supports it.
• sort() is classified as a Harper divergence on the reading that §4's call makes the argument list optional. §5.6 is ambiguous; it may be that sort should require a key, in which case the reference parser is the one that is wrong.
• Ledger rows 11 and 12 are execution-level (duplicate results from indexed elements, contains coercing numbers) and no parse-only harness can witness them. The report says so rather than leaving them looking untested.
• A recording of another repo's output now lives in this one. conformance/fixtures/harper-parse.json
  is Harper's parse output committed into the spec repo. That is what makes CI independent of a
  Harper checkout, and it is what the planning ruling chose; the cost is that the fixture is stale
  the moment Harper's parser changes, and only a deliberate conformance:record notices. If you
  would rather this harness (or just its fixture) lived in harper, now is the time to say so.
• The uninterpreted-value filter runs before every other rule. Harper types no bare literal
  (§5.2.2), which touches 70 cases and would otherwise decorate most other divergences. Those
  differences are removed from a case before the rules see it, and the case is marked
  "+ uninterpreted values". The risk of that choice is that it is one rule's worth of masking
  applied globally — if a second cause ever produces exactly the same difference shape, it will be
  attributed to this one.
• Provenance is asymmetric, on purpose. Harper's side of every comparison is the recorded
  fixture, so it is exactly the commit the report names. The reference side is parsed live at
  replay time from src/, which the provenance row dates only as of the recording — the report
  says so, and conformance:check is what keeps the two from drifting apart silently.
• One review nit declined. The reviewer would move conformance/referenceRunner.ts's module
  docstring into the design note. It is now two sentences stating the two invariants the class
  exists to hold — every request settles, and only one parse is in flight — which is the part a
  reader cannot get from the code. The rationale behind them is in conformance/design.md.
• Not fixed here, found on the way: npm run build and npm run typecheck already fail on kris/rql-v2 — src/*.ts imports carry .ts extensions without allowImportingTsExtensions, and tsc cannot emit with that flag set. npm run typecheck:conformance (new) does pass and covers src/ too, so the code is type-clean; the build configuration is the problem, and picking between .js import specifiers and a strip-only publish is a call for the repo owner.

Verification

• npm test — 111 parser tests, 103 conformance tests, all passing. The conformance suite includes an end-to-end replay that asserts the committed report is byte-for-byte what the pipeline produces, negative controls that an unexplained difference stays unclassified, and a check that no classification rule is dead.
• npm run typecheck:conformance — clean over conformance/, test/conformance/ and src/.
• npm run conformance:check — clean. Two --record runs against the same Harper commit, at different --concurrency, produced byte-identical recorded outcomes (the fixture differs only in its recordedAt stamp) and the same 190/33/117/9/0 totals.
• Recorded against Harper 11a1c489 (v5.2.2-14-g11a1c4891) on Node v26.2.0.
• Independent pre-push review ran over five rounds; the footer below reflects only the last one,
  which was a narrow delta. Across all five the outside legs were Codex (every round),
  Cursor/Grok (rounds 1 and 4) and the Harper domain pass (rounds 3 and 4). 21 findings were
  raised, 20 fixed — including two hang blockers in the replay supervisor — and the one left open
  is the comment nit above. Gemini was unauthenticated on the recording host in every round.

Refs #4

_{Review-Coverage: authored=claude; ran=codex; blocked=gemini(auth); declined=cursor-grok,cursor-composer,domain; rounds=5 @ 97e7b8b}

_{Human-Review-Need: 4 @ 97e7b8b}