RQL 2.0: specification draft + TypeScript reference parser - #86
Open
kriszyp wants to merge 14 commits into
Open
Conversation
Ports Harper's REST query-string parser (resources/search.ts:1221-1616) into a standalone, reentrant TypeScript package under src/. Key changes from the original: - parseQuery encapsulates all mutable state (lastIndex, regex instances) in a per-call closure so concurrent invocations are independent. - Query extends URLSearchParams; fast path returns URLSearchParams-backed Query (conditions unset), parsed path populates conditions[]. - Object.defineProperty shadows URLSearchParams.prototype.sort so a Sort object can be assigned on the same property name. - group-by now has a proper break (original falls through into sort). - 45 tests pass: all assertions from query-parse.test.js and the tier1 REST-query-parsing block, plus reentrancy and URLSearchParams shape tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Grammar (ABNF draft), Core comparator/coercion/composition semantics, canonical Query AST (extends URLSearchParams, dual-shape conditions), conformance profiles, and the v1 migration appendix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v1's normalized call form for comparators (lt(price,10)) is removed in 2.0; comparators are infix-only (open set), call syntax is the closed set of result-shaping functions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Language-neutral canonical model (no URLSearchParams, no dual-shape fast path, sort as ordered list, projection as mode+fields) - Comparator reduction: eq/lt/le/gt/ge/contains/starts_with/ends_with/in with uniform not_ negation; strict-vs-coercive becomes verbatim-vs- interpreted value literals; ne/equals/between/aliases -> Appendix B compatibility desugarings - Chaining and between desugar to plain same-path conditions - Percent-encoding layering normative (split-then-decode; %2E works — verified Harper already matches, earlier report was wrong) - prop[]= demoted to host accommodation - New Appendix D: tracked Harper divergences Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conjunction does not distribute over existential array matching: skiLengths=ge=175&=le=180 (one element in range) differs from two separate conditions (any elements witness each). Chaining and prop[group] sub-queries canonicalize to a new ElementMatch construct; between desugars into it. Adds the bare =op= and-continuation spelling (new in 2.0; Harper-lineage grammar rejects it — verified). README: v2 status note. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The bare =op= spelling was motivated by a typo. Also: chain legs take a single value (no lists), and the comparator name is required — Harper's nameless-leg comparator inheritance is now a tracked divergence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Drops Query extends URLSearchParams; parseQuery returns plain ParseResult.
- Parser creates fresh QP/VP regex instances per call for reentrancy.
- VP switching: VP used only after a non-eq comparator or FIQL name is
committed, preventing premature consumption of FIQL second-= tokens.
- Chaining (&= / |=) desugars to ElementMatch{path, some:Group} with
element-relative inner conditions (path:[] for scalar-field legs).
Preserves existential semantics: some ONE array element satisfies all legs.
- between/not_between desugar to ElementMatch (not a flat ge+le pair).
- Bracket prop access (prop[cond]) produces ElementMatch; single-condition
bracket normalises to plain Condition with concatenated path.
- Aliases and not_ prefix handled via resolveFiqlName at parse time.
- All 68 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- QP tokenizer now emits &= / |= as single tokens; chain handling inside
groups (and after verbatim = conditions) was unreachable and errored
- parseCondGroup builds ElementMatch for chains (element scoping was
lost inside (...)/[...])
- Interpreted mode auto-converts round-trip numerals, keeping a==3
distinct from verbatim a=3 (§5.2)
- Nameless chain legs (a=ge=1&=5) are a syntax error per the grammar
- Nested projections are records mode (brand{name} trims, not values)
- Singleton ElementMatch normalizes to a plain Condition on close
- ParseResult.filter narrowed to Group
74 tests pass (was 68).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- B1: negation scopes over the condition's own traversal (top-level ¬∃, in-scope predicate negation, recursive); flattening invariant exempts negated inner conditions; elem-cond surface (tags[=not_eq=urgent]) gives ∃¬ a Core spelling; 'not' reserved in Appendix C - B2: comparators are scalar predicates — contains is string containment, list handling comes only from §5.5 existential traversal - B3: interpretation is schema-free; schema binding is a non-canonical execution concern; conformance vectors schema-free - B4: semantic markers (type:, wildcard *, sort +/-) recognized on raw tokens; %2B sort recommendation removed - S1: ABNF reconciled — standalone calls, chained-cond binding, seg-char/ type-name defined, tokenization rules, trailing comma, chained lists - S2: malformed literals / limit args / duplicate calls are syntax errors; round-trip numeral rule explicit - S3: group-by removed from Core (reserved name, parse-rejected) - S4/S5: nested projections always records mode; wildcard stems verbatim - S6: determinism invariant scoped to desugaring, canonicalization deferred to §7; S7 covered by elem-cond - §5.5: result multiplicity sentence (each record at most once) - Appendix A: collection-matching and closed-converter rows added - Appendix D: rows 10-12 (harper#2433 chained-leg coercion, #2434 index duplicates, contains numeric coercion); row 2/9 updated with PR #2437 verification; examples de-personalized (ratings/reviews) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Negated-inner flattening exemption (§5.3/§6): elem-conds (ic.path=[]) and
negated inner conditions are never normalized to plain Conditions.
pushElementMatch + both bracket-singleton handlers now guard on
!ic.negated && ic.path.length > 0.
- Elem-cond surface (§4 scoped-body): inside prop[...] (isScoped=true),
=fiql-name=value with no preceding path gives Condition{path:[]};
&=/|= after an elem-cond decomposes to conjunction + new elem-cond.
- Malformed typed literals are syntax errors (§5.2.2): boolean:yes,
number:abc, number:, invalid date: → QueryError.
- Limit validation (§5.6): parseNonNegInt enforces non-negative decimal
integer; 2-arg form requires end >= start.
- Duplicate call functions (§5.6): seenCalls Set detects second
select()/sort()/limit() and throws QueryError.
- Nested tuple projection reserved (§5.7): select(rel{[x,y]}) → QueryError.
- Tests: rename ski-lengths example to ratings; add reviews[...] object-
element test; pin chained-value-list, raw-token-marker, malformed-literal,
limit-validation, and duplicate-call suites. 103/103 pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nce ledger not(body) is pure sugar desugared by De Morgan onto leaf negated flags and ElementMatch.negated — canonical model unchanged, gives negated scopes a surface (not(scores[=ge=10&=le=20])). Removed from Appendix C reserved names; §5.6 carves it out of the call-function namespace; harper support filed as HarperFast/harper#2441. Appendix D is now a pointer: implementation divergence ledgers live with the implementations (Harper: HarperFast/harper#2440), keeping the spec vendor-independent. Also: singleton flattening clarified to include element conditions (scores[=ge=10] ≡ scores=ge=10). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
not(...) (§5.4): term-form De Morgan negation, works at top level, inside groups, and inside scoped bodies. negateGroup/negateTerm recurse inward: Condition.negated toggles, and/or groups swap (De Morgan), ElementMatch.negated toggles. Nested not() cancels. Empty not() is a QueryError. not is excluded from the call-function namespace so unknown-call-function never fires on it. Singleton ElementMatch flattening (§5.3/§5.5/§6): drop the path.length>0 guard from pushElementMatch and both bracket-singleton handlers. Plain Conditions on list paths are already existential (§5.5), so elem-cond path=[] flattens safely: scores[=ge=10] ≡ scores=ge=10. Negated inner and negated scope remain unflattened. Update tests accordingly. 111/111 pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Maps PostgREST's URL filter syntax onto the canonical model: every filter operator lands in Core or as an extension comparator, and both dialects' logical layers are the same Group tree in prefix vs infix notation. Records the real gaps (projection aliasing/casting, nulls ordering, aggregates, resource embedding) and three semantic deltas to preserve deliberately (embedded-filter join defaults, SQL null tri-state -> map RQL negation to IS DISTINCT FROM, value-list any/all vs element scoping). Serves as evidence the canonical representation is dialect-neutral and as guidance for accepting a second surface syntax. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resurrects this project as RQL 2.0 — a specification and reference implementation of the query-language lineage that Harper's REST syntax descends from.
Philosophy (revised during review): the spec defines the ideal language — the cleanest coherent semantics for the syntax in production use — rather than reverse-engineering any single implementation. The reference implementation optimizes for correctness and clarity for direct users; it is deliberately language-neutral in its data model so reference implementations in other languages can follow. Harper keeps its own perf-tuned parser and converges over time; its known deviations are tracked in a divergence-ledger epic (HarperFast/harper#2440) linked from Appendix D, keeping the spec vendor-independent.
specification/rql-2.0.md— draft spec: ABNF grammar, comparator/value semantics, element-scoped matching, the canonical parsed representation, Core/Extensions conformance profiles, v1 migration (Appendix A), compatibility aliases (Appendix B), implementation divergence ledgers (Appendix D → harper#2440), and a non-normative PostgREST dialect mapping (Appendix E).src/— zero-dependency TypeScript reference parser producing the spec's canonical model: plain serializable data ({filter, sort, select, limit, offset}), no host-framework types.test/v2/— 111 tests: the conformance suite (surface string → canonical representation), seeded from Harper's parser test suites plus new coverage.Language highlights
not(...) group/scope negation — pure sugar desugared by De Morgan onto leaf
not_flags and negated element scopes; Harper support filed as HarperFast/harper#2441.Small orthogonal core: 9 comparators (
eq,lt,le,gt,ge,contains,starts_with,ends_with,in) with uniformnot_negation.ne/equals/not_equal/between/abbreviations are compatibility aliases with exact desugarings — one canonical form per meaning.Verbatim vs. interpreted values, not strict-vs-coercive comparators:
a=3binds the string"3";a==3binds the number3. The distinction lives in the value literal, so there is exactly one equality.Element-scoped matching: conditions on list-valued properties match existentially, and conjunction doesn't distribute over that quantifier — so
ratings=ge=3&=le=4(one element in range) is semantically distinct from two separate conditions (any elements witness each). Chains andprop[cond&cond]sub-queries canonicalize to anElementMatchnode;betweendesugars into it.Operator model: comparators are infix-only with an open, execution-validated name set; call syntax (
select/sort/limit) is a closed, parse-validated set of result-shaping functions. (In 1.x these were one category; see Appendix A.)Percent-encoding layering: paths split on literal
.before per-segment decoding, so%2Eexpresses a literal dot in a property name.PostgREST dialect mapping (Appendix E) — every PostgREST filter operator maps into Core or an extension comparator, and both dialects' logical layers are the same
Grouptree (prefix vs infix). Recorded as evidence the canonical model is dialect-neutral, with the real gaps (projection aliasing/casting, nulls ordering, resource embedding) and semantic deltas (embedded-filter join defaults, SQL null tri-state) named rather than papered over.Open questions for review
inshould also demote to an alias of or-groupedeq(kept Core for idiom and index-scan affinity).(a=in=(1,2)&b=2)) throw a syntax error — the group sub-parser lacks value-mode scanning for parenthesized lists. Fails loudly, fix planned.Test plan
npm test— 111/111 passing (includes the ski-lengths element-scoping case, verbatim/interpreted distinction,%2Ehandling, alias desugaring equivalences)🤖 Generated with Claude Code
🤖 Generated with Claude Code