Skip to content

design: single source of truth for SynaptExtraction IL (schema ↔ validators) #9

Description

@laynepenney

Context

Sprint 31 shipped @synapt-dev/extract 0.1.0 with handwritten JSON Schema + handwritten validators in TypeScript and Python. Atlas's adversarial sweep on extract#8 found substantial drift between four sources:

  • Schema too loose vs validators
  • Validators too loose vs schema
  • Implementation drift between Python jsonschema and TS ajv on format keywords

Sprint 31 patched the highest-impact drift but cannot eliminate the class — every new field has 4 places to write it (schema, TS validator, Python validator, hosted schema copy) and 6 pairs that can drift.

Goal

Single source of truth: schema, validators (TS + Python), and generated types all derived from one canonical source. Adding a field is one edit. Drift is impossible by construction.

Architecture options

  1. JSON Schema as source → generate validators (data-driven, schema canonical). Semantic checks stay declared once, ported to both languages.
  2. TypeBox as source → generate schema + TS types. Python validates against generated schema.
  3. Pydantic v2 as source → generate schema and TS types via .model_json_schema(). Python primary.
  4. Hybrid: schema canonical for structural; handwritten + ported semantic layer for cross-cutting.

Acceptance test

Atlas's fixture corpus from extract#8 review. Every fixture must produce identical verdicts across the hosted JSON Schema (third-party validator), TypeScript runtime, and Python runtime.

Scope

Sprint 32 design + implementation. Ship 0.2.0 from this work. 0.1.x stays as the handwritten beta. Migration notes in CHANGELOG.

Out of scope

  • New IL fields (separate stories)
  • Prompt-system refactor (independent)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions