Feature: Manifest as source - #124
Merged
Merged
Conversation
The first half of the manifest-as-source flip: every core entity's neutral schema now exists as an authored module under src/manifest/, one file per entity, carrying the design prose that previously lived on the decorated classes — the interpretation-hint essays included, since those are the part most worth reading where they are edited. Definitions carry their own schema only; WeNode's shared relations (comments, signals, participants, calls, mentions) are declared once in shared.ts and merged at assembly, exactly as the class hierarchy gives them to subclasses. Beside each schema ride codegen facts about the AD4M class it will generate — base class, optional fields, union aliases, accessor-method declarations — kept out of the neutral schema itself, which stays free of TypeScript. Bootstrapped mechanically from the generated manifest and the class sources rather than transcribed: a wrong predicate would not error anywhere, it would silently orphan data. Verified identical to the generated CORE_MANIFEST — sorted-key deep equality plus declaration order of every property map — before anything starts consuming it. The generated file and the classes remain the live source until the codegen lands in the next commits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…anifest
The second half of the flip, closing the loop the old generator's own comment
promised ("the manifest is meant to become the source of truth"). The decorated
classes under src/entities/ and src/blocks/ are now build artifacts of the
authored manifest modules — same paths, same export names, so nothing importing
them notices.
scripts/generateClasses.mjs emits each class from its definition: decorators
from the neutral schema (format 'file' → the file-storage language, readAs
'dataUri' → the transform, hints and identity riding through), TypeScript shape
from the codegen facts beside it (optionality, union aliases like SignalMode
emitted from their declared values, HasManyMethods interfaces, typed relation
arrays), and doc comments lifted from the manifest module — where they are
edited — into the class, where IDE hovers read them. The `initial:` decorator
option is gone from generated output: SHACL derives initial values from field
initialisers, so it was redundant where present and absent where not, and the
equivalence suite proves the schemas identical without it.
SpacePreference's AGENT_DEFAULT/FOLLOW_SPACE sentinels move to its manifest
module — real code cannot live in a generated file — and the class re-exports
them, so consumers keep their imports. The trailing closed-vocabulary comments
(origin, role, preference, valueType) became real doc comments in the manifest;
the CollectionBlock cleanup note and ThemeData rationale moved with the prose.
The old direction is dismantled: generateCoreManifest.mjs and the generated
coreManifest.ts are gone, '@we/models/generated/coreManifest' is now
'@we/models/manifest' (six import sites), and coreManifest.test.ts reverses its
meaning — it now holds the generated classes and the manifest's *runtime*
compilation in exhaustive agreement through two independent code paths, so a
codegen bug and a compiler bug cannot make the same mistake silently. All 176
backend-ad4m tests pass, including that suite; 2239 monorepo-wide.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The equivalence suite holds the semantics of the generated classes and the manifest in agreement, but prose is invisible to it: a doc comment edited in a manifest module without rerunning generate:classes would leave the class file — where IDE hovers read it — telling the old story. Every doc block the codegen lifts (the class doc and each member's, mirroring its own extraction) must now appear in the generated class, whitespace-normalised; a stale generation fails naming the command that fixes it. Deliberately not every comment in a module: prose that belongs to the manifest alone, like the docs on SpacePreference's sentinel constants, stays there on purpose. Also formats the authored manifest modules to the repo's prettier shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…, classes held to them Completes the contract the flip implied. Until now the manifest was the source of the schemas but the *types* consumers rely on still rode the AD4M classes — the one implementation doubled as the definition. generate:classes now also emits src/manifest/types.ts: one neutral interface per entity over a minimal hand-written ModelInstance base (id, author, createdAt/updatedAt as unknown — their representation is a backend's choice — save and delete; mutation is assign-then-save, so there is deliberately no update). Fields only, also deliberately: relation accessor methods and query sugar are backend ergonomics, not the contract. conformance.ts holds the generated AD4M classes to those interfaces with type-level assertions, reached from the manifest entry point so the dts build typechecks them — a class drifting from its interface fails the build rather than waiting to be noticed. The guard caught a real contract error on its first run: a base `update` method the classes never had. A new backend now has a typed target: implement the interfaces the way backend-inmemory runtime-compiles its own, register under the same names, and consumers never notice. Relocating the AD4M classes out of @we/models was considered and deliberately deferred — the block layer imports /classes, the registry would cycle with backend-ad4m, and proxy typing would need ad4m's typed include projections rebuilt; that is a PR of its own, recorded in the summary doc. Emitter output is lint-clean as generated, so a regeneration never dirties the tree. Package map and CONVENTIONS updated; @we/models is no longer described as "AD4M-decorated" but as what it now is: a neutral manifest with the AD4M lane as one generated artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ies off the classes
The type-level completion of the flip. @we/backend-shared gains modelContract.ts
— the companion to the query IR it has always carried: ModelInstance (the
instance base every record satisfies), ModelStatic<T> (the static CRUD surface,
update and delete statics included), and the typed-query generics that make
findAll(p, { include: { $likeCount: … } }) return rows whose $likeCount is a
number. The generics mirror the AD4M ORM's proven shapes — they were structural
all along — keyed off the neutral interfaces instead of any backend's metadata.
Where the contract is looser, that is the neutrality: dataset handles are
unknown, write values tolerate backend representations (a file payload into a
storage field, an explicit updatedAt stamp), timestamps commit to no format.
The generated neutral interfaces gain the accessor trios consumers actually
call — addParticipants and friends on WeNodeModel, per-relation add/remove/set
where methodRelations declares them, setX for to-ones — because accessors are
contract where stores use them. The extraction had missed WeNode's own
HasManyMethods declaration; the compiler found it within minutes of the proxies
being retyped.
And the proxies ARE retyped: export type Space = SpaceModel, defineEntity as
ModelStatic<SpaceModel>. The @coasys classes are no longer the public type
surface of @we/models. Held to the zero-regression bar — full build plus
explicit tsc on the heavy consumers, zero errors — and the migration surfaced
two pre-existing lies that `any` had been hiding: myMentions.createdAt declared
string while hydration returns epoch numbers, and block serialization typing
its AD4M-lane work against the neutral stand-in. Both fixed at the source.
One limit, recorded rather than papered over: static conformance cannot be
asserted structurally, because AD4M's statics are this-polymorphic generics and
a detached method carrying that constraint satisfies no interface member. The
instance side is asserted; the static side is guaranteed at runtime, where the
proxy binds this at call time and the suite exercises the statics through the
same proxies production uses.
This also fells two of relocation's three blockers (no more dependency cycle,
no more typed-query loss). The last — block persistence behind a port — is the
genuine prerequisite, recorded in the PR doc.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…the contract The inmemory backend's compiled entities now declare the neutral surface: instances typed as ModelInstance plus the open remainder (truthfully — every row is stamped with id/author/createdAt/updatedAt and hydration attaches save/delete), findAll/findOne carrying the same generic include-projection typing the contract declares, and a type-level assertion pinning EntityClassLike to ModelStatic<ModelInstance>. This is the assertion the AD4M lane cannot make structurally (its statics are this-polymorphic), so the inmemory backend is where the contract is compiler-verified end to end — a contract with one checked implementation is a description; with two it is a constraint. It is also the test double for the whole application, so this is where typing buys the most test fidelity per line. Making the assertion pass surfaced a real hole, not a typing nit: the backend had no static delete or count. Stores call MutedAgent.delete(dataset, id) — which resolved to nothing here and threw at runtime in any preview or test that exercised muting. Both implemented; delete of a missing id is a no-op, matching the instance method. The contract's update return widens to T | null — an update is a statement about a record that must exist, the inmemory lane answers null for a missing id, and the AD4M lane's Promise<T> narrows into it. No caller uses the return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The declare-const pair was half runtime: the declaration erases but the assignment survives type stripping, so every test importing entities.ts threw ReferenceError on a value that never existed. A Satisfies type alias asserts the same thing and is erased everywhere; the dts build still checks it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The documented layering said block-system was agnostic; its persistence engine said otherwise — serialization.ts read decorator metadata, called AD4M's transaction static, wrote files through perspective.createExpression, and classified blocks with an isSubjectInstance ASK, with @coasys declared in two package.jsons. Every one of those now has a neutral answer, because the manifest exists: - Which fields an entity declares, and which hold file content, comes from CORE_MANIFEST — registrations carry the entity name the facts are looked up under. This is what frees the registry to hold the entity PROXIES rather than the AD4M classes: the old identity test pinned "must be real classes, stand-ins resolve no metadata"; with metadata out of class identity, the stand-ins resolve everything, and the test now pins the new load-bearing pair instead (registration ↔ manifest entity ↔ file-field markers). - Transactions run through runModelTransaction, a runner the backend adapter registers beside its models (AD4M registers Ad4mModel.transaction; the default is a passthrough — writes land individually, atomicity is absent, which is the honest behaviour of a backend without batching). - File content goes through a registered ModelFileStore. AD4M stores through its file-storage language; the inmemory backend keeps content inline as data URIs, which the resolve pass recognises as already-renderable — previews get working image blocks for the first time. - Block classification is a class-scoped findOne per registered model: a find by id under class X answers null when the record is not an X, so the first non-null answer is the classification and the instance in one call — replacing an AD4M-specific ASK plus a second fetch. The 39 pipeline tests now run against fakes of the neutral surface with no @coasys mock at all, and @we/block-shared and @we/block-solid drop the dependency from their manifests. Types follow: dataset handles are opaque, instances are the contract base plus each type's declared fields. The reconcileBlocks seam cast in SpaceStore is gone — the neutral store now hands its neutral instance to a neutral pipeline. This was the block persistence port the PR doc named as relocation's true prerequisite — it turned out to be a registry, not a port: the capabilities (transaction, file store) register beside the models, and the pipeline speaks only the model contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s is purely neutral The final structural step. The generated AD4M classes, WeNode, and the class emitter move into @we/backend-ad4m (src/models, scripts/generateClasses.mjs) — beside the adapter that registers them, which with the block layer neutral is the only code left with a reason to hold a real class. @we/models keeps what is neutral: the authored manifest, the generated type contract (a new generate:types emits src/manifest/types.ts, union aliases like SignalMode included), the entity proxies, the capability registry, and the utils — and drops @coasys/ad4m from its dependencies entirely. The package map's row is now **Agnostic** with nothing to footnote, and the dependency graph enforces it. What had ridden on the classes rehomes: the SpacePreference sentinels and union aliases export from the manifest that declares them; DatasetProxy stops aliasing PerspectiveProxy and becomes what it always was to its consumers — an opaque handle passed along, never opened (no member access existed anywhere); the Ad4mModel type re-export dies and its one consumer types against ModelInstance; the registry's ModelClass loosens to a structural handle, with the AD4M lane narrowing at its own compiler boundary; signalAggregate types against the neutral interfaces. Conformance moves with the classes — asserted from the adapter's side now, pulled into every build graph through the models barrel — and the equivalence and prose-staleness suites follow them (204 backend-ad4m tests). The model documentation in CLAUDE.md now derives from the manifest itself rather than from ts-morph over class sources — what the docs say a model is, is what the manifest declares, definitionally; a backend that never generates classes documents identically. One visible improvement: relation targets the old parser missed (HasOne → LocationBlock) now appear. Emitted output lints clean as generated; both generators are stable under regeneration. 2268 tests, zero type errors monorepo-wide. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The store surface changed under the rebase — the base branch's final touch-ups reworked the generation flow (rows now stay collapsed after generating; canAutoGenerateFields is gone) — and the generated context follows the rebased source rather than either branch's snapshot of it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for coasys-we ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
…ct shape ModelClass in @we/models is now a constructor plus an index signature — the registry stores what a backend hands it, and what that class satisfies is asserted where it is built. An index signature does not satisfy a *declared* member, so this file's minimal structural ModelClass (whose whole content is a declared `create`) stopped accepting the real getModel: we-preview's connector failed to typecheck, the one host that passes it in. The dep now takes what the registry actually returns and narrows once inside applyFixture, keeping the internal typing (ModelInstance, .id, addChildren) and leaving every host cast-free — which is what the comment this replaces was protecting when Ad4mModel, with its real statics, was the registry's type. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Manifest as source: one authored definition per model, everything else derived and verified
Summary
Until now every core model existed twice: a hand-written AD4M-decorated class, and the neutral
CORE_MANIFESTgenerated from it for the tooling that consumes models as data — the modelwizard, AI shape generation, interpretation hints, space-defined entities. Two representations of
one vocabulary, held in agreement by a test, with the generator's own comment admitting the
intended end-state ("the manifest is meant to become the source of truth"). This PR is that
end-state: each core entity is authored once, as a manifest module under
packages/models/src/manifest/— schema, defaults, closed vocabularies, interpretation hints andthe design prose together — and everything else is derived from it: the decorated classes, the
TypeScript contract, and the model documentation in CLAUDE.md.
What the single source buys, concretely:
nowhere — it silently writes data where nothing looks for it. The manifest is now compiled by
two independent code paths (the class generator, and the runtime
compileManifestthe wizarduses) and the equivalence suite compares their output exhaustively — predicates, cardinality,
storage behaviour, hints, identity flags, per-instance defaults — so a bug in either path fails
loudly instead of drifting. A prose-staleness suite does the same for the doc comments.
manifest, and conformance assertions hold the implementations to them at build time. This paid
for itself before the PR was finished: it caught a wrong contract assumption (an instance
update()that never existed), a missingHasManyMethodsdeclaration the extraction haddropped, two pre-existing type lies that
anyhad been hiding (myMentions.createdAtdeclaredstring, actually epoch numbers; block serialization typed against the wrong lane), and —the best catch — two missing statics on the test backend that made
MutedAgent.delete()throw at runtime in any preview exercising muting.
manifests; the core vocabulary now speaks the same IR, compiled by the same machinery, typed by
the same query generics. One format to maintain, one set of invariants to test.
place, beside the schema they describe, instead of being scattered through decorator options
across 28 class files.
rather than from ts-morph parsing of class sources — what the docs say a model is, is what the
manifest declares, definitionally. The switch immediately fixed an omission the old parser had
(relation targets:
location: HasOne → LocationBlock).The flip was bootstrapped mechanically, not transcribed: the authored modules were extracted from
the generated manifest (data) and the hand-written classes (prose, optionality, bases, union
aliases), then verified identical to the old
CORE_MANIFEST— sorted-key deep equality plusthe declaration order of every property map — before anything consumed them.
Changes
packages/models/src/manifest/(new) — one authored module per entity: the schema(
EntitySchema), the design prose (interpretation-hint essays included — they are edited herenow), and codegen facts (
CoreEntityDefindefs.ts): base class, TS-optional fields, unionaliases like
SignalModewith their values, accessor-method interfaces, typed relation arrays.shared.tsdeclares WeNode's five shared relations once;index.tsassemblesCORE_MANIFEST,merging them into every WeNode-based entity exactly as the class hierarchy does.
@we/backend-ad4m/src/models/, rebuilt withpnpm --filter @we/backend-ad4m generate:classes.Same class names, same behaviour; doc comments are lifted from the manifest modules into the
classes so IDE hovers keep working, and the emitter's output is lint-clean as generated, so a
regeneration never dirties the tree. The only semantic-adjacent textual change is the removal of
redundant
initial:decorator options (SHACL derives initial values from field initialisers;the equivalence suite proves the schemas identical without them). The conformance assertions
live beside the classes and are pulled into the build's type graph through the models barrel,
so a class drifting from its interface fails the build.
packages/modelsreorganises around the manifest — it now holds the authored modules, thegenerated type contract (
generate:types→src/manifest/types.ts), the entity proxies(typed by that contract), the registries backends plug into, and the utils. Import path
@we/models/generated/coreManifestbecomes@we/models/manifest. Things that rode on theclass files rehomed with their meaning intact: the
SpacePreferencesentinels and union aliasesexport from the manifest that declares them;
DatasetProxybecomes the opaque handle it alwayswas to its consumers (no member access existed anywhere).
@we/backend-shared/modelContract.ts(new) — the type-level companion to the query IR thatpackage has always carried:
ModelInstance,ModelStatic<T>(the static CRUD surface,update/deletestatics included), and the typed-query generics (TypedWhere,TypedIncludeMap,IncludeExtras, …) that makefindAll(p, { include: { $likeCount: … } })return rows whose$likeCountis anumber.The generics mirror the AD4M ORM's proven shapes, keyed off the manifest-generated interfaces —
so dynamic entities and the test backend get the same typed query surface the hand-written
classes had. Where the contract is looser (dataset handles opaque, write values tolerant of
storage representations, timestamps uncommitted), that looseness is deliberate and documented.
@we/backend-inmemoryis the double thewhole application runs on in tests and we-preview; its compiled entities now declare the same
contract surface production code is typed against, pinned by a type-level assertion. Making the
assertion pass exposed the missing
delete/countstatics above — exactly the class of gap atest double should not be allowed to have. Its file store keeps content inline as data URIs,
which means previews get working image blocks for the first time.
fields exist and which hold file content comes from
CORE_MANIFEST(registrations carry theirentity name); transactions run through
runModelTransactionand file content through aregistered
ModelFileStore— capabilities the adapter registers beside its models; blockclassification became a class-scoped
findOnethat classifies and fetches in one call. Thisretired a whole failure mode the old identity test existed to pin ("stand-in resolves no
metadata → blocks persist empty → posts come back blank"): metadata is data now, so it cannot
be lost to class identity. The 39 pipeline tests run against plain fakes of the model surface —
no
@coasysmock — and the layering the package map documents for the block layer is nowenforced by its actual dependencies. The
reconcileBlockscall in SpaceStore loses its cast.coreManifest.test.tsreverses its meaning: it now holds the generated classes and themanifest's runtime compilation in exhaustive agreement (two independent compilers, so one bug
cannot hide in both). Staleness fails with the command that fixes it.
generatedClasses.test.ts(new) — prose staleness: every doc block the codegen lifts mustappear in the generated class, whitespace-normalised.
src/manifest/, rungenerate:types+generate:classes.Known follow-ups
statics are
this-polymorphic generics, and a detached method carrying that constraintsatisfies no interface member. A small upstream typing change in ad4m (a declared static-surface
interface) would enable it for every AD4M consumer; meanwhile the test backend's assertion plus
the runtime suites carry the guarantee.
WeNode's hand-written interface declares a'reactions'accessor for a relation that does notexist on the class — a pre-existing phantom, left untouched here, worth deleting separately.
origin,role,preference,valueType) became doccomments; promoting them to manifest
options(and generated unions) is a possible later step,pending confirmation that
optionsstays out of the compiled SHACL.devafter the model-authoring PR landed (nine commits, one generated-fileconflict taken from dev and regenerated; the branch's interim formatting commit dropped as
redundant with dev's own lint fixes). The base branch's final touch-ups — the reworked
generation flow among them — are reflected in the regenerated context.
Test plan
CORE_MANIFEST(sortedkeys + property declaration order) before the flip.
coreManifest.test.ts— all 29 entities: schema summaries (predicates, cardinality, storage,transforms, hints, identity, flags) and per-instance field defaults identical between the
generated classes and the manifest's runtime compilation.
tsc --noEmiton app-shell, block-shared, block-solid, editor, backend-ad4m,backend-inmemory, models), build clean,
we-validate-schemasclean,pnpm lintcleanincluding regenerated output (both generators are lint-stable), ai-context regenerated from
the manifest.
equivalence, prose-staleness and shape-staleness suites at their new home.
with an image, edit it, delete it), since that pipeline changed transport underneath.