diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 696decd..e07299b 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,60 +1,72 @@ # Architecture -This document describes the internal design of `@laboverwire/stitch` — how layers compose, how data flows, and the invariants each layer depends on. If you're adding a feature or tracking down a bug that crosses layer boundaries, start here. +This document describes the internal design of `@laboverwire/stitch` — what this +package is, where the seam between it and the WASM engine sits, and the few +responsibilities the TypeScript layer actually owns. -For the public API surface, see [README.md](./README.md). For a history of changes and 0.1 → 0.2 migration notes, see [CHANGELOG.md](./CHANGELOG.md). +For the public API surface, see [README.md](./README.md) and +[docs/api.md](./docs/api.md). For a history of changes, see +[CHANGELOG.md](./CHANGELOG.md). --- ## What the library is -Stitch is a reactive state-sync library for browser apps that need: +As of 0.5.0, `@laboverwire/stitch` is a **thin, framework-agnostic binding +layer** over [`@laboverwire/stitch-wasm`](https://www.npmjs.com/package/@laboverwire/stitch-wasm) +(`^0.2.1`) — a Rust/WASM package compiled from the sibling `stitch-rs` repo. The +WASM package owns everything that used to live here: -1. **Synchronous reads for the UI** — the render loop can call `store.read(...)` / `store.getSnapshot(...)` and get data back without awaiting. -2. **Durable local state** — IndexedDB survives reloads, offline sessions, and reconnect storms. -3. **Live multi-device sync** — changes propagate over MQTT and reconcile with other clients. -4. **Offline tolerance** — mutations queue locally and drain when connected. +1. **Synchronous reads for the UI** — an in-memory store services `read` / + `getSnapshot` without awaiting. +2. **Durable local state** — IndexedDB persistence (optionally AES-GCM encrypted). +3. **Live multi-device sync** — MQTT v5 over WebSocket, with reconcile. +4. **Offline tolerance** — a durable offline queue that drains on reconnect. -The central trick is that "source of truth for the UI" and "source of truth for durability" are different concerns, handled by different WASM databases. A synchronous in-memory `mqdb-wasm` DB services reads; an async `mqdb-wasm` IndexedDB-backed DB owns persistence; the MQTT layer is purely about moving mutations between clients. +This package contributes two things and nothing else: + +- a small TypeScript adapter (`src/store.ts`) that wraps the WASM `Store` behind + a stable `Store` interface, normalizes a few types, and tolerates access + before `initialize()` resolves; +- React (`@laboverwire/stitch/react`) and Vue (`@laboverwire/stitch/vue`) + bindings that subscribe to scoped entities. + +**There is no store logic in this repo.** No memory store, persistence layer, +sync engine, remote-sync layer, or offline queue — those files were removed in +0.5.0 and their behaviour now lives in `stitch-wasm`. If a bug concerns +persistence, MQTT sync, the offline queue, reconciliation, corruption recovery, +or topic parsing, it is in `stitch-rs`, not here. --- -## Layer stack +## The seam ```mermaid flowchart TB - subgraph UI["UI layer"] + subgraph UI["Binding layer (this package)"] direction LR - React["React hooks
src/react/"] - Vue["Vue composables
src/vue/"] + React["React hooks + StoreProvider/AuthProvider
src/react/"] + Vue["Vue composables + StoreRoot/StitchAuth
src/vue/"] end - Store["StoreImpl — public Store facade
src/store.ts"] - - subgraph Internals["Internal layers"] - direction LR - Memory["MemoryStore
in-memory mqdb-wasm · sync reads
src/memory-store.ts"] - Persistence["PersistenceLayer
mqdb-wasm + IndexedDB · async
src/persistence-layer.ts"] - Queue["OfflineQueue
pending_sync · consolidation
src/offline-queue.ts"] - Remote["RemoteSyncLayer
scope/CRUD orchestration
src/remote-sync-layer.ts"] - end + Adapter["StitchStore — Store<S> adapter
src/store.ts"] + MemoryView["MemoryView — snapshot cache
src/store.ts"] - Engine["SyncEngine
mqtt5-wasm · request/response
src/sync-engine.ts"] + Wasm[("@laboverwire/stitch-wasm
in-memory store · IndexedDB · MQTT · offline queue")] - UI --> Store - Store --> Memory - Store --> Persistence - Store --> Queue - Store --> Remote - Remote --> Engine + UI --> Adapter + Adapter --> MemoryView + Adapter --> Wasm + MemoryView --> Wasm ``` -Two independent mqdb-wasm `Database` instances live inside a store: - -- The **memory DB** (`MemoryStore._db`) holds only the currently active scope. Synchronous API (`createSync`, `listSync`, `readSync`). Rebuilt on every `replaceScope` call. -- The **persistence DB** (`PersistenceLayer.db`) holds everything ever stored locally. Async API. Survives page reloads via IndexedDB. - -`persistence-store.ts` and `persistence-bridge.ts` are the pre-0.2 monolith and are `@deprecated`. New code should not reach for them. +`createStore(config, options)` — the only value export from the package root — +calls the WASM `createStore(config, options)` and wraps the result in +`StitchStore`. Everything else the root exports is a **type**: `StoreConfig`, +`EntityDefinition`, `SchemaField`, `ForeignKeyDefinition`, `ConnectionStatus`, +`SortField`, `SortDirection`, `ListFilter`, `Store`, `StoreOptions`, +`PersistenceConfig`, `RemoteConfig`, `MemoryStore`, `EntitySchema`, +`DefaultSchema`, `EntityKey`, `OriginTag`. --- @@ -62,412 +74,163 @@ Two independent mqdb-wasm `Database` instances live inside a store: | File | Role | |---|---| -| `types.ts` | All public type exports. `Store`, `StoreConfig`, `StoreOptions`, `EntitySchema`, plus the internal `MemoryStore` / `PersistenceLayer` / `RemoteSyncLayer` / `OfflineQueue` / `SyncEngine` interfaces. | -| `store.ts` | `StoreImpl` — the public facade. Composes all layers. Owns the init promise cache, scope-lifecycle routing, and CRUD fan-out. `createStore()` returns this. | -| `memory-store.ts` | `MemoryStoreImpl` — in-memory WASM DB, sync reads, scope-indexed subscriptions, batch-notify support. Holds the `_Database` constructor reference so `loadScope` can swap instances. | -| `persistence-layer.ts` | `PersistenceLayerImpl` — IndexedDB-backed WASM DB with a serialized op queue, corruption recovery, and an entity-subscription bus that bridges WASM events into plain JS callbacks. | -| `remote-sync-layer.ts` | `RemoteSyncLayerImpl` — translates scope/CRUD operations into MQTT topic requests and routes inbound mutations back to the store. Owns reconcile + initial-sync logic. | -| `sync-engine.ts` | Low-level MQTT5 client wrapper. Topic subscription, request-response with correlation IDs, enhanced auth flow for JWT tickets, backoff strategy. Framework-agnostic — takes the WASM module as an argument. | -| `offline-queue.ts` | Two implementations: `createPersistentOfflineQueue` (writes to `pending_sync`) and `createInMemoryOfflineQueue`. Consolidation logic: collapses insert+updates, insert+delete, stacked updates before flushing. | -| `internal-utils.ts` | `stripNulls`, `isTransientSyncError`. Canonical utilities — an older `stripNulls` copy still exists in `memory-store.ts`. | -| `internal-wasm-error.ts` | `MqdbError` and `wrapWasmError` — used at every WASM call site. | -| `react/` | React bindings. `context.ts` + `provider.tsx` + `hooks/*`. | -| `vue/` | Vue 3 bindings. `injection-key.ts` + `StoreRoot.ts` + `StitchAuth.ts` + `composables/*`. | +| `src/types.ts` | Public type exports — `Store`, `StoreConfig`, `StoreOptions`, the schema-generic helpers, and the trimmed `MemoryStore` view interface. | +| `src/store.ts` | `StitchStore` (the `Store` adapter) and `MemoryView` (the snapshot cache). `createStore()` lives here. | +| `src/index.ts` | Package root: re-exports `createStore` and the public types. | +| `src/internal-list-apply.ts` | `applyEvent` — the pure list-diff helper the hooks use to fold subscription events into a rendered list. | +| `src/react/` | React bindings — `context.ts`, `provider.tsx`, `hooks/*`. | +| `src/vue/` | Vue 3 bindings — `injection-key.ts`, `StoreRoot.ts`, `StitchAuth.ts`, `composables/*`. | --- -## Core abstractions - -### Scope - -A **scope** is a single instance of the root entity. The root entity's `id` **is** the `scopeId`. - -- The `StoreConfig.scope` block declares which entity is the root (`rootEntity`), which entities belong under it (`childEntities`), and which field on the children points at the root (`scopeField`). -- `replaceScope(scopeId)` loads that scope's bundle from persistence (and the server, if connected), populates the memory DB, and starts streaming live mutations for it. Any previously-active scope is replaced wholesale. -- `closeScope(scopeId)` unsubscribes the MQTT topics for that scope and clears the in-memory data. - -Only one scope is live at a time in the memory DB. This keeps the memory WASM instance small and the sync boundaries predictable, at the cost of requiring a scope switch to rebuild in-memory state. - -### Origin tag - -Every mutation carries an `originTag: string | null` that controls which layers act on it. The tag is set on `MemoryStoreImpl.originTag` before a sync-WASM call and read back by the subscription callback that fires. - -| Tag | Meaning | -|---|---| -| `null` / `undefined` | Local user mutation — propagate to persistence and remote. | -| `'remote'` | Inbound from MQTT — write to memory, skip persistence (already written). Prevents echo loops. | -| `'load'` | From `replaceScope` / `loadScope` — populate memory silently, don't persist. | -| `'clear'` | Scope teardown — don't fire CRUD side effects. | - -The `persistence-bridge` and `PersistenceLayer` subscription callbacks check `originTag` and short-circuit when appropriate to prevent write amplification. - -### Entity schema types +## What the adapter owns -`types.ts` exports `EntitySchema = Record` and `EntityKey`. `createStore(...)` returns a `Store` whose `read` / `getSnapshot` / `create` / `update` / `delete` / `subscribeToEntity` / `subscribeToScope` / `list` are all typed via `S[EntityKey]`. +`StitchStore` forwards nearly every call straight to the WASM store. It adds +four responsibilities, and only these: -`listRootEntities` stays untyped (`Record[]`) because the root-entity type name isn't statically inferable from `StoreConfig`; callers typically cast. This is an intentional simplification — encoding the root entity as a separate generic parameter would require `Store` everywhere and pollute every hook signature. +### 1. Pre-init tolerance ---- +The WASM store requires `initialize()` before use and throws otherwise. The +adapter never lets that throw reach a caller, so React/Vue hooks are safe to +mount before the provider's `initialize()` resolves: -## Data flow +- **Synchronous reads return empties** before init: `read` → `null`, + `getSnapshot` → `[]`, `getSnapshotAsMap` → `{}`, `getChildCount` / `getVersion` + → `0`, `connectionStatus` → `'offline'`, `isReconnecting` → `false`. +- **Async methods await readiness** — `#afterReady(fn)` runs `fn` immediately if + ready, otherwise chains it onto an internal `#readyPromise` that + `initialize()` resolves. +- **`subscribe*` defer wiring** via `deferrableSubscribe(isReady, whenReady, + subscribeNow, poke)`: if the store is ready it subscribes immediately; + otherwise it waits on the ready promise, subscribes on resolution, and calls + `poke` so `useSyncExternalStore`-style consumers re-read the now-available + state. The returned unsubscribe cancels either the pending wire or the live + subscription. -### Local mutation (create / update / delete) +### 2. Status normalization -```mermaid -sequenceDiagram - autonumber - participant UI as UI - participant Store as StoreImpl - participant Memory as MemoryStore - participant Persistence as PersistenceLayer - participant Queue as OfflineQueue - participant Remote as RemoteSyncLayer - participant MQTT as MQTT broker - - UI->>Store: create(task, projectId, data) - Store->>Memory: createSync(entity, record) - Memory-->>UI: subscribers notified (sync re-render) - Store->>Persistence: create(entity, record) - Note over Persistence: serialized through _opQueue - Store->>Queue: queue(op=insert, entity, id, scopeId, data) - - alt connection is connected - Store->>Remote: syncCreate(entity, scopeId, data) - Remote->>MQTT: publish $DB/task/create - MQTT-->>Remote: ack - Remote-->>Store: ok - Store->>Queue: remove(entity, id, scopeId, insert) - else offline or disconnected - Queue->>Queue: stays queued, drained on next connect - end -``` +`normalizeStatus` maps the WASM store's PascalCase status (`Connected`, +`Connecting`, `Disconnected`, `Error`) onto the lowercase `ConnectionStatus` +union, defaulting anything else to `'offline'`. Consumers always see lowercase. -Error branches on the outbound sync: -- `OwnershipError` (403) — remove from queue, swallow (never retried). -- Transient (timeout / disconnected / FK violation) — leave queued, retry on next flush. -- `"not found"` on update — upsert: treat as create on remote. +### 3. Reconnect signature preservation -UI re-renders happen off the memory-store subscription. Persistence writes are non-blocking for the UI. +`reconnect(serverUrl, getTicket?)` runs any registered reconnect validator, +resolves the ticket via `getTicket`, and forwards to +`inner.reconnect(serverUrl, ticket)`. The WASM API takes a resolved ticket +string; the adapter keeps the `() => Promise` shape the bindings expect. -### Remote mutation (inbound MQTT) +### 4. Capability flags -```mermaid -sequenceDiagram - autonumber - participant MQTT as MQTT broker - participant Engine as SyncEngine - participant Remote as RemoteSyncLayer - participant Store as StoreImpl - participant Persistence as PersistenceLayer - participant Memory as MemoryStore - participant UI as UI - - MQTT->>Engine: inbound publish on $DB/task/.../events/updated - Note right of Engine: filter by x-origin-client-id
drop own echoes - Engine->>Remote: mutation event - Remote->>Store: handleRemoteMutation(mutation) - Store->>Remote: applyMutationToDb(mutation, localAccessor) - Remote->>Persistence: localAccessor.create / update / delete - Persistence-->>Store: persistence.subscribe callback fires - Store->>Memory: write with tag='remote' (setupPersistenceSubscriptions) - Memory-->>UI: subscribers notified (re-render) -``` +`hasPersistence` / `hasRemote` are derived once in the constructor from whether +`options.persistence` / `options.remote` were provided, so they are readable +before init without touching the WASM store. -Round trip: inbound MQTT → persistence write → persistence WASM event → `StoreImpl.setupPersistenceSubscriptions` relays into memory → UI. The `'remote'` origin tag prevents the memory-side write from looping back out through the offline queue. +### MemoryView — the snapshot cache -### Scope replacement - -```mermaid -flowchart TB - Start(["store.replaceScope(scopeId)"]) --> SameCheck{{"currentScopeId
=== scopeId?"}} - SameCheck -- yes --> NoOp(["return"]) - SameCheck -- no --> Switch["closeScope(previous) in background
currentScopeId = scopeId"] - - Switch --> RemoteCheck{{"remote
connected?"}} - - RemoteCheck -- yes --> Suppress["persistence.suppressNotifications = true"] - Suppress --> OpenScope["remote.openScope(scopeId)
returns ScopeState: root, children, bufferedMutations"] - OpenScope --> UpsertRoot["localAccessor.create/update root
in persistence"] - UpsertRoot --> ReconcileChildren["for each childEntity:
remote.reconcileChildren(…)"] - ReconcileChildren --> ApplyBuffered["apply bufferedMutations
arrived between subscribe + fetch"] - ApplyBuffered --> LoadBundle["bundle = loadScopeFromPersistence(scopeId)"] - LoadBundle --> MemoryLoad["memory.loadScope(scopeId, bundle, 'load')
fresh DB · createSync each · swap · notify"] - MemoryLoad --> LoadRoot["loadRootIntoMemory(rootEntity, scopeId)"] - LoadRoot --> Unsuppress["persistence.suppressNotifications = false"] - Unsuppress --> Done([ready]) - - RemoteCheck -- no --> LoadBundleOffline["bundle = loadScopeFromPersistence(scopeId)"] - LoadBundleOffline --> MemoryLoadOffline["memory.loadScope(scopeId, bundle, 'load')"] - MemoryLoadOffline --> LoadRootOffline["loadRootIntoMemory(rootEntity, scopeId)"] - LoadRootOffline --> Done -``` - -`memory.loadScope` is destructive by design — it constructs a fresh `new Database()`, registers schemas, `createSync`s each loaded record, then swaps `_db`. Old subscribers are still attached to the new DB via `setupSubscriptions()`. After the swap, each entity's subscribers are explicitly notified so consumers re-read their snapshots. +`MemoryView` implements the trimmed `MemoryStore` interface (`getSnapshot`, +`getSnapshotAsMap`, `subscribeToScope`). It keeps a per-`(scopeId, entity)` cache +keyed on the WASM store's reactivity token: `getVersion(scopeId, entity)` returns +a numeric version, and a cached snapshot is returned verbatim until that version +changes. This referential stability is what lets `useSyncExternalStore` avoid +re-render loops. Before the store is `ready()`, snapshots return the shared +`EMPTY_ARRAY` / `EMPTY_MAP` constants. --- -## Subscription machinery - -Three layers publish events; the framework hooks consume from the memory layer. - -```mermaid -flowchart LR - WasmMem[("Memory WASM DB
mqdb-wasm in-memory")] -- WASM subscribe --> MemHandle["MemoryStore.handleChangeEvent"] - MemHandle --> MemScope["subscribeToScope callbacks
per (scopeId, entity)"] - MemHandle --> MemGlobal["subscribeToEntity
memory-only stores"] - MemHandle --> MemMut["onMutation (internal)"] - - WasmPersist[("Persistence WASM DB
mqdb-wasm + IndexedDB")] -- WASM subscribe --> PersistEntity["PersistenceLayer.entitySubscriptions"] - PersistEntity --> Bridge["persistence-bridge
writes to memory w/ tag='remote'"] - PersistEntity --> StoreSub["Store.subscribeToEntity
persistence-backed stores"] - - MemScope --> Hooks["useEntitySnapshot
useEntitySnapshotAsMap
subscribeToScope"] - MemGlobal --> Hooks - StoreSub --> HookList["useRootEntityList
useScopedEntities
useChildCounts
useTopLevelEntities"] -``` - -Key contract: `Store.subscribeToEntity(entity, cb)` bridges both sources — when persistence is configured, `persistence.subscribe` carries normal create/update/delete events while `memory.onMutation` is kept attached but filtered to deliver only `'load'` and `'clear'` tags (which bypass the persistence bridge). Without persistence it falls back to `memory.onMutation` alone. The early-subscriber migration in `initialize()` rebinds pre-init subscribers onto persistence once it opens while keeping the memory hook alive so `replaceScope` loads still fire. - -### Memory-store subscriptions - -`MemoryStore.setupSubscriptions()` iterates `this.allEntities` (which includes the root plus child + top-level entities) and attaches a single WASM subscription per entity: - -``` -db.subscribe('#', entity, event => handleChangeEvent(entity, event)) -``` - -`handleChangeEvent` derives the `scopeId` (from `data[scopeField]` for children, from `event.id` for the root), bumps a per-(scope, entity) version counter, and calls: - -- `notifySubscribers(scopeId, entity)` → per-scope subscribers (`subscribeToScope`) and global subscribers (`subscribeToEntity`) -- `emitMutation(event)` → raw `onMutation` listeners (internal; `Store.onMutation` is not public) - -Batching: `beginBatch()`/`endBatch()` defer notification. The batched set is keyed `scopeId\0entity` and flushed in `endBatch`. `loadScope` notifies synchronously without using the batch because the batch would be empty (batched entries are populated only by the WASM subscription callback). - -### Persistence-layer subscriptions - -`PersistenceLayer.subscribe(entity, cb)` attaches to `entitySubscriptions`, a plain JS `Map>`. One WASM subscription per entity (in `setupWasmSubscriptions`) fan-outs to all registered JS callbacks. - -`notifyAllEntitySubscribers()` is called at the end of `store.initialize` and again at the end of `onConnected` after the initial remote sync resolves. It fires `(data: null, op: 'update')` to every entity's callbacks so late-joined subscribers can hydrate and hooks relying on `listRootEntities` (which returns `[]` until `initialSyncDone`) refresh once the broker's state has landed. Consumers interpret `data === null` as "bulk refresh, re-fetch". - -`setSuppressNotifications(true)` silences the WASM → JS fan-out during `replaceScope`'s reconcile window. Any events during reconcile are swallowed; the explicit `memory.loadScope` notification at the end is the source of truth. - -### `Store.subscribeToEntity` — the unified surface - -```ts -store.subscribeToEntity(entity, (data: Record | null, op) => { ... }) -``` - -- When persistence is configured, attaches to both `persistence.subscribe(entity, cb)` (carries every persisted mutation) and `memory.onMutation` (filtered to `'load'` and `'clear'` origin tags — the two tags that bypass persistence). This preserves `replaceScope` coverage without double-firing for normal local or remote mutations. -- When no persistence, listens on `memory.onMutation` alone and filters by entity name. -- Before initialize (`_persistence === null`), registers an "early subscriber" (`_earlySubscribers` array) that is migrated onto the persistence layer once it opens; the memory hook remains attached through the migration. - -Data is forwarded with a coerced `null` when the underlying callback signals a bulk-refresh. Ops are `'insert' | 'update' | 'delete'` normalized from whichever source fired. - -### `Store.subscribeToScope` - -Simpler: delegates to `memory.subscribeToScope(scopeId, entity, cb)`. Callback is `() => void` — it only signals "something in this scope's entity set changed"; consumers re-read the snapshot. +## Behavioural contracts the bindings depend on + +- **Event delivery is asynchronous.** `subscribeToEntity` / `subscribeToScope` + callbacks fire one tick after the mutating call resolves, not synchronously. + Hooks and tests must await a tick before asserting on subscription output. The + bindings treat a `subscribeToEntity` callback with `data === null` as a + "bulk-refresh, re-fetch" cue. +- **`getVersion` is an opaque reactivity token**, not a count — it exists only to + invalidate `MemoryView`'s cache. Treat it as monotonic-ish, not as a record + count or a sequential revision. +- **`replaceScope` is destructive inside the WASM store** — it rebuilds the + in-memory scope. Consumers holding references to pre-replace records must + re-read after the promise resolves; the hook layer re-subscribes automatically. +- **Pre-init reads are silent empties, not errors.** A `getSnapshot` returning + `[]` before `initialize()` resolves is expected, not a data-loss bug. --- -## Concurrency & serialization +## Scope model -### `PersistenceLayer._opQueue` +Configured via `StoreConfig.scope`: -Every async DB operation is threaded through `serialized(label, fn, timeoutMs=10000)`: +- `rootEntity` — the top-level entity type; its `id` **is** the `scopeId`. +- `childEntities` — entity types scoped under the root via `scopeField`. +- `scopeField` — the field on children that references the root's `id`. -```ts -this._opQueue = this._opQueue.then(async () => { - await new Promise(r => setTimeout(r, 0)); - if (this._dbNeedsRecovery) await this.recoverDb(); - return Promise.race([fn(), timeoutGuard]); -}); -``` - -This guarantees only one DB op is in flight at a time. The 10s timeout flags the DB for recovery on the next serialized call — `recoverDb()` re-opens the IndexedDB connection, re-registers schemas, and re-attaches WASM subscriptions before the next op runs. - -### `StoreImpl._initPromise` - -`store.initialize()` is idempotent. The first caller kicks off `doInitialize()` and stores the resulting promise on `_initPromise`. Subsequent concurrent callers (React `` double-invoke, nested providers) await the same promise instead of constructing a second persistence layer that would orphan already-migrated subscribers. On success the cache is cleared; on failure the cache is also cleared so a retry can run. +Entity categories: **root** (scoped parent), **child** (scoped via `scopeField`), +**top-level** (`topLevelEntities`, synced globally), **local-only** +(`localOnlyEntities`, never touch MQTT). Scope open/close, reconciliation, +offline-queue consolidation, `_version` LWW conflict resolution, and the MQTT +topic layout are all implemented in `stitch-wasm`; this package only forwards +`replaceScope` / `closeScope` / `loadScope` / `clearScope`. -### Memory-store origin tags - -`originTag` is a class field mutated around a single sync WASM call: - -```ts -this.originTag = tag ?? null; -this.db.createSync(entity, record); // WASM event fires; callback reads this.originTag -this.originTag = null; -``` - -This works because `createSync` is synchronous — the WASM event and its handler run before the next line executes. If the library ever adopts async variants internally, the tag machinery needs to move to an argument-threaded pattern. - ---- - -## Offline queue - -Any local mutation that has a `scopeId` gets queued in `pending_sync` (or in memory, when no persistence is configured). The queue is only drained when the remote layer reports `connected`. - -### Consolidation - -Before flushing, `OfflineQueue.flush()` groups pending rows by (entity, entityId) and collapses them: - -- **Insert + N updates** → single insert with merged fields (last write wins). -- **Insert + delete** → just a delete... actually: dropped entirely (the record never made it to the server, no-op). -- **N updates** → single update with merged fields. -- **Update + delete** → delete only. - -This keeps the wire protocol bounded regardless of how long the client was offline. Order across entities is preserved by `createdAt`, then by op priority (`insert=0, update=1, delete=2`) to respect FK constraints on replay. - -### Double flush on connect - -`StoreImpl.onConnected()` calls `flush(sender)` twice: - -1. First flush replays consolidated mutations. Some will hit "not found" on the server (e.g., updates to records whose insert hasn't been acknowledged yet) and emit upsert compensations. -2. Second flush drains those compensations. - -### Ownership errors - -`OwnershipError` (a 403 from the server — the client doesn't own the record it's trying to mutate) is treated specially: the pending row is removed and the error is **swallowed**, not retried. This prevents infinite loops when authorization changes mid-flight. - ---- - -## Reconciliation - -On `replaceScope` (with remote connected) or on `onConnected`, the local state and server state are compared: - -- **Server record absent locally** → write to persistence via `localAccessor.create`. -- **Server record present locally, same version** → skip. -- **Server record present locally, server version newer** → overwrite. -- **Server record present locally with a pending insert in the queue** → keep local (server will receive it on flush). -- **Local record not on server** → delete locally (unless there's a pending insert). - -Version comparison uses the `versionField` (numeric, monotonic) plus `updatedAtField` (timestamp) as a tiebreak. - -During reconcile, persistence notifications are suppressed so subscribers don't see partial intermediate states. After reconcile completes, `memory.loadScope` fires one explicit notification per entity. - ---- - -## Connection lifecycle - -### State machine - -```mermaid -stateDiagram-v2 - [*] --> offline - offline --> connecting: connect() - connecting --> connected: success - connecting --> offline: auth error (never retried) - connecting --> connecting: network error (backoff) - connected --> connecting: network loss (backoff) - connected --> offline: disconnect() - connected --> offline: auth error
(sessionInvalidHandler fires) -``` - -### Backoff - -`min(1000 * 2^n, 30000)ms` with 25% jitter for 5 attempts, then a fixed 15s interval. Auth errors bypass the state machine entirely — the connection is torn down and `sessionInvalidHandler` fires. The consumer app is expected to re-obtain credentials before calling `store.reconnect(serverUrl, getTicket)`. - -### Visibility-change reconnect - -`` / `` listen to `document.visibilitychange`. If the tab was hidden for more than 30s and the current state is not `connected`, a reconnect is triggered on return. This catches the common case of a laptop waking from sleep with a stale WebSocket. - -### `beforeunload` - -Both providers attach a `beforeunload` listener that calls `store.disconnect()` — this sends a clean MQTT `DISCONNECT` packet so the broker releases the session immediately instead of waiting for a keepalive timeout. +`StoreConfig.responseTopicPrefix` (default `$DB/clients`) is still a config field +and is forwarded to the WASM store, but the response-topic parsing lives in the +WASM, not here. --- -## Corruption & recovery - -`PersistenceLayer.isDbCorrupted(err)` inspects errors (walking through `MqdbError.cause`) and matches: +## Binding layer -- `err.name === 'RuntimeError'` — raw wasm-bindgen panic -- `msg` matches `/transaction.*null|arg0 is null|transaction error|index out of bounds|database is busy|unreachable/i` +Both framework layers consume the same `Store` interface and add no store logic: -When detected, the error path triggers `recoverDb()`: +- **React** (`src/react/`) — `StoreProvider` owns the store lifecycle + (`initialize`, connection-status tracking, visibility-change reconnect, + `beforeunload` disconnect); `AuthProvider` binds `setAuthenticatedUser` / + session-invalid / reconnect-validator handlers and tears down via + `resetForLogout` on logout. Hooks: `useEntitySnapshot`, + `useEntitySnapshotAsMap`, `useScopedEntities`, `useRootEntityList`, + `useTopLevelEntities`, `useChildCounts`, `useConnectionStatus`, `useSyncScope`, + `useStore`. +- **Vue** (`src/vue/`) — `StoreRoot` and `StitchAuth` mirror the two React + providers; the composables mirror the hooks. -1. Set `_dbNeedsRecovery = true`. -2. The next `serialized` call runs `recoverDb()` before the user's op. -3. `recoverDb()` re-opens `Database.openPersistent(this.dbName)`, re-runs `setupSchemas()` (async), and re-attaches WASM subscriptions. -4. The original op is retried once; if it fails again, the error is wrapped in `MqdbError` and thrown. +`useEntitySnapshot` reads through `store.memory` (the `MemoryView`) with +`useSyncExternalStore`; the version-keyed cache is what keeps its snapshot +referentially stable across renders. The list hooks +(`useScopedEntities` / `useRootEntityList` / `useTopLevelEntities` / +`useChildCounts`) fetch via `store.list` / `store.listRootEntities` and fold +subsequent `subscribeToEntity` events in with `applyEvent`. -The memory store has an analogous `_corrupted` flag and `tryRecover()` path. Memory corruption invalidates all cached snapshots (`clearAllCaches()`) and notifies via `onCorruption` so the persistence layer can mirror the recovery. - ---- - -## WASM integration - -### `mqdb-wasm` - -The library uses the same WASM package for both memory and persistence, distinguished by which constructor is called: - -- `new wasmMod.Database()` — in-memory backend. `memory-store.ts` uses this; supports sync methods (`createSync`, `readSync`, `listSync`). -- `await wasmMod.Database.openPersistent(dbName)` — IndexedDB-backed backend. `persistence-layer.ts` uses this; sync methods throw `"sync operations require memory backend"`. Use `addSchemaAsync` / `addForeignKeyAsync` / `addIndexAsync` for DDL. - -Both instances share the same WASM module (one load per page). `memory-store.ts` holds a module-level reference to the init function (`_initWasm`) and the `Database` constructor so `loadScope` can synchronously construct new instances after initial init. - -### `mqtt5-wasm` - -Dynamically imported inside `SyncEngine.connect(serverUrl, wasmModule, getTicket?)`. The engine handles: - -- MQTT5 CONNECT with enhanced authentication (`AUTH` / `RE-AUTH` flow) when `getTicket` is provided — the ticket is the JWT, sent as auth data. -- Request-response correlation via `$DB/clients/{clientId}/{requestId}` subscription (configurable via `responseTopicPrefix`). -- Topic matching against the `$DB/{entity}/…` topic tree. -- `x-origin-client-id` user property on every published message so clients can filter their own echoes. - -The engine is framework-agnostic and receives the WASM module as an argument. This lets tests (and future SSR scenarios) inject a different module without the engine knowing the difference. - -### Error wrapping - -Every `this.db.*` call site in `memory-store.ts` and `persistence-layer.ts` is wrapped so that the raw WASM throw (typically a JS string, occasionally a `RuntimeError`) is coerced into an `MqdbError`: - -```ts -try { - await this.db.list(entity, options); -} catch (err) { - throw wrapWasmError(`list:${entity}`, err); -} -``` - -`MqdbError` exposes `.name = 'MqdbError'`, `.method` (e.g. `'list:project'`), a qualified `.message`, and the original throw on `.cause`. All of stitch's own "is this corrupted?" checks unwrap `.cause` before pattern-matching so wrapping is free of downstream cost. - ---- - -## Invariants - -A checklist of things that must stay true. Breaking any of these usually breaks something subtle elsewhere. - -1. **`memory.allEntities` contains the root entity.** Pre-0.2 it didn't; the result was that `subscribeToEntity('root', …)` never fired for root mutations. Dropping the root from this list must also come with a different dispatch path for the root entity. -2. **`memory.loadScope` explicitly notifies all loaded entities' subscribers.** Without this, consumers that subscribed before `replaceScope` resolved see empty snapshots until some unrelated state change forces a re-read. -3. **`Store.initialize` caches `_initPromise`.** Concurrent callers must share one init. The cache must clear on both success and failure. -4. **`persistence-layer.setupSchemas` uses `*Async` methods.** The sync variants throw on IndexedDB-backed DBs. -5. **The memory and persistence DBs are always opened with the same config.** Any entity or foreign key declared in `StoreConfig.entities` exists in both. `pending_sync` is auto-declared in persistence only; if user code needs it in memory it must be opted-in via `localOnlyEntities`. -6. **`persistence-layer.close()` calls `db.free()`.** Otherwise IndexedDB connections leak and subsequent `deleteDatabase` calls block — a real problem for tests and for `resetForLogout` flows. -7. **Origin-tag propagation.** The `persistence-bridge` callback must check the tag and skip writes when tag ∈ `{'remote', 'load', 'clear'}`. Remove this check and inbound MQTT mutations bounce back out as outbound ones. -8. **`x-origin-client-id` filtering in `SyncEngine`.** The engine must reject inbound messages whose origin matches its own `clientId`. Without this, every local mutation loops through MQTT → persistence → memory twice. -9. **`replaceScope` is destructive.** `memory.loadScope` wipes and rebuilds. Callers holding references to pre-replace records must re-read after the promise resolves. The hook layer handles this by re-subscribing automatically; direct `store.*` callers are responsible. -10. **`Store.subscribeToEntity` early subscribers are migrated on init.** If init order changes such that `_earlySubscribers` migration is skipped, subscriptions silently vanish. +Because the adapter tolerates pre-init access, these hooks are safe to mount +before the provider finishes initializing — a subscription established early +wires up and re-reads once `initialize()` resolves. --- ## Testing -Integration and unit tests run in real Chromium via Playwright + Vitest browser mode. jsdom / happy-dom don't work — the WASM layer requires real `web_sys::window()` and real IndexedDB, and the `WebAssembly.instantiateStreaming` path wants a real `Response` with `application/wasm` MIME type. - -Run: +Tests run in real Chromium via Playwright + Vitest browser mode — jsdom / +happy-dom do not work because the WASM layer needs a real `window`, real +IndexedDB, and a real `application/wasm` response. `vitest.config.ts` loads the +WASM through `vite-plugin-wasm` + `vite-plugin-top-level-await` (the same two +plugins a consuming Vite app needs — see +[docs/vite-consumer.md](./docs/vite-consumer.md)). ```bash npm test # one-shot, real Chromium npm run test:watch # watch mode -npm run test:ui # Vitest UI in a browser tab ``` -Tests are in `tests/integration/` (cross-layer behavior) and `tests/unit/` (isolated primitives like `MqdbError` wrapping or schema-type inference via `expectTypeOf`). Fixtures in `tests/helpers/` provide a canonical `projectTaskConfig()` plus a `uniqueDbName()` for test isolation. - -The `tests/setup.ts` `beforeEach` hook deletes any leftover IndexedDB databases between tests with a per-db timeout — stuck handles are tolerated but don't hang the suite. +Tests live in `tests/integration/` (cross-cutting behaviour through the adapter, +including the React/Vue init-ordering coverage) and `tests/unit/` (isolated +primitives like `applyEvent` and schema-type inference via `expectTypeOf`). +Fixtures in `tests/helpers/` provide `projectTaskConfig()` and a `uniqueDbName()` +for test isolation. --- -## Surface removed in 0.3 +## Where the engine internals live -The pre-0.2 monolithic path (`createPersistenceStore` / `PersistenceStore` / `PersistenceStoreImpl`), its React bindings (`StitchProvider`, `SyncStoreProvider`, `StitchContext`, `StitchContextValue`, `useStitch`, `SyncStoreContext`, `SyncStoreContextValue`, `useSyncStore`, `usePersistenceToMemorySync`), the bundled Vue ``, and the standalone `createPersistenceBridge` / `PersistenceBridge` helpers were all deleted in 0.3. New code uses `createStore` + `` / `` (React) or `` + `` (Vue). There is no migration shim — a 0.2 consumer upgrading to 0.3 will see import errors at the removed names and must port to the unified API. +Everything below the seam — the dual in-memory/IndexedDB stores, MQTT v5 +enhanced-auth, request/response correlation, the offline queue and its +consolidation, reconciliation and `_version` LWW, corruption recovery, and the +`$DB/…` topic tree — is implemented in `stitch-wasm` (the `stitch-rs` repo). See +that repo's `ARCHITECTURE.md` for the engine design. diff --git a/CHANGELOG.md b/CHANGELOG.md index 6aa02e0..3f03e5f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,13 +2,45 @@ ## Unreleased +### Added + +- _Nothing yet._ + +## 0.5.0 + ### Changed (breaking) -- **`responseTopicPrefix` default moved off `$SYS`.** Default changed from `$SYS/responses` to `$DB/clients`. The previous default published responses under `$SYS`, which the MQTT 5 spec reserves for broker-internal use (§4.7.2) — production brokers (EMQX, HiveMQ, AWS IoT Core) reject client publishes under `$SYS` by default ACL, so the old default only worked against permissive or custom-configured brokers. Per-request response topic shape is unchanged: `{prefix}/{clientId}/{requestId}`. Deployments overriding `responseTopicPrefix` explicitly are unaffected. Deployments relying on the old default must either set `responseTopicPrefix: '$SYS/responses'` to preserve current behavior or update broker ACLs to accept the new `$DB/clients/...` topic. +- **Store backend replaced by `@laboverwire/stitch-wasm`.** The entire TypeScript store — memory cache, IndexedDB persistence, MQTT sync, offline queue — has been removed. `@laboverwire/stitch` is now a thin, framework-agnostic binding layer: `src/store.ts` is a small adapter that wraps the Rust/WASM `Store` from `@laboverwire/stitch-wasm` (`^0.2.1`), and the package ships the same React and Vue bindings. All store/sync/persistence/offline-queue/MQTT logic now lives inside the wasm (compiled from the sibling `stitch-rs` repo). +- **`StoreOptions.remote` shape changed.** `remote` is now `{ url, clientId?, ticket?, username?, password? }` instead of `{ serverUrl, getTicket }`. `url` is a `ws://`|`wss://` MQTT endpoint; `ticket` is a JWT for MQTT v5 enhanced auth; `username`/`password` drive classic MQTT password auth. `persistence` is `{ dbName, passphrase? }` — supplying `passphrase` enables AES-GCM encryption. +- **`responseTopicPrefix` default moved off `$SYS`.** Default changed from `$SYS/responses` to `$DB/clients`. The previous default published responses under `$SYS`, which the MQTT 5 spec reserves for broker-internal use (§4.7.2) — production brokers (EMQX, HiveMQ, AWS IoT Core) reject client publishes under `$SYS` by default ACL, so the old default only worked against permissive or custom-configured brokers. Per-request response topic shape is unchanged: `{prefix}/{clientId}/{requestId}`. The `responseTopicPrefix` config field still exists on `StoreConfig` and is forwarded to the wasm; only the implementation moved. Deployments overriding `responseTopicPrefix` explicitly are unaffected. Deployments relying on the old default must either set `responseTopicPrefix: '$SYS/responses'` to preserve current behavior or update broker ACLs to accept the new `$DB/clients/...` topic. +- **Subscription callbacks are now asynchronous.** `subscribeToEntity` and `subscribeToScope` callbacks fire one tick after the mutating call resolves, not synchronously within it. Callers that relied on synchronous delivery must not assume the callback has run by the time `create`/`update`/`delete` returns. +- **`connectionStatus` values are lowercase** — `'connected' | 'connecting' | 'disconnected' | 'error' | 'offline'`. + +### Added + +- **`@laboverwire/stitch-wasm` dependency** (`^0.2.1`) — the Rust/WASM store, MQTT client, and IndexedDB persistence, bundled as a single wasm-bindgen module. +- **Pre-initialize tolerance in the adapter.** The wasm store requires `initialize()` before use, but the TS adapter tolerates pre-init access: synchronous reads return empties (`[]` / `{}` / `null` / `0` / `'offline'`) before init, `subscribe*` defer wiring until `initialize()` resolves and then trigger a re-read, and async methods await init. React and Vue hooks are therefore safe to mount before the provider finishes initializing. +- **`Store.getVersion(scopeId, entity): number`** — reactivity token used by the memory snapshot cache. +- **`Store.pendingMutationCount(scopeId): Promise`** — offline-queue depth for a scope. + +### Removed + +- **In-package store internals.** `src/memory-store.ts`, `src/persistence-layer.ts`, `src/remote-sync-layer.ts`, `src/sync-engine.ts`, `src/offline-queue.ts`, `src/internal-utils.ts`, and `src/internal-wasm-error.ts` deleted — their logic now lives in `@laboverwire/stitch-wasm`. +- **`mqdb-wasm` and `mqtt5-wasm` direct dependencies.** Both are now bundled inside `@laboverwire/stitch-wasm`. +- **Internal factory exports.** `createMemoryStore`, `createSyncEngine`, `createPersistenceLayer`, `createRemoteSyncLayer`, `createPersistentOfflineQueue`, `createInMemoryOfflineQueue` removed. The only value export is now `createStore`. +- **`OwnershipError` and `MqdbError` classes** removed — error handling for the wasm store no longer surfaces these TS types. +- **Session/auth-cache `Store` methods removed.** `getCachedUser`, `setCachedUser`, `clearCachedUser`, `hasPendingLogout`, `setPendingLogout`, and `flushPendingLogout` are no longer on `Store`. They were thin `sessionStorage` wrappers; apps that relied on them must reimplement the caching in application code. +- **Internal-layer interface types.** `SyncEngine`, `PersistenceLayer`, `RemoteSyncLayer`, `OfflineQueue`, `MutationSender`, `LocalAccessor`, `PendingMutation`, `ConsolidatedMutation`, `ScopeBundle`, `ScopeState`, `SyncMutation`, `MutationEvent` no longer exported. The public type surface is now `StoreConfig`, `EntityDefinition`, `SchemaField`, `ForeignKeyDefinition`, `ConnectionStatus`, `SortField`, `SortDirection`, `ListFilter`, `Store`, `StoreOptions`, `PersistenceConfig`, `RemoteConfig`, `MemoryStore`, `EntitySchema`, `DefaultSchema`, `EntityKey`, and `OriginTag`. The `MemoryStore` type still exists but is trimmed to `{ getSnapshot, getSnapshotAsMap, subscribeToScope }`. +- **Consumer Vite `server.fs.allow` / `optimizeDeps.exclude` guidance obsolete.** `stitch-wasm` ships a wasm-bindgen bundler-target ESM module (ESM wasm import proposal), so a consuming Vite app must instead add `vite-plugin-wasm` and `vite-plugin-top-level-await`, and set `build.target: 'esnext'` for production builds (the wasm module and the top-level-await plugin emit top-level `await`). The old `mqdb-wasm`/`mqtt5-wasm` filesystem-allow and dep-exclude instructions no longer apply and must be removed. This repo's `vitest.config.ts` now uses those two plugins and the browser test suite loads the real wasm. ### Fixed -- **Brittle response-topic regex.** `handleResponseMessage` in `src/sync-engine.ts` built its match regex by prepending `\\$` to `responseTopicPrefix`, which only worked because the default started with `$`. Any prefix not beginning with `$` (e.g. a custom override like `responses`) produced an invalid regex (`\r` was interpreted as carriage return). Replaced with a `String.prototype.startsWith`-based check that works for any prefix value. As a side effect, the handler now binds to the live `clientId` rather than the previous `[^/]+` wildcard segment — defense in depth, since the response subscription is already scoped per client. +- **React/Vue init-order safety.** Because the adapter tolerates pre-init access (see Added), hooks like `useEntitySnapshot`, `useScopedEntities`, and `useRootEntityList` no longer read stale or throwing state when a component mounts before the provider's `initialize()` resolves — `subscribe*` wires up and re-reads once init completes. +- **Brittle response-topic parsing** now lives in `@laboverwire/stitch-wasm`. The former `src/sync-engine.ts` regex that split `{prefix}/{clientId}/{requestId}` on a fixed segment count is gone along with the file; the wasm handles response-topic matching against the configured `responseTopicPrefix`. + +### Unchanged + +- **React bindings** — `StoreProvider`, `AuthProvider`, `useEntitySnapshot`, `useScopedEntities`, `useRootEntityList`, `useTopLevelEntities`, `useChildCounts`, `useConnectionStatus`, `useSyncScope`, `useStore` — and **Vue bindings** — `StoreRoot`, `StitchAuth`, matching composables, and `useStore` — keep the same API. `Store.reconnect(serverUrl, getTicket?)` also keeps its signature: the adapter resolves the ticket via `getTicket` and runs any reconnect validator. ## 0.4.3 diff --git a/README.md b/README.md index 85e308e..ae69d22 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # @laboverwire/stitch -Reactive state synchronization library. Bridges an in-memory store, IndexedDB persistence, and MQTT-based remote sync into a single `Store` interface. Optional React and Vue 3 bindings. +Reactive state synchronization for React and Vue. A thin, framework-agnostic binding layer over [`@laboverwire/stitch-wasm`](https://www.npmjs.com/package/@laboverwire/stitch-wasm) — a Rust/WASM package that owns the in-memory store, IndexedDB persistence, and MQTT-based remote sync. This package wraps that WASM `Store` in a small adapter and ships React and Vue 3 bindings on top; all store, sync, persistence, and offline-queue logic lives inside the WASM. ```bash npm install @laboverwire/stitch @@ -11,7 +11,7 @@ import { createStore } from '@laboverwire/stitch'; const store = createStore(config, { persistence: { dbName: 'my-app' }, - remote: { serverUrl: 'wss://mqtt.example.com', getTicket: () => fetchAuthTicket() }, + remote: { url: 'wss://mqtt.example.com', ticket: await fetchAuthTicket() }, }); await store.initialize(); @@ -20,15 +20,18 @@ await store.replaceScope('project-abc'); const tasks = store.getSnapshot('task', 'project-abc'); ``` +`remote.url` is a `ws://`/`wss://` MQTT endpoint. Pass `ticket` (a JWT) for MQTT v5 enhanced auth, or `username`/`password` for classic password auth. A `persistence.passphrase` enables AES-GCM encryption of the IndexedDB store. + ## Documentation - [Configuration](./docs/configuration.md) — `StoreConfig`, `StoreOptions`, scope model, typed schemas -- [Store API](./docs/api.md) — full method reference and error handling +- [Store API](./docs/api.md) — full method reference - [React bindings](./docs/react.md) — providers, hooks, runnable example - [Vue 3 bindings](./docs/vue.md) — providers, composables, runnable example - [Concepts](./docs/concepts.md) — origin tags, offline queue, reconciliation, connection resilience -- [Vite consumer guide](./docs/vite-consumer.md) — using the package via source alias from a monorepo -- [Architecture](./ARCHITECTURE.md) — internal layer composition, data flow, invariants +- [Vite consumer guide](./docs/vite-consumer.md) — consuming apps must add `vite-plugin-wasm` (and `vite-plugin-top-level-await`) so the bundler can load the WASM module +- [Architecture](./ARCHITECTURE.md) — layer composition, data flow, invariants +- [Upgrading to 0.5.0](./docs/upgrading-to-0.5.md) — breaking changes and step-by-step upgrade from 0.4.x - [Changelog](./CHANGELOG.md) - [Releasing](./RELEASING.md) diff --git a/docs/api.md b/docs/api.md index 7d5ab7a..94012ac 100644 --- a/docs/api.md +++ b/docs/api.md @@ -1,32 +1,55 @@ # Store API +`@laboverwire/stitch` is a thin binding layer over the `@laboverwire/stitch-wasm` +store. `createStore(config, options?)` returns a `Store`; all methods below are on +that instance. + ## Lifecycle ```ts -store.initialize() // idempotent: concurrent callers share one init promise -store.destroy() // tear down all layers +store.initialize() // must be awaited before use; returns Promise +store.destroy() // tear down the store; returns Promise store.ready // boolean, true after initialization ``` +## Pre-init tolerance + +The adapter tolerates access before `initialize()` resolves, so React/Vue hooks are +safe to mount before the provider finishes initializing: + +- Synchronous reads return empties: `read` → `null`, `getSnapshot` → `[]`, + `getSnapshotAsMap` → `{}`, `getChildCount` → `0`, `getVersion` → `0`, + `connectionStatus` → `'offline'`, `isReconnecting` → `false`. +- `subscribe*` calls defer wiring until `initialize()` resolves, then poke a + re-read so `useSyncExternalStore`-style consumers pick up the first snapshot. +- Async methods await initialization before running. + ## CRUD ```ts store.create(entity, scopeId, data, tag?) // returns Promise -store.update(entity, id, fields, tag?) // partial update -store.delete(entity, id, tag?) // delete by id +store.update(entity, id, fields, tag?) // partial update, Promise +store.delete(entity, id, tag?) // delete by id, Promise ``` +`tag` is an `OriginTag` (`'remote' | 'load' | 'clear'`). + ## Queries ```ts store.read(entity, id) // single record or null (sync, from memory) store.getSnapshot(entity, scopeId) // all records for entity in scope (sync) store.getSnapshotAsMap(entity, scopeId) // same as map keyed by id (sync) -store.list(entity, filter?) // filtered list from persistence (async) +store.list(entity, filter?) // filtered list (async) store.listRootEntities(sort?) // all root entities (async) -store.getChildCount(entity, scopeId) // count children in scope (async) +store.getChildCount(entity, scopeId) // count children in scope (sync, returns number) +store.getVersion(scopeId, entity) // reactivity token for the scope+entity (sync, returns number) ``` +`getVersion` returns an opaque reactivity token the memory snapshot cache compares +for equality to decide when a snapshot must be re-read. It changes whenever the +scope+entity's data changes; treat it as opaque, not a sequential count. + ## Subscriptions ```ts @@ -34,7 +57,8 @@ store.subscribeToScope(scopeId, entity, cb) // fires when the given scope+enti store.subscribeToEntity(entity, cb) // fires on every create/update/delete; callback: (data | null, op) => void ``` -Both return an unsubscribe function. +Both return an unsubscribe function. Callbacks are delivered **asynchronously** — +one tick after the mutating call resolves, not synchronously. ## Batch operations @@ -46,70 +70,55 @@ store.endBatch() // flush batch and notify subscribers ## Scope management ```ts -store.replaceScope(scopeId) // subscribe MQTT + fetch + reconcile + load (rebuilds memory DB) -store.closeScope(scopeId) // unsubscribe + clear in-memory data -store.loadScope(scopeId, data) // manually load scope data (no network) -store.clearScope(scopeId) // clear in-memory scope data +store.replaceScope(scopeId) // subscribe + fetch + reconcile + load, rebuilds memory DB (async) +store.closeScope(scopeId) // unsubscribe + clear in-memory data (async) +store.loadScope(scopeId, data) // manually load scope data, no network (async) +store.clearScope(scopeId) // clear in-memory scope data (async) ``` +`loadScope` takes `data` as `Record[]>` (entity +name → array of records). All four return `Promise`. + ## Connection ```ts -store.connectionStatus // current ConnectionStatus +store.connectionStatus // current ConnectionStatus (sync) store.isReconnecting // boolean store.subscribeToConnectionStatus(cb) // returns unsubscribe -store.disconnect() // close MQTT connection -store.reconnect(serverUrl, getTicket?) // reconnect with new credentials +store.disconnect() // close connection, Promise +store.reconnect(serverUrl, getTicket?) // reconnect with new credentials, Promise ``` +`ConnectionStatus` is one of the lowercase values `'connected' | 'connecting' | +'disconnected' | 'error' | 'offline'`. `reconnect` runs any configured reconnect +validator, resolves the ticket via `getTicket`, and reconnects. + ## Authentication & session ```ts store.setAuthenticatedUser(userId) store.setSessionInvalidHandler(handler) store.setReconnectValidator(validator) -store.resetForLogout() - -store.getCachedUser() // from sessionStorage (15min TTL) -store.setCachedUser(user) -store.clearCachedUser() - -store.hasPendingLogout() -store.setPendingLogout(pending) -store.flushPendingLogout(logoutFn) +store.resetForLogout() // Promise ``` -## Local state +## Local state & pending sync ```ts -store.readLocalState(entity, id) // read from local-only entities -store.updateLocalState(entity, id, fields) // write to local-only entities +store.readLocalState(entity, id) // read from local-only entities (async) +store.updateLocalState(entity, id, fields) // write to local-only entities (async) +store.pendingMutationCount(scopeId) // pending offline mutations for scope, Promise ``` ## Advanced ```ts -store.request(topic, payload) // raw MQTT request-response +store.request(topic, payload) // raw MQTT request-response (async) store.hasPersistence // boolean store.hasRemote // boolean -store.memory // underlying MemoryStore +store.memory // underlying MemoryStore view store.config // StoreConfig ``` -## Errors - -All throws from the underlying WASM layer are coerced into `MqdbError` so consumers get a real `Error` object with a `.stack`, a method-qualified message, and the original exception preserved on `.cause`: - -```ts -import { MqdbError } from '@laboverwire/stitch'; - -try { - await store.list('task', { sort: [{ field: 'bogus', direction: 'asc' }] }); -} catch (err) { - if (err instanceof MqdbError) { - console.error(err.method); // "list:task" - console.error(err.message); // "mqdb.list:task: unknown field: 'bogus'" - console.error(err.cause); // the raw WASM throw - } -} -``` +`store.memory` is a read-only `MemoryStore` view exposing `getSnapshot`, +`getSnapshotAsMap`, and `subscribeToScope`. diff --git a/docs/concepts.md b/docs/concepts.md index 38f2497..8663995 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -1,5 +1,7 @@ # Key concepts +Sync, persistence, offline queueing, and reconciliation are implemented in the `@laboverwire/stitch-wasm` store. `@laboverwire/stitch` is a thin binding layer that wraps that store — exposing it through `createStore` plus the React and Vue bindings. The concepts below describe how the wasm store behaves; the `StoreConfig` and origin tags you pass through this package drive it. + ## Origin tags Mutations carry an `originTag` controlling propagation: @@ -11,7 +13,7 @@ Mutations carry an `originTag` controlling propagation: ## Offline queue -Local mutations are queued in `pending_sync` and flushed when connected. Before flushing, mutations are consolidated: +Local mutations are queued in `pending_sync` and flushed when connected. `pendingMutationCount(scopeId)` reports how many are outstanding. Before flushing, mutations are consolidated: - Multiple updates to same entity → merged (last write wins per field) - Insert + updates → single insert with merged data diff --git a/docs/configuration.md b/docs/configuration.md index 2a46570..35d310d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -20,8 +20,8 @@ | Field | Type | Description | |---|---|---| -| `persistence` | `{ dbName: string }` | Enable IndexedDB persistence | -| `remote` | `{ serverUrl: string, getTicket?: () => Promise }` | Enable MQTT sync with optional JWT auth | +| `persistence` | `{ dbName: string, passphrase?: string }` | Enable IndexedDB persistence. A `passphrase` turns on AES-GCM encryption at rest | +| `remote` | `{ url: string, clientId?: string, ticket?: string, username?: string, password?: string }` | Enable MQTT sync. `url` is a `ws://`/`wss://` MQTT endpoint; `ticket` is a JWT for MQTT v5 enhanced auth; `username`/`password` drive classic MQTT password auth | ## Scope model diff --git a/docs/design/MQDB-AGENT-BRIEF.md b/docs/design/MQDB-AGENT-BRIEF.md index 4a0ca53..9279211 100644 --- a/docs/design/MQDB-AGENT-BRIEF.md +++ b/docs/design/MQDB-AGENT-BRIEF.md @@ -2,6 +2,8 @@ **Audience:** whoever (human or agent) is implementing the MQDB server-side changes for stitch's reactive `replaceScope`. This document is **self-contained**: all wire-format excerpts and protocol contracts are quoted inline so you do not need to read stitch's source tree to do the work. Pointers into stitch source are provided for verification only. +> **Note (stitch 0.5.0):** the TypeScript store/sync/persistence layer was replaced by the `@laboverwire/stitch-wasm` package (compiled from the sibling `stitch-rs` repo). `@laboverwire/stitch` is now a thin binding layer. The wire format below is unchanged, but the source pointers that used to name `src/sync-engine.ts` / `src/remote-sync-layer.ts` now refer to that wasm implementation — those `.ts` files were removed. + **Scope:** the changes needed in MQDB to support `docs/design/reactive-scope-open.md`. Peer mode (`docs/design/reactive-peer-mode.md`) is a separate workstream and out of scope for this brief. --- @@ -40,7 +42,7 @@ $DB/clients/{clientId}/{requestId} # resp `{eventType}` in the topic and `operation` in the payload must agree per the mapping in §3.2. -Source of truth in stitch: `parseScopedTopic` at `src/sync-engine.ts:487-507` (regex matches both root-own and child topics). +Source of truth in stitch: the scoped-topic parser in `@laboverwire/stitch-wasm` (formerly `parseScopedTopic` in `src/sync-engine.ts`, removed in 0.5.0) — matches both root-own and child topics. ### 3.2 ChangeEvent payload (the one MQDB must emit on event topics) @@ -58,7 +60,7 @@ interface ChangeEvent { } ``` -Validation in stitch (`src/sync-engine.ts:460-470`): +Validation in stitch (in `@laboverwire/stitch-wasm`; formerly `isValidChangeEvent` in `src/sync-engine.ts`): - `id` is a non-empty string. - `operation` is exactly one of `'Create' | 'Update' | 'Delete'` (note capitalization). @@ -66,7 +68,7 @@ Validation in stitch (`src/sync-engine.ts:460-470`): Anything that fails validation is silently dropped by stitch. Make sure replay events conform. -**Topic-to-operation mapping** (`src/sync-engine.ts:540-545`): the topic suffix is lowercase past tense, the payload field is capitalized infinitive: +**Topic-to-operation mapping** (handled in `@laboverwire/stitch-wasm`): the topic suffix is lowercase past tense, the payload field is capitalized infinitive: | Topic suffix | Payload `operation` | |---|---| @@ -84,7 +86,7 @@ Stitch sets these on outgoing publishes and reads them on incoming messages. MQD |---|---|---|---| | `x-origin-client-id` | Yes (own clientId) | Yes — used in `isOwnMutation` filter | Prevents loop-back: stitch ignores any event whose `x-origin-client-id` matches its own clientId. | -`isOwnMutation` (`src/sync-engine.ts:519-523`) returns true if **either** `event.sender === clientId` **or** the user property `x-origin-client-id === clientId`. Both mechanisms are honored; either is sufficient. +`isOwnMutation` (in `@laboverwire/stitch-wasm`) returns true if **either** `event.sender === clientId` **or** the user property `x-origin-client-id === clientId`. Both mechanisms are honored; either is sufficient. For replay events MQDB emits, see §3.5 — the `sender` value is reserved. @@ -105,7 +107,7 @@ interface RequestResponse { } ``` -Source: `request()` at `src/sync-engine.ts:745-783` (request side), `checkResponseAndAuth` at `src/sync-engine.ts:628-640` (response side). +Source: the request/reply envelope and response status handling in `@laboverwire/stitch-wasm` (formerly `request()` and `checkResponseAndAuth` in `src/sync-engine.ts`, removed in 0.5.0). ### 3.5 NEW: Synthetic replay events @@ -114,7 +116,7 @@ When MQDB processes a `hello` (§4) and emits replay events, those events: - Use the **same topics** as live mutations (`events/created` / `events/updated` / `events/deleted`). - Use the **same payload shape** as §3.2. - Carry `sender: '__server_replay__'` in the payload **AND** set the MQTT user property `x-origin-client-id` to a server-fixed sentinel (suggest: `'__mqdb_server__'`). Neither value matches any real client id, so all clients (including the requesting one) will accept the events. -- The fan-out cost is intentional: existing peers on the scope receive replay events too. They apply them via the existing `_version` LWW compare (`src/remote-sync-layer.ts:386-399`), which no-ops when the version matches. Bandwidth cost accepted as the price of one channel. +- The fan-out cost is intentional: existing peers on the scope receive replay events too. They apply them via the existing `_version` LWW compare (in `@laboverwire/stitch-wasm`; formerly `src/remote-sync-layer.ts`), which no-ops when the version matches. Bandwidth cost accepted as the price of one channel. ### 3.6 NEW: `hello` payload (client → server) @@ -281,16 +283,4 @@ These are flagged in `reactive-scope-open.md` §5 but worth pulling forward: - `docs/design/reactive-scope-open.md` — full design rationale, client-side changes, phasing, open questions. - `docs/design/reactive-peer-mode.md` — adjacent design (peer-coordinated mode); §8/Q2 has tombstone retention discussion that applies here. -- `src/types.ts` — `ChangeEvent`, `SyncMutation`, `OwnershipError` definitions. -- `src/sync-engine.ts` — wire-format truth: - - L18-26 — `ChangeEvent` interface. - - L368-421 — root + top-level event handlers (parsing). - - L423-437 — response message handler (for the request/reply path). - - L439-457 — scope subscription (`subscribeToScope`). - - L460-470 — `isValidChangeEvent` validator. - - L487-507 — `parseScopedTopic` regex. - - L509-523 — `extractOriginClientId` / `isOwnMutation`. - - L525-557 — main scoped event handler (`handleWatchMessage`). - - L628-640 — response status/error handling (`checkResponseAndAuth`). - - L654-711 — current mutation request/reply (`createEntity`, `updateEntity`, `deleteEntity`, `bumpScopeVersion`). - - L745-783 — `request()` envelope. +- `@laboverwire/stitch-wasm` (compiled from the sibling `stitch-rs` repo) — wire-format truth. As of stitch 0.5.0 the store/sync/persistence layers moved out of this repo into the wasm; the `ChangeEvent` shape, scoped-topic parsing, change-event validation (`isValidChangeEvent`), origin-client filtering (`extractOriginClientId` / `isOwnMutation`), the scoped event handler, the request/reply envelope (`request()` / `checkResponseAndAuth`), and the mutation request handlers (`createEntity` / `updateEntity` / `deleteEntity` / `bumpScopeVersion`) all live there. The former TypeScript sources were removed: `src/sync-engine.ts` (which defined the private `ChangeEvent` interface) and `src/remote-sync-layer.ts`, along with the `SyncMutation` and `OwnershipError` type defs in `src/types.ts`. The inline excerpts in §3 remain the authoritative contract regardless of implementation language. diff --git a/docs/design/reactive-peer-mode.md b/docs/design/reactive-peer-mode.md index 0ed1ae7..4a6400d 100644 --- a/docs/design/reactive-peer-mode.md +++ b/docs/design/reactive-peer-mode.md @@ -6,6 +6,8 @@ This document is a **design proposal** to be reviewed and corrected before implementation. Open questions are flagged inline. +> **Superseded context — predates the 0.5.0 backend swap.** This design was written against the old TypeScript store implementation. As of `@laboverwire/stitch` 0.5.0 the store / sync / persistence / offline-queue / MQTT stack moved into the Rust/WASM package `@laboverwire/stitch-wasm`; `@laboverwire/stitch` is now a thin binding layer (`src/store.ts` adapts the wasm `Store`). The five TypeScript files this doc references — `sync-engine.ts`, `remote-sync-layer.ts`, `persistence-layer.ts`, `offline-queue.ts`, `memory-store.ts` — no longer exist here; that logic now lives inside stitch-wasm (compiled from the sibling `stitch-rs` repo), and the module names, file paths, and line numbers this doc attributes to **those five deleted files** must be re-mapped onto the Rust codebase before implementation. This does **not** apply blanket to all of §7/§10: `src/types.ts` and `src/store.ts` still exist in this package. The `syncMode` config field and HLC type in §7.4 would still be added to `src/types.ts`, and the `replaceScope` branch in §7.5 lives in the `src/store.ts` wasm adapter (or is forwarded into the wasm) — not on the Rust side. The design intent below is unchanged. + --- ## 1. Motivation @@ -15,7 +17,7 @@ Stitch today depends on MQDB as a central coordinator: clients publish CRUD requ For stitch to be adoptable without committing the user to MQDB, the library needs a mode where: - The MQTT broker is just a message bus (any MQTT 5 broker works). -- Clients each persist their own state (already true via `PersistenceLayer` / IndexedDB). +- Clients each persist their own state (already true — stitch persists each client's state to IndexedDB). - Clients coordinate among themselves to converge on a shared view of the data. The primary use case is **single-user, multi-device sync** (one user editing on phone + laptop + tablet). Multi-user collaboration on shared data has the same coordination shape, just with more concurrent writers. @@ -67,11 +69,11 @@ Client publishes to the events topic at QoS 1 with `retain=false`: } ``` -Broker fans out to all subscribers (peers and the publisher itself; publisher filters its own via the existing `x-origin-client-id` user property check at `sync-engine.ts:519-523`). +Broker fans out to all subscribers (peers and the publisher itself; publisher filters its own via the existing `x-origin-client-id` user property check, now handled inside stitch-wasm). -**No await on a response.** The publish promise resolves on PUBACK; that is the success signal. The offline queue (`src/offline-queue.ts`) handles broker disconnection. +**No await on a response.** The publish promise resolves on PUBACK; that is the success signal. The offline queue (now inside stitch-wasm) handles broker disconnection. -Compare to today's `createEntity` (sync-engine.ts:654-672) which sets a `responseTopic` and awaits a JSON `{status, code, data}` reply with a 10s timeout. That whole shape is gone. +Compare to today's `createEntity` (in the wasm sync engine) which sets a `responseTopic` and awaits a JSON `{status, code, data}` reply with a 10s timeout. That whole shape is gone. ### 4.2 Bootstrap (new client opens scope) @@ -92,8 +94,8 @@ Client A (new) Peers B, C (online with state) events/deleted 6. Receive events on events/# (already subscribed in step 1) -7. Apply each via existing - applyMutationToDb (LWW by hlc) +7. Apply each via applyMutationToDb + (LWW by hlc), now inside stitch-wasm 8. UI re-renders reactively as each entity arrives ``` @@ -106,17 +108,17 @@ The `manifest` is `Array<{id, hlc}>` — what the new client already has from lo Identical to bootstrap. On reconnect, client publishes a fresh `hello` with its current manifest. Peers reactively send the delta. No special offline-detection path needed. -The existing `OfflineQueue` flush already handles outgoing-mutations-during-offline correctly; this design touches only the inbound side. +The offline queue flush (now inside stitch-wasm) already handles outgoing-mutations-during-offline correctly; this design touches only the inbound side. ### 4.4 Deletes Two modes of delete: -**Locally-initiated**: client publishes `events/deleted` with `{operation: "Delete", entity, id, hlc}`. Peers apply via existing `applyMutationToDb` delete branch. +**Locally-initiated**: client publishes `events/deleted` with `{operation: "Delete", entity, id, hlc}`. Peers apply via the `applyMutationToDb` delete branch, now inside stitch-wasm. **Tombstone replay** (during bootstrap): when a peer responds to `hello` and detects an entity in the new client's manifest that the peer no longer has, the peer needs to know whether the entity was deleted (vs. simply never existed). This requires the peer to retain a tombstone. -**Tombstone storage** (open question — see §9): each client persists tombstones in a local table (e.g. `_tombstones` keyed by `{entity, id, hlc}`) for some retention period. Old tombstones are pruned. PersistenceLayer would need a tombstone API. +**Tombstone storage** (open question — see §9): each client persists tombstones in a local table (e.g. `_tombstones` keyed by `{entity, id, hlc}`) for some retention period. Old tombstones are pruned. The wasm persistence layer would need a tombstone API. **Resurrection failure mode**: if every peer prunes a tombstone (TTL expires) while at least one peer still has the deleted entity in its local IndexedDB — most likely because that peer was offline at the time of the original delete and longer than the TTL — bootstrap will replay the entity to anyone whose manifest lacks it. The delete effectively un-happens. This is the central correctness cost of bounded tombstone retention; any TTL chosen in §9/Q2 must be large enough to cover the longest expected offline window, or the system must accept rare resurrections. There is no in-protocol fix without unbounded tombstones or a coordinator. @@ -160,7 +162,7 @@ else ### 5.3 Replacing `_version` -Today, `applyMutationToDb` (remote-sync-layer.ts:386-399) compares numeric `_version` and `updatedAt` for LWW: +Today, `applyMutationToDb` (in the wasm remote-sync layer) compares numeric `_version` and `updatedAt` for LWW: ```ts if (typeof remoteVersion === 'number') { @@ -204,21 +206,23 @@ App authors who need a "loaded" gate can implement one themselves: e.g. observe Throws in peer mode: `Error('request() is not supported in peer sync mode')`. `Store.request` on the public API documents this as a no-op in peer mode. -### 6.4 `bumpScopeVersion` +### 6.4 Scope-level versioning -No-op in peer mode. Per-entity HLC handles ordering; there is no scope-level version. +Scope-level version bumping is an internal wasm concept, not a member of the public `Store` interface. In peer mode it is a no-op: per-entity HLC handles ordering, so there is no scope-level version to bump. -### 6.5 `fetchList` / `fetchOne` +### 6.5 `list` / `listRootEntities` -Not used in peer mode. The "initial state" comes from local IndexedDB (already-persisted) plus reactive `hello` replies from peers. There is no `fetchList(rootEntity)` to enumerate all root entities the user has access to — that information must arrive via the same `hello` mechanism, scoped at the user level (see §10 for top-level entities). +The public read methods `list(entity, filter?)` and `listRootEntities(sort?)` still exist in peer mode, but they read from local IndexedDB (via the wasm) rather than a server. The "initial state" comes from that already-persisted local data plus reactive `hello` replies from peers. In peer mode there is no server-backed enumeration of all root entities the user has access to — that information must arrive via the same `hello` mechanism, scoped at the user level (see §10 for top-level entities). ## 7. What changes in the existing code -### 7.1 `src/sync-engine.ts` +> §7.1–§7.3 describe the pre-0.5.0 TypeScript modules (`sync-engine.ts`, `remote-sync-layer.ts`, `persistence-layer.ts`). That code no longer exists here — it now lives inside `@laboverwire/stitch-wasm` (the `stitch-rs` repo) — so the module names and line numbers in those subsections are historical and must be re-mapped onto the Rust codebase before implementation. §7.4 (`src/types.ts`) and §7.5 (`src/store.ts`) reference files that **still exist** in this TS package; those changes stay on the TypeScript side. The design intent per module is still accurate. + +### 7.1 sync engine (formerly `src/sync-engine.ts`) - Constructor reads `config.syncMode`, branches on it for the relevant methods. -- `createEntity` / `updateEntity` / `deleteEntity` in peer mode: compute the events topic, publish QoS 1 retain=false, no responseTopic, no await on reply. `data.id` (already injected by `StoreImpl.create` at `store.ts:239`) is the entity ID; no server-generated ID. -- `bumpScopeVersion`: no-op in peer mode. +- `createEntity` / `updateEntity` / `deleteEntity` in peer mode: compute the events topic, publish QoS 1 retain=false, no responseTopic, no await on reply. `data.id` (already injected on create before dispatch) is the entity ID; no server-generated ID. +- scope-level version bump: no-op in peer mode. - `openScope`: in peer mode, subscribe to `events/#`, publish `hello` with manifest (read from local accessor), return `{root: null, children: {}, version: 0, bufferedMutations: []}`. The state arrives reactively after return. - `subscribeToTopLevel`: same `events/#` subscription. - `handleWatchMessage`: same path. In peer mode, "buffered while awaitingState" doesn't apply (we don't await state). @@ -227,27 +231,30 @@ Not used in peer mode. The "initial state" comes from local IndexedDB (already-p Estimated diff: ~250-400 LOC added, mostly contained in mode-branching `if` blocks. Most of the existing connection / reconnect / auth machinery is unchanged. -### 7.2 `src/remote-sync-layer.ts` +### 7.2 remote-sync layer (formerly `src/remote-sync-layer.ts`) - `applyMutationToDb`: replace numeric `_version` compare with HLC compare. Single localized change. - `syncRootEntityList`: in peer mode, this becomes "subscribe to top-level pattern, publish hello, react." Major refactor. May warrant extraction into a separate helper. - `reconcileChildren`: not invoked in peer mode (no synchronous server snapshot to reconcile against). -### 7.3 `src/persistence-layer.ts` +### 7.3 persistence layer (formerly `src/persistence-layer.ts`) - New: tombstone storage. A `_tombstones` table keyed by `(entity, id)`, holding `{hlc, deletedAt}`. New methods: `markDeleted(entity, id, hlc)`, `getTombstone(entity, id)`, `pruneTombstones(beforeTimestamp)`. - `delete(entity, id)` writes a tombstone before (or instead of) actually removing the row in peer mode. -### 7.4 `src/types.ts` +### 7.4 `src/types.ts` (still exists in this package) - `StoreConfig.syncMode?: 'mqdb' | 'peer'`. - HLC type definition. - Entity-level HLC field (replaces or augments `_version` in peer mode). -### 7.5 `src/store.ts` +### 7.5 `src/store.ts` (still exists — thin wasm adapter) + +The current `src/store.ts` is a thin adapter over the wasm `Store`: its `replaceScope` simply forwards to the wasm via `return this.#afterReady(() => this.#inner.replaceScope(scopeId));`, with no `sync.openScope` call and no reconcile logic (all of that moved into the wasm). The description below reflects the **pre-0.5.0** store.ts, which had the mutation flow and the `openScope` long path inline; it no longer matches the file that currently bears this path. -- No changes to mutation flow (already mode-agnostic at this layer). -- `replaceScope` may need a small branch: in peer mode, skip the `await sync.openScope` long path and just subscribe + publish `hello`. +- Pre-0.5.0: no changes to mutation flow (already mode-agnostic at that layer). +- Pre-0.5.0: `replaceScope` needed a small branch — in peer mode, skip the `await sync.openScope` long path and just subscribe + publish `hello`. +- Post-0.5.0: the mode branch belongs inside the wasm's `replaceScope`; the TS adapter forwards the call unchanged. ## 8. Open questions @@ -289,7 +296,7 @@ Recommendation: each peer publishes hello on every connect. Symmetric and doesn' ### Q4: Multiple-scope users -Today, `syncRootEntityList` enumerates all root entities a user has access to via `fetchList(rootEntity)` — server-side filtering by `userScopeField`. In peer mode, there's no server to query. +Today, `listRootEntities` enumerates all root entities a user has access to — server-side filtering by `userScopeField`. In peer mode, there's no server to query. Options: - The user's "list of accessible scopes" is enumerated by subscribing to a top-level topic and reactively collecting hellos / mutation events. New top-level topic: `$DB/{root}/+/hello` — every scope's hello announcements are visible. Client builds the scope list reactively. @@ -304,7 +311,7 @@ If many mutations happen at the exact same `wallNow` ms, `counter` increments. J ### Q6: `sender` field vs HLC.nodeId -Today the `sender` field is used for own-message filtering (`isOwnMutation` at sync-engine.ts:519-523). In peer mode, HLC.nodeId could play the same role. Should we collapse them? +Today the `sender` field is used for own-message filtering (`isOwnMutation`, now handled inside stitch-wasm). In peer mode, HLC.nodeId could play the same role. Should we collapse them? Recommendation: keep both for now (HLC for ordering, sender for filtering) — simpler diff, no semantic change. Collapse if the duplication bothers us in code review. @@ -313,25 +320,25 @@ Recommendation: keep both for now (HLC for ordering, sender for filtering) — s - **Peer that has unique data and is permanently offline.** That data is unreachable. No protocol short of CRDT replication to all peers solves this without a server. - **Concurrent writes during a network partition** that resolve to LWW. One side's edit wins; the other is lost. CRDTs would merge both. If multi-user concurrent edits during partition is a real requirement, we need CRDTs and this design doesn't apply. - **Strong consistency.** Stitch in peer mode is eventually consistent. Apps requiring read-your-writes consistency on a different device need MQDB mode. -- **Cross-scope queries / search.** No `fetchList(entity, filters)` against a global index. Apps needing this need MQDB mode. +- **Cross-scope queries / search.** No `list(entity, filter?)` against a global server-side index. Apps needing this need MQDB mode. These are honest trade-offs of going masterless. They should be documented prominently in the user-facing docs once this ships. ## 10. Estimated scope -Rough LOC budget for v1 (single-scope, no top-level entity discovery): +Rough LOC budget for v1 (single-scope, no top-level entity discovery). The first three files reflect the pre-0.5.0 TypeScript layout and must be re-mapped onto the stitch-rs modules that now own this logic. On the TypeScript side of this package, only `src/types.ts` takes a real change (the `syncMode` field + HLC type); `src/store.ts` forwards `replaceScope` to the wasm unchanged, so its mode branch lives in the wasm: -| File | Net change | +| Module (pre-0.5.0 file) | Net change | |---|---| -| `src/sync-engine.ts` | +250 | -| `src/remote-sync-layer.ts` | +60 (HLC compare, hello plumbing) | -| `src/persistence-layer.ts` | +120 (tombstone API) | -| `src/store.ts` | +30 (replaceScope branch) | -| `src/types.ts` | +40 (HLC type, syncMode field) | -| New: `src/hlc.ts` | +60 (HLC algorithm) | +| sync engine (`src/sync-engine.ts`, now in wasm) | +250 | +| remote-sync layer (`src/remote-sync-layer.ts`, now in wasm) | +60 (HLC compare, hello plumbing) | +| persistence layer (`src/persistence-layer.ts`, now in wasm) | +120 (tombstone API) | +| `src/store.ts` (still exists) | ~0 (replaceScope forwarded to wasm unchanged; mode branch lives in wasm) | +| `src/types.ts` (still exists) | +40 (HLC type, syncMode field) | +| New: HLC module (in wasm) | +60 (HLC algorithm) | | Tests | +500 (unit + integration with embedded broker e.g. aedes) | -~1100 LOC. Self-contained behind a config flag — MQDB mode is unaffected. +~1050 LOC. Self-contained behind a config flag — MQDB mode is unaffected. ## 11. Implementation phasing @@ -339,8 +346,8 @@ Once this design is approved: 1. **Phase 0**: this doc, plus the `$SYS` fix (already done in `fix/response-topic-default`). 2. **Phase 1**: HLC type, `_hlc` field on peer-mode records, mode-gated branch in `applyMutationToDb` (default `mqdb` keeps numeric `_version`). Mode persistence + `ModeMismatchError` on boot + `Store.clearLocalData()` recovery API (§12.1). Tests. -3. **Phase 2**: Tombstone storage in PersistenceLayer. Tests. -4. **Phase 3**: Peer-mode SyncEngine — mutations as fire-and-forget events, `request()` throws. +3. **Phase 2**: Tombstone storage in the persistence layer. Tests. +4. **Phase 3**: Peer-mode sync engine — mutations as fire-and-forget events, `request()` throws. 5. **Phase 4**: `hello` protocol — client publishes, server reacts, manifest diffing. 6. **Phase 5**: Integration tests with embedded MQTT broker (aedes or similar). 7. **Phase 6**: Documentation updates (README, configuration.md, ARCHITECTURE.md). @@ -378,7 +385,7 @@ Message format: `` `stitch: store at "${dbName}" was initialized in ${expected} **Recovery path** — stitch does not currently expose an API to wipe its local state. Phase 1 will add `Store.clearLocalData(): Promise`, with this contract: - Disconnects the MQTT client (idempotent — safe to call when already disconnected). -- Deletes the IndexedDB database underlying `MemoryStore` + `PersistenceLayer`, including the offline queue and persisted mode metadata. +- Deletes the IndexedDB database underlying the memory store + persistence layer, including the offline queue and persisted mode metadata. - Clears the stitch-owned `sessionStorage` keys: `stitch_client_id`, `stitch_cached_user`, `stitch_pending_logout`. - After the promise resolves, the existing `Store` instance is **unusable** — all further method calls reject with `StoreDisposedError`. The caller must obtain a new instance via `createStore()`. diff --git a/docs/design/reactive-scope-open.md b/docs/design/reactive-scope-open.md index 874bbef..8d79f3b 100644 --- a/docs/design/reactive-scope-open.md +++ b/docs/design/reactive-scope-open.md @@ -2,49 +2,59 @@ **Status**: Draft for review. Not implemented. -**Purpose**: Remove the blocking `Promise.all` of N request/reply round-trips inside `replaceScope` by having the server stream per-record events instead of replying to batched `fetchList` queries. The client subscribes once and reacts to whatever arrives — no `Promise.all`, no per-call timeout, no "loaded" gate. +**Purpose**: Remove the blocking `Promise.all` of N request/reply round-trips inside scope-open by having the server stream per-record events instead of replying to batched `fetchList` queries. The client subscribes once and reacts to whatever arrives — no `Promise.all`, no per-call timeout, no "loaded" gate. This is a **design proposal** for the existing server-backed sync flow. It is intentionally orthogonal to the peer-mode design in `reactive-peer-mode.md` — they share a target shape (one streaming channel, no client-side deadlines) but address different deployments. +> **Where the work lands (0.5.0 architecture).** As of 0.5.0 the TypeScript store backend has been removed. `@laboverwire/stitch` is a thin, framework-agnostic binding layer: `src/store.ts` is a small adapter that wraps the Rust/WASM `Store` from `@laboverwire/stitch-wasm` (`^0.2.1`). **All store / sync / persistence / offline-queue / MQTT logic now lives inside the wasm**, compiled from the sibling `stitch-rs` repo — not in this package. The files an earlier draft of this doc proposed editing (`src/sync-engine.ts`, `src/remote-sync-layer.ts`, `src/memory-store.ts`, `src/persistence-layer.ts`, `src/offline-queue.ts`) no longer exist here. The **client-side** changes in this design are implemented in `stitch-rs`; this repo's adapter is essentially untouched because `replaceScope` already forwards to the wasm and already returns `Promise`. The **server-side** changes (the load-bearing part) are in MQDB. Line references below point at the wasm/protocol behaviour, not at TypeScript in this package; where a concrete line in this repo is cited it is the thin delegating call, not the logic. + --- ## 1. Motivation -### 1.1 What `replaceScope` does today +### 1.1 What scope-open does today + +In this package, `store.replaceScope` (`src/store.ts:245`) is a one-line delegate to the wasm store: + +```ts +replaceScope(scopeId: string): Promise { + return this.#afterReady(() => this.#inner.replaceScope(scopeId)); +} +``` -`store.replaceScope` (`store.ts:484`) → `sync.openScope` (`sync-engine.ts:563`) does: +There is no `Promise.all`, no buffering, and no reconcile path in this repo — all of that is internal to `@laboverwire/stitch-wasm` and is not observable or editable from here. The description below is the protocol-level behaviour, inferred from the pre-0.5.0 TypeScript implementation (now relocated into `stitch-rs`) and the wire contract. Opening a scope does: 1. Subscribe to `${prefix}/${rootEntity}/${scopeId}/#` (streaming, reactive). -2. `await Promise.all([fetchOne(rootEntity, scopeId), ...childEntities.map(e => fetchList(e, scopeId))])` — N parallel one-shot request/reply round-trips, each capped at 10 s by the timeout in `request()` (`sync-engine.ts:752`). -3. While step 2 is in flight, mutation events arriving on the events topic are buffered in `this.buffered` (line 552) instead of being applied immediately. -4. After step 2 resolves, `openScope` drains the buffer, returns the assembled `ScopeState`, and `replaceScope` reconciles it into IndexedDB + MemoryStore. +2. Issue N parallel one-shot request/reply round-trips — one `fetchOne` for the root plus one `fetchList` per child entity — each capped by the wasm's per-request timeout. +3. While step 2 is in flight, mutation events arriving on the events topic are buffered inside the wasm instead of being applied immediately. +4. After step 2 resolves, the wasm drains the buffer, reconciles the assembled snapshot into IndexedDB and its in-memory cache, and the `replaceScope` promise resolves. -The user-visible "wait" on `await store.replaceScope(id)` is step 2: stitch holds the promise until every child entity's list reply has arrived. With slow networks or large scopes this dominates time-to-interactive. +The user-visible "wait" on `await store.replaceScope(id)` is step 2: the wasm holds the promise until every child entity's list reply has arrived. With slow networks or large scopes this dominates time-to-interactive. ### 1.2 Why this is the wrong shape -Stitch already has a fully reactive subscription path (Pattern A in §2 of `reactive-peer-mode.md`). The events topic streams mutations with no timer, no deadline, callback-per-message. `openScope` *also* uses it, but only for events that occur *during* step 2 — once step 2 resolves, the buffer drains and from then on the subscription is the only path. +Stitch already has a fully reactive subscription path (Pattern A in §2 of `reactive-peer-mode.md`). The events topic streams mutations with no timer, no deadline, callback-per-message. Scope-open *also* uses it, but only for events that occur *during* step 2 — once step 2 resolves, the buffer drains and from then on the subscription is the only path. There is no good reason for the initial state to come through a different channel from subsequent mutations. The split exists because the server emits a batched reply to `fetchList` instead of a stream of per-record events. Collapsing both paths onto the events topic produces: - One channel for both bootstrap and ongoing sync. -- No `Promise.all`, no 10 s deadline on scope-open. +- No `Promise.all`, no per-request deadline on scope-open. - `replaceScope` returns as soon as the subscription is established. UI populates reactively as records stream in. - Same bootstrap shape as peer mode (§4.2 of `reactive-peer-mode.md`), with the server filling the peer role. ### 1.3 What this design does NOT change -- **Mutations stay request/reply.** Writes need synchronous validation (ownership, constraint violations, server-canonical fields). Removing the await from `createEntity` / `updateEntity` / `deleteEntity` is a separate, harder design problem and is explicitly out of scope here. -- **The 10 s `request()` timeout stays.** It still applies to mutations and to any other request/reply call. This design simply removes scope-open from the set of callers. +- **Mutations stay request/reply.** Writes need synchronous validation (ownership, constraint violations, server-canonical fields). Removing the await from `create` / `update` / `delete` is a separate, harder design problem and is explicitly out of scope here. +- **The wasm request timeout stays.** It still applies to mutations and to any other request/reply call inside the wasm. This design simply removes scope-open from the set of callers. - **The events topic structure stays.** Same `${prefix}/${rootEntity}/${scopeId}/events/{type}` topics, same payload shape, same `x-origin-client-id` filter. Only the *bootstrap delivery* changes. ## 2. Wire-level flow ### 2.1 Client publishes a hello -When `replaceScope(scopeId)` is called and the client is connected, stitch publishes one message to a new topic: +When `replaceScope(scopeId)` is called and the client is connected, the wasm publishes one message to a new topic: ``` ${prefix}/${rootEntity}/${scopeId}/hello @@ -70,86 +80,89 @@ QoS 1, `retain=false`. No response topic, no correlation data — this is fire-a The server (MQDB or any compatible backend) subscribes to `${prefix}/+/+/hello` (or its tenant-scoped equivalent). On receiving a hello: -1. Validate the requesting client's auth and access to `scopeId` using existing `userScopeField` rules. If denied, publish a single error event on the client's response topic (`${responsePrefix}/${clientId}/scope-open-error`) and stop. The error event carries `{scopeId, code, message}`. (Mechanism mirrors today's 401/403 paths in `checkResponseAndAuth` at `sync-engine.ts:628`.) +1. Validate the requesting client's auth and access to `scopeId` using existing `userScopeField` rules. If denied, publish a single error event on the client's response topic (`${responseTopicPrefix}/{clientId}/scope-open-error`) and stop. The error event carries `{scopeId, code, message}`. (Mechanism mirrors today's 401/403 auth-failure paths in the wasm's remote-sync code.) 2. Diff the requesting client's manifest against server-side state for that scope: - For each record the server has that the client lacks, or has at a lower `_version`: publish a synthetic `events/created` (or `events/updated`) event on the scope topic carrying the full record. The `sender` field on these synthetic events is `'__server_replay__'` (a reserved value clients treat as non-self). - For each record in the client's manifest that the server has marked deleted: publish a synthetic `events/deleted` event. - For each record in the client's manifest that the server's state matches at the same `_version`: emit nothing. 3. Once the diff is fully published, the server is done. There is no "I'm finished" message — the client never asks for one. -The synthetic events are indistinguishable on the wire from real mutation events. They flow through `handleWatchMessage` (`sync-engine.ts:447`) and into `applyMutationToDb`, where the existing `_version` LWW compare resolves any conflict against locally-pending writes. +The synthetic events are indistinguishable on the wire from real mutation events. They flow through the wasm's remote-mutation handler and into its apply path, where the existing `_version` LWW compare resolves any conflict against locally-pending writes. ### 2.3 Client reacts -After publishing the hello, `replaceScope` returns. The client's events-topic subscription (already established in step 1 of `openScope`) delivers each replay event as it arrives. Each event is applied via the existing `applyMutationToDb` path — same code as a normal remote mutation. +After publishing the hello, `replaceScope` returns. The client's events-topic subscription (already established when the scope opened) delivers each replay event as it arrives. Each event is applied via the wasm's normal remote-mutation path — the same code as a live remote mutation. -There is no "scope is loaded" promise resolution. The UI populates as records stream in. Apps that need a "loaded" gate implement one themselves (e.g., observe MemoryStore for stability) — same recommendation as peer mode (§6.2 of `reactive-peer-mode.md`). +There is no "scope is loaded" promise resolution. The UI populates as records stream in. Because `subscribeToScope` / `subscribeToEntity` callbacks are delivered **asynchronously** (one tick after each event is applied — see the 0.5.0 behavioural change), the UI reflects each applied record a tick behind the apply, not synchronously. Apps that need a "loaded" gate implement one themselves (e.g., observe the memory snapshot for stability) — same recommendation as peer mode (§6.2 of `reactive-peer-mode.md`). ### 2.4 Concurrent mutations during replay -A peer (or the user themselves on another device) mutates a record while the server is mid-replay. The mutation event flows on the same scope events topic. The client receives both the replay event and the live mutation event in publish order, applies both via `applyMutationToDb`, and `_version` LWW resolves any overlap. No special handling needed. +A peer (or the user themselves on another device) mutates a record while the server is mid-replay. The mutation event flows on the same scope events topic. The client receives both the replay event and the live mutation event in publish order, applies both via the wasm's remote-mutation path, and `_version` LWW resolves any overlap. No special handling needed. -### 2.5 `bumpScopeVersion` goes away +### 2.5 The per-mutation version-bump round-trip goes away -Today every child mutation triggers a *second* request/reply (`sync-engine.ts:668, 685, 695`) to bump the root entity's version field. In the new flow: +Today every child mutation triggers a *second* request/reply inside the wasm to bump the root entity's version field. In the new flow: - The server bumps the scope version implicitly when it processes any child mutation. - The server emits an `events/updated` event for the root entity on the same topic, carrying the new `_version` (and `updatedAt`). - The client receives that event reactively. No round-trip. -This halves the per-mutation latency and removes a hidden second 10 s timeout from every awaited write. +This halves the per-mutation latency and removes a hidden second timeout from every awaited write. ## 3. API surface changes ### 3.1 `StoreConfig` -No new config field. The existing `syncMode` field from peer mode could in principle gain a `'mqdb-reactive'` value, but a cleaner cut is: +No new config field, and no reliance on an existing sync-mode selector — `StoreConfig` (`src/types.ts:21-38`) has no such field today. The peer-mode design (`reactive-peer-mode.md` §6.1 / §12) *proposes* introducing a `syncMode` field (`'mqdb'` vs `'peer'`); if that lands, this reactive server-backed flow becomes the behaviour of the `'mqdb'` mode: + +- This change replaces the classic MQDB scope-open semantics. Old MQDB servers that don't support hello are handled via capability negotiation (§5). +- The proposed `'peer'` mode is unchanged. -- This change replaces `syncMode: 'mqdb'` semantics. Old MQDB servers that don't support hello are handled via capability negotiation (§5). -- `syncMode: 'peer'` is unchanged. +Until `syncMode` exists, this design is simply the new behaviour of the single (MQDB-backed) mode, gated by server capability negotiation. ### 3.2 `replaceScope` semantics | Before | After | |---|---| -| Awaits `openScope` → server returns full ScopeState → reconcile + load → resolve. UI sees complete state on resolve. | Subscribes to events, publishes hello, returns immediately. UI starts with whatever's in local IndexedDB. State populates over time as server replay events arrive. | +| Awaits the wasm's batched bootstrap → server returns full state → wasm reconciles + loads → resolve. UI sees complete state on resolve. | Subscribes to events, publishes hello, returns immediately. UI starts with whatever's in local IndexedDB. State populates over time as server replay events arrive (each surfaced a tick later via the async subscription callbacks). | -This is a deliberate API contract change. App code that today does `await store.replaceScope(id); /* assume loaded */` must handle "still populating" UI states. Same trade-off and same guidance as peer mode. +The public signature is unchanged — `replaceScope(scopeId): Promise` already (`src/types.ts:140`). What changes is *when* the promise resolves and what the app can assume afterward. App code that today does `await store.replaceScope(id); /* assume loaded */` must handle "still populating" UI states. Same trade-off and same guidance as peer mode. ### 3.3 `request()` -Unchanged. Mutations still use it. The 10 s timeout still applies to mutations, `bumpScopeVersion` (now unused — see §2.5), and any other one-shot request/reply. +Unchanged. Mutations still use it. The per-request timeout still applies to mutations and any other one-shot request/reply — but it lives inside the wasm; in this repo `request()` (`src/store.ts:327`) is a one-line delegate. This design simply removes scope-open (and the per-mutation version-bump, §2.5) from the set of callers. ### 3.4 `fetchList` / `fetchOne` -No longer called by `openScope`. They remain available for ad-hoc queries that don't want to subscribe (rare today, may be removed entirely if no caller remains). +These are wasm-internal protocol requests, not public API of `@laboverwire/stitch`. Today the wasm uses them to bootstrap a scope. Under this design the wasm stops using them for scope-open. They may remain available inside the wasm for ad-hoc queries that don't want to subscribe (rare today, may be removed entirely if no caller remains). + +### 3.5 Return type + +The public `Store.replaceScope` already returns `Promise` (`src/types.ts:140`), so nothing in this package's type surface changes. The old TypeScript backend used internal snapshot types (`ScopeState` and friends) to assemble bootstrap state; those were removed from the public API in 0.5.0 and now exist only inside the wasm — there is nothing in `src/types.ts` to reshape for this design. The engine has nothing meaningful to return synchronously; state populates via the normal mutation-apply path. -### 3.5 `OpenScopeResult` +## 4. What changes, and where -`openScope` returns immediately. The current `ScopeState` shape (`root`, `children`, `version`, `bufferedMutations`) is replaced by `Promise` — the engine has nothing meaningful to return synchronously. State populates via the existing mutation handler. +The client-side changes land in `stitch-rs` (compiled into `@laboverwire/stitch-wasm`). The server-side changes land in MQDB. This package's TypeScript adapter is essentially untouched. -## 4. What changes in the existing code +### 4.1 Client sync engine (`stitch-rs` / `@laboverwire/stitch-wasm`) -### 4.1 `src/sync-engine.ts` +- **Scope-open**: drop the `Promise.all` of `fetchOne` + `fetchList` round-trips. Subscribe to the scope topic, publish a hello message, return. The buffer-around-snapshot machinery becomes unnecessary because there is no longer an "await snapshot" phase to buffer around. +- **Per-mutation version bump**: delete. The server handles version bumping implicitly (§2.5). `create` / `update` / `delete` drop the trailing version-bump request — one round-trip per mutation instead of two. +- **New**: handle a `scope-open-error` response on the response topic so scope-open auth failures still surface (as a rejected `replaceScope` promise or as an emitted store-level error event, depending on the chosen contract — see Q1). +- **Top-level discovery**: apply the same hello-based pattern at the top-level entity discovery topic. Open question — see §5. -- `openScope`: drop the `Promise.all` block (lines 576–589). Subscribe to the scope topic, publish a hello message, return. The `awaitingState` / `buffered` machinery (lines 568–570, 595–610) becomes unnecessary because there is no longer an "await snapshot" phase to buffer around. -- `bumpScopeVersion`: delete. The server handles version bumping implicitly. -- `createEntity` / `updateEntity` / `deleteEntity`: drop the trailing `await this.bumpScopeVersion(scopeId)` call (lines 668, 685, 695). One round-trip per mutation instead of two. -- New: handle a `scope-open-error` response on the response topic so that scope-open auth failures still surface as a rejected promise on `replaceScope` (or as an emitted error event on the store, depending on the chosen contract — see Q1). -- `subscribeToTopLevel`: same hello-based pattern applied at the top-level entity discovery topic (`${prefix}/${rootEntity}/+/hello` for the server's perspective, `${prefix}/${rootEntity}/discovery/hello` or similar for the client). Open question — see §5. +Estimated diff: net negative LOC. The subscribe-and-replay path is mostly *removal* of the buffering / `Promise.all` logic. -Estimated diff: net negative LOC. The subscribe-and-replay path is mostly *removal* of the buffering / Promise.all logic. +### 4.2 Client remote-mutation apply (`stitch-rs`) -### 4.2 `src/remote-sync-layer.ts` +- **Apply path**: unchanged. Replay events are treated identically to live remote mutations. The `sender === '__server_replay__'` value is not own-mutation, so the existing self-filter already accepts it. +- **Snapshot reconcile**: the batched-snapshot reconcile step — "delete locally records the server doesn't have" — is removed. That job still needs to happen, but it now happens reactively as the client receives `events/deleted` for those records during replay. -- `openScope` callers: no longer receive a fully-populated `ScopeState`. Reconciliation work currently in `replaceScope` (lines 504–545) — `reconcileChildren`, the `bufferedMutations` drain — moves into the streaming path. Each replay event flows through `applyMutationToDb` like any other remote mutation. -- `applyMutationToDb`: unchanged. Treats replay events identically to live mutations. The `sender === '__server_replay__'` value is not own-mutation; the existing filter at `isOwnMutation` (sync-engine.ts:417) already accepts it. -- `reconcileChildren`: only meaningful in the old batched-snapshot flow. Likely deletable. The same job — "delete locally records the server doesn't have" — still needs to happen, but it now happens reactively as the client receives `events/deleted` for those records during replay. +### 4.3 TypeScript binding layer (`@laboverwire/stitch`) -### 4.3 `src/store.ts` +Essentially no change. `replaceScope` (`src/store.ts:245`) already delegates to `this.#inner.replaceScope(scopeId)` and already returns `Promise`; the new semantics (resolves once the subscription is established, state streams in afterward) are entirely a property of the wasm implementation behind that delegating call. `src/types.ts` needs no edit — `Store.replaceScope` is already `Promise` and the internal snapshot types are already gone. -- `replaceScope` (line 484): the long path from line 502 onward (snapshot reconciliation, `loadScopeFromPersistence`, `loadRootIntoMemory`) collapses to: ensure subscription is established, ensure local IndexedDB state is loaded into MemoryStore, return. State updates flow through the existing mutation handler. -- The `setSuppressNotifications(true)` block (line 499) becomes unnecessary — there is no longer a batched-load-then-notify cycle to suppress around. +The only change that would touch this repo is optional: if the contract for surfacing scope-open auth failures is a store-level error event (§5 Q1), that adds one method to the `Store` interface here plus a one-line forward in the adapter. Otherwise this package is untouched. ### 4.4 Server (MQDB) @@ -159,7 +172,7 @@ Headline list (full breakdown in §4.4.1): - Subscribe to `${prefix}/+/+/hello` and implement the manifest diff + replay logic in §2.2. - Emit per-record events on the scope topic instead of replying to `fetchList` / `fetchOne` requests. -- Emit version-bump events on child mutations (replaces the per-mutation client `bumpScopeVersion` round-trip). +- Emit version-bump events on child mutations (replaces the per-mutation client version-bump round-trip). - Surface auth failures via `scope-open-error` events on the response topic. - Retain tombstones for deleted records so deletions can be replayed. - Advertise protocol version on CONNACK so clients can branch. @@ -172,7 +185,7 @@ For a fresh reader: this is the complete server-facing contract. Each item is so **New protocol handlers** - [ ] **Subscribe to `${prefix}/+/+/hello`** (per-tenant equivalent acceptable). Source: §2.1. -- [ ] **On hello: validate auth.** Use existing `userScopeField` rules. On failure, publish `{scopeId, code, message}` to `${responsePrefix}/${clientId}/scope-open-error`. Do not publish replay events. Source: §2.2 step 1. +- [ ] **On hello: validate auth.** Use existing `userScopeField` rules. On failure, publish `{scopeId, code, message}` to `${responseTopicPrefix}/{clientId}/scope-open-error`. Do not publish replay events. Source: §2.2 step 1. - [ ] **On hello: compute manifest diff.** For each record server-side: if absent from client manifest or at lower `_version`, queue for replay. For each record in client manifest: if server has marked deleted (tombstone) at any version newer than client's, queue a delete-replay. If versions match, emit nothing. Source: §2.2 step 2. - [ ] **Replay is stateless.** No per-client progress tracking. If the client reconnects mid-replay and republishes hello, the server replays the diff from scratch. Source: §2.4 (implicit; surface explicitly). @@ -180,7 +193,7 @@ For a fresh reader: this is the complete server-facing contract. Each item is so - [ ] **Synthetic replay events use real `events/{type}` topics** — not a per-client replay channel. They are indistinguishable on the wire from live mutation events, so existing subscribers see them too and apply them via LWW (idempotent — `_version` matches → no-op). The fan-out cost to existing peers is accepted as the price of keeping a single channel. Source: §2.2 step 2 (implicit). - [ ] **Synthetic events carry `sender: '__server_replay__'`** as an MQTT user property (and/or payload field, matching the existing `sender` shape). This reserved value is not "own mutation" for any client, so all clients receive and apply it. The server is responsible for setting it. Source: §2.2 step 2 + §4.2. -- [ ] **Emit version-bump events on child mutations.** When the server processes a child create / update / delete, it bumps the root's `_version` (and `updatedAt`) and publishes an `events/updated` for the root entity carrying the new version. Replaces the client-driven `bumpScopeVersion` round-trip. Source: §2.5. +- [ ] **Emit version-bump events on child mutations.** When the server processes a child create / update / delete, it bumps the root's `_version` (and `updatedAt`) and publishes an `events/updated` for the root entity carrying the new version. Replaces the client-driven version-bump round-trip. Source: §2.5. - [ ] **Emit per-record events instead of (or in addition to) batched `fetchList` / `fetchOne` replies.** During the deprecation window (§8 Phase 4), both paths can coexist behind capability negotiation. After the window, `fetchList` / `fetchOne` handlers are removed. Source: §3.4 + §8. **New persistence requirements** @@ -197,7 +210,7 @@ For a fresh reader: this is the complete server-facing contract. Each item is so **Deprecation track** -- [ ] **Phase 4 deletes legacy handlers.** Once capability negotiation has been live for one major version and telemetry confirms no clients are falling back, the server's `fetchList`, `fetchOne`, and (once `bumpScopeVersion` is removed client-side) the `bumpScopeVersion` request handlers can be deleted. Source: §8 Phase 4–5. +- [ ] **Phase 4 deletes legacy handlers.** Once capability negotiation has been live for one major version and telemetry confirms no clients are falling back, the server's `fetchList`, `fetchOne`, and (once the client version-bump is removed) the version-bump request handlers can be deleted. Source: §8 Phase 4–5. **Out of scope for v1** @@ -207,11 +220,11 @@ For a fresh reader: this is the complete server-facing contract. Each item is so ### Q1: Auth failure surfacing -Today `openScope` rejects with the server's 401/403 reply. In the reactive flow, `replaceScope` has already returned by the time the server validates the hello. +Today scope-open rejects with the server's 401/403 reply. In the reactive flow, `replaceScope` has already returned by the time the server validates the hello. Options: -- **Reject `replaceScope` only on transport errors; surface auth failures via a store-level event** (e.g. `store.onScopeError(cb)`). Apps that care register a listener. +- **Reject `replaceScope` only on transport errors; surface auth failures via a store-level event** (e.g. `store.onScopeError(cb)`). Apps that care register a listener. This is the one change that would touch the TS adapter (§4.3). - **Hold `replaceScope`'s promise open until either the first replay event or an error arrives, with a short timeout** — sneaks a timer back in. Reject. - **Pre-validate auth at connect time** — server publishes a list of accessible scopes on a per-client topic at session start; `replaceScope` rejects synchronously if `scopeId` is not in that list. Adds a connect-time round-trip but no per-scope-open round-trip. @@ -233,16 +246,16 @@ Recommendation: capability negotiation. The fallback path keeps the old code int `syncRootEntityList` today iterates all root entities a user can access via `fetchList(rootEntity)` with `userScopeField` filtering on the server. Same problem at this level. -Recommendation: same hello mechanism at the discovery topic. Client publishes a discovery hello with whatever roots it already has locally; server replays the delta. Out-of-scope details mirror peer mode's §10 (top-level entities). +Recommendation: same hello mechanism at the discovery topic. Client publishes a discovery hello with whatever roots it already has locally; server replays the delta. Out-of-scope details mirror the top-level-entity discussion in `reactive-peer-mode.md`. ### Q4: Replay ordering -The server publishes replay events in some order. If a child references a root that hasn't replayed yet, the client may briefly see a child without its root in MemoryStore. +The server publishes replay events in some order. If a child references a root that hasn't replayed yet, the client may briefly see a child without its root in the memory cache. Options: - **Server publishes root first, then children** — natural ordering. Documented contract. -- **Client tolerates out-of-order** — already partially true (the existing mutation handler doesn't enforce parent-before-child). Document as "same guarantees as live mutations: eventual consistency, brief inconsistency tolerated." +- **Client tolerates out-of-order** — already partially true (the mutation handler doesn't enforce parent-before-child). Document as "same guarantees as live mutations: eventual consistency, brief inconsistency tolerated." Recommendation: document the second. Mirrors normal mutation behavior; no special handling. @@ -263,21 +276,21 @@ Recommendation: accept for v1. The manifest diff keeps the common case (returnin - **Reactive mutations.** Writes still use request/reply. Surfacing write validation results without an await is a separate, harder problem. - **Offline scope-open.** If the client is disconnected, `replaceScope` falls back to whatever's in local IndexedDB. Same as today. Reconnect republishes the hello. - **Cross-scope queries.** Same as today; out of scope. -- **Strong consistency on scope-open.** The reactive flow is eventually consistent. Apps requiring "see all data before render" need an app-level loaded-gate (observe MemoryStore for stability). +- **Strong consistency on scope-open.** The reactive flow is eventually consistent. Apps requiring "see all data before render" need an app-level loaded-gate (observe the memory snapshot for stability). Note that subscription callbacks are async (one tick behind each applied event), so any such gate must poll/observe rather than assume synchronous delivery. ## 7. Estimated scope -Rough LOC budget for the client-side change: +Rough LOC budget. The concrete, line-level budget belongs to the `stitch-rs` repo, not this package; the numbers below are indicative. -| File | Net change | +| Component | Net change | |---|---| -| `src/sync-engine.ts` | -150 (remove Promise.all + buffering + bumpScopeVersion) | -| `src/remote-sync-layer.ts` | -100 (remove reconcileChildren snapshot path) | -| `src/store.ts` | -60 (collapse replaceScope reconcile path) | -| `src/types.ts` | ±10 (ScopeState removal, OpenScopeResult shape) | -| Tests | +400 (new replay tests, capability fallback tests) | +| `stitch-rs` scope-open / sync module | −250 (remove `Promise.all` + buffering + client version-bump) | +| `stitch-rs` reconcile path | −100 (remove batched-snapshot reconcile) | +| `@laboverwire/stitch` TS adapter (`src/store.ts`) | ~0 — `replaceScope` already delegates and returns `Promise` | +| `@laboverwire/stitch` types (`src/types.ts`) | 0 — `Store.replaceScope` already `Promise`; internal snapshot types already removed in 0.5.0 | +| Tests (`stitch-rs` unit + this repo's browser suite) | +400 (new replay tests, capability fallback tests) | -Net: roughly LOC-neutral but structurally simpler. +Net: roughly LOC-neutral but structurally simpler. Almost all of it is in `stitch-rs`. Server-side scope is **not estimated here** — depends on MQDB's existing architecture and whoever owns it. This document captures only the protocol contract. @@ -285,8 +298,8 @@ Server-side scope is **not estimated here** — depends on MQDB's existing archi 1. **Phase 0**: this doc + agreement with MQDB owner on protocol. 2. **Phase 1**: server implements hello handling and per-record replay events behind a feature flag. Existing `fetchList` path stays in parallel. -3. **Phase 2**: capability negotiation. Stitch detects support and routes scope-open through the new path when available. -4. **Phase 3**: `bumpScopeVersion` deleted from stitch; server handles version bump implicitly. +3. **Phase 2**: capability negotiation. The wasm detects support and routes scope-open through the new path when available. +4. **Phase 3**: the client-side per-mutation version-bump is deleted from the wasm; server handles version bump implicitly. 5. **Phase 4**: deprecation window for old `fetchList`-based scope-open. Eventually removed. 6. **Phase 5**: top-level entity discovery moved to same hello pattern (§ Q3). @@ -310,6 +323,6 @@ The differences are entirely upstream of the events topic: | Auth validation | Server-side at hello + per-mutation reply | None (each peer trusts its own state) | | Mutation channel | Request/reply (this design preserves it) | Fire-and-forget events | -Once both ship, the events-topic stream is identical from the client's perspective in both modes. The mode flag (`syncMode`) controls only the source-of-truth and versioning concerns spelled out in §12 of `reactive-peer-mode.md`. +Once both ship, the events-topic stream is identical from the client's perspective in both modes. The proposed `syncMode` flag (`reactive-peer-mode.md` §12) would control only the source-of-truth and versioning concerns spelled out there. This is the practical argument for sequencing: ship reactive scope-open first, then peer mode reuses the streaming infrastructure with only the source-of-truth and versioning differences to design. diff --git a/docs/react.md b/docs/react.md index 21ce93d..be98572 100644 --- a/docs/react.md +++ b/docs/react.md @@ -4,9 +4,23 @@ npm install react@^19.0.0 ``` +The store backend ships as WebAssembly (`@laboverwire/stitch-wasm`), so a consuming +app's Vite config must enable `vite-plugin-wasm` and `vite-plugin-top-level-await`: + +```ts +import wasm from 'vite-plugin-wasm'; +import topLevelAwait from 'vite-plugin-top-level-await'; + +export default defineConfig({ + plugins: [react(), wasm(), topLevelAwait()], + build: { target: 'esnext' }, +}); +``` + ## Quick start ```tsx +import { useEffect } from 'react'; import { createStore } from '@laboverwire/stitch'; import type { StoreConfig } from '@laboverwire/stitch'; import { @@ -47,9 +61,11 @@ const config: StoreConfig = { }, }; +const authTicket = ''; + const store = createStore(config, { persistence: { dbName: 'my-app' }, - remote: { serverUrl: 'wss://mqtt.example.com', getTicket: () => fetchAuthTicket() }, + remote: { url: 'wss://mqtt.example.com', ticket: authTicket }, }); function App() { @@ -65,7 +81,7 @@ function App() { function ProjectView({ scopeId }: { scopeId: string }) { const { store } = useStore(); const { syncing, openScope } = useSyncScope(store, scopeId); - const tasks = useEntitySnapshot(store, scopeId, 'task'); + const tasks = useEntitySnapshot(store, scopeId, 'task') as Task[]; useEffect(() => { void openScope(); }, [openScope]); @@ -78,7 +94,24 @@ function ProjectView({ scopeId }: { scopeId: string }) { } ``` -Because `createStore()` is generic, `tasks` above is typed as `Task[]` — no `as string` casts per field. +The schema generic on `createStore()` types the direct `Store` methods +(`read`, `getSnapshot`, `list`, etc.), but it does **not** reach the snapshot hooks. The +entity-snapshot hooks are not schema-generic: `useEntitySnapshot(store, scopeId, 'task')` +returns `Record[]` regardless of the store's schema, and `store` from +`useStore()` is a plain non-generic `Store`. Cast the hook result to your app's record type +(e.g. `as Task[]`) when you need field-level typing, as in the example above. + +`remote.ticket` is a JWT for MQTT v5 enhanced auth; use `remote.username`/`remote.password` +for classic MQTT password auth instead. `remote.url` is a `ws://`/`wss://` endpoint. Passing +`persistence.passphrase` enables AES-GCM encryption of the local database. + +## Pre-init tolerance + +Hooks are safe to mount before the store finishes initializing. `` calls +`store.initialize()` on mount, but the adapter tolerates access before that resolves: +synchronous reads return empties (`[]` / `{}` / `null` / `0` / `'offline'`), and `subscribe*` +calls defer wiring until initialization completes and then trigger a re-read. You don't need +to gate hooks behind an "initialized" flag. ## Providers @@ -103,6 +136,10 @@ Compose them; nest `` inside ``: ``` +The optional `serverUrl`/`getTicket` props on `` drive reconnect-on-wake: +after the tab has been hidden past the stale threshold, the provider calls +`store.reconnect(serverUrl, getTicket)`. + ## Hooks | Hook | Description | diff --git a/docs/upgrading-to-0.5.md b/docs/upgrading-to-0.5.md new file mode 100644 index 0000000..0f4a591 --- /dev/null +++ b/docs/upgrading-to-0.5.md @@ -0,0 +1,151 @@ +# Upgrading to 0.5.0 + +0.5.0 moves the entire store engine — the in-memory store, IndexedDB persistence, +MQTT sync, offline queue, and reconciliation — into the Rust/WASM package +[`@laboverwire/stitch-wasm`](https://www.npmjs.com/package/@laboverwire/stitch-wasm). +`@laboverwire/stitch` is now a thin binding layer over it. + +This is a breaking release. The two changes that affect **every** app are the Vite +plugins (step 1) and the `remote` option shape (step 2); the rest apply only if you +used the specific API involved. `createStore(config, options)`, the `StoreConfig` +shape, the scope model, and the entire React/Vue binding surface +(`StoreProvider`/`AuthProvider` + hooks, `StoreRoot`/`StitchAuth` + composables) +are unchanged. + +## 1. Add the WASM Vite plugins (required) + +`stitch-wasm` ships as a `wasm-bindgen` bundler-target ESM module, so your bundler +must load a WASM ESM import. Install the two plugins and set the build target: + +```bash +npm install -D vite-plugin-wasm vite-plugin-top-level-await +``` + +```ts +// vite.config.ts +import { defineConfig } from 'vite'; +import wasm from 'vite-plugin-wasm'; +import topLevelAwait from 'vite-plugin-top-level-await'; + +export default defineConfig({ + plugins: [wasm(), topLevelAwait()], + build: { target: 'esnext' }, // the wasm module + plugin emit top-level await +}); +``` + +If you previously excluded or filesystem-allowed the old WASM deps, **remove** that: + +```ts +// delete these — they no longer apply: +optimizeDeps: { exclude: ['mqdb-wasm', 'mqtt5-wasm'] }, +``` + +See [Vite consumer guide](./vite-consumer.md) for the source-alias/monorepo variant. + +## 2. Update the `remote` option shape + +The `remote` block changed, and its auth field changed from a *function* to a +*resolved value*: + +```ts +// Before (0.4.x) +createStore(config, { + persistence: { dbName: 'my-app' }, + remote: { + serverUrl: 'wss://mqtt.example.com', + getTicket: async () => fetchAuthTicket(), // a function + }, +}); + +// After (0.5.0) +createStore(config, { + persistence: { dbName: 'my-app', passphrase }, // passphrase optional (AES-GCM) + remote: { + url: 'wss://mqtt.example.com', + ticket: await fetchAuthTicket(), // a resolved JWT string + // or: username, password for classic MQTT auth + }, +}); +``` + +Resolve the ticket **before** calling `createStore`. Ticket *refresh* on +wake-from-background is unchanged: it still flows through the provider's +`getTicket` prop (`` / +``), and `store.reconnect(serverUrl, getTicket?)` +keeps its signature. + +## 3. Replace `OwnershipError` / `MqdbError` handling + +Both classes were removed. Error checks that used `instanceof` must inspect the +error another way: + +```ts +// Before +try { await store.update(entity, id, fields); } +catch (e) { if (e instanceof OwnershipError) redirectToLogin(); } + +// After — match on the error message/shape surfaced by the wasm store +try { await store.update(entity, id, fields); } +catch (e) { if (String((e as Error)?.message).includes('ownership')) redirectToLogin(); } +``` + +## 4. Reimplement session/auth-cache helpers in your app + +These six `Store` methods were removed: `getCachedUser`, `setCachedUser`, +`clearCachedUser`, `hasPendingLogout`, `setPendingLogout`, `flushPendingLogout`. +They were thin `sessionStorage` wrappers. If you used them, move the caching into +your own app code: + +```ts +// Example replacement for the cached-user pair +const CACHE_KEY = 'my-app-cached-user'; +const getCachedUser = () => JSON.parse(sessionStorage.getItem(CACHE_KEY) ?? 'null'); +const setCachedUser = (u: unknown) => sessionStorage.setItem(CACHE_KEY, JSON.stringify(u)); +``` + +## 5. Drop internal factory and layer-type imports + +Only `createStore` is exported now. Remove any imports of the internal factories +or layer interfaces: + +```ts +// removed — delete these imports +import { + createMemoryStore, createSyncEngine, createPersistenceLayer, + createRemoteSyncLayer, createPersistentOfflineQueue, createInMemoryOfflineQueue, +} from '@laboverwire/stitch'; +import type { + SyncEngine, PersistenceLayer, RemoteSyncLayer, OfflineQueue, + MutationSender, LocalAccessor, PendingMutation, ConsolidatedMutation, + ScopeBundle, ScopeState, SyncMutation, MutationEvent, +} from '@laboverwire/stitch'; +``` + +The exported type surface is now: `StoreConfig`, `EntityDefinition`, `SchemaField`, +`ForeignKeyDefinition`, `ConnectionStatus`, `SortField`, `SortDirection`, +`ListFilter`, `Store`, `StoreOptions`, `PersistenceConfig`, `RemoteConfig`, +`MemoryStore`, `EntitySchema`, `DefaultSchema`, `EntityKey`, `OriginTag`. The +`MemoryStore` type is trimmed to `{ getSnapshot, getSnapshotAsMap, subscribeToScope }`. + +## 6. Account for behavioural changes + +- **Subscription callbacks are now asynchronous.** `subscribeToEntity` / + `subscribeToScope` callbacks fire one tick after the mutating call resolves, not + synchronously. Code that assumed a callback had already run by the time + `create` / `update` / `delete` returned must `await` a tick (or react to the + callback) instead. +- **A few signatures changed shape.** `getChildCount` is now synchronous (returns + `number`); `disconnect`, `resetForLogout`, `loadScope`, and `clearScope` now + return `Promise`. These are usually source-compatible (`await` on a + non-promise is a no-op; calling an async method without `await` still works), but + the TypeScript types changed. `connectionStatus` is unchanged — still the + lowercase union `'connected' | 'connecting' | 'disconnected' | 'error' | 'offline'`. + +## 7. Remove direct `mqdb-wasm` / `mqtt5-wasm` dependencies + +If your `package.json` listed either directly, remove them — they are bundled +inside `@laboverwire/stitch-wasm` and no longer installed transitively. + +--- + +For the full list of changes, see the [0.5.0 changelog entry](../CHANGELOG.md). diff --git a/docs/vite-consumer.md b/docs/vite-consumer.md index 6ef2161..ac1a3f7 100644 --- a/docs/vite-consumer.md +++ b/docs/vite-consumer.md @@ -1,16 +1,42 @@ -# Using from a Vite consumer (source alias) +# Using from a Vite consumer -When you alias `@laboverwire/stitch` to the source (e.g. from a monorepo sibling or a vendored checkout) **and** the `mqdb-wasm` / `mqtt5-wasm` `node_modules` folder lives above your Vite project root, add that folder to `server.fs.allow`: +`@laboverwire/stitch` is a thin binding layer over `@laboverwire/stitch-wasm`, which ships as a `wasm-bindgen` bundler-target ESM module (it uses the ESM wasm import proposal). To load it, a consuming Vite app **must** add `vite-plugin-wasm` and `vite-plugin-top-level-await`: + +```bash +npm install -D vite-plugin-wasm vite-plugin-top-level-await +``` + +```ts +// vite.config.ts +import { defineConfig } from 'vite'; +import wasm from 'vite-plugin-wasm'; +import topLevelAwait from 'vite-plugin-top-level-await'; + +export default defineConfig({ + plugins: [wasm(), topLevelAwait()], + build: { target: 'esnext' }, +}); +``` + +This mirrors what this repo's own `vitest.config.ts` does — the browser test suite loads the real wasm through the same two plugins. Without them the wasm module fails to instantiate. `build.target: 'esnext'` is required for production builds: the wasm-bindgen module and `vite-plugin-top-level-await` emit top-level `await`, which older build targets can't down-level. + +## Source alias (monorepo sibling or vendored checkout) + +When you alias `@laboverwire/stitch` to the source (e.g. from a monorepo sibling or a vendored checkout) **and** the `@laboverwire/stitch-wasm` `node_modules` folder lives above your Vite project root, also add that folder to `server.fs.allow` so the `.wasm` binary resolves: ```ts // vite.config.ts import { defineConfig } from 'vite'; import { fileURLToPath } from 'node:url'; import { dirname, resolve } from 'node:path'; +import wasm from 'vite-plugin-wasm'; +import topLevelAwait from 'vite-plugin-top-level-await'; const here = dirname(fileURLToPath(import.meta.url)); export default defineConfig({ + plugins: [wasm(), topLevelAwait()], + build: { target: 'esnext' }, resolve: { alias: { '@laboverwire/stitch': resolve(here, '../path/to/stitch/src/index.ts'), @@ -19,10 +45,7 @@ export default defineConfig({ server: { fs: { allow: [resolve(here, '../path/to/stitch')] }, }, - optimizeDeps: { - exclude: ['mqdb-wasm', 'mqtt5-wasm'], - }, }); ``` -Without `fs.allow`, Vite serves WASM binaries with HTTP 403 and `WebAssembly.instantiateStreaming` fails. If you install `@laboverwire/stitch` as a normal npm dependency instead, neither the alias nor the `fs.allow` entry is needed. +If you install `@laboverwire/stitch` as a normal npm dependency instead, the alias and the `fs.allow` entry are not needed — but the `vite-plugin-wasm` / `vite-plugin-top-level-await` plugins are required either way. diff --git a/docs/vue.md b/docs/vue.md index 6259d35..7fa9470 100644 --- a/docs/vue.md +++ b/docs/vue.md @@ -16,6 +16,8 @@ interface Project { id: string; name: string } interface Task { id: string; projectId: string; title: string; done: boolean } type Schema = { project: Project; task: Task }; +const authTicket = ''; + const config: StoreConfig = { entities: { project: { @@ -44,7 +46,7 @@ const config: StoreConfig = { const store = createStore(config, { persistence: { dbName: 'my-app' }, - remote: { serverUrl: 'wss://mqtt.example.com', getTicket: () => fetchAuthTicket() }, + remote: { url: 'wss://mqtt.example.com', ticket: authTicket }, }); @@ -57,6 +59,11 @@ const store = createStore(config, { ``` +`StoreOptions` takes two optional blocks: + +- `persistence: { dbName, passphrase? }` — IndexedDB persistence; a `passphrase` turns on AES-GCM encryption at rest. +- `remote: { url, clientId?, ticket?, username?, password? }` — `url` is a `ws://`/`wss://` MQTT endpoint. `ticket` is a JWT for MQTT v5 enhanced auth; supply `username`/`password` instead for classic MQTT password auth. + ```vue