Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
9079c4f
feat(model): rename node kind project -> module (types, kind strings,…
Yona-Appletree Aug 1, 2026
2386287
refactor: sweep .project tree-path literals to .module
Yona-Appletree Aug 1, 2026
53f2e00
test: move "kind": "Project" fixtures and corpora to "Module"
Yona-Appletree Aug 1, 2026
3c8cbcb
chore(schema): regenerate for the module rename; icon/copy stragglers
Yona-Appletree Aug 1, 2026
a86b238
test: catch escaped-quote fixtures and manifest-kind strings missed b…
Yona-Appletree Aug 1, 2026
f4bc2ba
style: rustfmt after the rename sweeps
Yona-Appletree Aug 1, 2026
fe7426a
feat(model): project/module mitosis — container manifest, format gate…
Yona-Appletree Aug 1, 2026
82bef67
test(app): migrate server/link/studio/fw/cli fixtures to the mitosis …
Yona-Appletree Aug 1, 2026
1d64fc4
feat(studio): manifest-backed settings, rename patches the manifest; …
Yona-Appletree Aug 1, 2026
5a50dbd
feat(model): ModuleDef capability — bindings, exports, channel meta, …
Yona-Appletree Aug 1, 2026
3f1c90e
feat(engine): structural scope on RuntimeNodeEntry; delete dead Bus r…
Yona-Appletree Aug 1, 2026
7909f90
feat(engine): scoped channels — writer-shadowing resolution, suppress…
Yona-Appletree Aug 1, 2026
a0f1b33
feat(engine): real ModuleNode, R7 output interface, root de-specialis…
Yona-Appletree Aug 1, 2026
a0d7e47
docs: scoped-bus engine architecture ADR; R7 implemented note
Yona-Appletree Aug 1, 2026
33dd26f
ci: discard build-regenerated tailwind.css before the baseline auto-c…
Yona-Appletree Aug 1, 2026
30e45bb
Merge remote-tracking branch 'origin/main' into claude/yona-implement…
Yona-Appletree Aug 1, 2026
f7e4767
merge: origin/main (firmware-manifest, xt fixes); rename new NodeKind…
Yona-Appletree Aug 1, 2026
851127a
chore(studio): auto-refresh story baselines [validate-stories]
github-actions[bot] Aug 1, 2026
8c3a95b
feat(wire,engine,studio): structured scope on the probe surface (P7)
Yona-Appletree Aug 2, 2026
011ef33
feat(engine): panel writer store — lazy latching runtime writers (P8)
Yona-Appletree Aug 2, 2026
f0ae027
feat(panel): PanelWrite/PanelClear command channel + re-point panel c…
Yona-Appletree Aug 2, 2026
2185b13
feat(panel): persist latched panel state to /.lp/panel.json
Yona-Appletree Aug 2, 2026
7ada405
merge: origin/main into the modules roadmap branch
Yona-Appletree Aug 2, 2026
177dd4d
merge: origin/main (debug-slots P2-P6) into the modules roadmap branch
Yona-Appletree Aug 2, 2026
cad5a7d
fix(fw): declare wire proto 7 in the embedded manifest expectations
Yona-Appletree Aug 2, 2026
9508e6f
chore(studio): auto-refresh story baselines [validate-stories]
github-actions[bot] Aug 2, 2026
9dcb935
docs/debt: CI resolved the lps-probe perf question — load, not regres…
Yona-Appletree Aug 2, 2026
2b87b11
Merge remote-tracking branch 'origin/main' into claude/yona-implement…
Yona-Appletree Aug 3, 2026
d3010c5
docs: refresh the panel-state flash numbers after main's abort tier
Yona-Appletree Aug 3, 2026
3ab935e
docs/defects: fw-browser smoke check throws on its first poll
Yona-Appletree Aug 3, 2026
c0986c5
Merge remote-tracking branch 'origin/main' into claude/yona-implement…
Yona-Appletree Aug 3, 2026
989c874
chore(studio): auto-refresh story baselines [validate-stories]
github-actions[bot] Aug 3, 2026
1a9eb9c
test(panel): pin the authored-binding path a panel control depends on
Yona-Appletree Aug 3, 2026
107d51e
feat(examples): make fyeah-sign's Glow a real panel control
Yona-Appletree Aug 3, 2026
51ef8d6
chore(studio): auto-refresh story baselines [validate-stories]
github-actions[bot] Aug 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Project/module mitosis: container manifest and the relocated format gate

- Status: accepted
- Date: 2026-08-01
- Context: `docs/design/modules.md` §1/§6 (ratified 2026-07-31);
implementation plan `planning/2026-08-01-1003-modules-impl-roadmap` P2.

## Decision

The authored project root splits into two files ("mitosis"):

- **`project.json` — the container manifest, NOT a node.** Exactly the
workspace identity: `format`, optional `uid`, optional `name` (provenance
joins it in a later phase). No `kind` tag, no node envelope, unknown keys
rejected. Modeled as `lpc_model::ProjectManifest`: read by a streaming
`JsonSyntaxSource` probe, written by a hand-rolled deterministic writer.
It is deliberately **not** a `#[derive(Slotted)]` type — a second
shape+codec surface would link into every firmware image for three
fields, and serde surface is the flash lever.
- **`module.json` — the root module node** (`kind: "Module"`, `nodes`
map). `ModuleDef` loses `format`/`uid`/`name` to the container; carrying
them in a module artifact is now a parse error, so pre-mitosis roots
fail loudly instead of silently dropping identity fields.

**Format gate (settled D-A).** `PROJECT_FORMAT_VERSION` bumps 2 → 3
(bump-and-refuse; v2 schemas snapshotted to `schemas/history/v2/`). The
gate moves to the container: `ProjectRegistry::load_root` reads
`/project.json` through the streaming probe before anything parses — one
code path on host, browser, and device, proven on the emulated-firmware
path. A **missing or malformed container manifest is a hard refuse**,
never a skip: the manifest carries the gate, so the old
skip-on-malformed fallthrough would let unversioned projects load
ungated. Devices receive both files; the deploy path copies the whole
folder, so no firmware-side special case exists.

**Vendored module folders carry no format** (Q10 settled): a module
folder inside a project is gated by the host project's container; the
loader never re-runs the gate for child artifacts. Standalone opening of
a bare module folder wraps it in a workbench project and assumes the
current format (alpha posture).

**Schemas.** `project.schema.json` becomes the closed container schema
(`additionalProperties: false`, `format` const-pinned); a new
`module.schema.json` is the single-variant kind-tagged envelope over
`ModuleDef`; the conformance walk routes by filename.

**Studio.** The library's `package_manifest` rewrites on
`ProjectManifest` (read→modify→write is lossless because the vocabulary
is closed). The gallery rename now also patches the manifest `name` —
post-mitosis the manifest is library-owned workspace metadata, never an
authored def slot, so rename lives where the identity lives. The project
popup's settings rows render read-only from the manifest; the root def
contributes only its `nodes` count. Blank Created packages write a
minimal `module.json` so they stay loadable — gated to blank creates so
device pulls stay byte-faithful and adoption parity hashes cannot
diverge.

## Rejected alternatives

- **Deploy-time-only gating** (validate format when Studio pushes, skip
on device): loses the on-device refusal for projects that arrive by
other means (copied SD contents, partial syncs), and splits the gate
into two implementations. The streaming container probe costs the
device one tiny file read.
- **Slotted container type**: uniform codec machinery, but drags a second
shape and its serializer into every firmware image for three fields.
- **Keeping identity fields on the root module def**: the design's whole
point is that workspace concerns (who/when/what version) are not part
of the module's technical spec; it also kept the "Studio lets you
retype your project's uid" class of bugs structurally possible.

## Consequences

- Pre-mitosis projects (format 2 and earlier) refuse with a clear
format/manifest diagnostic; there is no migration (alpha posture).
- `read_module_format_json`/`ModuleFormatProbe` are gone; the manifest
probe is the single format authority.
- The persisted-state and panel phases (P8+) get a stable, non-authored
home for container-adjacent state (`.lp/`), and P3 adds provenance to
the manifest vocabulary.
83 changes: 83 additions & 0 deletions docs/adr/2026-08-01-scoped-bus-engine-architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Scoped-bus engine architecture: structural scopes, scoped resolver keys, module runtime

- Status: accepted
- Date: 2026-08-01
- Context: implements docs/design/modules.md R1–R7 (ratified 2026-07-31);
supersedes the two ADRs on the closed #218 spike branch —
`2026-07-28-scoped-buses.md` and `2026-07-28-effects-are-projects.md`
(branch `claude/composite-effects-planning-f4f51b`, kept as a harvest
source). The spike's *model* survived review; this records how the
restructured implementation differs.
- Plan: `planning/2026-08-01-1003-modules-impl-roadmap` P4–P6.

## Decision

**Scope is structural engine state, not a load-time side table.**
`RuntimeNodeEntry` carries the scope a node inhabits
(`ScopeRef::Module { owner }` / `ScopeRef::Sink { owner, entry }`) plus an
introduces-scope bit, assigned inside `ensure_runtime_spine` — the single
code path both fresh load and `apply_project_changes` run — so an edited
project can never wear different scopes than a reloaded one (pinned by a
load-vs-apply differential test). `Pending`/`Failed` entries carry scope
too (R1: the engine always answers), and payload reattach never touches
it. The spike's `BusScopes` table, built and dropped inside one loader
function, is the rejected alternative.

**The resolver stays scope-dumb; scope arrives as a richer key.**
`QueryKey::Bus` is `{ scope, channel }`: the reading node's scope is part
of the cache and cycle-detection identity, so same-named channels in
different scopes can neither collide in the cache nor fake a cycle. The
host answers "which providers win for a read from this scope"
(`NodeTree::providers_for_bus_read`): a pure outward writer-shadowing walk
(R5) that never descends into a scope — which is what makes sink-scope
no-demand (R2) hold *by construction*. The spike's probe-side filter for
inactive playlist entries is the rejected alternative; the pinned test
asserts a probe read with `include_values` never ticks a sink-scope
producer. The playlist ownership-suppression rule is deleted outright:
entry children publish into their entry's sink scope like any producer.

**Reading-scope rule.** A node reads from the scope it inhabits — except
scope *introducers*, whose bus reads face inward (the root's unscoped
reads are root-scope reads; a module export republishing an inner channel
reads it from the scope the module introduces). Write-side classification
is always the owner's inhabited scope (R4: produces write locally; a
module node resides in its parent's scope).

**A real `ModuleNode` runtime, root included.** Every module-kinded node
wears the mirror runtime (harvested from the spike's `ProjectNode`):
`produce` resolves the introduced scope's `visual.out` and forwards
render/sample dispatch to the producer; no writer renders cleared. The
loader registers the R7 surface: authored bindings (the contention pick),
authored exports, and the automatic `output` → `visual.out` fallback
publish for non-root modules (drop-in embedding). Root is no longer a
placeholder special case — the same runtime attaches on load and on the
apply path's reattach. The runtime is deliberately never feature-gated
(every project has a root module; C6 headroom after: ~244 KB).

**Primary visual is an engine-reported role.** `WireBusChannel` gains
`primary_visual`, decided once in the probe (the root scope's listing of
the vocabulary channel the root mirror reads). Studio and friends consume
the flag; `channel.name == "visual.out"` string tests are dead. The name
comparison survives in exactly one place, engine-side, next to the
vocabulary constant.

**Persisted scope identity.** A scope's stable string is its owner's tree
path; a sink scope keys by the authored playlist entry
(`…/entries[k]`) — stable under sibling reorder (names and keys, never
indices), stable across reattach/reload, and following the entry SLOT
rather than its content, so swapping what an entry plays keeps the
entry's panel state. This string becomes the panel-state key prefix in
the panel phases; it was chosen before any device persists one.

## Consequences

- Depth-2 composition works and is pinned (E5): module publishes and
exports count as writers in resolution like any producer — the spike's
latent `collect_writers` omission cannot recur silently.
- Two fallback writers in one scope (host visual + embedded module's
publish) resolve ambiguous-until-authored — the accepted consequence;
the pick is authorable on the module node (`ModuleDef.bindings`).
- Feedback via one channel in one scope reports as a cycle; chains that
need explicit topology use `node:` refs (E5 note).
- The wire still lists channels flat; the structured `WireScopeRef`
surface and per-scope listings land in the next phase (P7).
142 changes: 142 additions & 0 deletions docs/adr/2026-08-02-panel-writers-and-state-persistence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Panel writers: an unauthored runtime tier, persisted device-first in `/.lp/panel.json`

- Status: accepted
- Date: 2026-08-02
- Context: implements docs/design/panel.md P1–P4, P8–P11, P14 (ratified
alongside modules.md 2026-07-31). Builds on the scoped-bus engine
(`2026-08-01-scoped-bus-engine-architecture.md`), which supplies the
`(scope, channel)` identity this whole tier keys on, and follows the
runtime command-channel precedent from
`2026-07-27-runtime-node-command-channel.md`.
- Plan: `planning/2026-08-01-1003-modules-impl-roadmap` P8–P10.

## Decision

**A panel control is not an editor.** Turning a knob does not change the
project. It engages a *panel writer*: an unauthored, lazily materialized
runtime value source living on the Engine, keyed by `(scope, channel)`,
that outranks authored writers for that channel in that scope until
explicitly cleared.

Four consequences follow, and each one is load-bearing:

**The store is a side store on the Engine, not a binding.**
`apply_project_changes` rebuilds bindings from defs, so a panel writer
registered as a `BindingDraft` would be silently destroyed by the first
authoring edit — the user's dim would vanish because someone renamed a
node. `PanelWriterStore` therefore sits beside the binding set and
survives every edit that does not rebuild the Engine itself. A test pins
exactly that.

**Writers are lazy.** A writer materializes on first touch, never at load.
Eager materialization would make every public parameter self-shadow at
boot, and an outer scope could then never drive an inner channel —
modules.md R10 would be dead on arrival.

**Engagement REPLACES the scope's provider set, it does not join it.**
At the host seam, an engaged writer is returned alone. This is what makes
"panel wins" hold on `ByKey` merge channels too, where a merge would
otherwise blend the panel value with the authored ones and produce
something the user never asked for.

**The wire ops are runtime pokes, never authored ops.**
`WireProjectCommand::PanelWrite` / `PanelClear` are project-level arms —
they address a scope, not a node — and touch no overlay, no `PendingEdit`,
and no dirty flag. That is not merely tidy: it is what sidesteps
`PendingEdit` value-shadowing, which would otherwise fight multi-client
convergence. Clients learn state through ordinary probe pulls, so two
phones on one device agree, and a knob fight resolves last-writer-wins at
the engine with no locks and no ownership.

### Persistence is device-first, in the framework tier

Panel state persists to `/.lp/panel.json` *inside the project's own
filesystem* — the framework-owned tier, never an authored artifact. Both
sim tiers run on `LpFsMemory`, so **sims stay ephemeral by construction**
(settled D-B); unit tests are the correctness story there, and the device
walk confirms it. Persistent sims are recorded future work.

**Restore happens at Engine construction — before the first tick, and so
before the first render.** The requirement is verbatim from the design:
*4 a.m., Burning Man, LED scarf dimmed from a phone; unplug, replug — it
must come back dim, with not one bright frame.* A boot that renders even
one frame at authored brightness is non-conforming, which is why the seam
is `Project::new` (and `Project::reload`, which rebuilds the Engine) and
not a later ready-event. On device, `auto_load_project` runs before the
main loop, so this seam is the boot path.

**Keys are `scope-path / channel`** from `ScopeRef::persist_path` — tree
paths and authored entry keys, never runtime ids or indices — so state
survives reload, reattach, and sibling reorder. A sink scope keys by the
ENTRY, not the entry's child: swapping which node a playlist entry plays
keeps the entry's panel state. **State follows the slot, not the
content.** An entry naming a scope this project no longer has is dropped
on load: vendoring and renames degrade gracefully rather than failing a
boot.

**Version is bump-and-refuse.** An unknown version means the file is
ignored wholesale — no migration, matching the alpha posture everywhere
else in the format story (settles P-Q3). Losing panel state costs one
re-dim; a half-applied migration costs trust.

**Writes are throttled to ~10 s and gated on a mutation counter**, with a
flush on clean shutdown. The counter matters more than it looks: a clear
followed by a re-write inside one window leaves an identically-shaped
map, so comparing `(len, newest revision)` would miss it and silently
lose the newer value. An idle project writes nothing at all, however long
it runs.

**Momentary writers never persist** (P14). A gesture has no held value,
and a deadline that outlived a power cycle would be meaningless.

### The prerequisite that makes persistence safe at all

Writing inside the project filesystem fires an `FsEvent` back into
`Project::refresh_artifacts` → `apply_project_changes`. Without a filter,
**every ~10 s save would clear and re-register the whole binding graph,
and the rebuild would schedule the next save** — a permanent churn loop
costing the device its flash and its frame budget, triggered by nothing
more than leaving a knob engaged.

So `refresh_artifacts` drops `/.lp/**` events *first*, before anything
reads the batch, and returns early when nothing authored remains. This is
the same boundary `lpc_history::is_hashed_path` draws for the canonical
package hash and `SnapshotStore` draws for device copies; panel state
inherits both, so a dimmed scarf can never read as a modified project or
show up as a device diff. `Project::applied_refresh_count` exists so this
is *observable* rather than asserted — a test proves an authored write
moves it and a `/.lp/` write does not.

## Alternatives rejected

- **Panel state as overlay slot edits** (the transient/`SlotRole::Debug`
tier used by clock rate/scrub). Rejected on identity: panel writers are
`(scope, channel)` command-channel state, not per-node slot edits. This
is the boundary the debug-slots-taxonomy ADR itself draws — events go to
the command channel, Debug goes to the overlay, panel state goes to
`.lp/panel.json`. Riding the overlay would also re-introduce the
dirty-flag and `PendingEdit` coupling the whole design exists to avoid.
- **Panel writers as authored bindings.** Destroyed by
`apply_project_changes`; see above.
- **Client-side panel state.** Breaks P9 outright: two phones would show
different values for one control, and a device rebooting alone (no
client attached) could not restore anything.
- **Writers that accumulate or generate** (`phase += speed·dt`). Rejected
by P3: that behavior belongs to a node. The supported idiom is
`speed` → phasor node → `phase` → consumer.
- **Slew in this phase.** Deferred deliberately (P-Q1 stays open):
emission is immediate, which is correct on its own rather than a
placeholder, and the seam when it arrives is writer-side shaping — the
writer holds the raw value and shapes only what it emits, so nothing
downstream changes.

## Consequences

- Any future input source — MIDI, OSC, hardware encoders, play mode,
phones — enters through these same two ops with the same identity,
latch semantics, and persistence rules. No second control path.
- A control whose backing slot has no bus channel behind it still edits
its authored default through the slot path; the two coexist, and the
control's `panel_target` is what selects between them.
- Turning auto-save off records itself in the file, so the choice
survives a reboot rather than quietly re-enabling overnight.
1 change: 1 addition & 0 deletions docs/debt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ stay in place when retired; the log is the history).
| [s3-frame-cost-scales-per-fixture](s3-frame-cost-scales-per-fixture.md) | carried | 2026-07-31 | lpc-engine resolver + lpc-hardware registry | Frame cost is flat ~8.4 ms/fixture: per-frame dataflow re-resolution + per-frame endpoint-status recomputation; the shader JIT is ~1%, sends 11% |
| [per-lamp-data-stored-three-times](per-lamp-data-stored-three-times.md) | carried | 2026-07-31 | lpc-model fixture mapping slots + lpc-engine fixture/output nodes | a lamp's position is stored three times and its colour twice — 31.6 of the classic's 89.5 B/LED; the big half is a wire-visible slot schema, so it did not land with the measurement |
| [c6-on-legacy-ws281x-driver](c6-on-legacy-ws281x-driver.md) | retired | 2026-07-31 | lp-fw/fw-esp32c6/src/output | C6 ran its own single-channel WS281x driver, not `lp-ws281x`; retired 2026-08-01 when the C6 moved onto the shared core and gained its second channel |
| [panel-state-serde-flash-cost](panel-state-serde-flash-cost.md) | carried | 2026-08-02 | lpa-server/panel_state + serde surface | a SECOND LpValue JSON codec in the image costs 50,512 B of C6 flash; the first one is already there and unused by panel state |
| [example-shaders-not-compile-gated](example-shaders-not-compile-gated.md) | carried | 2026-07-29 | examples GLSL + CI + lps-filetests | an example shader can compile on the host yet fail on 4 of 5 targets; the break surfaces only when a human opens it in Studio |
| [clock-transport-has-no-transport-ui](clock-transport-has-no-transport-ui.md) | carried | 2026-05-12 | clock node + studio faces | scrubbing a show means typing seconds into a generic slider; the misfit also keeps the `Debug` name provisional |
| [project-reload-drops-debug-silently](project-reload-drops-debug-silently.md) | carried | 2026-07-04 | lpa-server project lifecycle | the documented recovery path discards every pending edit and Debug override with no return value, event, or notice |
Expand Down
Loading
Loading