feat: classic ESP32 level-4/5 RMT refill — the HLI experiment - #272
Draft
Yona-Appletree wants to merge 8 commits into
Draft
feat: classic ESP32 level-4/5 RMT refill — the HLI experiment#272Yona-Appletree wants to merge 8 commits into
Yona-Appletree wants to merge 8 commits into
Conversation
…efill lp-ws281x-hli: the exact repr(C) state the classic-ESP32 level-4 vector will read (HliBank/HliChannel), and a pure-Rust model of its algorithm — prefill, threshold service (entry-delay telemetry, boundary flip, guard, byte-granular fill), end classification. Host suite pins the model against lp-ws281x's driver as oracle: byte-identical wire streams across tail shapes, guard trips on lost refills, boundary-wrapped entry delays, and shift-bucket equivalence with lag_bucket for power-of-two halves. The firmware will derive every asm field offset from these structs via offset_of!, so layout cannot drift from the vector that consumes it. Part of the classic HLI experiment (plan 2026-08-01-1459-rmt-priority-hli, P5 reopened as experiment). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ness The experiment proper. hli_refill (additive to the server build) swaps the endpoint layer's refill path onto a hand-written Xtensa level-4 vector: - vector.rs: __naked_level_4_interrupt via xtensa-lx-rt's PROVIDE seam (the sanctioned esp-hal-world hook), RMT routed to CPU interrupt 24 (extern, level-triggered, level 4) through the peripheral's own matrix register. Call0 discipline, no windowed instructions, a2..a13+SAR saved to a static DRAM area, code+literals in IRAM, ack-all storm guard, rfi 4. Field offsets arrive as global_asm const operands from offset_of! on the lp-ws281x-hli contract structs — nothing transcribed. - app.rs: the shared_driver-shaped thread side (configure/send/abort/ telemetry); the start path runs the host-tested reference model, so the asm's spec executes on silicon every frame beside the asm itself. Telemetry prints the P4 line format + src=hli4. - stress.rs (hli_stress, entrypoint-replacing like radio_ram_probe): radio-linked cells on one boot — L3 idle/scan then a live matrix switch to the level-4 vector, idle/scan again; 4x30 LEDs at the P4 baseline pace, C6's S2 scan generator. Coexistence facts recorded in the module docs: esp-sync critical sections are rsil 5, so level 4 escapes only the rsil<=3 class (embassy locks, level<=3 handler execution) — which class dominates is what the harness measures. Feature off: section sizes byte-identical (verified against a stashed build). Feature on vs telemetry build: .bss +816 B (bank + save area), .data -1088 B / .rwtext -452 B (idle L3 refill path GC'd), flash +264 B. Clippy matrix extended and green for all six configurations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ll on the slot Silicon finding (radio-off, quad-strips-v3): the level-4 handler's entry delay is ZERO words on every refill — so the L3 algorithm's pre-fill-only guard placement degenerated to skips==refills (43k/43k): the reader was always still sitting on the guard slot. A refill path that fast would have had no flicker protection at all on lost interrupts. Model + vector now retry after the fill (by which time the reader has moved), and `skips` counts only refills left unguarded by both attempts. Re-measured: skips=0 across 43k refills, trips=0, entry_max=0 (ch6: 1), lag 3.0/3 vs the L3 path's 15.0/17 — and the L3 baseline's per-frame 55-word ch0 start-path delay is entirely absent at level 4, which classifies that masking as rsil<=3-class. Logs in .hli-logs/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
First head-to-head (run3, same boot, scan cells 150 s): the surprise is real and honest — under identical WiFi-scan load the level-4 path trips MORE (1.8-3.1% per channel) than level-3 (0.8-1.1%), despite far lighter entry-delay tails (max 46 vs 58 words, nothing near the 64-word deadline on either path, lag_max 4 vs 21). Neither path's trips are explained by recorded late entries, so some events are being lost or mis-sequenced rather than delayed. Instrument for the re-measure: a sel_mismatch counter (reader's half at service vs the armed boundary's half) in contract+model+vector, printed as an appended selmis= field. If threshold events are missed/duplicated, pos and boundary disagree by a half and this counts it. Also: ADR draft (docs/adr/2026-08-02-classic-hli-refill.md, Status: proposed) with the full provenance chain, the rsil-5 ceiling finding, the level choice, coexistence contract, RAM ledger, and the argued I2S/DMA alternative; stress-cell logs under .hli-logs/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ecommendation Run4 (instrumented) reproduces run3: under identical scan load the level-4 path truncates 2-3x MORE than level-3 (worst-ch 2.79% vs 1.40%), selmis=0 (no event bookkeeping desync at any serviced entry). Analysis shows both paths' entry-delay histograms are censored at exactly the deadline: an over-deadline refill ends the frame via the guard, and the end-beats-threshold precedence swallows the late event's delay — so trips measure the rsil-5-class outage population the histograms structurally miss. The level-4 excess remains unexplained after the one instrumented re-measure the stop conditions allow; known confound: cell order within the boot. ADR carries the full table, the censoring analysis, the follow-up experiment design (interleaved cells + de-censored trip-adjacent capture), and an advisory park-don't-ship recommendation for G2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Making the v3_rmt interrupt-bit helpers pub(crate) for the hli module left three outlined copies in the feature-off .text (+24 B vs the base commit). #[inline] drops them; section sizes are now byte-identical to a pristine 099b422 build again — the untouched-image invariant the experiment promised, verified rather than assumed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI's Lint (x64) fmt-check caught attribute-wrapping and import-order nits in the new/edited files (host clippy and the esp32v3 clippy matrix don't run rustfmt). Token-identical reflow; firmware rebuilt to confirm. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Captured in the pinned CI environment by run: https://github.com/light-player/lightplayer/actions/runs/30735787145
Contributor
CI refreshed the story baselines on this branchThe Review every PNG in the PR's Files changed view (swipe / onion-skin). 1067 more file(s)
…(list truncated; see the commit) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plan: lp2025/2026-08-01-1459-rmt-priority-hli
P5-reopened-as-experiment: G1 recorded NO-GO the same evening; Yona then authorized this as an overnight EXPERIMENT on its own branch/PR so it can be gracefully dropped if the numbers do not justify it. Closing this PR is the removal path.
What this is
A clean-room level-4 high-priority-interrupt RMT refill for the classic ESP32, behind
fw-esp32v3cargo featureshli_refill/hli_stress(both default OFF; the shipping image is proven untouched — section-identical build). The full honest chain — the WLED-was-MIT tip, the firewalled provenance verdict (WLED contains no HLI shim at any revision; its WiFi stability is NeoPixelBus/LGPL I2S-DMA; neither was ever opened here), the clean-room implementation from Espressif Apache references, the level choice, the esp-hal/esp-rtos coexistence contract, the RAM cost, the measurements, and the argued I2S/DMA alternative — is indocs/adr/2026-08-02-classic-hli-refill.md(Status: proposed; G2 decides ship vs park).Pieces
lp-fw/lp-ws281x-hli(new, host-tested): therepr(C)state the vector reads + a pure-Rust reference model, differential-tested againstlp-ws281x(byte-identical wire streams, guard-trip/entry-delay/bucket semantics). Asm offsets come fromoffset_of!on these structs — nothing transcribed.fw-esp32v3/src/output/rmt/hli/: the vector (global_asm!, call0 discipline, CPU interrupt 24 via xtensa-lx-rt's__naked_level_4_interruptPROVIDE seam — the sanctioned hook; ack-all storm guard;rfi 4) and ashared_driver-shaped thread side the endpoint layer swaps in.lp-ws281xcore untouched.hli_stress: entrypoint-replacing harness (radio_ram_probe pattern) — same boot, same board, 4×30 LEDs at the P4 pace, level-3 cells then a live matrix switch to level 4, WiFi-scan load cells ≥150 s.Key findings
rsil 5ceiling.esp-sync's critical sections (⇒critical_section::with, esp-radio'swifi_int_disable, esp-storage flash windows) run atrsil 5— so level 4 and 5 are masked by them, unlike esp-idf (which stops at 3, the origin of HLI folklore). Level 4 escapes thersil ≤ 3class: embassy/PriorityLocklocks and all level ≤ 3 handler execution time.skips == refillsmeasured), so the vector retries the guard after the fill; skips returned to 0. Recorded in the ADR as a design consequence unique to HLI speed.Two full runs (run3; run4 instrumented). Per-cell deltas, run4 (frames per channel ≈2.8 k per scan cell):
rsil 5-class outages the histograms structurally miss — consistent with level 4 not helping, but not with it measuring worse. The selmis instrument (one focused re-measure) ruled out event bookkeeping desync; the remaining delta is unexplained, with a known confound (cell order within the boot / uncontrolled scan environment). Stop-condition honored: recorded, not chased further.Measurements ledger
.bss+816 B (bank + save area);.data−1,088 B /.rwtext−452 B (idle L3 path GCs out) ⇒ net DRAM −272 B; flash.text+264 B. Staging: one frame-sizedVecper channel (~90 B at product load)..hli-logs/(feature-off baseline, hli radio-off, stress cells).CI
clippy-fw-esp32v3extended in-recipe:esp32,hli_stressin the no-default loop,hli_refill,ws281x_telemetryin the additive loop — same job, per the plan's constraint. Host tests:lp-ws281x-hli(9 tests) in the default workspace.🤖 Generated with Claude Code