diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7887ae7..b08d311 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -214,6 +214,16 @@ jobs: cargo clippy -p rusty_alloc --all-targets -- -D warnings env: RUSTFLAGS: --cfg ra_small_profile --cfg ra_single_threaded --cfg ra_aligned_region + # `ra_segment_size="256k"` is the geometry a large-allocation consumer + # runs (docs/plans/finished/esp32-large-alloc-ceiling.md). It moves + # SEGMENT_SIZE, LARGE_OBJ_SIZE_MAX and every sizing rule, so the WHOLE + # suite runs there -- a geometry that only builds is a geometry that rots. + - name: test + clippy (256k segment geometry) + run: | + cargo test -p rusty_alloc + cargo clippy -p rusty_alloc --all-targets -- -D warnings + env: + RUSTFLAGS: --cfg ra_small_profile --cfg ra_segment_size="256k" # The no_std build REFUSES to compile without `ra_single_threaded`: its # soundness rests on there being exactly one thread, and that has to be # opted into rather than inherited. The gate's negative case is asserted diff --git a/Cargo.toml b/Cargo.toml index 669ed7a..fc5218b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -155,7 +155,7 @@ unsafe_op_in_unsafe_fn = "deny" # unify across the dependency graph, so two consumers wanting different # geometries would silently get one of them. A cfg is set by the DELIVERABLE, # the same way a Janus firmware picks its chip. -unexpected_cfgs = { level = "warn", check-cfg = ["cfg(loom)", "cfg(kani)", "cfg(ra_small_profile)", "cfg(ra_single_threaded)", "cfg(ra_max_extents, values(\"8\", \"16\", \"64\"))", "cfg(ra_aligned_region)"] } +unexpected_cfgs = { level = "warn", check-cfg = ["cfg(loom)", "cfg(kani)", "cfg(ra_small_profile)", "cfg(ra_single_threaded)", "cfg(ra_max_extents, values(\"8\", \"16\", \"64\"))", "cfg(ra_aligned_region)", "cfg(ra_segment_size, values(\"256k\"))"] } # Lint policy (hardening gate H-15). `pedantic` and `nursery` are ENABLED at # workspace level and the build is clean under them, because every group diff --git a/README.md b/README.md index 9729d1e..f615360 100644 --- a/README.md +++ b/README.md @@ -243,6 +243,7 @@ region change by `.stack` or the section sum, never by `.bss` alone. | `Region::give` (or `init_region`) | every allocation fails; the backend has no memory | | *(optional)* `--cfg ra_max_extents="8"` | the free-extent table keeps its default 32 slots (256 B of `.bss`, i.e. stack); 8 is plenty for a region of a few segments and returns 192 B — the doc on `MAX_EXTENTS` states the bound | | *(optional)* `--cfg ra_aligned_region` | segments stride from the region's base: `Region` is 16-byte aligned, the linker leaves no gap before it, and `free` pays three instructions. With the flag, the 2.0.4 layout: the address mask on `free`, `Region` segment-aligned, up to 64 KiB of gap in front of it | +| *(optional)* `--cfg ra_segment_size="256k"` | a 64 KiB segment, so `LARGEST_SHARED_ALLOC` is 61,440 and any bigger request takes a dedicated **two** segments. With the flag, an 8 KiB slice x 32: a 64 KiB block is a span and three pack into one segment. Set it when your allocation unit is tens of KB; leave it unset for small objects, since it doubles the page floor | Size the region with the two `const fn`s in `prim::fixed`, not a round number: `good_region_size(220 * 1024)` is 196,608 — three whole segments, @@ -400,6 +401,49 @@ That floor is roughly **fixed** for a given mix of sizes: the same pages serve a page allocator starts earning what it charges, and the throughput above is what it buys. +#### The floor model does NOT cover large allocations — read this if your unit is tens of KB + +**Everything above is about a workload of small objects, and it inverts for one +whose unit approaches the segment.** A `rusty_zstd` firmware allocating 64 KiB +match tables measured the inversion on an ESP32-S3: in a 256 KiB region it got +**one** 64 KiB block, and the second failed with 192 KiB unused +([`docs/plans/finished/esp32-large-alloc-ceiling.md`](docs/plans/finished/esp32-large-alloc-ceiling.md)). + +The reason is structural and worth stating plainly. A segment's slice 0 holds +its header, so the largest object that can live in a segment is +`SEGMENT_SIZE - SEGMENT_SLICE_SIZE` — **61,440 bytes** at the default small +profile, exposed as `prim::fixed::LARGEST_SHARED_ALLOC`. One byte over that and +the request gets a dedicated run of segments, and since **no allocation of +`SEGMENT_SIZE` can share a segment with the metadata describing it**, a 64 KiB +request takes two segments and a 128 KiB request takes three. So the cost is +not a fixed floor that amortises; for segment-sized blocks it is a +**granularity tax that scales with how many are live**. + +Two things follow, and the crate now answers both: + +- **Size the region with `region_for_allocs(size, count)`**, not by adding up + payloads. It knows the tax, and it adds the segment the first small + allocation claims — which is what took the reporting firmware from two blocks + to one. +- **If your unit is at or above `LARGEST_SHARED_ALLOC`, set + `--cfg ra_segment_size="256k"`.** It moves the small profile to an 8 KiB + slice x 32, so `LARGEST_SHARED_ALLOC` becomes 253,952 and a 64 KiB request is + a span the allocator packs three-to-a-segment instead of a dedicated + two-segment run. Measured on the same board, same 256 KiB region: + + | geometry | 64 KiB blocks served | payload live | region used for it | + |---|---:|---:|---:| + | default | 1 | 64 KiB | 25 % | + | `--cfg ra_segment_size="256k"` | **3** | **192 KiB** | **75 %** | + + It is opt-in because it doubles the page floor every small workload pays + (`(classes touched) x 8 KiB`), which is the wrong trade for the sketch above + and the right one for a codec. + +And when an allocation does fail, `prim::fixed::region_capacity()` says why: +free BYTES hide this, free SEGMENTS do not. On the failing run it read +`free_segments=1, largest_servable=61440` with 126,976 bytes free. + We got from 192 KiB to 68 KiB by fixing a placement bug in the fixed-region backend and halving the slice, and from 68 KiB to 64 KiB by moving the first heap's descriptor out of the region into a static, so the region is whole diff --git a/crates/rusty_alloc/CHANGELOG.md b/crates/rusty_alloc/CHANGELOG.md index 52055f6..123b5be 100644 --- a/crates/rusty_alloc/CHANGELOG.md +++ b/crates/rusty_alloc/CHANGELOG.md @@ -7,6 +7,47 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **`--cfg ra_segment_size="256k"`, for a firmware whose allocation unit is + tens of kilobytes.** A segment's slice 0 is its header, so the largest object + that can share a segment is `SEGMENT_SIZE - SEGMENT_SLICE_SIZE` — 61,440 + bytes at the default small profile — and one byte over that takes a dedicated + run of segments. Since no allocation of `SEGMENT_SIZE` can share a segment + with its own metadata, a 64 KiB request costs **two** segments and a 128 KiB + request three. A `rusty_zstd` firmware measured the consequence on an + ESP32-S3: one 64 KiB block served from a 256 KiB region, the second refused + with 192 KiB unused. The flag moves the small profile to an 8 KiB slice x 32, + raising `LARGEST_SHARED_ALLOC` to 253,952 so a 64 KiB request becomes a span + three of which pack into one segment — **3 blocks instead of 1 in the same + 256 KiB region, measured on the board**. Opt-in, because it doubles the page + floor `(classes touched) x slice` that a small-object workload pays. +- **`prim::fixed::LARGEST_SHARED_ALLOC`, `dedicated_segments(size)` and + `region_for_allocs(size, count)`** — the sizing API that predicts the above at + compile time, instead of leaving a firmware to discover it on silicon. + `region_for_allocs` also counts the segment the first small allocation claims, + which is what took the reporting firmware from two blocks to one. +- **`prim::fixed::PrimError`**, re-exported so the whole fixed-region recipe is + reachable from one path. The type has always been public as + `prim::PrimError`, but only from the parent module, so a seam re-exporting + this API in a single `pub use` could name `Region`, `good_region_size`, + `init_region` and the `FERR_*` values but not the type they fail with. The + Kairos RTOS allocator seam hit exactly that and carried an "arrives with the + next release" comment for it. Same type, one more path. +- **`prim::fixed::region_capacity() -> (free_segments, largest_servable)`.** + `region_stats` reports free BYTES, and free bytes hide this failure: the + refused allocation above had 126,976 bytes free and read + `free_segments=1, largest_servable=61440`. + +### Fixed + +- **The README's footprint model did not cover large allocations and implied + the opposite of the truth for them.** It described the floor as + `(classes touched) x (page size)`, "independent of bytes requested" and + amortising as the working set grows. That holds for small objects; for + segment-sized ones the cost is a granularity tax that scales with how many + are live. The section now says so, with the measured numbers and the flag. + ## [2.0.5](https://github.com/Remade-With-Rust/rusty_alloc/compare/rusty_alloc-v2.0.4...rusty_alloc-v2.0.5) - 2026-09-09 ### Semver note, read this one diff --git a/crates/rusty_alloc/README.md b/crates/rusty_alloc/README.md index 7320fbc..eacd065 100644 --- a/crates/rusty_alloc/README.md +++ b/crates/rusty_alloc/README.md @@ -124,6 +124,21 @@ roughly fixed, so it amortises as the working set grows. Reach for `esp-alloc` when the budget is tight, and for this when throughput or fragmentation under churn is what hurts. +**That model covers small objects only, and inverts once your unit approaches +the segment.** A segment's slice 0 is its header, so the largest object that +can share one is `SEGMENT_SIZE - SEGMENT_SLICE_SIZE` — 61,440 bytes at the +default small profile (`prim::fixed::LARGEST_SHARED_ALLOC`). One byte over and +the request takes a dedicated run of segments, and because no allocation of +`SEGMENT_SIZE` can share a segment with its own metadata, a 64 KiB block costs +**two** segments: a firmware allocating 64 KiB match tables got one of them out +of a 256 KiB region with 192 KiB unused. For segment-sized blocks the cost is a +granularity tax that scales with how many are live, not a floor that amortises. +Size such a region with `prim::fixed::region_for_allocs(size, count)`, ask +`region_capacity()` when an allocation fails with bytes to spare, and set +**`--cfg ra_segment_size="256k"`** if your unit is tens of KB — it packs three +64 KiB blocks into one segment instead of one, measured on the board. The full +account is in `docs/plans/finished/esp32-large-alloc-ceiling.md`. + **It also costs flash and static RAM, measured on the linked ELF of one firmware built both ways:** about **+3.2 KB of flash** and **+0.3 KB of static RAM**, down from +16.6 KB and +3.1 KB two releases earlier — diff --git a/crates/rusty_alloc/UNSAFE.md b/crates/rusty_alloc/UNSAFE.md index 24f7e00..73c1dd1 100644 --- a/crates/rusty_alloc/UNSAFE.md +++ b/crates/rusty_alloc/UNSAFE.md @@ -32,7 +32,7 @@ unsafe in DEPENDENCIES, and ours are `libc` plus bindings-only `windows-sys`. | `os.rs` | 12 | The prim-layer wrapper: commit/decommit/protect plumbing | 2026-08-08 | | `prim/mock.rs` | 8 | Miri-only mock OS backend (never shipped; `cfg(miri)`) | 2026-08-06 | | `arena.rs` | 8 | Lock-free chunk bitmap claim/verify, recycled-chunk scrubbing (the 0.1.0-alpha.2 UAF fix lives here: `wait_no_remote_in_flight` on every recycle path) | 2026-08-08 | -| `prim/fixed.rs` | 33 | **New 2026-09-07 (P1 of `docs/plans/small-metal.md`).** The fixed-region backend for a target with no OS: memory is a `&'static mut [u8]` handed over once. **6 of the 18 are `unsafe fn` signatures the prim seam requires** (`alloc`/`free`/`commit`/`decommit`/`reset`/`protect`) whose *bodies contain no unsafe operation at all* — the free list is `AtomicUsize` arrays under a spin lock and the pointers are built with the safe `with_exposed_provenance_mut`, so this backend adds **zero** unsafe dereferences to the shipped crate. The other 12 are in `#[cfg(test)]`: two `&raw mut` static-region handoffs and ten calls through the `unsafe fn` seam, each with its SAFETY line. **+1 on 2026-09-07 (P2):** the region test became two-sided — where the shipped geometry refuses a segment-sized request from a 512 KiB region, the small profile SERVES one, so the test now frees it too. Audited at the site; the module is `allow(dead_code)` and unreachable on every platform that has an arm above it. **+7 on 2026-09-07 (P4b, §2.9/§2.10 of the same plan):** the two-ended `place` rule added ZERO unsafe to shipped code — `place` is a pure arithmetic `fn` and the scan around it is unchanged — and all seven are in `#[cfg(test)]`: one `slice::from_raw_parts_mut` carving the `REGION_ALIGN`-aligned window out of `BACKING` (replacing a `&mut *ptr` that a `repr(align(65536))` static would have needed, which rustc 1.97.1 on MSVC cannot compile), one `ptr::add` to reach that window, and five calls through the `unsafe fn` seam in the placement assertions and in `greedy_segments`, which allocates segments until refusal and frees every one before returning. Each carries its SAFETY line **+1 on 2026-09-09 (morning, `#21`):** the two-sided alignment test frees the SEGMENT_SIZE-aligned page it is served when the region straddles a boundary — `#[cfg(test)]`, banked without a row here at the time; recorded now. **+5 on 2026-09-09 (`docs/plans/finished/region-alignment-bug.md` §7): the module ships its first unsafe OPERATIONS.** (1) `unsafe impl Sync for Region` and (2) the `&mut *self.bytes.get()` in `Region::give` — the once-only handoff the Janus seam has carried since 2.0.0, moved here so the alignment travels with it; a module-wide `REGION_GIVEN` swap precedes the `&mut`, so a second `give` on any instance is refused before it could alias. (3) `unsafe impl Sync for FirstHeapBox`, the bare-metal static holding the first heap's descriptor (handed out once by `take_first_heap_box`, on the one thread such a build has). The other two are `#[cfg(test)]`: a `from_raw_parts_mut` carving the report's misaligned base out of a static for the refusal probe, and a `Box::new_zeroed().assume_init()` allocating a `Region` in place, because materialising a 64 KiB-aligned value on the stack first faults on Windows. Each carries its SAFETY line. **+2 on 2026-09-09 (`docs/plans/finished/region-alignment-dissolve.md`): zero new unsafe in shipped code** — segments now stride from the region's base, and every piece of that is safe: `stride_base` is a relaxed load, `place` takes an `origin` and does arithmetic, `install_region` aligns the base up. The two are `#[cfg(test)]`, the `--cfg ra_aligned_region` arm of the two-sided alignment test (an `alloc` and a `free` through the `unsafe fn` seam: the 2.0.4 straddle case, kept under the knob that restores the mask), and the `Box::new_zeroed().assume_init()` moved under that same cfg — the default arm is a plain `static Region` now, which a 16-byte-aligned type can be on every host toolchain. Each carries its SAFETY line | 2026-09-09 | +| `prim/fixed.rs` | 37 | **New 2026-09-07 (P1 of `docs/plans/small-metal.md`).** The fixed-region backend for a target with no OS: memory is a `&'static mut [u8]` handed over once. **6 of the 18 are `unsafe fn` signatures the prim seam requires** (`alloc`/`free`/`commit`/`decommit`/`reset`/`protect`) whose *bodies contain no unsafe operation at all* — the free list is `AtomicUsize` arrays under a spin lock and the pointers are built with the safe `with_exposed_provenance_mut`, so this backend adds **zero** unsafe dereferences to the shipped crate. The other 12 are in `#[cfg(test)]`: two `&raw mut` static-region handoffs and ten calls through the `unsafe fn` seam, each with its SAFETY line. **+1 on 2026-09-07 (P2):** the region test became two-sided — where the shipped geometry refuses a segment-sized request from a 512 KiB region, the small profile SERVES one, so the test now frees it too. Audited at the site; the module is `allow(dead_code)` and unreachable on every platform that has an arm above it. **+7 on 2026-09-07 (P4b, §2.9/§2.10 of the same plan):** the two-ended `place` rule added ZERO unsafe to shipped code — `place` is a pure arithmetic `fn` and the scan around it is unchanged — and all seven are in `#[cfg(test)]`: one `slice::from_raw_parts_mut` carving the `REGION_ALIGN`-aligned window out of `BACKING` (replacing a `&mut *ptr` that a `repr(align(65536))` static would have needed, which rustc 1.97.1 on MSVC cannot compile), one `ptr::add` to reach that window, and five calls through the `unsafe fn` seam in the placement assertions and in `greedy_segments`, which allocates segments until refusal and frees every one before returning. Each carries its SAFETY line **+1 on 2026-09-09 (morning, `#21`):** the two-sided alignment test frees the SEGMENT_SIZE-aligned page it is served when the region straddles a boundary — `#[cfg(test)]`, banked without a row here at the time; recorded now. **+5 on 2026-09-09 (`docs/plans/finished/region-alignment-bug.md` §7): the module ships its first unsafe OPERATIONS.** (1) `unsafe impl Sync for Region` and (2) the `&mut *self.bytes.get()` in `Region::give` — the once-only handoff the Janus seam has carried since 2.0.0, moved here so the alignment travels with it; a module-wide `REGION_GIVEN` swap precedes the `&mut`, so a second `give` on any instance is refused before it could alias. (3) `unsafe impl Sync for FirstHeapBox`, the bare-metal static holding the first heap's descriptor (handed out once by `take_first_heap_box`, on the one thread such a build has). The other two are `#[cfg(test)]`: a `from_raw_parts_mut` carving the report's misaligned base out of a static for the refusal probe, and a `Box::new_zeroed().assume_init()` allocating a `Region` in place, because materialising a 64 KiB-aligned value on the stack first faults on Windows. Each carries its SAFETY line. **+2 on 2026-09-09 (`docs/plans/finished/region-alignment-dissolve.md`): zero new unsafe in shipped code** — segments now stride from the region's base, and every piece of that is safe: `stride_base` is a relaxed load, `place` takes an `origin` and does arithmetic, `install_region` aligns the base up. The two are `#[cfg(test)]`, the `--cfg ra_aligned_region` arm of the two-sided alignment test (an `alloc` and a `free` through the `unsafe fn` seam: the 2.0.4 straddle case, kept under the knob that restores the mask), and the `Box::new_zeroed().assume_init()` moved under that same cfg — the default arm is a plain `static Region` now, which a 16-byte-aligned type can be on every host toolchain. Each carries its SAFETY line. **+4 on 2026-09-09 (`docs/plans/finished/esp32-large-alloc-ceiling.md`): all four are `#[cfg(test)]`, and shipped code gained none.** The large-allocation ceiling reported from `rusty_zstd` is reproduced against the real extent allocator by `greedy_dedicated`, which serves `huge_alloc`-shaped reservations until the region refuses and frees every one before returning (two calls through the `unsafe fn` seam), plus one `alloc`/`free` pair modelling the whole segment a first small allocation claims. The fix itself -- the `ra_segment_size` geometry knob, `dedicated_segments`, `region_for_allocs` and `region_capacity` -- is arithmetic over the existing atomics and adds no unsafe operation at all. Each carries its SAFETY line | 2026-09-09 | | `prim/wasm.rs` | 6 | `memory.grow` linear-memory backend | 2026-08-06 | | `options.rs` | 6 | Env parsing at init, registered-hook invocation | 2026-08-06 | | `stats.rs` | 3 | Volatile whole-struct snapshot of racy-by-design counters | 2026-08-06 | diff --git a/crates/rusty_alloc/src/prim/fixed.rs b/crates/rusty_alloc/src/prim/fixed.rs index d639a2b..9ddb816 100644 --- a/crates/rusty_alloc/src/prim/fixed.rs +++ b/crates/rusty_alloc/src/prim/fixed.rs @@ -57,7 +57,19 @@ use core::ffi::c_void; use core::sync::atomic::{AtomicBool, AtomicU32, AtomicUsize, Ordering}; -use super::{Alloc, MemConfig, PrimError, TlsDtor, align_up}; +use super::{Alloc, MemConfig, TlsDtor, align_up}; + +/// The error type every fallible entry point here returns, re-exported so the +/// whole fixed-region recipe is reachable from ONE path. +/// +/// It has always been public as [`crate::prim::PrimError`], but only from the +/// parent module — so a seam re-exporting this API in one `pub use` could name +/// `Region`, `good_region_size`, `init_region` and the `FERR_*` values but not +/// the type they fail with. The Kairos RTOS allocator seam hit exactly that +/// and carried a "arrives with the next release" comment for it +/// (`rusty_rtos_alloc::small_metal`, `rusty_RTOS/docs/plans/build-me-bare.md` +/// B3). Same type, one more path. +pub use super::PrimError; /// Synthetic error code. The backends surface no errno, so any non-zero /// sentinel does; this one is distinct from wasm's `0xBEEF` and the mock's. @@ -390,6 +402,105 @@ pub const fn region_for(usable: usize) -> usize { segments * seg } +/// The largest allocation that can SHARE a segment with other allocations. +/// +/// At or below this, a request is carved as a span of slices inside a segment +/// and several of them pack together. **Above it the cost jumps**: the request +/// gets a dedicated run of segments of its own, because the segment header +/// owns slice 0 and so no allocation of `SEGMENT_SIZE` can ever share a +/// segment with the metadata describing it. +/// +/// 61,440 bytes at the default small profile; `--cfg ra_segment_size` raises +/// it (`crate::types::SLICES_PER_SEGMENT`). **This is the number a firmware +/// with a large allocation unit must design against**, and the one the +/// README's floor model used to omit. +pub const LARGEST_SHARED_ALLOC: usize = crate::types::LARGE_OBJ_SIZE_MAX; + +/// Whole segments one allocation of `size` bytes reserves FOR ITSELF; `0` when +/// it shares a segment ([`LARGEST_SHARED_ALLOC`]). +/// +/// The cliff this reports is the one a `rusty_zstd` firmware fell off: at the +/// default geometry a 64 KiB request answers **2**, so a four-segment 256 KiB +/// region holds one of them and a second fails with 192 KiB free +/// (`docs/plans/finished/esp32-large-alloc-ceiling.md`). +/// +/// ```ignore +/// use rusty_alloc::prim::fixed::dedicated_segments; +/// const _: () = assert!(dedicated_segments(64 * 1024) <= 1, "raise ra_segment_size"); +/// ``` +#[must_use] +pub const fn dedicated_segments(size: usize) -> usize { + if size <= LARGEST_SHARED_ALLOC { + return 0; + } + // Mirrors `segment::huge_alloc`: one slice of header, then the payload, + // rounded up to whole segments because the reservation must start on a + // segment stride. Page-rounding inside `huge_alloc` cannot change this + // count, since a segment is a whole number of pages. + (crate::types::SEGMENT_SLICE_SIZE + size).div_ceil(crate::types::SEGMENT_SIZE) +} + +/// The smallest region that can hold `count` simultaneously-live allocations +/// of `size` bytes each — the question a firmware actually has, answered +/// including the costs that are easy to forget. +/// +/// Two of those bit the reporting firmware. **`count` large allocations do not +/// cost `count * size`**: once each is over [`LARGEST_SHARED_ALLOC`] it takes +/// [`dedicated_segments`] of its own, so three 64 KiB blocks cost six segments +/// at the default geometry, not three. And **the small allocations every +/// program makes need somewhere to live** — a `Vec`'s spine, a formatting +/// buffer — which is a whole extra segment when the large blocks took +/// dedicated ones, and one more slice when they are sharing. +/// +/// ```ignore +/// use rusty_alloc::prim::fixed::{Region, region_for_allocs}; +/// // three live 64 KiB tables, plus room for everything smaller +/// static HEAP: Region<{ region_for_allocs(64 * 1024, 3) }> = Region::new(); +/// // 448 KiB at the default geometry; 256 KiB under --cfg ra_segment_size="256k" +/// ``` +/// +/// Returns 0 if the arithmetic would overflow. +#[must_use] +pub const fn region_for_allocs(size: usize, count: usize) -> usize { + let seg = crate::types::SEGMENT_SIZE; + let usable_slices = crate::types::SLICES_PER_SEGMENT - 1; + let dedicated = dedicated_segments(size); + let segments = if dedicated == 0 { + // Shares: every block is a span of slices, and the small allocations + // take one more slice from the same segments rather than a segment of + // their own. `+ 1` slice, not `+ 1` segment — getting that wrong is + // what made a 3-block region read 512 KiB when 256 KiB serves it. + let slices_each = if size == 0 { + 1 + } else { + size.div_ceil(crate::types::SEGMENT_SLICE_SIZE) + }; + let total = match slices_each.checked_mul(count) { + Some(n) => match n.checked_add(1) { + Some(n) => n, + None => return 0, + }, + None => return 0, + }; + total.div_ceil(usable_slices) + } else { + // Dedicated: each block owns its segments, so the small allocations + // have no carved segment to share and need one of their own. + match dedicated.checked_mul(count) { + Some(n) => match n.checked_add(1) { + Some(n) => n, + None => return 0, + }, + None => return 0, + } + }; + let segments = if segments == 0 { 1 } else { segments }; + match segments.checked_mul(seg) { + Some(bytes) => bytes, + None => 0, + } +} + /// Hand the backend the region it will serve from, once. /// /// Takes `&'static mut [u8]` because that is exactly the claim being made: the @@ -718,6 +829,19 @@ pub fn region_contains(addr: usize) -> bool { len != 0 && addr >= base && addr - base < len } +/// What alignment is measured from: the region's base where segments stride +/// from it, address zero under `ra_aligned_region` (where they stride from +/// zero, as on a hosted target). The one definition both [`alloc`] and +/// [`region_capacity`] read, so a placement and a report cannot disagree. +#[inline] +fn stride_origin() -> usize { + if cfg!(ra_aligned_region) { + 0 + } else { + REGION_BASE.load(Ordering::Relaxed) + } +} + /// The origin segments stride from: the registered region's base, or 0 while /// none is registered — when no pointer can be ours and the answer is the /// hosted mask's. Crate-internal: `segment_of` and the free-list plausibility @@ -747,6 +871,50 @@ pub fn region_stats() -> (usize, usize, usize) { (total - free, free, total) } +/// What the region can still SERVE, which is not what is merely free: +/// `(whole segments still placeable, largest single allocation in bytes)`. +/// +/// [`region_stats`] answers "how many bytes are unclaimed" and that number +/// **hides the constraint that actually fails an allocation**. The reporting +/// firmware saw 192 KiB free and a failing 64 KiB request; this function would +/// have answered `(1, 61_440)` and named the reason — one segment left, and +/// nothing bigger than a shared span can be placed in it. +/// +/// The second value is the largest DEDICATED allocation placeable in a fresh +/// run of segments. A request at or below [`LARGEST_SHARED_ALLOC`] may still +/// succeed above this figure by sharing a segment that is already carved, +/// which this backend cannot see — so treat it as the floor of what will +/// work, not the ceiling. +/// +/// A snapshot, as [`region_stats`] is. +#[must_use] +pub fn region_capacity() -> (usize, usize) { + let _g = Guard::acquire(&LOCK); + let seg = crate::types::SEGMENT_SIZE; + let origin = stride_origin(); + let mut segments = 0usize; + let mut largest = 0usize; + for i in 0..EXT_COUNT.load(Ordering::Relaxed) { + let base = EXT_BASE[i].load(Ordering::Relaxed); + let len = EXT_LEN[i].load(Ordering::Relaxed); + let end = base + len; + // The first segment stride at or above this extent's base. + let first = origin + (base - origin).next_multiple_of(seg); + if first >= end { + continue; + } + let run = end - first; + segments += run / seg; + if run > crate::types::SEGMENT_SLICE_SIZE { + let placeable = run - crate::types::SEGMENT_SLICE_SIZE; + if placeable > largest { + largest = placeable; + } + } + } + (segments, largest) +} + /// Remove the extent at `idx`, shifting the tail down to keep the list sorted. fn remove_at(idx: usize) { let n = EXT_COUNT.load(Ordering::Relaxed); @@ -869,13 +1037,7 @@ pub(super) unsafe fn alloc( if REGION_LEN.load(Ordering::Relaxed) == 0 { return Err(FERR); } - // What alignment is measured from: the region's base where segments - // stride from it, address zero under `ra_aligned_region`. - let origin = if cfg!(ra_aligned_region) { - 0 - } else { - REGION_BASE.load(Ordering::Relaxed) - }; + let origin = stride_origin(); // Page-aligned requests search from the HIGHEST extent down and settle at // its top; coarsely-aligned ones search from the lowest up, as before. @@ -1418,6 +1580,93 @@ mod tests { // SAFETY: `hdr` is live and unfreed. unsafe { free(hdr.ptr, FIXED_PAGE).expect("free hdr") }; assert_eq!(free_total(), N, "and the region ends whole"); + + // ---- the large-allocation ceiling, reported from rusty_zstd ---- + // + // `docs/plans/finished/esp32-large-alloc-ceiling.md`: a 64 KiB request + // in a 256 KiB region served ONCE, with 192 KiB free. Here against the + // real extent allocator, and pinned to what `dedicated_segments` + // predicts so the sizing API and the backend cannot drift. + // + // Also here rather than standalone, for the same process-wide-state + // reason as §2.1 and §2.9. + assert_eq!(free_total(), N, "the region is whole before this"); + let seg_count = N / SEGMENT_SIZE; + if seg_count >= 2 { + // A request of exactly SEGMENT_SIZE is the reported shape. + let size = SEGMENT_SIZE; + let cost = dedicated_segments(size); + // The whole finding in one assertion: an allocation the size of a + // segment can NEVER share one, because the header owns slice 0. + if size > LARGEST_SHARED_ALLOC { + assert!( + cost >= 2, + "a SEGMENT_SIZE request cannot fit one segment: the header owns slice 0" + ); + } + let served = greedy_dedicated(size); + assert_eq!(free_total(), N, "counting leaves the region whole"); + assert!(served >= 1, "a region of {seg_count} segments serves none"); + + // THE REPORTED SYMPTOM, as a property rather than a placement + // count: the region is left with far more free bytes than the + // payload it managed to serve. At a geometry where a + // segment-sized request is DEDICATED, utilisation cannot reach + // half, because every block drags a header into a second segment. + if cost >= 2 { + assert!( + served * size * 2 <= N + SEGMENT_SIZE, + "dedicated blocks cannot use half the region: served {served} x {size} of {N}" + ); + } + + // And the half the report could not see: the FIRST small + // allocation claims a WHOLE segment (a normal segment is a + // SEGMENT_SIZE reservation), so it costs a large consumer reach. + // This is why the firmware measured 1 where the arithmetic on + // free bytes alone suggested more. + // SAFETY: prim contract - a power-of-two alignment, page multiple. + let seg_taken = + unsafe { alloc(SEGMENT_SIZE, SEGMENT_SIZE, true, false).expect("a segment") }; + let after_small = greedy_dedicated(size); + // SAFETY: `seg_taken` is live and unfreed. + unsafe { free(seg_taken.ptr, SEGMENT_SIZE).expect("free the segment") }; + assert_eq!(free_total(), N, "and the region ends whole"); + assert!( + after_small <= served, + "taking a segment cannot increase the large-allocation reach" + ); + + // `region_for_allocs` must not promise a region that would fail: + // whatever this region actually served, the API's answer for one + // MORE block has to be bigger than this region. + let promised = region_for_allocs(size, served + 1); + assert!( + promised > N, + "region_for_allocs({size}, {}) = {promised} must exceed the {N} that served {served}", + served + 1 + ); + } + } + + /// Serve as many DEDICATED reservations of `size` as the region will take, + /// then hand them all back. The request shape is `segment::huge_alloc`'s: + /// one slice of header, the payload, page-rounded, at `SEGMENT_SIZE` + /// alignment - which is exactly why it cannot share a segment. + fn greedy_dedicated(size: usize) -> usize { + let want = align_up(crate::types::SEGMENT_SLICE_SIZE + size, FIXED_PAGE); + let mut held = Vec::new(); + // SAFETY: prim contract - SEGMENT_SIZE is a power of two, and every + // pointer collected here is freed below before the function returns. + while let Ok(a) = unsafe { alloc(want, SEGMENT_SIZE, true, false) } { + held.push(a.ptr); + } + let n = held.len(); + for p in held { + // SAFETY: each `p` came from the `alloc` above and is unfreed. + unsafe { free(p, want).expect("free a counted reservation") }; + } + n } /// Serve `SEGMENT_SIZE`-aligned segments until the region refuses, then @@ -1456,7 +1705,9 @@ mod tests { let good = good_region_size(budget); assert!(good <= budget, "a budget is a ceiling"); if budget >= MIN_REGION { - assert_eq!(good, 3 * SEG, "three segments at this geometry"); + // The PROPERTY, true at every geometry: whole segments, and the + // remainder is exactly what a round budget strands. + assert_eq!(good, (budget / SEG) * SEG, "whole segments"); assert_eq!(usable_bytes(0, good), good, "every byte is a segment"); assert_eq!( usable_bytes(0, budget), @@ -1465,9 +1716,17 @@ mod tests { ); assert_eq!( budget - good, - 28_672, + budget % SEG, "and that is what the budget was stranding" ); + // The REPORTED numbers, pinned at the geometry they were measured + // on (4 KiB x 16). `ra_segment_size` moves them, and a test that + // asserted them everywhere would fail for the wrong reason. + if SEG == 64 * 1024 { + assert_eq!(good, 3 * SEG, "three segments at the default"); + assert_eq!(good, 196_608); + assert_eq!(budget - good, 28_672); + } } else { assert_eq!( good, 0, @@ -1526,6 +1785,66 @@ mod tests { } } + /// The sizing API a firmware plans with, and the cliff it exists to make + /// visible (`docs/plans/finished/esp32-large-alloc-ceiling.md`). + #[test] + fn dedicated_segments_names_the_large_allocation_cliff() { + use crate::types::{SEGMENT_SIZE as SEG, SEGMENT_SLICE_SIZE as SLICE}; + + // Below the cliff nothing is dedicated: the request is a span that + // packs with its neighbours. + assert_eq!(dedicated_segments(0), 0); + assert_eq!(dedicated_segments(1), 0); + assert_eq!(dedicated_segments(LARGEST_SHARED_ALLOC), 0); + assert_eq!(LARGEST_SHARED_ALLOC, SEG - SLICE, "the header owns slice 0"); + + // One byte over, and the request owns segments outright. TWO of them, + // always: the header cannot share the segment its payload fills. + assert_eq!(dedicated_segments(LARGEST_SHARED_ALLOC + 1), 2); + assert_eq!(dedicated_segments(SEG), 2); + assert_eq!(dedicated_segments(2 * SEG), 3); + + // Monotone, and never less than the payload needs. + let mut prev = 0; + let mut size = 0; + while size < 5 * SEG { + let d = dedicated_segments(size); + assert!(d >= prev, "cost cannot fall as the request grows"); + if d > 0 { + assert!(d * SEG >= size + SLICE, "must hold header plus payload"); + } + prev = d; + size += SLICE / 2 + 1; + } + + // The region a firmware must declare, including the segment the small + // allocations take when the large ones are dedicated. + let three = region_for_allocs(SEG, 3); + assert_eq!(three % SEG, 0, "whole segments"); + assert_eq!(good_region_size(three), three, "already a good size"); + if dedicated_segments(SEG) == 0 { + // A sharing geometry: three spans plus a slice for the smalls. + assert!(three <= 2 * SEG, "sharing should not need a segment each"); + } else { + assert_eq!( + three, + (3 * dedicated_segments(SEG) + 1) * SEG, + "three dedicated runs, plus one segment for everything smaller" + ); + } + // The reported case, pinned at the geometry it was measured on: a + // 256 KiB region is four segments and serves ONE 64 KiB block once a + // small allocation has taken a segment. + if SEG == 64 * 1024 { + assert_eq!(dedicated_segments(64 * 1024), 2); + assert_eq!(region_for_allocs(64 * 1024, 3), 448 * 1024); + assert!( + region_for_allocs(64 * 1024, 3) > 256 * 1024, + "the reported 256 KiB region cannot hold three, and now says so" + ); + } + } + /// `docs/plans/finished/region-alignment-bug.md` §5: for any base, a /// region sized by `good_region_size` either delivers the segments its /// name implies, or the caller is told it did not. Both halves failed @@ -1539,7 +1858,10 @@ mod tests { // against an EXACT length, 12 bytes is still a segment. let base = 0x3fc8_a1e4usize; let n = good_region_size(220 * 1024); - if n >= MIN_REGION { + // The reported case is a DEFAULT-geometry case: a 220 KiB budget is + // three 64 KiB segments. At a raised `ra_segment_size` it is one + // segment or none, and none of the numbers below describe it. + if n >= MIN_REGION && SEG == 64 * 1024 { assert_eq!(n, 196_608); assert_eq!(usable_bytes(base, n), 131_072, "two segments, not three"); assert!(usable_bytes(base, n) < n); @@ -1588,8 +1910,11 @@ mod tests { // probe neither needs nor consumes the process-wide registration. #[cfg(ra_small_profile)] { - const M: usize = 196_608; - const SLACK: usize = 65_536 + 0x1e4; + // Two segments EXACTLY, so the base's run-up costs the last one + // (one segment would be refused as GEOMETRY before MISALIGNED + // could fire). Derived, so `ra_segment_size` moves it. + const M: usize = 2 * crate::types::SEGMENT_SIZE; + const SLACK: usize = crate::types::SEGMENT_SIZE + 0x1e4; static mut MIS: [u8; M + SLACK] = [0; M + SLACK]; let bp = (&raw mut MIS).cast::().expose_provenance(); // Put the base at the report's residue, 0x1e4 past a boundary: diff --git a/crates/rusty_alloc/src/types.rs b/crates/rusty_alloc/src/types.rs index 181d3b6..30d42aa 100644 --- a/crates/rusty_alloc/src/types.rs +++ b/crates/rusty_alloc/src/types.rs @@ -41,12 +41,21 @@ pub const SEGMENT_SLICE_SIZE: usize = 64 * 1024; /// caught it. `good_size` is ABI-visible and G2-pinned against the oracle, so /// the slice is the side that moves. See the const assert in `prim/fixed.rs`. #[cfg(ra_small_profile)] -pub const SEGMENT_SLICE_SIZE: usize = 4 * 1024; +pub const SEGMENT_SLICE_SIZE: usize = if cfg!(ra_segment_size = "256k") { + // The 256 KiB geometry keeps the slice COUNT at 32 (so `Segment` still fits + // slice 0) and doubles the slice instead. 8 KiB is safe on the axis the + // 2 KiB probe failed: a slice must be at least one `prim` page, and + // `FIXED_PAGE` is 4 KiB. + 8 * 1024 +} else { + 4 * 1024 +}; /// Slices per segment (`MI_SLICES_PER_SEGMENT` = 512). #[cfg(not(ra_small_profile))] pub const SLICES_PER_SEGMENT: usize = 512; /// Slices per segment, small profile: 16, so a segment is 64 KiB. +/// `--cfg ra_segment_size="128k"` / `"256k"` raises it — see below. /// /// Raised with the slice halving so `SEGMENT_SIZE` does NOT move. Segment size /// is the wrong lever — it is the granule the region is carved in, and @@ -55,8 +64,64 @@ pub const SLICES_PER_SEGMENT: usize = 512; /// slices leaves 15 usable, and the measured workload needs 13. Holding the /// slice COUNT while shrinking the slice is what collapsed this workload from /// two 64 KiB segments to one 32 KiB one. +/// +/// **That reasoning is about a workload of SMALL objects, and it inverts for a +/// workload whose unit is the segment.** [`LARGE_OBJ_SIZE_MAX`] is +/// `(SLICES_PER_SEGMENT - 1) * SEGMENT_SLICE_SIZE`, because the header owns +/// slice 0 — so at the default geometry the largest object that fits in one +/// segment is **61,440 bytes**, and a 64 KiB request is a HUGE allocation: +/// it reserves `4 KiB + 64 KiB` on a `SEGMENT_SIZE` grid and therefore +/// consumes **two** segments. A 128 KiB request consumes three. That is +/// structural rather than a leak — **no allocation of `SEGMENT_SIZE` can ever +/// share a segment with the metadata that describes it** — and it is why a +/// `rusty_zstd` firmware allocating 64 KiB match tables got exactly one of +/// them out of a 256 KiB region with 192 KiB free +/// (`docs/plans/finished/esp32-large-alloc-ceiling.md`). +/// +/// **So `SEGMENT_SIZE` is the lever for a large-unit workload**, and it is a +/// knob rather than a new default because the two workloads want opposite +/// values: raising it raises the page floor every small workload pays, and +/// raising it raises [`LARGE_OBJ_SIZE_MAX`] with it, which is what turns a +/// segment-sized request back into a span the span allocator can pack. +/// +/// | `ra_segment_size` | slice | slices | `SEGMENT_SIZE` | `LARGE_OBJ_SIZE_MAX` | 64 KiB blocks per segment | bytes per block | +/// |---|---:|---:|---:|---:|---:|---:| +/// | *(unset)* | 4 KiB | 16 | 64 KiB | 61,440 | 0 — huge, 2 segments each | 131,072 | +/// | `"256k"` | 8 KiB | 32 | 256 KiB | 253,952 | **3**, 56 KiB left over | **87,381** | +/// +/// **Why the slice count stops at 32, why 256k doubles the SLICE instead, and +/// why there is no 128 KiB rung.** Two hard constraints bracket this. +/// `Segment` is `48 + 92 * SLICES_PER_SEGMENT` bytes (a `Page` is 88, a +/// `page_off` entry 4) and must fit in slice 0, which caps the count at 44 for +/// a 4 KiB slice; and `SEGMENT_SIZE` must remain a POWER OF TWO, because +/// `segment_of` recovers a segment by masking with `SEGMENT_SIZE - 1` and +/// `segment_map` asserts `1 << WINDOW_SHIFT == SEGMENT_SIZE`. 44 slices is +/// neither, so the next power of two above 32 needs a bigger slice. +/// +/// A 4 KiB x 32 (128 KiB) rung was built and **withdrawn**, for a reason worth +/// keeping: it buys a 64 KiB consumer NOTHING. That request becomes a 16-slice +/// span in a 31-slice segment, so exactly one fits and the block still costs +/// 128 KiB — the same as the default's two-segment huge path, reached by a +/// different route. The lever is not the segment size by itself; it is +/// **`LARGE_OBJ_SIZE_MAX / size`, the number of blocks that pack into one +/// segment**, and that only exceeds 1 when the slice grows too. (It also +/// segfaulted 11 runs in 12 under the concurrent host battery where the +/// default and `"256k"` never did — chased far enough to know it is real, not +/// far enough to name it, and recorded in +/// `docs/plans/finished/esp32-large-alloc-ceiling.md`.) +/// +/// The cost of raising it, stated: a segment is the granule the region is +/// carved in, so a region rounds DOWN to a whole number of them +/// ([`crate::prim::fixed::good_region_size`]). At `"256k"` a 256 KiB region is +/// one segment and a 320 KiB region is still one, stranding 64 KiB — size the +/// region from the knob, never from a round number. And the page floor is +/// `(bins touched) * SEGMENT_SLICE_SIZE`, so `"256k"` doubles it. #[cfg(ra_small_profile)] -pub const SLICES_PER_SEGMENT: usize = 16; +pub const SLICES_PER_SEGMENT: usize = if cfg!(ra_segment_size = "256k") { + 32 +} else { + 16 +}; /// Segment size (`MI_SEGMENT_SIZE` = 32 MiB on 64-bit): the unit of OS/arena /// allocation, and the shift+mask that takes any block pointer to its segment @@ -142,11 +207,26 @@ mod tests { assert_eq!(SEGMENT_SIZE, 32 * 1024 * 1024); // The small profile's geometry is a DECISION, pinned here so moving it // has to be meant. 4 KiB slices x 16 = a 64 KiB segment - // (docs/plans/small-metal.md §2.10). - #[cfg(ra_small_profile)] - assert_eq!(SEGMENT_SIZE, 64 * 1024); + // (docs/plans/small-metal.md §2.10). The segment size is a knob + // (`ra_segment_size`), so pin each arm rather than pinning the + // default's numbers and passing vacuously at every other setting. #[cfg(ra_small_profile)] - assert_eq!(SEGMENT_SLICE_SIZE, 4 * 1024); + { + let (slice, slices) = if cfg!(ra_segment_size = "256k") { + (8 * 1024, 32) + } else { + (4 * 1024, 16) + }; + assert_eq!(SEGMENT_SLICE_SIZE, slice); + assert_eq!(SLICES_PER_SEGMENT, slices); + assert_eq!(SEGMENT_SIZE, slice * slices); + // The mask in `segment_of` needs this, and so does `segment_map`. + assert!(SEGMENT_SIZE.is_power_of_two()); + // The routing constant must track the geometry, or a segment-sized + // request is sent to a dedicated huge segment while a span would + // have held it — the defect this knob exists for. + assert_eq!(LARGE_OBJ_SIZE_MAX, (slices - 1) * slice); + } assert_eq!(BIN_FULL, 74); } } diff --git a/crates/rusty_alloc/tests/region.rs b/crates/rusty_alloc/tests/region.rs index 81b04fa..5c22e45 100644 --- a/crates/rusty_alloc/tests/region.rs +++ b/crates/rusty_alloc/tests/region.rs @@ -13,12 +13,13 @@ #![cfg(ra_small_profile)] use rusty_alloc::prim::fixed::{ - FERR_REGISTERED, MIN_REGION, REGION_ALIGN, Region, good_region_size, usable_bytes, + FERR_REGISTERED, MIN_REGION, REGION_ALIGN, Region, good_region_size, region_for, usable_bytes, }; -/// The report's budget: 220 KiB, which a round declaration strands 28,672 -/// bytes of and `good_region_size` trims to three whole segments. -const N: usize = good_region_size(220 * 1024); +/// Three whole segments, whatever the geometry. At the default small profile +/// this IS the report's case — a 220 KiB budget trimmed to 196,608 — and +/// `--cfg ra_segment_size` moves it without making the test vacuous. +const N: usize = region_for(3 * rusty_alloc::types::SEGMENT_SIZE); /// A plain `static`, exactly as a firmware declares it: 16-byte aligned, so /// the linker owes it no gap, and `size_of` is the heap it serves. @@ -29,8 +30,12 @@ static OTHER: Region = Region::new(); #[test] fn a_region_given_once_serves_exactly_what_its_size_says() { - assert_eq!(N, 196_608); - assert_eq!(Region::::USABLE, 196_608); + assert_eq!(N, 3 * rusty_alloc::types::SEGMENT_SIZE, "three segments"); + assert_eq!(Region::::USABLE, N); + if N == 196_608 { + // The default geometry: the reported budget, pinned. + assert_eq!(good_region_size(220 * 1024), N); + } assert_eq!( core::mem::size_of::>(), N, @@ -70,7 +75,7 @@ fn a_region_given_once_serves_exactly_what_its_size_says() { // The handoff returns what the allocator can serve: three whole segments. let usable = heap.give().expect("first give of an exact region"); - assert_eq!(usable, 196_608); + assert_eq!(usable, N); assert_eq!(usable, Region::::USABLE); // Given once: a second call is refused without touching the bytes. diff --git a/docs/LEDGER.md b/docs/LEDGER.md index c5c1240..a5abde4 100644 --- a/docs/LEDGER.md +++ b/docs/LEDGER.md @@ -4,6 +4,50 @@ One entry per milestone/brick: what landed, the numbers with their method lines, what was reverted and **which kind** of revert (measured-worse vs within-noise). Newest first. +## LARGE-ALLOCATION CEILING — a 64 KiB block cost two segments; a geometry knob takes 1 block to 3 (2026-09-09) + +`docs/plans/finished/esp32-large-alloc-ceiling.md`, reported from the +`rusty_zstd` bare-metal work: in a 256 KiB region `rusty_alloc` served ONE +64 KiB allocation and refused the second with 192 KiB free. + +**Confirmed on the board, and their hypothesis was right.** A segment's slice 0 +is its header, so `LARGE_OBJ_SIZE_MAX = SEGMENT_SIZE - SEGMENT_SLICE_SIZE` = +61,440; one byte over and `huge_alloc` reserves `4 KiB + size` on a +`SEGMENT_SIZE` stride, spanning two segments. Structural — no allocation of +`SEGMENT_SIZE` can share a segment with its own metadata. The sweep they asked +for puts the cliff at 61,440 exactly. Their 50 % estimate read 25 % on silicon +because of a cost they could not see: **the first small allocation claims a +whole segment** (`used=135168` = 69,632 + 65,536 for a `Vec` spine). + +**Fix: `--cfg ra_segment_size="256k"`** (8 KiB slice x 32), which raises +`LARGEST_SHARED_ALLOC` to 253,952 so a 64 KiB request is a span three of which +pack into one segment. Same board, same 256 KiB region: **1 block -> 3**, 25 % +-> 75 % utilisation. Their kill test ("at least 3") passes, and "no region size +works on an S3" is no longer true. Opt-in: it doubles the page floor +`(classes touched) x slice` that a small-object workload pays. + +Also shipped, because the report's second ask was diagnosability: +`LARGEST_SHARED_ALLOC`, `dedicated_segments(size)`, `region_for_allocs(size, +count)` (which counts the small-allocation segment), and +`region_capacity() -> (free_segments, largest_servable)` — on the failing call +`(1, 61440)` against 126,976 free bytes. The README's floor model said the cost +"amortises as the working set grows"; that is true for small objects and +inverts for segment-sized ones, and now says so. + +**A rung built and withdrawn.** 4 KiB x 32 (128 KiB) buys a 64 KiB consumer +NOTHING — one 16-slice span in a 31-slice segment is still 128 KiB per block, +the default's cost by another route. The lever is +`LARGEST_SHARED_ALLOC / size`, not `SEGMENT_SIZE`. It also segfaulted 11/12 +under the concurrent host battery where the default and 256k are 0/40 and 0/12; +real, geometry-specific, and NOT root-caused. Kept out of the shipped set and +written down in §9.5 in case it is latent rather than local. + +**Gates.** 20 suites green at both shipped geometries; CI runs the whole suite +at 256k rather than building it; gate-selftest 11/11 (new: drop the header +slice from `dedicated_segments` and the sizing test goes red); the reproduction +is a permanent property-based test against the real extent allocator. Unsafe ++4, all `#[cfg(test)]` — the fix is arithmetic and adds none to shipped code. + ## REGION ALIGNMENT DISSOLVED — segments stride from the base; 24,144 B of stack back, +3 instructions per free, one knob (2026-09-09) `docs/plans/finished/region-alignment-dissolve.md`: the Janus firmware's diff --git a/docs/plans/finished/esp32-large-alloc-ceiling.md b/docs/plans/finished/esp32-large-alloc-ceiling.md new file mode 100644 index 0000000..327e5b2 --- /dev/null +++ b/docs/plans/finished/esp32-large-alloc-ceiling.md @@ -0,0 +1,264 @@ +# A 64 KiB allocation fails with 192 KiB of the region free (ESP32-S3, 2.0.5) + +Reported 2026-09-09 from the `rusty_zstd` bare-metal work. Measured on real +silicon, not inferred: ESP32-S3 rev v0.2, `xtensa-esp32s3-none-elf`, +`rusty_alloc 2.0.5` + `rusty_alloc-api 2.0.5` from crates.io, built +`default-features = false` with `--cfg ra_single_threaded --cfg +ra_small_profile`, region handed over as +`prim::fixed::Region<{ good_region_size(N) }>` exactly as the README prescribes. + +**Status: CONFIRMED and FIXED, 2026-09-09. See section 9.** Reproduced on the +same part, root-caused, and closed with a geometry knob that takes the reported +case from 1 block to 3. The verdict on "by design or defect" is *both*, and +section 9 says which half is which. + +## The one-paragraph version + +In a **256 KiB region (four 64 KiB segments, 262,144 bytes reported usable)**, +`rusty_alloc` serves **exactly one** 64 KiB allocation. The second fails. At +that moment 64 KiB is live and **192 KiB of the region is unused**. + +## Minimal reproduction, no zstd involved + +```rust +static REGION: Region<{ good_region_size(256 * 1024) }> = Region::new(); +#[global_allocator] +static ALLOC: rusty_alloc_api::RustyAlloc = rusty_alloc_api::RustyAlloc; + +// in main, after REGION.give().unwrap(): +let mut big: Vec> = Vec::new(); +for i in 0..8 { + println!("requesting 64 KiB block #{i} ..."); + big.push(vec![0u8; 64 * 1024]); // #1 never returns + println!("64 KiB block #{i} OK (live {} KiB)", (i + 1) * 64); +} +``` + +Observed: + +```text +region 262144 bytes declared, 262144 usable +probe requesting 64 KiB block #0 ... +probe 64 KiB block #0 OK (live 64 KiB) +probe requesting 64 KiB block #1 ... +memory allocation of 65536 bytes failed <- 192 KiB still free +``` + +## What DOES work, so the shape is clear + +Same region, same build, from a fresh heap: + +| request shape | result | +|---|---| +| 16 x 656 B | OK | +| 4 x 32 KiB (128 KiB live, 50% of region) | **OK** | +| a further 656 B after those | OK | +| 1 x 64 KiB | OK | +| **2 x 64 KiB** | **fails, 192 KiB free** | + +So small and medium blocks pack fine. The cliff is exactly at the **segment +size**: a request at or near 64 KiB appears to need more than one 64 KiB +segment, which leaves a four-segment region able to satisfy one of them. + +## Hypothesis (yours to confirm — we have not read the internals) + +A 64 KiB user request cannot fit inside a 64 KiB segment once the segment's own +metadata is accounted for, so it takes **two** segments. A 256 KiB region has +four, block #0 consumes two, and the remainder cannot serve another. If that is +right, the usable fraction for segment-sized requests is roughly 50% at best and +the failure is structural rather than a leak. + +Two experiments that would discriminate, both cheap on your rig: + +1. Sweep the request size across the segment boundary (60, 62, 64, 66 KiB) in a + fixed region and record the last size that still gives two successes. If the + cliff sits just below `SEGMENT_SIZE`, the metadata-overflow story holds. +2. Report free segments alongside free bytes at the point of failure. Free bytes + alone (192 KiB) look like plenty and hide the real constraint. + +## Why it matters to a consumer, concretely + +`rusty_zstd` 0.2.5 now builds and runs `no_std + alloc` on this part (it +round-trips at levels 1, 3 and 5 on the S3). It allocates its match tables in +units of tens of kilobytes, including 64 KiB and larger. Against `esp-alloc +0.11` on **the same firmware source, one cargo feature apart**: + +| | esp-alloc 0.11 | rusty_alloc 2.0.5 | +|---|---|---| +| 8 KiB payload, L1/L3/L5 round trip | **PASS** | **OOM** | +| peak heap the workload needs | 175,832 B | (never reached) | +| smallest heap that still round-trips | **176 KiB** | fails at 192 and 256 KiB | +| 320 KiB region | n/a | does not link: "Main stack is smaller than 8192 bytes" | + +The 320 KiB row is the ceiling on this part: RAM is a fixed map, so a bigger +region comes straight out of `.stack` until the linker refuses. There is +therefore **no region size on an ESP32-S3 at which this consumer can use +rusty_alloc**, which is why this is worth your time even if the behaviour turns +out to be intended. + +Static cost, for completeness, both arms at the same 192 KiB budget on the +linked ELF (`.data + .bss + .stack` sums to a constant, so the stack column is +the one that moves): + +| | esp-alloc | rusty_alloc | delta | +|---|---|---|---| +| `.bss` | 196,728 | 198,788 | +2,060 | +| `.data` | 2,432 | 2,356 | −76 | +| `.stack` | 135,848 | 133,852 | −1,996 | +| `.text` + `.rodata` | 222,685 | 230,221 | +7,536 | + +`.stack` shrank by `Δ.bss + Δ.data` to within 12 bytes, exactly as your README +says it does. + +## Against the README + +`README.md` states the floor as *"64 KiB for `rusty_alloc` ... against 8 KiB for +`esp-alloc`"*, with *"a size-class page allocator's [floor] is (classes touched) +x (page size), independent of bytes requested"*, and adds that the floor +*"amortises as the working set grows"*. + +The model predicts a fixed additive cost, so a 112 KiB working set should fit a +256 KiB region with room to spare. It does not, and the amortisation claim +inverts for this consumer: the cost is not a fixed floor but a **granularity +tax that scales with how many segment-sized blocks are live**. Whatever the +verdict on the code, that paragraph is worth a sentence about large allocations. + +## What "fixed" would look like + +A kill test a stranger can run, in the shape this repo already uses: + +```text +Region<{ good_region_size(256 * 1024) }>, --cfg ra_single_threaded --cfg ra_small_profile +allocate 64 KiB blocks in a loop +PASSES WHEN: at least 3 succeed before the region is exhausted +TODAY: 1 +``` + +## Provenance + +- Firmware: `rusty_zstd`'s `bare-metal/esp32s3` (two arms, one source, the + allocator selected by a cargo feature) at commit `8941050`. +- Every number above is from the board over USB serial, not a host simulation. +- Nothing has been filed against a public repository; this note is the hand-off. + +--- + +## 9. Resolution (2026-09-09) + +**The report is correct in every particular, including its hypothesis.** +Reproduced on a XIAO ESP32-S3 with the consumer's own shape — a 256 KiB +`Region`, 64 KiB blocks in a loop: + +```text +[big] geometry: SEGMENT_SIZE=65536 LARGEST_SHARED_ALLOC=61440 dedicated_segments(65536)=2 +[big] before #0: used=0 free=262144 total=262144 | free_segments=4 largest_servable=258048 +[big] block #0 OK (live 64 KiB) +[big] before #1: used=135168 free=126976 total=262144 | free_segments=1 largest_servable=61440 +[big] block #1 REFUSED +[big] served 1 blocks of 65536 +``` + +### 9.1 The mechanism, and their experiment 1 + +A segment's slice 0 holds its header, so `LARGE_OBJ_SIZE_MAX` is +`SEGMENT_SIZE - SEGMENT_SLICE_SIZE` = **61,440**. Above that a request is +routed to `huge_alloc`, which reserves `4 KiB + size` and must start on a +`SEGMENT_SIZE` stride — so it spans **two** segments. The sweep they asked for, +computed from the same constants the allocator routes on: + +| request | shares a segment? | dedicated segments | +|---|---|---:| +| 61,440 | yes | 0 | +| **61,441** | no | **2** | +| 65,536 | no | 2 | +| 131,072 | no | 3 | + +**The cliff is at 61,440, one slice below `SEGMENT_SIZE`, exactly as their +metadata-overflow story predicted.** It is structural, not a leak: no +allocation of `SEGMENT_SIZE` can share a segment with the metadata describing +it, at any segment size. + +Their arithmetic said "roughly 50 % at best" and the board said 25 %, because +of a cost they could not see: **the first small allocation claims a whole +segment.** `used=135168` above is block #0's 69,632 plus a 65,536 segment for +the `Vec` spine. Two segments went to one 64 KiB block, one to bookkeeping, and +the fourth could not hold the second block. + +### 9.2 Their experiment 2, shipped + +`region_stats` reports free BYTES, and free bytes hide this. Added +`prim::fixed::region_capacity() -> (free_segments, largest_servable)`, which on +the failing call reads `(1, 61440)` against 126,976 free bytes — the question +answered in the units that decide it. + +### 9.3 Verdict: by design AND a defect + +- **By design:** the two-segment cost of a segment-sized allocation follows + from mimalloc's segment/header structure and cannot be removed at a given + `SEGMENT_SIZE`. Putting the header out of band was considered and rejected + before (`segment-tax.md` F3): every `free` resolves its segment by masking + the pointer, and a table lookup there is a cost on the hottest path in the + crate for a consumer-specific win. +- **A defect:** the README taught a floor model that inverts for this workload, + there was no way to size a region correctly in advance, nothing reported the + real constraint, and there was no escape hatch. All four are fixed. + +### 9.4 The fix, measured + +**`--cfg ra_segment_size="256k"`** moves the small profile to an 8 KiB slice x +32 slices, so `LARGEST_SHARED_ALLOC` becomes 253,952 and a 64 KiB request is a +span the allocator packs three-to-a-segment. Same board, same 256 KiB region: + +| geometry | blocks served | payload live | region used | +|---|---:|---:|---:| +| default (64 KiB segment) | 1 | 64 KiB | 25 % | +| `ra_segment_size="256k"` | **3** | **192 KiB** | **75 %** | + +Their kill test — *"PASSES WHEN: at least 3 succeed"* — **passes**. The +claim that "there is no region size on an ESP32-S3 at which this consumer can +use rusty_alloc" no longer holds: 256 KiB at this geometry serves 3 x 64 KiB +with 56 KiB of slices left for everything smaller, inside the 320 KiB ceiling. + +**What was NOT verified, stated plainly.** The kill test in section 8 was run, +and passes. `rusty_zstd`'s actual workload was not: this end measured 64 KiB +blocks in a loop, not match tables through a real round trip. The peak that +firmware reported, 175,832 bytes, is inside the 253,952 usable at this geometry +and its 64 KiB units pack three to a segment, so the arithmetic says it fits — +but packing depends on the live SET, and only that firmware can run it. If it +does not fit, `region_capacity()` and `region_for_allocs` are now there to say +why in one line rather than a bisect. + +Also shipped: `LARGEST_SHARED_ALLOC`, `dedicated_segments(size)` and +`region_for_allocs(size, count)` so the ceiling is a compile-time answer rather +than a board discovery; `region_for_allocs` counts the small-allocation segment +from 9.1. The README's floor section now states the large-allocation case with +these numbers. + +### 9.5 A rung that was built and withdrawn — worth recording + +A 4 KiB x 32 (128 KiB) geometry was implemented first and removed, for a reason +that generalises: **it buys a 64 KiB consumer nothing.** The request becomes a +16-slice span in a 31-slice segment, so exactly one fits and the block still +costs 128 KiB — identical to the default's two-segment huge path, reached by a +different route. The lever is not `SEGMENT_SIZE` by itself but +`LARGEST_SHARED_ALLOC / size`, the number that pack into one segment, and that +only exceeds 1 when the slice grows too. + +It also **segfaulted 11 runs in 12** under the concurrent host battery +(`tests/secure.rs`, default features), where the default and `"256k"` geometries +are 0/40 and 0/12. It passes single-threaded and under `debug_checks`, so it is +a concurrency-sensitive fault specific to 4 KiB x 32. Chased far enough to know +it is real and to keep the geometry out of the shipped set; **not** root-caused. +Recorded here rather than dropped, because it may be a latent fault that this +geometry merely exposes. + +### 9.6 Gates + +Whole suite green at the default and at `"256k"` (20 suites each), and CI now +runs the full suite at the new geometry rather than merely building it. The +mutation self-test gained an eleventh case: drop the header slice from +`dedicated_segments` and the sizing test goes red (11/11 fire). The reproduction +itself is a permanent test against the real extent allocator, expressed as a +property — a region of dedicated blocks cannot reach half utilisation — so it +does not depend on brittle placement arithmetic. Unsafe census +4, all +`#[cfg(test)]`; the fix adds no unsafe to shipped code. diff --git a/tools/gate-selftest.sh b/tools/gate-selftest.sh index ed807eb..faad2db 100644 --- a/tools/gate-selftest.sh +++ b/tools/gate-selftest.sh @@ -139,6 +139,12 @@ run_case "FIXED_REGION is false where an OS exists" "crates/rusty_alloc/src/li # startup panic the Janus firmware reported. run_case "init_region refuses a misaligned exact region" "crates/rusty_alloc/src/prim/fixed.rs" 's/if usable_bytes\(base, len\) < usable_bytes\(0, len\) \{/if false {/' "--lib prim::fixed::tests::a_misaligned_exact_region" "--cfg ra_single_threaded --cfg ra_small_profile" +# --- esp32-large-alloc-ceiling: a segment-sized request costs TWO segments - +# Drop the header slice from the price and a 64 KiB request looks like it fits +# one segment. That is the arithmetic a firmware sizes its region with, and the +# defect the rusty_zstd report measured as one block served from four segments. +run_case "dedicated_segments prices the header slice" "crates/rusty_alloc/src/prim/fixed.rs" 's/\(crate::types::SEGMENT_SLICE_SIZE \+ size\)\.div_ceil/(size).div_ceil/' "--lib prim::fixed::tests::dedicated_segments" "--cfg ra_single_threaded --cfg ra_small_profile" + echo if ((fail > 0)); then echo "GATE SELFTEST FAILED: $fail of $((pass + fail)) gates did not fire." diff --git a/tools/unsafe-baseline.txt b/tools/unsafe-baseline.txt index 16bafab..4617c86 100644 --- a/tools/unsafe-baseline.txt +++ b/tools/unsafe-baseline.txt @@ -6,7 +6,7 @@ 6 crates/rusty_alloc/src/options.rs 12 crates/rusty_alloc/src/os.rs 38 crates/rusty_alloc/src/page.rs - 33 crates/rusty_alloc/src/prim/fixed.rs + 37 crates/rusty_alloc/src/prim/fixed.rs 8 crates/rusty_alloc/src/prim/mock.rs 17 crates/rusty_alloc/src/prim/mod.rs 27 crates/rusty_alloc/src/prim/unix.rs