Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion technical-documentation/engineering/rendering-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,14 @@ The trail ends here: the D3D11 fast path that replaced all of it is measured in

## Measurement hazards

### A synthetic fixture cannot price a decoder

The thread-count experiment above was run first on the generated fixture and concluded that thread count **did not matter at all** — one thread measured −0.0 % wall and −12.6 % CPU, which read as "the decoder has so much slack the knob does nothing". The same experiment on the public bundle inverts it completely: one thread is **+70 % wall**.

The fixture is flat fills and sharp text, generated to exercise the compositor. It is trivially decodable, so the decoder was never close to being the constraint and the knob had nothing to act on. It is a fine instrument for a composite measurement and a misleading one for a decode measurement — and nothing in the run's own gates says so: drift, spread and output equality were all clean on the wrong answer.

**Rule: match the fixture to the stage under test.** If the thing being measured is decode, the source has to be something a decoder actually works at.

### A baseline you built yourself is not a control

The most expensive mistake in this record's macOS chapter, and it passed every gate the harness has. An entry-point change measured **−17.2 %** against what was called "before" — three cycles, closing drift 0.9990, byte-identical output, MAD of 6 ms. Every check green. The "before" was **a variant of the same branch**, rebuilt from an incremental `dist/`. Measured again against `origin/main` built clean, the same change was worth **−0.1 %**, and the 3.9 s it claimed to remove turned out to be absent from the *shipped, unmodified* application too — 4208 ms in the morning, 481 ms in the evening, same binary, most likely memory pressure on an 8 GiB machine.
Expand Down Expand Up @@ -562,6 +570,18 @@ Unit tests never look at a pixel. The `native*` arms write real files: export th

## Rejected routes

### Capping the macOS decoder's thread count

**What it was.** After the export moved to the software H.264 decoder it runs with `thread_count = 0`, which in libavcodec means *automatic* — the decoder picks, from the CPU count and its own threading model, and the number it actually chose was never read back here. The export's CPU-seconds went 8.4 → 29.8. Since the walk is bound by the encoder and the decoder has seconds of slack, capping its threads looked like free CPU. **What the measurement said.** It is not free and it does not return CPU. Public bundle, S4, three cycles with a floor inside each, closing drift 0.9979, output identical across variants:

| decode threads | cost | CPU s |
|---|---:|---:|
| **auto (default)** | **1.044×** | 30.3 |
| 2 | 1.056× | 29.2 |
| 1 | **1.775×** | 27.4 |

**One thread costs +70 % of wall clock to return 9.6 % of CPU** — it throws away nearly the whole decode gain, landing back near the 2.002× the shipped build measures. The premise was wrong about the size of the effect, not its sign: CPU-seconds do fall, by 30.3 → 27.4, but nothing like proportionally, because decoding N frames costs roughly the same total work however many threads share it. Threads mostly redistribute that work rather than reduce it, and the ~10 % that does disappear is plausibly the thread pool's own overhead — which was not isolated, so treat the mechanism behind that last 10 % as unexplained rather than established. The profile shows the mechanism cleanly — at one thread `decode.screen` goes 1.05 s → 6.17 s while `enc.send_frame` goes 7.61 s → 2.16 s, the decoder eating the slack the encoder-bound pipeline left it, until the slack runs out. **One-line reason not to re-propose:** threads do not buy CPU-seconds back, and by the time the cap is low enough to matter it has already cost the export more than the optimisation gained.

### A dedicated encode thread on macOS

**What it was.** After the decode change above, the macOS export profile was two waits and almost nothing else: `enc.send_frame` 7.658 s (40.6 %, the CPU waiting on VideoToolbox) and `gpu.wait` 7.501 s (39.8 %, the CPU waiting on Metal). Two different engines, one serial loop. Put the encoder on its own thread — bounded queue, backpressure, error propagation, `av_frame_free` on the consumer side — and the two waits overlap. It is also the move that was worth −30 % on the Linux path. **What the measurement said.** **+0.2 %.** Three cycles, an ffmpeg floor inside each, closing drift 1.0003, machine 84–85 % idle: serial 23 039 ms / 1.308× floor, threaded 22 995 ms / 1.305×. Output byte-identical either way. A probe on the producer's blocking says exactly why: blocking on a full queue measured **7.631 s** where blocking inside `avcodec_send_frame` had measured **7.658 s** — 0.4 % apart. The thread **moved** the wait, it did not remove it. The export is bound by VideoToolbox's encode throughput, not by CPU serialisation: the main thread's own work is 9.64 s across 3600 frames (2.68 ms/frame, enough to feed 373 fps) against an encoder that absorbs about 191. **One-line reason not to re-propose:** the encoder is the constraint and no scheduling changes that; it could pay on an M-series Pro/Max/Ultra with more encode blocks, but that is a hypothesis and shipping unmeasured concurrency on it is the mistake this record exists to prevent.
Expand Down Expand Up @@ -668,7 +688,7 @@ the bench runs on the reference machine.
- **macOS export startup can cost 4 s, and nobody has reproduced it on demand.** Measured repeatedly at 4208–4502 ms between the CLI's `started` event and the first composed frame — 18 % of a 60 s export, 71 % of a 5 s one — then gone, on the same shipped binary, hours later (481 ms). It is not the compositor (init is 2.4 ms, runtime MSL compilation included), not the `<video>` metadata probes (13 ms and 6 ms), not the CLI prologue (24 ms total), and not the renderer entry point (measured at −0.1 %). It correlates with memory pressure on an 8 GiB machine — `387M unused / 2613M compressor` while it reproduced, `564M unused / 1837M compressor` after — which would fit faulting ~1.8 MB of module chunks out of a 274 MB `app.asar` while the compressor thrashes: seconds of wall clock, no CPU in either process, cost independent of the media. Untested. Recreating the pressure deliberately and watching it return is what would settle it, and then whether asar size is the lever.
- **10-bit and HEVC decode on macOS are unmeasured.** The export's decode predicate is `codec_id == H264 && format == YUV420P`, so both keep VideoToolbox untested. HEVC is the case most likely to invert the result, since its software decoder is materially more expensive. 10-bit needs work beyond the predicate first: `mac_frames::CpuFrames` converts to 8-bit NV12, so routing 10-bit through the software path would silently truncate — the predicate is currently what prevents that.
- **The macOS preview's decode backend has never been measured.** `DecodeIntent` splits preview from export precisely so the preview could keep the old arbitration; the export won on throughput, but the preview scrubs, where seek latency after `avcodec_flush_buffers` may matter more, and it shares the machine with the editor UI. Changing it without measuring it would be the same mistake the export change corrects.
- **The energy cost of software decode on macOS is unmeasured.** The export burns 3.5× the **CPU-seconds** it used to (8.4 → 29.8 s), and that is the only thing measured. It does not follow that energy moved by the same factor, and the earlier version of this line said it did: on an M-series the P and E cores draw very differently, clock is not fixed, and a shorter run at higher occupancy can spend less total energy than a longer one — racing to idle. Whether this trade costs or saves battery is genuinely unknown; `powermetrics` would answer it and needs sudo. Capping `thread_count` below the core count is the obvious lever, and untried: the walk is encoder-bound, so the decoder only has to be fast enough to stop being the constraint.
- **The energy cost of software decode on macOS is unmeasured, and the CPU figure is not a proxy for it.** The export burns 3.5× the **CPU-seconds** it used to (8.4 → 29.8 s), and that is the only thing measured. It does not follow that energy moved by the same factor: on an M-series the P and E cores draw very differently, clock is not fixed, and a shorter run at higher occupancy can spend less total energy than a longer one — racing to idle. Nor is the jump waste: VideoToolbox does the same decoding in a fixed-function block that CPU accounting never sees, so the work did not grow, it moved somewhere visible and got 12× faster on the way. Capping the decoder's threads does **not** recover it (see Rejected routes); what it would buy is lower peak core occupancy — how unusable the machine feels during an export — which is a different question and also unmeasured. `powermetrics` would answer the energy half and needs sudo.
- **The C0→C8 table rests on one run, on one machine.** [Recorded above](#one-admissible-run--2026-07-27) and admissible on its own gate, but a single sweep: the C5–C7 plateau is the part most likely to move under a second run, since the layers it prices are individually smaller than the machine's own noise. A repeat on a cool machine — and on the discrete-GPU box that is [owed anyway](#known-gaps) — would settle whether those three are genuinely free or merely under the floor.
- **The fixture media is not versioned, and its cursor track has no provenance entry.** `crates/fixture/fixture.json` documents the exact `-c copy` cuts for `screen.mp4` and `webcam.mp4` (which is what made the run above reproducible), but says nothing about `screen.cursor.json`, which C7 needs. It happens to be the raw, uncut `.cursor.json` beside the origin recording — the loader windows it itself at `offset 100_000 ms, 6 s` ([`bench.rs:70`](../../crates/poc-d3d/src/bench.rs)), matching the manifest's `cut_offset_s: 100`. That is recoverable by reading the code, not by reading the manifest; the manifest should carry it.
- **One bench fixture is still corrupt, from a bug since fixed.** Two concurrent saves used to be able to interleave and truncate a project file, which destroyed at least two real ones (a valid JSON prefix followed by the tail of a longer version). `proj_de6ffaaa` (`os_parity`) is still in that state — 4006 bytes, 3485 of them valid JSON — with a byte-exact backup beside it (`*.corrupt-backup-20260716`); recovery is mechanical (truncate to the 3485-byte prefix). The bug itself is gone: `DocumentService` now serialises saves through a per-project write queue and writes atomically (unique temp file → `fsync` → rename), which is why Gate G0 was run on `proj_5b3ac6bc` instead. The reference project for M1–M4 is `proj_a7468696`.
Expand Down
Loading