From c748b2f7a3cf47057e81926516692abae8506cf8 Mon Sep 17 00:00:00 2001
From: Etienne Lescot <etiennelescot@gmail.com>
Date: Fri, 4 Sep 2026 13:23:21 +0200
Subject: [PATCH 1/2] docs(perf): capping the macOS decoder's threads is a
 loss, and why the fixture said otherwise
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

#592 proposed capping `thread_count` on the export's software decoder to
give back the CPU-seconds that switching to it cost. Measured, it does
neither: it does not return the CPU, and it costs the wall clock the
decode change bought.

    decode threads      cost      CPU s
    auto (default)     1.044x      30.3
    2                  1.056x      29.2
    1                  1.775x      27.4

One thread is +70 % of wall clock to return 9.6 % of CPU — back near the
2.002x the shipped build measures. The premise was wrong: decoding N
frames costs the same total work however many threads do it, so threads
move wall clock and core occupancy, never CPU-seconds.

The stage profile shows the mechanism. At one thread `decode.screen` goes
1.05 s -> 6.17 s while `enc.send_frame` goes 7.61 s -> 2.16 s: the decoder
eats the slack an encoder-bound pipeline was leaving it, and the total
holds — until the slack runs out, which on real content it does.

A SECOND HAZARD, and the one worth reading. This experiment was run first
on the generated fixture and concluded thread count did not matter at all
(one thread: -0.0 % wall, -12.6 % CPU). The same experiment on the public
bundle inverts it. The fixture is flat fills and sharp text, built to
exercise the compositor; it is trivially decodable, so the decoder was
never near being the constraint and the knob had nothing to act on. Every
gate on that run was clean — drift, spread, output equality — on an answer
that was wrong. Match the fixture to the stage under test.

The known gap on energy is corrected at the same time. It said the CPU
jump was 3.5x the energy; it is not a proxy at all, and the jump is not
waste either — VideoToolbox does the same decoding in a fixed-function
block that CPU accounting never sees, so the work moved somewhere visible
and got 12x faster on the way. What capping threads would buy is lower
peak core occupancy, which is a different question and still unmeasured.
---
 .../engineering/rendering-performance.md      | 22 ++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/technical-documentation/engineering/rendering-performance.md b/technical-documentation/engineering/rendering-performance.md
index 8016e24ff..878db160d 100644
--- a/technical-documentation/engineering/rendering-performance.md
+++ b/technical-documentation/engineering/rendering-performance.md
@@ -376,6 +376,14 @@ The trail ends here: the D3D11 fast path that replaced all of it is measured in
 
 ## Measurement hazards
 
+### A synthetic fixture cannot price a decoder
+
+The thread-count experiment above was run first on the generated fixture and concluded that thread count **did not matter at all** — one thread measured −0.0 % wall and −12.6 % CPU, which read as "the decoder has so much slack the knob does nothing". The same experiment on the public bundle inverts it completely: one thread is **+70 % wall**.
+
+The fixture is flat fills and sharp text, generated to exercise the compositor. It is trivially decodable, so the decoder was never close to being the constraint and the knob had nothing to act on. It is a fine instrument for a composite measurement and a misleading one for a decode measurement — and nothing in the run's own gates says so: drift, spread and output equality were all clean on the wrong answer.
+
+**Rule: match the fixture to the stage under test.** If the thing being measured is decode, the source has to be something a decoder actually works at.
+
 ### A baseline you built yourself is not a control
 
 The most expensive mistake in this record's macOS chapter, and it passed every gate the harness has. An entry-point change measured **−17.2 %** against what was called "before" — three cycles, closing drift 0.9990, byte-identical output, MAD of 6 ms. Every check green. The "before" was **a variant of the same branch**, rebuilt from an incremental `dist/`. Measured again against `origin/main` built clean, the same change was worth **−0.1 %**, and the 3.9 s it claimed to remove turned out to be absent from the *shipped, unmodified* application too — 4208 ms in the morning, 481 ms in the evening, same binary, most likely memory pressure on an 8 GiB machine.
@@ -562,6 +570,18 @@ Unit tests never look at a pixel. The `native*` arms write real files: export th
 
 ## Rejected routes
 
+### Capping the macOS decoder's thread count
+
+**What it was.** After the export moved to the software H.264 decoder it runs with `thread_count = 0`, i.e. one thread per core, and the export's CPU-seconds went 8.4 → 29.8. Since the walk is bound by the encoder and the decoder has seconds of slack, capping its threads looked like free CPU. **What the measurement said.** It is not free and it does not return CPU. Public bundle, S4, three cycles with a floor inside each, closing drift 0.9979, output identical across variants:
+
+| decode threads | cost | CPU s |
+|---|---:|---:|
+| **auto (default)** | **1.044×** | 30.3 |
+| 2 | 1.056× | 29.2 |
+| 1 | **1.775×** | 27.4 |
+
+**One thread costs +70 % of wall clock to return 9.6 % of CPU** — it throws away nearly the whole decode gain, landing back near the 2.002× the shipped build measures. The premise was simply wrong: decoding N frames costs the same total work however many threads do it, so threads move wall clock and core occupancy, never CPU-seconds. The profile shows the mechanism cleanly — at one thread `decode.screen` goes 1.05 s → 6.17 s while `enc.send_frame` goes 7.61 s → 2.16 s, the decoder eating the slack the encoder-bound pipeline left it, until the slack runs out. **One-line reason not to re-propose:** threads do not buy CPU-seconds back, and by the time the cap is low enough to matter it has already cost the export more than the optimisation gained.
+
 ### A dedicated encode thread on macOS
 
 **What it was.** After the decode change above, the macOS export profile was two waits and almost nothing else: `enc.send_frame` 7.658 s (40.6 %, the CPU waiting on VideoToolbox) and `gpu.wait` 7.501 s (39.8 %, the CPU waiting on Metal). Two different engines, one serial loop. Put the encoder on its own thread — bounded queue, backpressure, error propagation, `av_frame_free` on the consumer side — and the two waits overlap. It is also the move that was worth −30 % on the Linux path. **What the measurement said.** **+0.2 %.** Three cycles, an ffmpeg floor inside each, closing drift 1.0003, machine 84–85 % idle: serial 23 039 ms / 1.308× floor, threaded 22 995 ms / 1.305×. Output byte-identical either way. A probe on the producer's blocking says exactly why: blocking on a full queue measured **7.631 s** where blocking inside `avcodec_send_frame` had measured **7.658 s** — 0.4 % apart. The thread **moved** the wait, it did not remove it. The export is bound by VideoToolbox's encode throughput, not by CPU serialisation: the main thread's own work is 9.64 s across 3600 frames (2.68 ms/frame, enough to feed 373 fps) against an encoder that absorbs about 191. **One-line reason not to re-propose:** the encoder is the constraint and no scheduling changes that; it could pay on an M-series Pro/Max/Ultra with more encode blocks, but that is a hypothesis and shipping unmeasured concurrency on it is the mistake this record exists to prevent.
@@ -668,7 +688,7 @@ the bench runs on the reference machine.
 - **macOS export startup can cost 4 s, and nobody has reproduced it on demand.** Measured repeatedly at 4208–4502 ms between the CLI's `started` event and the first composed frame — 18 % of a 60 s export, 71 % of a 5 s one — then gone, on the same shipped binary, hours later (481 ms). It is not the compositor (init is 2.4 ms, runtime MSL compilation included), not the `<video>` metadata probes (13 ms and 6 ms), not the CLI prologue (24 ms total), and not the renderer entry point (measured at −0.1 %). It correlates with memory pressure on an 8 GiB machine — `387M unused / 2613M compressor` while it reproduced, `564M unused / 1837M compressor` after — which would fit faulting ~1.8 MB of module chunks out of a 274 MB `app.asar` while the compressor thrashes: seconds of wall clock, no CPU in either process, cost independent of the media. Untested. Recreating the pressure deliberately and watching it return is what would settle it, and then whether asar size is the lever.
 - **10-bit and HEVC decode on macOS are unmeasured.** The export's decode predicate is `codec_id == H264 && format == YUV420P`, so both keep VideoToolbox untested. HEVC is the case most likely to invert the result, since its software decoder is materially more expensive. 10-bit needs work beyond the predicate first: `mac_frames::CpuFrames` converts to 8-bit NV12, so routing 10-bit through the software path would silently truncate — the predicate is currently what prevents that.
 - **The macOS preview's decode backend has never been measured.** `DecodeIntent` splits preview from export precisely so the preview could keep the old arbitration; the export won on throughput, but the preview scrubs, where seek latency after `avcodec_flush_buffers` may matter more, and it shares the machine with the editor UI. Changing it without measuring it would be the same mistake the export change corrects.
-- **The energy cost of software decode on macOS is unmeasured.** The export burns 3.5× the **CPU-seconds** it used to (8.4 → 29.8 s), and that is the only thing measured. It does not follow that energy moved by the same factor, and the earlier version of this line said it did: on an M-series the P and E cores draw very differently, clock is not fixed, and a shorter run at higher occupancy can spend less total energy than a longer one — racing to idle. Whether this trade costs or saves battery is genuinely unknown; `powermetrics` would answer it and needs sudo. Capping `thread_count` below the core count is the obvious lever, and untried: the walk is encoder-bound, so the decoder only has to be fast enough to stop being the constraint.
+- **The energy cost of software decode on macOS is unmeasured, and the CPU figure is not a proxy for it.** The export burns 3.5× the **CPU-seconds** it used to (8.4 → 29.8 s), and that is the only thing measured. It does not follow that energy moved by the same factor: on an M-series the P and E cores draw very differently, clock is not fixed, and a shorter run at higher occupancy can spend less total energy than a longer one — racing to idle. Nor is the jump waste: VideoToolbox does the same decoding in a fixed-function block that CPU accounting never sees, so the work did not grow, it moved somewhere visible and got 12× faster on the way. Capping the decoder's threads does **not** recover it (see Rejected routes); what it would buy is lower peak core occupancy — how unusable the machine feels during an export — which is a different question and also unmeasured. `powermetrics` would answer the energy half and needs sudo.
 - **The C0→C8 table rests on one run, on one machine.** [Recorded above](#one-admissible-run--2026-07-27) and admissible on its own gate, but a single sweep: the C5–C7 plateau is the part most likely to move under a second run, since the layers it prices are individually smaller than the machine's own noise. A repeat on a cool machine — and on the discrete-GPU box that is [owed anyway](#known-gaps) — would settle whether those three are genuinely free or merely under the floor.
 - **The fixture media is not versioned, and its cursor track has no provenance entry.** `crates/fixture/fixture.json` documents the exact `-c copy` cuts for `screen.mp4` and `webcam.mp4` (which is what made the run above reproducible), but says nothing about `screen.cursor.json`, which C7 needs. It happens to be the raw, uncut `.cursor.json` beside the origin recording — the loader windows it itself at `offset 100_000 ms, 6 s` ([`bench.rs:70`](../../crates/poc-d3d/src/bench.rs)), matching the manifest's `cut_offset_s: 100`. That is recoverable by reading the code, not by reading the manifest; the manifest should carry it.
 - **One bench fixture is still corrupt, from a bug since fixed.** Two concurrent saves used to be able to interleave and truncate a project file, which destroyed at least two real ones (a valid JSON prefix followed by the tail of a longer version). `proj_de6ffaaa` (`os_parity`) is still in that state — 4006 bytes, 3485 of them valid JSON — with a byte-exact backup beside it (`*.corrupt-backup-20260716`); recovery is mechanical (truncate to the 3485-byte prefix). The bug itself is gone: `DocumentService` now serialises saves through a per-project write queue and writes atomically (unique temp file → `fsync` → rename), which is why Gate G0 was run on `proj_5b3ac6bc` instead. The reference project for M1–M4 is `proj_a7468696`.

From 6f8c53ca6813dd46162cb71eedf86fe95fe70e23 Mon Sep 17 00:00:00 2001
From: Etienne Lescot <etiennelescot@gmail.com>
Date: Fri, 4 Sep 2026 13:49:18 +0200
Subject: [PATCH 2/2] docs(perf): stop overstating the thread result, and stop
 guessing the thread count
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Two review findings, both of them catching me overstating my own
measurement.

**"threads move wall clock and core occupancy, never CPU-seconds" is
contradicted by the table directly above it.** CPU-seconds go 30.3 → 27.4
at one thread — a 9.6 % fall. The premise was wrong about the size of the
effect, not its sign, and the text now says so: threads mostly
redistribute the decode work rather than reduce it, which is why 8× fewer
of them buys ~10 % rather than ~90 %. That last 10 % is plausibly the
thread pool's own overhead, but it was not isolated, so it is now marked
unexplained rather than asserted.

**`thread_count = 0` was described as "one thread per core".** libavcodec
documents it as *automatic* — the decoder chooses, from the CPU count and
its own threading model — and the number it actually picked was never
read back. The doc says that instead of inventing a mapping.

Neither changes the conclusion: one thread costs +70 % of wall clock, and
the default stays.
---
 technical-documentation/engineering/rendering-performance.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/technical-documentation/engineering/rendering-performance.md b/technical-documentation/engineering/rendering-performance.md
index 878db160d..ea5b63cf1 100644
--- a/technical-documentation/engineering/rendering-performance.md
+++ b/technical-documentation/engineering/rendering-performance.md
@@ -572,7 +572,7 @@ Unit tests never look at a pixel. The `native*` arms write real files: export th
 
 ### Capping the macOS decoder's thread count
 
-**What it was.** After the export moved to the software H.264 decoder it runs with `thread_count = 0`, i.e. one thread per core, and the export's CPU-seconds went 8.4 → 29.8. Since the walk is bound by the encoder and the decoder has seconds of slack, capping its threads looked like free CPU. **What the measurement said.** It is not free and it does not return CPU. Public bundle, S4, three cycles with a floor inside each, closing drift 0.9979, output identical across variants:
+**What it was.** After the export moved to the software H.264 decoder it runs with `thread_count = 0`, which in libavcodec means *automatic* — the decoder picks, from the CPU count and its own threading model, and the number it actually chose was never read back here. The export's CPU-seconds went 8.4 → 29.8. Since the walk is bound by the encoder and the decoder has seconds of slack, capping its threads looked like free CPU. **What the measurement said.** It is not free and it does not return CPU. Public bundle, S4, three cycles with a floor inside each, closing drift 0.9979, output identical across variants:
 
 | decode threads | cost | CPU s |
 |---|---:|---:|
@@ -580,7 +580,7 @@ Unit tests never look at a pixel. The `native*` arms write real files: export th
 | 2 | 1.056× | 29.2 |
 | 1 | **1.775×** | 27.4 |
 
-**One thread costs +70 % of wall clock to return 9.6 % of CPU** — it throws away nearly the whole decode gain, landing back near the 2.002× the shipped build measures. The premise was simply wrong: decoding N frames costs the same total work however many threads do it, so threads move wall clock and core occupancy, never CPU-seconds. The profile shows the mechanism cleanly — at one thread `decode.screen` goes 1.05 s → 6.17 s while `enc.send_frame` goes 7.61 s → 2.16 s, the decoder eating the slack the encoder-bound pipeline left it, until the slack runs out. **One-line reason not to re-propose:** threads do not buy CPU-seconds back, and by the time the cap is low enough to matter it has already cost the export more than the optimisation gained.
+**One thread costs +70 % of wall clock to return 9.6 % of CPU** — it throws away nearly the whole decode gain, landing back near the 2.002× the shipped build measures. The premise was wrong about the size of the effect, not its sign: CPU-seconds do fall, by 30.3 → 27.4, but nothing like proportionally, because decoding N frames costs roughly the same total work however many threads share it. Threads mostly redistribute that work rather than reduce it, and the ~10 % that does disappear is plausibly the thread pool's own overhead — which was not isolated, so treat the mechanism behind that last 10 % as unexplained rather than established. The profile shows the mechanism cleanly — at one thread `decode.screen` goes 1.05 s → 6.17 s while `enc.send_frame` goes 7.61 s → 2.16 s, the decoder eating the slack the encoder-bound pipeline left it, until the slack runs out. **One-line reason not to re-propose:** threads do not buy CPU-seconds back, and by the time the cap is low enough to matter it has already cost the export more than the optimisation gained.
 
 ### A dedicated encode thread on macOS