Skip to content

perf(pipeline)+build: R5 campaign — pool return, ceyx v0.1.16 double pin bump - #4

Merged
jhangyu merged 37 commits into
mainfrom
gc-r5-wp9
Sep 6, 2026
Merged

perf(pipeline)+build: R5 campaign — pool return, ceyx v0.1.16 double pin bump#4
jhangyu merged 37 commits into
mainfrom
gc-r5-wp9

Conversation

@jhangyu

@jhangyu jhangyu commented Sep 6, 2026

Copy link
Copy Markdown
Owner

GC-remediation campaign R5 + accumulated local-main backlog (user-cleared 1a).

Key changes: WP6b pooled-buffer return on encoded publish; ceyx release pin bump v0.1.15 → v0.1.16 (all 9 archive digests re-derived by --ceyx-release latest); both workflow ceyx refs → 9f9f557 (= v0.1.16 tag commit, AC10.5-verified); TC registry renumber 1079-1082.

CI purpose: AC10.3 — green on the bumped pin. Merge happens only on explicit user instruction after green (post-merge re-verify is a separate gate).

🤖 Generated with Claude Code

claudecode_project and others added 30 commits September 6, 2026 15:23
…rialize stall attribution

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…roof (T2)

Independent verifier tests for H1-A (spec docs/logs/2026-09-06/h1h2-spec.md
§1.6, plan ticket T2): TC-999/TC-1000 pin pixelPayloadFromOrientedImage
byte-equivalence across all 8 EXIF orientations plus the downscale arm;
TC-998/TC-998b prove exactly one materialize| event per decode via
PerfLog.testSink. Red evidence and mutation proofs in docs/logs/2026-09-06/
(gitignored, not part of this commit).
Comment-only. H2-A changed the ceyx pool arm's payload from a
TransferableTypedData copy to a native pointer view; the length check and
DecodedRgba construction are backing-store-agnostic and unchanged, and
the legacy Isolate.run arm stays on TransferableTypedData as the A/B
control.

Co-Authored-By: Claude <noreply@anthropic.com>
…1b/1002/1003

Collision with an in-flight uncommitted TC-995..998 allocation in the main
tree's intent_coalescing_test.dart (a parallel campaign sequentially
allocating in the same range). Per team-lead: TC-1001..1009 verified
hit-free in both trees. Test logic and evidence unchanged.
… masking

Race A: PhotoPayloadCache gains an onEvicted callback, fired only from the
byte-budget eviction loop and only after the entry is gone. The controller
answers it by recomputing the item's state from the same containers a notifier
born at that instant would read -- the single sanctioned path where a stage may
move backwards. _markStage stays strictly forward-only. retainOnly/clear
deliberately do not report: those ids have left the retention union and their
notifiers are disposed on the same pass.

Race B: notifier disposal gains a 100ms grace for ids a view touched through
stateFor(). A row built inside the sidebar's debounce window is live even though
the wanted set has not caught up, so a navigation-side sweep could orphan it
silently. The map's bound becomes retention union + ids touched within the
grace, still finite; no new timer, and retention semantics are untouched.

TC-1010/TC-1011 (Race A) and TC-1012 (Race B) land red-proven from round 1;
TC-1013 pins that the grace expires rather than leaks. payload_state_test.dart's
TC-986/987/988 now drive the injected clock past the grace -- additive lines
only, every assertion byte-identical, and the tight <= union bound is kept
deliberately unrelaxed so it can still fail.

Co-Authored-By: Claude <noreply@anthropic.com>
…ound settle waits

Round-2 speedups from the test-speedup campaign (contract:
docs/logs/2026-09-06/test-speedup-contract.md, review r2-review.md):
- photo_export_service: 24-90MB RGBA fixtures shrunk to minimum sizes
  that still exceed each resize target
- dart_image_loader: per-test sample classification hoisted to one
  shared setUpAll previewCache
- dng extractor family: memoize synthetic JPEG/payload by size, cache
  repeated sample reads, hoist deterministic endian pair to setUpAll;
  sony_ifd_chain 7008x4672 -> 2900x1936 (still clears 2800 floor)
- view/layout tests: bare pumpAndSettle -> bounded 20-frame settle
- preload tests: oversized real delays trimmed (600->320, 300->280ms)

Measured: dng family 46.3s -> 10.6s CPU; export+loader ~45s -> ~10s.
probe_first_navigation is excluded pending a contention-flake fix.
Roster unchanged (253 tests, zero added/removed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the fixed 50ms sleep with a bounded poll on the asserted
quantity (imageCache.currentSize == 9) — under CPU contention pending
precaches satisfied containsKey while currentSize was still 0
(reproduced 3/3 in a 14-file batch, r2-review.md). A 20ms quiescence
drain after the poll preserves the upper-bound direction of the frozen
==9 expect. Also trims oversized real delays (60->20ms, 350->280ms).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…g it

SidebarThumbnailController now accepts an optional deriveQueueWidth
ctor param (default kSecondaryStageWidth) and exposes
setDeriveQueueWidth() to re-push at runtime, so the pipeline's
single stage-width source of truth (StageWidths) can drive it.
DeriveQueue itself is untouched (jobTimeout preserved, clear() not
reintroduced). TC-1022/1023.
…C-1014)

A payload landing no longer wakes every context.watch<AppState>() consumer;
it wakes exactly the item that landed through payloadStateFor(id), which
photo_viewport.dart already consumes via ValueListenableBuilder. Verified
the other 9 watch sites read selection/scan/settings state, unrelated to
landing. ImagePreloadController.preloadImages's notifyLoaded param becomes
optional (VoidCallback?) since AppState no longer supplies one; the sole
other override (test/providers/app_state_exif_test.dart's _SilentPreload)
is updated to match.
…hasFullSize

Both getters could report stale true after eviction and had zero consumers
in lib/ or test/. The distinct SidebarThumbnailController hasPayload
constructor parameter (backed by cache.contains) is unaffected.

Co-Authored-By: Claude <noreply@anthropic.com>
New assert-only claim protocol for lib/services/image_pipeline: tracks
who currently owns the right to produce a payload for an id
(producer/laneQueue/offLaneEncode), with a double-hold assertion on
acquire, an owner assertion on transfer/release, and a
generation-tolerant release (mirrors InflightBytesBudget's epoch) so a
release racing a clear() is a silent no-op instead of an over-release.
Enforcement compiles out in release builds; zero production behavior
change.
AppState's fire-and-forget async _initPrefs hits the real shared_preferences
platform channel when no mock handler is registered, occasionally throwing
MissingPluginException inside this test's FakeAsync/runAsync window. Add
SharedPreferences.setMockInitialValues({}) before constructing AppState,
matching the guard already used in app_state_test.dart.
Probes whether the A3/A5->A8 deferred lane hand-off window lets a second
producer start a duplicate RAW decode for the same id. Result: not
reproducible across 8 pump-depth scenarios; positive control proves the
counting instrument itself can register a duplicate when one genuinely
occurs. Findings recorded separately in docs/logs (untracked).
…s into 15

Round 3+4 of the test-speedup campaign (target: 'flutter test
test/services/image_pipeline -j 1' < 30s; baseline 97s, now 48s):
- thread the ImagePreloadController navigationDebounce test seam
  through the preload family (real 250-600ms waits -> zero/40ms with
  scaled margins; P1 window test deliberately kept on the production
  default with red->green evidence)
- convert fixed sleeps to bounded condition polls (sidebar/lane/
  retention/cost-memo files)
- merge per-subsystem test files to cut -j 1 isolate-startup overhead
  (each source wrapped in a group('<file>.dart') scope; leaf test-name
  multisets verified unchanged per target; helper collisions renamed)

572 unique tests before == 572 after (name-level diff, zero removals).
TC-1021 red is another campaign's in-flight lib/stage_widths.dart edit,
unrelated. The lib seam itself is NOT in this commit (its file carries
another campaign's uncommitted refactor; seam rides with that landing).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>


0f61a49 deleted the 60 source files but 9 of the 15 merge targets were
never git-add'ed by the merge workers; without this commit HEAD has the
deletions and not the replacements.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The registry landed in 01d4d3d matched a mutation to a claim by id plus
owner tag, and both repeat across generations. A caller that crossed an
await -- `_finishOffLane` is unawaited by design and can outlive a
reset() -- could therefore arrive to find the id re-claimed by a fresh
producer, trip the wrong-owner assert against that innocent producer,
and (in release builds, where the assert is compiled out) go on to
delete or re-tag its live claim: exactly the second producer the class
exists to prevent.

release/handOffToLane/transfer now accept the PayloadClaim their caller
was handed and refuse to touch the map when the id is held by a
different claim object. Identity is stronger than a generation compare:
it also catches a lane hand-off and re-acquire inside one generation.
handOffToLane additionally declines to arm the awaiting-lane set on a
stale call, which would otherwise miscount the next legitimate producer
as a duplicate.

The claim parameter is OPTIONAL, so this change is additive: the
registry unit test from 01d4d3d passes unchanged, and callers with no
suspension point between acquiring and mutating keep the old id-only
matching.

Docs corrected in the same change: the class no longer claims to be
"assert-only, zero production behaviour change". Owner assertions remain
debug-only, but the identity guards run in release and deliberately
change behaviour by refusing stale mutations -- that refusal IS the fix.
PayloadClaim.generation is now marked diagnostic-only; nothing reads it
to make a decision.

Co-Authored-By: Claude <noreply@anthropic.com>
… payload ownership to PayloadClaimRegistry

StageWidths.derive(decodeLaneWidth) is now the only place that decides how wide any bounded stage is, and ImagePreloadController._applyStageWidths is the only place that pushes those numbers into the decode lane, the native decode pool, the EncodeStage and the sidebar DeriveQueue. The encodeStageWidth constructor parameter (a second, independent width source) is removed. Clamping is LOW-SIDE ONLY: TC-966 pins pass-through above kMaxDecodeLaneWidth, and the ceiling on the user's setting belongs to AppState, where the preference is read. _applyStageWidths carries no equality guard -- TC-938/TC-966 pin push-on-every-call for the pool sink.

CO-LANDS, implemented by a teammate: the payload ownership migration in this file, replacing the bare Set<String> _loadingKeys with PayloadClaimRegistry across all hand-off points (acquire / transfer / handOffToLane / assumeFromLane / release). It shares this file with the width work and cannot be committed separately.

ALSO INCLUDED, authored by another session: the navigationDebounce test seam (constructor parameter, field and tier-two wiring). It is committed here rather than excluded because test files already on main pass navigationDebounce: and would not compile without it.

Co-Authored-By: Claude <noreply@anthropic.com>
Four cases covering the guards added in c2145f9 and the ownership
protocol the controller migration in a10af62 now routes through:

  TC-1035  releasing a claim owned by another party trips the owner
           assertion. This is AC2's fallback: the duplicate-producer
           window the probe in this same file went looking for was
           ruled NOT-REPRODUCIBLE through the public seams, so the
           acceptance criterion is met by an assertion guard rather
           than a red repro.
  TC-1036  a stale release leaves the current holder untouched --
           returns false, asserts nothing, deletes nothing.
  TC-1037  a stale handOffToLane likewise, and additionally must not
           arm the awaiting-lane set, which would miscount the next
           legitimate producer as a duplicate.
  TC-1037b a stale transfer does not re-tag the current holder.

Every case was seen RED before being seen green, by neutering the
specific guard it pins and observing the failure; TC-1035's red proof
was taken at the CONTROLLER seam (site A7's owner argument mutated),
not only at module level. Evidence:
docs/logs/2026-09-06/p3-claim-redproof.txt (not version controlled).

Also renames a stale `_loadingKeys` reference in the probe's
negative-finding prose: that field no longer exists after a10af62.

KNOWN LIMIT, recorded so nobody reads more into the counter than it
carries: debugDuplicateProducerCount can inflate via lane-entry
replacement, so it is an observation signal and not hard evidence of
how many duplicate producers occurred.

Co-Authored-By: Claude <noreply@anthropic.com>
…ode parameter

Binding user ruling 2026-09-06: every stage pool width derives EQUAL to the
one user-configured decode parameter. StageWidths.derive now clamps once
(<1 -> 1) and returns that same width for decodeLane, encode and derive; the
pinned kSecondaryStageWidth constant is deleted. This supersedes the pinned-2
secondary-stage semantics and is a deliberate behaviour change at default
widths, not a refactor.

ImagePreloadController builds EncodeStage at the derived width in its
initialiser list, so no stage is ever briefly wrong before the constructor
body's _applyStageWidths push. Push-on-every-call (TC-938/966), the low-only
clamp, the sidebar debounce and the claim registry are untouched.

SidebarThumbnailController's deriveQueueWidth constructor default becomes a
documented local literal 2, reached only by that class's own unit tests --
production always receives the derived value via _stageWidths.derive.

TC-1024/1025 were seen red against the old semantics (expected 3/5, actual 2)
before this change and green after.

The decode_lane_pool_test TC-1020/1021 rewrite was authored by a teammate and
co-lands here for coherence: without it this commit's tests would be red,
since HEAD's version still pins the constant 2.
The window-resolution PixelPayload was materialized on every expensive decode
and then discarded unread whenever the re-encode succeeded -- the overwhelmingly
common case. SourceDecode now carries a PixelFallbackBuilder thunk plus a
rawDecodeRan flag in place of the eager payload, and reencodePayload invokes it
only on its four failure exits.

The PHASE 13 one-buffer ruling is untouched: encodePhase still returns exactly
one SourcePayload and the controller still writes exactly that object once, so
payload object identity -- the tier-1 ImageCache key and the tier-2 registry
anchor -- is never swapped. Only WHEN the fallback candidate is constructed
moved; never when a payload is published.

The closure captures only objects the record already keeps alive across the
encode (decoded aliases fullRes.rgba on the identity path, fullRes.image is held
to publish on the rotated path), so it extends no lifetime.

Counters pixelFallbackBuilds / debugPixelFallbackRetained are the observability
seam, kept separate from reencodeFallbacks. Mutation-proved: restoring the eager
build fails the success-path test.

Co-Authored-By: Claude <noreply@anthropic.com>
claudecode_project and others added 7 commits September 6, 2026 21:49
…, TC-1052/TC-1053)

Co-Authored-By: Claude <noreply@anthropic.com>
…s to 9f9f557

Two pins move together, deliberately. scripts/ceyx_release_pin.json pins the
prebuilt BINARIES the --fetch-native legs download; the workflow ref: on each
jhangyu/ceyx checkout pins the SOURCE Halcyon compiles its Dart against.
Bumping only the first gives CI new binaries against old Dart bindings;
bumping only the second gives new Dart calling symbols the pinned binaries do
not export, which the guarded lookup degrades silently to unsupported -- a
GREEN CI with the feature missing.

Pin written by scripts/build_apps.py --ceyx-release latest (RC=0) and checked
by --ceyx-release verify (RC=0, 9/9 entries re-downloaded and re-extracted);
no digest was hand-edited. All 9 archive digests and the artifacts.lock digest
re-derived; the 5 decoder members changed.

Release v0.1.16 was cut from ceyx 9f9f557, which is exactly what both workflow
refs now point at.

Co-Authored-By: Claude <noreply@anthropic.com>
test_pin_file_untouched is a review-ledger guard, not a correctness test: it
fails whenever scripts/ceyx_release_pin.json changes, so that a pin bump can
never land unreviewed. It fired on PR #4 exactly as designed -- 4bf7063
rewrote the pin to ceyx v0.1.16. The ledger entry moves; the test does not
get relaxed.

New digest recomputed locally with the same CRLF-normalizing method the test
itself uses (read_bytes, replace b"\r\n" -> b"\n", sha256), never
transcribed. The normalization matters because this repo has no .gitattributes
forcing LF: windows-latest checks the pin out with CRLF while macos/ubuntu get
LF, so hashing raw bytes would make the guard platform-dependent.

python3 scripts/ci.py selftest: Ran 32 tests, OK, RC=0.

Co-Authored-By: Claude <noreply@anthropic.com>
Aligns CI with the standing compile-only rule (CLAUDE.md: "CI is
compile-only ... Functional tests ... are NOT run in CI"). The verify
phase running `flutter test -j 1` was the violation. verify() now runs
pub get -> analyze only; the "Analyze & Test" job/step name and its
UTF-8-decoding comment are updated to match.
Follow-up to d449102 (CI compile-only). Root fact: the 8/31 rewrite
implemented verify() WITH `flutter test -j 1` from day one, contradicting
the CLAUDE.md 2026-08-31 compile-only decree the whole time -- nothing
mechanical enforced it. Adds TestNoTestExecutionInCI to
scripts/ci/tests/test_policy.py: fails if any argv list literal under
scripts/ci/ or scripts/ci.py invokes `flutter test` / `dart test`, so
selftest itself is now the guard against this regressing again.

Proved red-capable: temporarily re-added the test leg to phases.py,
reran `python3 scripts/ci.py selftest` (FAILED, test_no_test_execution_in_ci
red), reverted via a cp-backup (no working-tree diff after revert), reran
selftest (OK). Evidence: docs/logs/2026-09-06/r5-policy-red.txt,
r5-policy-green.txt, r5-policy-green2.txt.
@jhangyu
jhangyu merged commit c67c36c into main Sep 6, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant