Phase 13 reading arm - #4
Merged
Merged
Conversation
… cannot merge aggregate() keyed runs on (model_name, version, verifier_*), which gave docTR stock, docTR tuned and docTR+parseq — and PP-OCRv6 stock vs tuned thresholds — identical identities. They aggregated into one averaged row with nothing to flag it. Phase 13 adds ~10 such arms. OCROutput now carries config_id (human label, rendered in the report) and config_hash (auto digest of the runner's declared config()). The hash is what protects the aggregation: a label alone only works if someone remembers to change it when they change a knob, and forgetting is silent. Runner.config() is abstract so a runner cannot inherit an empty declaration, and config_hash() lives on the base so no engine can hash differently. The blank-config_hash check runs BEFORE the tuple comparison, for the same reason D-8.4's blank-version check does: uniformly blank rows collapse to one consistent identity and would pass. Scoring path untouched — matching.py, metrics.py and normalize() are unchanged, so the cross-engine comparison stays fair. 28 new tests (441 passed, was 413). No new ruff errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e the string Measures reading quality with detection error held at zero: gt_v1's human-drawn boxes are supplied, so a recognizer with no detector of its own becomes testable. Scoring reuses the frozen normalize() + score() + aggregate() — no second path. Match/MatchResult move to contract.py (matching.py re-exports them) so the reading arm can reuse the one scoring path without importing the matcher it does not use; config_digest() moves there too so runners and readers cannot hash differently. Pure moves — no scoring logic changed, no reported results invalidated. Fairness rails, structural rather than advisory: - crop preprocessing is module-level (pad 0.10 of box height, 48px, bicubic, RGB); crop_for_reading() takes no preprocessing arguments and a Reader sees only the finished crop. The live crop settings are hashed into every row's config_hash, so a changed pipeline is a different arm that aggregate() refuses to blend. - detection metrics are undefined here: no matching.py import, no IoU, iou_thr=None, and the written artifact carries arm/oracle_detector/crop_preprocessing so a reader result can never be mistaken for an end-to-end one. - control boxes (the arm's only hallucination floor, since oracle boxes make location invention impossible) are refused on text-bearing frames: the GT records where text is, never where it is absent. The floor is therefore measured on the blank control set only — all ct_axial. - cost is billed per crop, not per image; N boxes are N reader calls. Deferred to 13c/13d: the "reader charset covers A-Z0-9-" check needs a real reader, and putting it here would make the harness import tests/. Tests: 31 new, synthetic fixtures only (fake images, fake tokens). Full suite 472 passed, 2 skipped, 1 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A third table over rows the harness already produces: boxes vs GT at the
matcher's IoU bar, strings ignored, so a loss can be attributed to finding the
text rather than reading it. No engine re-run, no inference path, and no change
to matching.py / metrics.py / contract.py — detection recall is found/gt_total
and precision is found/(found+added), both derived from counts score() already
returns.
The invariant the table exists for: a stratum with no text-bearing images has NO
recall denominator (ct_scout, mg_tomo), and a per-image mean over it reads a
meaningless saturated 100% — the defect that made the 2026-08-06 ct_scout sweep
compare noise against noise. Structurally prevented here: recall is a pooled
sum(found)/sum(gt_total), a zero denominator yields None, and None renders `n/a`
with its reason in a footnote. Never a defaulted 100%.
Blank negative controls are partitioned out of both rates before any sum and
reported as the hallucination floor on their own line — pooled and per stratum,
since invention rates differ by two orders of magnitude across strata and one
pooled number hides which stratum invents. When a run has no blank controls the
floor itself reads `n/a` with a reason: "0 boxes on 0 frames" would read as a
measured floor of zero, which is the no-invention-floor blind spot again.
Rows no IoU matcher produced are excluded and counted, never folded in — both
kinds would print a better detector than was measured:
- box-free rows: matching.py returns hallucinations=[] unconditionally, so
they contribute added_count=0 and read as flawless precision;
- reading-arm rows (iou_thr=None, Phase 13b): that arm is handed the GT box
and runs no matcher, so every token is found by construction.
Also, from review of this change:
- the D-8.4 / D-9.1 / D-13.5 / D-8.3 identity guards move into
_reject_unidentified_rows() and are called by BOTH aggregate() and the new
detection_view(), so the new public rollup cannot become the one door
through which two arms of an engine get pooled into one number (rule #9);
- the detector count is `n_text_images` under a "Text imgs" header, never
`n_images` — the sibling _group_stats already publishes `n_images` meaning
every image, and two denominators under one name is how a detection number
gets read against an end-to-end one;
- a missing `detection` key renders a "Not rendered" warning rather than
silently dropping the section.
Known and left alone: `negative_control` means "zero GT tokens", not "a human
confirmed this frame blank", so an image absent from the loaded gt.csv lands in
the floor. That is the frozen scoring path's meaning, shared with the existing
run-wide floor; documented, pointing at gt_set.sha256 as the control that pins
the image set.
Tests are synthetic and PHI-free throughout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…scored set Step 6 measures every end-to-end engine twice — stock config and tuned — so the bake-off ranks engines rather than our configuration of them. The open question was where the tuning happens, and the answer decides whether the tuned number is a measurement or a bound. D-13.4 resolved: tune on dev_v1, a slice annotated from OUTSIDE gt_v1. Sweeping thresholds against gt.csv fits the test set. Holding a slice out of gt_v1 was rejected on arithmetic — only 102 of 199 images carry text and four strata have <=3 text-bearing images, so a stratum-spanning holdout either deletes a stratum from the scored set or leaves it untuned. Full reasoning in experiments/D-13.4_resolution.md, ready for plan.md §4. build_gt.py wrote gt_set.sha256 and gt_summary.txt under hardcoded names while honouring --out, so building a second artifact would have overwritten gt_v1's frozen scope hash — a rule-#8 invalidation performed by a script. Sibling names now derive from --out; `--out gt.csv` still produces the same four filenames byte for byte, verified against the committed artifacts. Six identities, three engines. docTR and PP-OCRv6 are stock with no argument and tuned by argument; EasyOCR runs backwards — it has shipped 0.2/0.2 against library defaults 0.7/0.4 since Phase 9, so its no-argument runner IS the tuned arm and its stock arm is built from the library defaults. config_id is DERIVED from the knobs, never a constructor argument: a label that can be passed can be passed wrong, and a label in the digest would split one config into two identities. Unset knobs are declared as None (= the engine's own default) rather than as copied literals, which cannot go stale on a version bump. PP-OCR's return_word_box / enable_mkldnn / three orientation stages stay set identically on both arms — correctness requirements, not tuning — and a test enforces it, so the report's "stock thresholds, not PaddleOCR out of the box" is checked rather than asserted in prose. The sweep refuses to run against gt_v1 by path AND by content hash, with no override flag. A candidate must beat stock on the pooled metric and regress no stratum beyond a disclosed tolerance: a gate tuned on CT once deleted 29-47% of genuine ultrasound text, and a pooled mean hides exactly that. "Nothing beat stock" is a reportable outcome, and tuned_configs.json stays empty until --freeze writes a measured winner, so <engine>:tuned raises rather than quietly resolving to stock values under a tuned label. An audit subagent found three defects, all fixed and pinned: the latency column read a p50 key aggregate never emits; freeze_winners merged without deleting, leaving a stale config served under a fresh dev-set stamp; and the winner was never compared to stock, so a grid where everything lost would crown the least-bad config. The sweep has NOT run — it needs the dev slice annotated. No engine version changed, no gt_v1 artifact changed. Tests: 51 new, synthetic fixtures only. Full suite 492 passed, 2 skipped, 1 xfailed. No new ruff errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ide with gt_v1's review_gt.py hardcoded ground_truth/review/ with no flag, so annotating the D-13.4 tuning slice would have written its records into gt_v1's directory. build_gt's completeness gate is bidirectional: the dev build would then see gt_v1's 199 records as ORPHANS and hard-fail, and the gt_v1 build would see the dev records the same way. Both builds blocked, and the only workaround was renaming a PHI directory by hand between runs. --review-dir must name a directory directly inside ground_truth/. Records are PHI and the Read-deny rules and .gitignore entries that protect them (ground_truth/review_*/) are written around that directory, so the flag can rename the records dir, never relocate it. Only an EXPLICIT --review-dir is validated. Left unset the module constant stands untouched — it is repo-anchored by construction, and policing it would also reject the value the tests bind in its place. test_review_gt.py's two exact CLI allowlists are updated rather than loosened: they exist so that adding any option is a deliberate reviewed edit, and --review-dir cannot reach the socket. HOST/PORT remain unreachable from the command line. Full suite 547 passed. No new ruff errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tomo, not ct_axial The default quota asked for 4 ct_axial blank controls. Measured against the real manifest: 66 ct_axial series exist and gt_v1 uses all 66, so ZERO are available and the draw came back SHORT with no invention floor at all. Tuning a threshold on a set with no blank frames optimizes recall against noise with nothing to catch it — the ct_scout failure (CLAUDE.md §8), which is the specific mistake this quota existed to prevent. Even if spares existed, drawing from ct_axial would tune against frames inside the scored set and contaminate the headline hallucination floor — what D-13.4 forbids. BLANK_CONTROL_STRATA is now (ct_scout, mg_tomo): both returned zero text-bearing images in gt_v1, which is exactly what makes them useless as tuning signal and right as a floor. 113 and 80 unused candidates respectively. Two sources rather than one, so a human finding text in one does not wipe out the floor — and the summary now says out loud that a human must confirm they are blank, since a frame with text silently turns the floor into a recall measurement. Total draw is unchanged at 26 images. Tests: the two that encoded the old reasoning are rewritten, not loosened — one now asserts ct_axial is NOT drawn from and says why. 549 passed. No new ruff errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both are crop-in/string-out Readers, not engines: they run under an oracle detector and are tagged arm="reader", never blended into the end-to-end ranking. - harness/readers/read_svtrv2.py — OpenOCR task='rec' (SVTRv2_server). Passes drop_score=0.0; that argument is inert on the rec path in openocr-python 0.1.5 (OpenOCR forwards it to the 'ocr' task only), so a test asserts the library property directly rather than trusting the inert setting. Never constructs the 'ocr' task, which is what writes ./e2e_results/system_results.txt. - harness/readers/read_doctr_parseq.py — docTR's PARSeq recognition head only, no detector. Shares model_name "doctr" with DoctrRunner on purpose; config_id "parseq" + config_hash keep the identities apart (D-13.5). - tests/test_readers.py — shared scaffold: contract, version sourcing, charset covers A-Z0-9-, determinism, exact read of a synthetic CMFN-00421, no stray output file, distinct identities. Resolved versions for run metadata: openocr-python 0.1.5 (SVTRv2_server, ppocr_keys_v1.txt) and docTR v1.0.1 (parseq). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ray-file check From the fresh-context review of both readers: - tests: an import failure silently emptied REGISTERED_READERS and deleted all five shared tests with no skip and no failure. Now asserted explicitly. - tests: the stray-file test reused the already-built shared reader, so it covered read() only and never the construction path that resolves configs and materializes a checkpoint. It now builds its own reader inside the redirected CWD. - read_svtrv2: charset() included CTCLabelDecode's literal 'blank' token, so it was not a set of characters. Single-character entries only. - read_svtrv2: the docstring claimed the module never logs. OpenOCR's own logger writes PHI-free INFO lines to stdout at construction; documented rather than hidden. - read_doctr_parseq: documented split_wide_crops — docTR re-splits crops wider than 8x their height (>~384px at CROP_HEIGHT=48) and SVTRv2 has no equivalent stage, so a wide-token difference between the two readers may be that, not reading quality. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Arm C of the reading arm, and the measured evidence D-11.1 has been missing. harness/readers/read_qwen3vl.py - Qwen3VLReader: one crop in, one string out, weights from the local HF cache at a pinned commit sha (local_files_only on both from_pretrained calls). No HTTP client, no api key, no remote endpoint — a test greps the executable source for all of them. - The prompt is a versioned module constant, not a string at the call site: two prompts are two arms (13a), and the full text is in config() so a diff shows what changed. - Abstain contract: <<NOTEXT>> (and its punctuated/quoted/cased near-misses) -> "" -> scored as an omission. Free-form refusal prose is deliberately NOT matched — guessing at intent could hide an invention, so the floor stays an upper bound instead. - No preface stripping: "The text reads: X" is returned intact and scores as the misread it is. Cleaning it up would flatter this arm against recognizers that cannot be chatty. - No confidence anywhere. output_scores/return_dict_in_generate are unset, so a decode logprob can never reach OCRWord.confidence or share a threshold with a detector score. - Identity (rule #9): version from transformers.__version__, model revision + torch version + dtype + thread count + prompt in config(), model_name derived from model_id so an 8B run of the same class cannot report under the 4B's name. experiments/qwen3vl_cpu_timing.py (PHI-free, synthetic crops) - mean/median/p95 seconds per crop, reusing the harness's own _latency_stats so "p95" means the same thing here as in every other result. Records load average: these cores are shared, and a busy box reads ~1.8x slower. experiments/reader_negative_control.py (PHI-touching — Arnav runs it) - The hallucination floor. A reader handed only GT boxes cannot invent a location, so its added_count is structurally zero; this lays a deterministic grid of token-shaped control boxes over each confirmed-blank frame and counts non-empty reads. - Box height comes from the median GT token height per stratum, not a fraction of the frame: at 3.5% of a mammo render the box is ~90px and crop_for_reading DOWNSCALES it, while every real token crop is an upscale. - --gt is required and read for two non-scoring things: the image_id set (to refuse any frame marked has_text=0 that actually has tokens — read_image's own guard cannot fire when this passes an empty ground truth) and those median heights. token_text is never read. - Floor runs take config_id="floor-Nbox" so their rows can never pool with an accuracy run's; the control-box spec is not part of arm_config_hash. - The blank set is every has_text=0 row (97 in text_presence_v2), not the 64 ct_axial the spec assumes. Tests are synthetic-only: 50 fast tests through fake model/processor objects plus a processor-only check that the chat template really emits one vision placeholder, and a weight-backed tier behind QWEN3VL_WEIGHTS=1 (known token, determinism, abstention on a blank crop, charset). Full suite 558 passed, 32 skipped; ruff clean on all new files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both branches added harness/readers/read_*.py independently and the package __init__ came out byte-identical, so pyproject.toml was the only content conflict: two extras added at the same point. Both survive — [openocr] and [qwen3vl]. Beyond resolving it, three integration edits so the two halves are actually one arm: - Qwen3VLReader joins 13c's SHARED reader scaffold (contract, version sourcing, charset, determinism, known token) instead of being tested only by its own file. It registers OPT-IN under QWEN3VL_WEIGHTS=1: the shared tier reads three crops per reader, and at ~16 GB resident and ~40 s per crop this one would turn a 30-second suite into a multi-minute one and OOM a box that already has a model loaded. test_every_reader_is_ registered asserts the third reader IS there when the opt-in is set, so the opt-in cannot silently skip it. - charset() implemented for the VLM: vocabulary entries that decode to exactly one character. Needed for the shared charset check, and it loads only the processor — the processor/model split (_ensure_processor) exists so asking a real question of the real checkpoint does not materialize 16 GB. - Fixed a collection crash in tests/test_readers.py, pre-existing and not from 13d: with no reader installed the parametrize list is empty, pytest hands its NOTSET sentinel to the ids= function, and the raise ERRORs the whole FILE at collection instead of skipping it — taking test_every_reader_is_registered, whose entire job is to notice missing readers, down with it. Also adds prompts/phase13/13d_RESULT.md: measured CPU timing (45.9 s/crop mean under load, ~26 s idle), the per-crop planning arithmetic, the OOM finding (fp32 4B is ~16 GB and was SIGKILLed twice under contention), the exact negative-control command, and what is still outstanding — the floor itself has NOT been run, and no accuracy number for this arm may be reported until it has. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the cloud reading arm. Every crop it reads leaves the environment, so
the gate is the feature, not the model wrapper around it.
harness/runners/read_gemini.py
GeminiReader(Reader): one crop in, one string out, scored by the existing
frozen path. Constructed with an explicit source= and no default:
- CropSource.REAL requires OCR_BAA_CLEARED_GEMINI == "1" (compared with ==,
never truthiness, so "0"/"no" cannot open it), checked at construction AND
before every call, above the transport — an injected transport, a subclass,
or a reader outliving its clearance cannot route around it.
- CropSource.SYNTHETIC needs no flag but transmits only crops whose sha256
was registered up front, so a real render fails on its pixels rather than
on the caller's honesty.
Vertex AI only: _live_transport builds genai.Client(vertexai=True, project,
location) and refuses to start without both. The Gemini Developer API /
AI Studio key surface is a different product and is NOT BAA-covered
(CLAUDE.md §4); an API key alone cannot reach this reader.
Rule #9 with no library __version__ to source from: Vertex serves only the
alias gemini-2.5-pro (verified 2026-08-11 — the dated preview revisions 404),
so the revision is pinned from the RESPONSE. The first reply's model_version
becomes the arm's version; reading version before that raises rather than
stamping a placeholder onto a result row; a later reply naming a different
revision aborts the run; an alias plus no reported revision is fatal.
Never persists or logs a response body — it holds read-back token text, which
is PHI on a real crop. Never opens a credential file. Never retries: a retry
is a second egress event on the same pixels.
harness/cost.py
+ gemini pricing rule (258 tokens/image unit, 768px tiling) and gemini_in /
gemini_out rates. Input side only — 2.5 Pro bills unswitchable thinking
tokens as output, so the reader accumulates real usage and usage_summary()
["measured_cost_usd"] is what the arm should be priced on. Measured 1676
input tokens/crop against ~303 estimated: the estimator is known-inaccurate
here and deliberately not retuned to match one run.
experiments/bench_reader_synthetic.py
Exact-match bench over 18 fixed synthetic CMFN/GRDN/ACC crops, cut by the
pinned crop_for_reading so arms are comparable. --show-predictions is
refused for a REAL Gemini reader whatever the flag says.
tests/test_read_gemini.py
38 tests, fully offline. The gate test is first and asserts the transport
was never called. Also covers: only "1" opens the gate; an injected
transport does not bypass it; a real crop cannot be laundered through the
synthetic path; response-side version pinning and mid-run drift; blocked and
truncated reads score as omissions rather than crashing; and source-level
greps that no credential file is opened and no response body persisted.
Measured on synthetic fixtures 2026-08-11, live against Vertex:
94.4% exact match (17/18), tying the docTR recognizer, and missing the same
token — GRDN0001, read GRDN00001 by Gemini and GRDNO001 by docTR. 7.71 s/crop
against 0.071, and $0.0028/crop against $0. Not a gt_v1 result: fixture glyphs
are clean and evenly spaced, burned-in overlay text is not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolves one conflict and adopts the readers convention that landed while this branch was out. pyproject.toml — the only conflict, in [tool.ruff.lint.per-file-ignores]. Both sides added a one-line-docstring E501 exemption at the same point; KEPT BOTH (13d's reader_negative_control.py and 13g's bench_reader_synthetic.py). The optional-dependencies block merged cleanly: all of doctr, easyocr, openocr, paddle, qwen3vl and tesseract survive. Adds the [gemini] extra, which this branch should have carried and did not — google-genai was installed by hand, so the arm was not reproducible from the repo. Pinned google-genai==2.17.0. Noted there that unlike every other extra it fixes an SDK, not a model: no weights, no revision, so it does not pin what the arm reads. READER, NOT RUNNER: harness/runners/read_gemini.py -> harness/readers/ read_gemini.py (git mv, history follows). It subclasses harness.reading. Reader, takes a crop and returns a string — the same shape as SVTRv2, docTR-PARSeq and Qwen3-VL. The old path straddled both conventions: a "read_" name in the "runners" directory. Its numbers belong in the Arm C reader table with those three, never in the end-to-end ranking, because every reader in that arm is handed oracle boxes. Registered in tests/test_readers.py's shared scaffold, GATED on GEMINI_LIVE=1. Qwen3-VL's gate is about cost; this one is about egress — reading a crop makes a network call to a BAA-covered endpoint, so an unconditional registration would mean anybody's pytest silently billed a Google project and sent pixels out of the environment. Following 13d's pattern, an opted-in reader that fails to import is now a failure rather than a silent skip. The instance is wired to the digest of exactly the one crop the shared tests use, so it can transmit that crop and nothing else. Adds charset(), and an explicit VERSION_SOURCE_EXEMPT entry for it. The shared version-sourcing check imports version_source and asserts it equals the library's __version__; no installed library carries a hosted model's version. Setting version_source="google.genai" would pass the test and be wrong — the SDK version says nothing about what the model returns, so it would attach confident, false provenance to every row. The exemption is a named list with a written reason, mirroring runners/base.py's precedent, so opting out stays a visible edit and an ACCIDENTAL version_source=None still fails. The revision is pinned from the API response instead. charset() returns printable ASCII and says why that is a different kind of answer than a recognizer's: there is no decoder dictionary to inspect behind an API. Worth stating rather than hiding behind a matching signature — this arm's risk is the opposite of a missing charset. A generative model can spell anything, including an identifier that is not in the image, and that is measured by the negative control. Restores this branch's prompt edits, which the merge replaced with main's newly-tracked copies: 13g_gemini_gated.md's amendment (the BAA decision was made), 13i_experiment_cli.md's Gemini-on-real-frames constraints, and the new 13g_RESULT.md. Full suite in the repo-root venv: 617 passed, 3 skipped, 1 xfailed. ruff: 27 errors, identical to main's pre-existing count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ation, and what is still outstanding Records the placement decision and its consequence (Arm C table, never the end-to-end ranking), the GEMINI_LIVE gate and why it is stricter than Qwen3-VL's, and the version-sourcing exemption. Adds the two sections the doc was missing: what a human actually runs against real frames — including that no driver exists until 13i, so the gate variable currently has nothing to feed — and an explicit outstanding list. The item that matters most there: this arm is generative and has NOT run its blank-frame negative control, so its 94.4% must not reach the final report until it has. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
617 passed / 3 skipped / 1 xfailed in the repo-root venv, 38 in the Gemini file. Also records which venv was used and why: google-genai went into a real worktree-local venv, never the shared benchmark one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… harness that produced it
Results (all against gt.csv afc65989..., scored-set e0bf9503..., 199 images / 2,351 tokens, CPU):
Arm A end-to-end, library-stock reading false-red found invented p95
doctr:stock 1.0.1 60.47% 39.53% 1,757 605 2.23s
pp-ocrv6_medium:stock 3.7.0 38.57% 61.43% 1,075 2,502 116.94s
easyocr:stock (floor) 1.7.2 23.97% 76.03% 818 707 24.69s
easyocr:tuned (shipped) 1.7.2 5.78% 94.22% 210 1,310 22.68s
Arm C oracle-box readers reading invents on 388 blank control crops
doctr-parseq v1.0.1 81.96% 242 (62.4%)
svtrv2 0.1.5 80.79% 34 (8.8%)
docTR wins pooled AND per-stratum (9 of 10 scored strata; sole loss us_other, -1.7pts on 61
tokens from one image). The contender lost on every axis: PP-OCRv6 invents more tokens than
exist and runs 52x slower. Oracle boxes lift docTR-family reading 60.47% -> 81.96%, so about
half the end-to-end error is detection, not recognition — the next win is a better detector.
Two findings that cut against this project's own prior decisions:
* The EasyOCR 0.2/0.2 threshold shipped since Phase 9 reads 4x worse than the library
default it overrode (5.78% vs 23.97%), with detection recall collapsing 34.79% -> 8.93%.
Box count barely moved (1,518 -> 1,472) while Found fell 818 -> 210, i.e. the boxes fit
worse rather than the text being missed.
* Nothing is frozen: tuned_configs.json engines is still {} and --freeze was never run, so
doctr:tuned and pp-ocrv6_medium:tuned still raise ArmNotFrozenError. Both dev-slice sweep
reports print "FROZEN" and are wrong.
Harness added here: preflight_13i.sh (three-stage escalating gate), run_overnight_13i.sh (the
chained 7-job run), run_morning_13i.sh (waits out an in-flight run then chains the next job),
morning_summary.sh (collapses every completed result into one PHI-free file). The overnight
chain died at 09:21 UTC to the 1h CLOUD_WORKSTATIONS_IDLE_TIMEOUT, not the scheduled
shutdown; run_morning_13i.sh documents the progress-counting trick that located the killed
run to the exact image, and that nothing here survives a VM stop.
Not run, and not silently: Qwen3-VL reader (~20h CPU), the PP-OCRv6 dev sweep (~2h), and
every BAA-gated cloud arm. Gemini cost analysis in slides.md is arithmetic over real crop
geometry with harness/cost.py's own price table — an input-side floor of $1.04 for 2,739
crops, never a measurement; that arm has never touched a real frame.
PHI: no pixels, no token text, no manifest rows. Every artifact staged here is counts, rates,
hashes, knob values or version strings. gt.csv and dev_v1.csv remain gitignored; only their
hashes are committed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
final commit