From 194a706eed78f7d4fad640f1a79d57bcd5bf696c Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Sun, 26 Jul 2026 06:46:09 +0100 Subject: [PATCH 1/3] docs: record the itd-88 lifeboat coverage experiment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Probe this repository (record-rich, 21/23 grounded) against two record-less public repos — spf13/cobra and psf/requests (4/23 and 3/23 grounded) — and read the disembark coverage aggregate as the experiment's result. The record delta is ~17-18 grounded brief sections; product/personas is the one section never grounded by extraction on any repo, confirming the intent's open question. The packer is built to the full 23-section list, personas carried as a human-owned blank. Raw probe/coverage JSON stays under .abcd/.work.local/scratch (gitignored). Assisted-by: Claude:claude-opus-4-8 --- .../2026-07-26-itd-88-coverage-experiment.md | 148 ++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md diff --git a/.abcd/development/research/2026-07-26-itd-88-coverage-experiment.md b/.abcd/development/research/2026-07-26-itd-88-coverage-experiment.md new file mode 100644 index 00000000..1d138518 --- /dev/null +++ b/.abcd/development/research/2026-07-26-itd-88-coverage-experiment.md @@ -0,0 +1,148 @@ +# itd-88 lifeboat coverage experiment — the cross-repo readout + +itd-88 inverts the lifeboat build: **probe before pack**. Rather than assume the +brief's 23-section structure fits reality, the experiment runs the same read-only +`disembark probe` over repositories of mixed record quality and reads the coverage +delta as *what keeping a record is worth*. This note records the first run of that +experiment and the section list the packer inherits from it. + +## Method + +Three repositories, probed read-only with `disembark probe --json`, then +aggregated with `disembark coverage ...`: + +- **this repository** — the record-rich case (Tier 0 git + Tier 1 conventions + + Tier 2 abcd-native), 489 commits. +- **[spf13/cobra](https://github.com/spf13/cobra)** — a well-known Go CLI library, + 1106 commits, no abcd record (Tier 0 + Tier 1 only). +- **[psf/requests](https://github.com/psf/requests)** — a well-known Python HTTP + library, 6486 commits, no abcd record (Tier 0 + Tier 1 only). + +The two foreign repositories are the record-less case the experiment targets: a +popular project of a different ecosystem each, cloned full-history so Tier 0 git +archaeology has commits to read, and carrying no `.abcd/` directory. Both were +byte-identical after probing — the working trees stayed clean and no `.abcd/` was +written — so the read-only, out-of-tree contract holds across ecosystems. Probing +the same repository twice yields byte-identical JSON, so a delta in the aggregate +is a delta in the repositories, never in the tool. + +## The aggregate + +23 brief sections × 3 repositories. Per-repository status counts: + +| repository | grounded | partial | blank | +|-----------------|----------|---------|-------| +| this repository | 21 | 2 | 0 | +| cobra | 4 | 7 | 12 | +| requests | 3 | 11 | 9 | + +The record delta is legible as a number: the record-rich repository grounds **21 +of 23** sections; the two git-plus-conventions repositories ground **4** and **3**. +Keeping an abcd-native record is worth roughly **17–18 grounded sections** over an +otherwise comparable repository that keeps only a README, docs, and git history. + +The coverage verdict names **0 of 23 sections blank in every probed repository** — +no section is universally ungroundable, because this repository authored a brief +page for nearly all of them. The always-blank cut therefore removes nothing. + +## What survives, and the one section that does not + +The `Kind` field (schema v2) partitions the 23 sections identically in all three +probes: **19 extractable** and **4 human-owned** (`product/mental-model`, +`product/personas`, `delivery/verification-matrix`, `delivery/out-of-scope`). A +human-owned blank is reported not as a failure but as *"yours to write, not an +extraction"* — the probe declines to pretend a repository could ground it. + +Reading the section × repo table for the sections that no repository grounds by +extraction (best status below `grounded` across all three) yields exactly one: + +- **`product/personas`** — `partial` here (abcd-native, low confidence), + `blank` on cobra, `blank` on requests. Its blank on both foreign repos searched + *"personas registry, press-release quote attributions"* and found nothing. This + is the section itd-88's Open Question predicted: *"`product/personas` is + predicted blank below abcd-native and only partial there; if that holds across + the corpus, the section is not derivable from a repository at all."* **The + experiment confirms the prediction.** Personas survives the strict always-blank + cut only because this repository hand-authored a personas page; on the + extraction test it is never grounded on any repository, and only ever partial. + +The remaining three human-owned sections (`mental-model`, +`verification-matrix`, `out-of-scope`) ground on this repository — someone wrote +the brief page — but stay blank on both foreign repos: they are authored, not +extracted. They survive because the record carries them, not because a repository +yields them. + +## The surviving section list + +Read as the packer's input, the aggregate's own always-blank verdict removes no +section, so **the packer is built to all 23 brief sections**: + +``` +product/press-release, product/context, product/mental-model, product/scope, +product/personas, constraints/platform, constraints/dependencies, +constraints/invariants, constraints/naming, evidence/what-worked, +evidence/what-didnt, evidence/open-questions, evidence/tradeoffs, surfaces, +internals, delivery/build-sequence, delivery/verification-matrix, +delivery/out-of-scope, glossary, graveyard, rescue/spine, docs/adrs, +activity/issues +``` + +The one section the experiment flags for review is `product/personas`: it clears +the always-blank cut only on the strength of this repository's authored page and +is never groundable by extraction. It stays in the section list as a **human-owned +blank** — the packer emits it with its searched-list and its question rather than +dropping it — which is the honest home for a section a repository cannot yield. + +## Load-bearing evidence lines + +The three probe lines that carry the finding: + +1. **cobra, `graveyard` — grounded (git, high):** *"4 reverted commits, 48 files + deleted (e.g. .circleci/config.yml)"*. A repository whose richest tier is Tier 0 + still grounds `graveyard` from git history alone, satisfying that acceptance + criterion on a real foreign repository. +2. **requests, `constraints/naming` — blank:** searched *"GLOSSARY.md, + docs/glossary\*, docs/naming\*, naming document"*, question *"What names and + reserved vocabulary are fixed? No naming document and no glossary found."* The + blank is a first-class result: it names what it searched and the question a + human must answer, rather than inventing a plausible section. +3. **cobra, `product/personas` — blank:** *"human-owned — yours to write, not an + extraction"*, searched *"personas registry, press-release quote + attributions"*. This is the evidence behind the one section the experiment + flags as never groundable by extraction. + +## Observed reality vs spc-3's assumptions + +- **The rich-vs-git-only framing holds, but the graveyard inverts it.** spc-3 + frames the delta as record-rich beating git-only everywhere. `graveyard` is the + exception: it is `grounded (git, high)` on cobra (4 reverts, 48 deletions) and + requests (39 reverts, 272 deletions) but only `partial (git, medium)` on this + repository (40 deletions, no reverts detected). `graveyard` grounds from Tier 0 + archaeology, which is orthogonal to record richness — a busy history with + reverts grounds it better than a curated one, so the record-rich repository + scores *lower* here than the poor ones. This confirms the tiers table's claim + that `graveyard` grounds from git alone, and refines the delta story: the record + premium is 17–18 sections *elsewhere*, and roughly zero on `graveyard`. +- **The always-blank cut is uninformative on a corpus that includes the author's + own repository.** Because this repository grounds nearly everything, no section + is blank-everywhere, so the aggregate's headline verdict prunes nothing. The + section that actually fails the derivability test (`personas`) is visible only by + reading the section × repo table for *never-grounded-by-extraction*, not from the + always-blank count. A future run over several record-less repositories *without* + the authoring repository in the corpus would let the always-blank cut do the + pruning the spec expects of it. +- **Corpus size (Open Question 2).** Two foreign repositories agree closely + (grounded 4 vs 3; the same human-owned sections blank), which is suggestive but + not yet a trustworthy population — the finding here is a first reading, not a + settled number. + +## Conclusion + +The premise survives contact with reality: an abcd-native record is worth roughly +17–18 grounded brief sections over a git-plus-conventions repository, and the probe +holds its honesty discipline — every grounded section cites a file, every blank +names what it searched and the question it raises, and it never fails merely +because a repository is poor. The section structure survives too: 22 of 23 sections +ground somewhere, and the packer is built to the full 23-section list, with +`product/personas` carried as the human-owned blank the experiment singles out as +never derivable from a repository. From 7b24b98befbd20440b335a96d0154bd33cbcec6e Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Sun, 26 Jul 2026 06:49:17 +0100 Subject: [PATCH 2/3] docs: ship itd-88 and close spc-3 behind the coverage experiment The lifeboat coverage experiment has run (see the 2026-07-26 research note), so spec close moves spc-3 open -> closed and reconciles itd-88 planned -> shipped, parking fidelity receipt rcp-4d07032fc6ab (the independent judge is dispatched separately; Audit Notes stay untouched). impact: additive is hand-stamped into the shipped itd-88 frontmatter: the Reconcile path cannot stamp it yet (known gap iss-126, the intent ship-path impact gap), so it is applied by hand here. Assisted-by: Claude:claude-opus-4-8 --- .../itd-88-lifeboat-coverage-experiment.md | 4 +++- .../{open => closed}/spc-3-lifeboat-coverage-experiment.md | 0 2 files changed, 3 insertions(+), 1 deletion(-) rename .abcd/development/intents/{planned => shipped}/itd-88-lifeboat-coverage-experiment.md (98%) rename .abcd/development/specs/{open => closed}/spc-3-lifeboat-coverage-experiment.md (100%) diff --git a/.abcd/development/intents/planned/itd-88-lifeboat-coverage-experiment.md b/.abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md similarity index 98% rename from .abcd/development/intents/planned/itd-88-lifeboat-coverage-experiment.md rename to .abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md index d9e7d7c7..ebda6101 100644 --- a/.abcd/development/intents/planned/itd-88-lifeboat-coverage-experiment.md +++ b/.abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md @@ -7,6 +7,7 @@ suggested_kind: null reclassification_history: [] builds_on: [] severity: major +impact: additive related_adrs: [adr-35] --- @@ -75,4 +76,5 @@ The honesty discipline is the other half. A rescue tool that invents a plausible ## Audit Notes -_Empty. Populated by intent-fidelity-reviewer when intent moves to shipped/._ + +Fidelity review OWED (receipt rcp-4d07032fc6ab). diff --git a/.abcd/development/specs/open/spc-3-lifeboat-coverage-experiment.md b/.abcd/development/specs/closed/spc-3-lifeboat-coverage-experiment.md similarity index 100% rename from .abcd/development/specs/open/spc-3-lifeboat-coverage-experiment.md rename to .abcd/development/specs/closed/spc-3-lifeboat-coverage-experiment.md From 85fb9e9e406521de8ec5410972cb2e0faecd58e4 Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Sun, 26 Jul 2026 07:03:19 +0100 Subject: [PATCH 3/3] docs: ingest itd-88's fidelity verdict into its Audit Notes Receipt rcp-4d07032fc6ab: 6 MET, 2 MET_WITH_CONCERNS, 0 NOT_MET. The gap audit's diverged entries (verb naming, graveyard grounding narrower than the four-signal promise, probe/coverage absent from the plugin surface doc) and the one missing claim (no Pass-B exemption marker in provenance) are recorded for the maintainer, per verifier-selects-gates-decide. Assisted-by: Claude:claude-fable-5 --- .../itd-88-lifeboat-coverage-experiment.md | 76 ++++++++++++++++++- 1 file changed, 74 insertions(+), 2 deletions(-) diff --git a/.abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md b/.abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md index ebda6101..a0835cd1 100644 --- a/.abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md +++ b/.abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md @@ -76,5 +76,77 @@ The honesty discipline is the other half. A rescue tool that invents a plausible ## Audit Notes - -Fidelity review OWED (receipt rcp-4d07032fc6ab). + +Fidelity review — receipt rcp-4d07032fc6ab (verifier abcd:intent-fidelity-reviewer claude-fable-5). + +Provenance: abcd:intent-fidelity-reviewer@claude-fable-5 · rubric_hash sha256:bda482993615f6ee00d06f9649bff9c9bc8f22c989683386437eea4db28369b2 · prompt_hash sha256:95792472ae74ca0469f69a51c618946e0d33cb1380032460099ed4b469d67e86 +Input attestations: diff:docs/itd-88-coverage-experiment @ 7b24b98befbd20440b335a96d0154bd33cbcec6e: internal/core/lifeboat/ (probe/coverage/plan/pack/graveyard), internal/surface/cli/cli.go disembark wiring, commands/abcd/disembark.md, .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md@-; rubric:.abcd/.work.local/reviews/rcp-4d07032fc6ab.request.md@sha256:bda482993615f6ee00d06f9649bff9c9bc8f22c989683386437eea4db28369b2; + +Acceptance rollup: MET 6 · MET_WITH_CONCERNS 2 · NOT_MET 0 · INCONCLUSIVE 0 + +Per-criterion verdicts: +- ac-1 — MET: Live probe of a git-only fixture (git init, one revert, nothing else) exits 0 and renders all 23 sections with every blank carrying its searched list and a human question; completeness and question-on-every-blank are test-enforced. + evidence: cmd: go run ./cmd/abcd disembark probe (exit 0) — "grounded 1 · partial 4 · blank 18 (of 23 sections) … searched: glossary, naming registry, reserved-vocabulary tables / ? Nothing probed grounds constraints/naming; a human must supply it." + evidence: internal/core/lifeboat/probe_test.go:353 — "func TestProbeNeverBlankSectionCarriesAQuestion" + evidence: internal/core/lifeboat/probe_test.go:322 — "func TestProbeReportsEverySection" +- ac-2 — MET: TestProbeLeavesEveryFileByteIdentical sha256-hashes every file before and after a probe and fails on any rewrite/create/remove; it and TestProbeNeverMutatesTheSource ran green (go test -run …, PASS), and all reads go through a contained read-only os.Root. + evidence: internal/core/lifeboat/probe_test.go:254 — "func TestProbeLeavesEveryFileByteIdentical … t.Errorf(\"probe rewrote %s (sha256 %s -> %s)\"" + evidence: cmd: go test -run 'TestProbeLeavesEveryFileByteIdentical|TestProbeNeverMutatesTheSource' ./internal/core/lifeboat/ — "--- PASS: TestProbeLeavesEveryFileByteIdentical / --- PASS: TestProbeNeverMutatesTheSource" + evidence: internal/core/lifeboat/probe.go:102 — "Probe must be side-effect-free and must never write to the source repository" +- ac-3 — MET: The anti-fiction rule is test-enforced (every non-blank row must cite evidence) and a live JSON sweep over abcd-cli, cobra, and requests found zero grounded/partial rows without a citation. + evidence: internal/core/lifeboat/grounding_test.go:105-108 — "if s.Status != StatusBlank && len(s.Evidence) == 0 { t.Errorf(\"section %s is %s but cites no evidence\"" + evidence: cmd: disembark probe --json over abcd-cli/cobra/requests, checked for evidence-less non-blank rows — "non-blank rows missing evidence: NONE (all three repos)" +- ac-4 — MET: Live `disembark coverage self.json cobra.json requests.json` renders one 23-section × 3-repo status table, the per-repo probe summaries put the delta in numbers (grounded 21 vs 4 vs 3), and the research note states it as 17–18 grounded sections. + evidence: cmd: go run ./cmd/abcd disembark coverage <3 reports> (exit 0) — "brief section abcd-cli cobra requests verdict … 0 of 23 sections are blank in every probed repo." + evidence: .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md:39-42 — "Keeping an abcd-native record is worth roughly **17–18 grounded sections**" + evidence: internal/core/lifeboat/coverage.go:203 — "func Aggregate(covs []Coverage) AggregateReport" +- ac-5 — MET: TestProbeIsDeterministic requires byte-identical JSON across two probes and ran green, and a live double probe of this repository produced byte-identical output under cmp. + evidence: internal/core/lifeboat/probe_test.go:385 — "func TestProbeIsDeterministic … if string(ja) != string(jb)" + evidence: cmd: disembark probe . --json twice, cmp — "DETERMINISTIC: byte-identical" +- ac-6 — MET_WITH_CONCERNS: A live git-only fixture grounds graveyard at (git, high) with only the git tier present, and TestProbeGraveyardFromGitAlone enforces it — but grounded status requires a revert: deletions alone yield partial (this repository itself scored partial), and the parenthetical's unmerged-branches and removed-dependencies signals live in the packer's archaeology layer, not the probe's grounding decision. + evidence: cmd: disembark probe — "tiers present: git … + graveyard grounded (git, high) / evidence: 1 files deleted (e.g. f.txt), 1 reverted commits" + evidence: internal/core/lifeboat/grounding_test.go:115 — "func TestProbeGraveyardFromGitAlone" + evidence: internal/core/lifeboat/sources_git.go:81-89 — "deletions without any revert are only partial evidence, not a grounded graveyard" + evidence: .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md:118-120 — "only `partial (git, medium)` on this repository (40 deletions, no reverts detected)" +- ac-7 — MET: IngestLessons drops any lesson whose evidence refs resolve to no layer-1/2 finding id ("no valid evidence refs") and TestIngestLessonsCiteOrDropped asserts the uncited lesson is dropped while the cited one is written; the test ran green. + evidence: internal/core/lifeboat/graveyard_lessons.go:116 — "drop(\"no valid evidence refs\")" + evidence: internal/core/lifeboat/graveyard_lessons_test.go:96 — "func TestIngestLessonsCiteOrDropped … Evidence: []string{\"no-such-id\"} … res.Dropped != 1" + evidence: cmd: go test -run TestIngestLessonsCiteOrDropped ./internal/core/lifeboat/ — "--- PASS: TestIngestLessonsCiteOrDropped" +- ac-8 — MET_WITH_CONCERNS: One code path holds — Pack calls the same Plan the dry-run renders (pack.go:84) — and parity is test-enforced as a hash chain (dry-run manifest hash = ManifestSHA256(planned files) = provenance hash = independent re-hash of the written tree), all green; the caveats are that the shipped verbs are `plan`/`pack ` rather than the promised `dry-run`/`to `, and the parity assertion spans TestPlanManifestReportsHashAndTotals plus TestPackProvenanceHashVerifies rather than one direct plan-vs-written-files comparison. + evidence: internal/core/lifeboat/pack.go:84 — "lb, err := Plan(repoAbs)" + evidence: internal/core/lifeboat/pack_test.go:96 — "func TestPackProvenanceHashVerifies … does not verify against the written tree" + evidence: internal/core/lifeboat/plan_test.go:741-743 — "if m.ManifestSHA256 != ManifestSHA256(lb.Files) { t.Error(\"manifest hash disagrees with the file set\")" + evidence: internal/surface/cli/cli.go:387-388 — "Use: \"plan [repo]\" … without writing anything (dry run)" + +Gap audit: +- honoured: + - Probe before pack, read-only and out-of-tree — reads a repo without touching it + evidence: internal/core/lifeboat/probe_test.go:254 — "TestProbeLeavesEveryFileByteIdentical" + evidence: .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md:24-25 — "Both were byte-identical after probing" + - A blank is a first-class result: what was searched and the question a human must answer + evidence: internal/core/lifeboat/coverage.go:112-121 — "searched: … / ? %s" + evidence: cmd: disembark probe — "every one of 18 blanks carries searched + question" + - The cross-repo table answers what keeping a record is worth, in a number + evidence: .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md:39-42 — "grounds **21 of 23** … the two git-plus-conventions repositories ground **4** and **3**" + - Stable, aggregatable coverage schema with schema_version, enforced at the aggregate + evidence: internal/core/lifeboat/coverage.go:16 — "SchemaVersion int `json:\"schema_version\"`" + evidence: internal/surface/cli/cli.go:369-375 — "missing schema_version … upgrade abcd" + - Graveyard as a first-class section with a code-enforced cite-or-be-dropped validator + evidence: internal/core/lifeboat/graveyard_lessons.go:116 — "drop(\"no valid evidence refs\")" + - Packer built after the aggregate settled the section list + evidence: .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md:77-78 — "the packer is built to all 23 brief sections" +- diverged: + - Packer surface promised as `disembark to ` with `dry-run`; shipped as `disembark plan [repo]` and `disembark pack ` + evidence: .abcd/development/intents/shipped/itd-88-lifeboat-coverage-experiment.md:41 — "The packer (`disembark to `)" + evidence: internal/surface/cli/cli.go:413 — "Use: \"pack \"" + - Graveyard grounding is narrower than the four-signal promise: only reverts ground; deletions alone are partial; unmerged branches and removed dependencies feed the pack-path archaeology, not the probe's grounding + evidence: internal/core/lifeboat/sources_git.go:81-89 — "deletions without any revert are only partial evidence" + evidence: internal/core/lifeboat/graveyard_archaeology_test.go:146 — "TestArchUnmergedBranchesOrderedByDivergence" + - Press release headlines `/abcd:disembark probe` and `/abcd:disembark coverage`; the markdown command surface documents only plan and pack — probe/coverage ship as CLI verbs only + evidence: commands/abcd/disembark.md:4 — "argument-hint: \" | plan \"" + evidence: internal/surface/cli/cli.go:306 — "Use: \"probe [repo]\"" + - The recorded delta compares record-rich against git+conventions repos, not a strictly git-only repo; the corpus is two foreign repositories + evidence: .abcd/development/research/2026-07-26-itd-88-coverage-experiment.md:134-137 — "suggestive but not yet a trustworthy population — the finding here is a first reading" +- missing: + - Pass B ships as a declared exemption in `_provenance.json`, never a silent gap — no exemption field or marker exists anywhere in the lifeboat package or the Provenance struct + evidence: internal/core/lifeboat/plan.go:65-80 — "type Provenance struct { SchemaVersion … Omissions } — no exemption field; grep 'exemption' across internal/core/lifeboat/ returns nothing" \ No newline at end of file