From eea3ef163b4fcb902b0e1208b3bebe7121d9ed79 Mon Sep 17 00:00:00 2001 From: Guy Corbaz Date: Sat, 1 Aug 2026 00:42:57 +0200 Subject: [PATCH 1/3] Story 5.6: the blocker proposes every pair, and the floor is an integer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The candidate generator is now an explicit component. `candidates()` returns every unordered pair of DISTINCT observation ids as a `BTreeSet`, total by decision rather than by omission: D13 disposes of the performance argument itself (90k pairs at 300 hosts is noise), and every exclusion a blocker makes is a false split it can never be talked out of. `CandidatePair` has private fields ordered by its constructor, so `new(a, b) == new(b, a)` holds by construction; `new(a, a)` is `None`, which gives the self-pair precondition its first holder — `decide_pair` answers it today and says only that the pair "arrives as an argument". D13's `blocking_recall >= 0.999` ships as `BLOCKING_RECALL_FLOOR_PER_MILLE: u32 = 999`, on D13's own milli-units corollary and on the architecture's ratified test name, which already carries no float. The `float-free` gate walks 4 files under identity/ now, unweakened. Measured against the committed corpus, in fixtures.rs's test module because the domain crate may not read files (D47): 24 traps, 23 name a pair, 10 must-merge, recall 1000 per-mille. The containment assertion and the recall value are two tests on purpose — in one, a missing pair panics before any recall is computed. 309 -> 332 tests (139 bin + 147 core + 46 xtask). Six gates green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HEE7mz4xufqYW5xKfgJBaF --- .../5-6-blocker-and-recall-assertion.md | 770 ++++++++++++++++++ .../sprint-status.yaml | 82 +- crates/opencmdb-bin/src/fixtures.rs | 196 +++++ crates/opencmdb-core/src/identity/blocking.rs | 608 ++++++++++++++ crates/opencmdb-core/src/identity/mod.rs | 1 + 5 files changed, 1654 insertions(+), 3 deletions(-) create mode 100644 _bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md create mode 100644 crates/opencmdb-core/src/identity/blocking.rs diff --git a/_bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md b/_bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md new file mode 100644 index 0000000..2a34530 --- /dev/null +++ b/_bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md @@ -0,0 +1,770 @@ +# Story 5.6: The blocker, and the recall assertion nobody writes + +Status: ready-for-dev + + + +## Story + +As the identity engine, +I want candidate generation to be an **explicit component** with a **measured recall floor**, +so that a false split cannot be born silently before any rule has had a chance to speak — and so +that abstention finally has a denominator. + +**This story writes the blocker and the recall assertion. It answers no pair and scores no trap.** +Story 5.7 owns the corpus wiring (`score_corpus`, `run_trap`, `Decision -> Outcome`); 5.8 owns the +unanswerable-level bucket; 5.9 persistence; 5.14 the operator surface; Epic 6 the `l2-*` rules. The +build order, quoted from `epics.md:1317`: *"the three debt stories (5.1, 5.2, 5.2b) -> the engine's +vocabulary (5.3, 5.4) -> the verdict algebra (5.4b) -> the pure join (5.5) -> **the blocker (5.6)** +-> wiring it to the corpus (5.7, 5.8) -> persistence (5.9, 5.10) -> the invariants (5.11, 5.12, +5.13) -> the operator-visible surface (5.14)"*. + +**Nothing under `fixtures/` moves.** No artefact bytes, no `MANIFEST.toml`, no re-hash. This story +**reads** the corpus (that is AC4) and never writes it. If any step appears to require re-authoring +a committed artefact, **STOP** — that is a finding, reported rather than absorbed. + +**`architecture.md` is NOT edited** (issue #54 for D13's short table; a milestone act). +**`architecture-views.md` is NOT regenerated** (issue #50). **`epics.md` is verify-only — an edit +there is a finding.** + +⚠️ **Branch from `master` only after PR #58 merges.** Measured at contexting: the working branch is +`bookkeeping-5.5-done`, one commit ahead of `master`, and **PR #58 is OPEN, not merged**. `master` +is at `0ebd50f` with **309 tests**. + +## What this story inherits, measured rather than assumed + +### 1. 🔴 The epic's own assertion reds the `float-free` gate — and this was measured, not reasoned + +`epics.md:1507` gives this story the assertion `blocking_recall >= 0.999`. Story 5.5 flagged it +forward on purpose (`deferred-work.md` §*Deferred from: story-5.5*, *"Owner: story 5.6, at +contexting, so it is a decision and not a surprise"*). **This is that decision.** + +Measured at contexting by writing a probe file under `crates/opencmdb-core/src/identity/` and +running `cargo xtask ci` — the probe was deleted and `git status` verified clean afterwards: + +| line, as written in the probe | gate | +|---|---| +| `let x = 0.999;` | 🔴 **bare float literal** | +| `assert!(true, "blocking_recall >= 0.999");` | 🔴 **bare float literal** — quoting the epic's AC *in an assertion message* reds | +| `assert!(true, "story 5.6 owns the blocker");` | 🔴 **bare float literal** — a story number with no letter suffix, inside a string, on a code line | +| `assert!(true, "story 5.7 answers it");` | 🔴 **the same** — the rule is *any* story number without a letter suffix, **not** just this story's | +| `/* … 0.999 … */` · `#[doc = "… 0.999 …"]` | 🔴 — block comments are **not** stripped, and `#[doc]` is a code line | +| `/// blocking_recall >= 0.999 in a doc comment` · `//! …` | ✅ green — `//`, `///` and `//!` are stripped | +| `assert!(true, "story 5.6b owns the blocker");` | ✅ green — **only** because of the trailing `b`; do not generalise | +| `const A: usize = 999;` · `const B: usize = 1000;` | ✅ green | +| `let recall_per_mille = 1000 * 10 / 10;` and `assert!(recall_per_mille >= 999, "…")` | ✅ green | +| `fn blocking_recall_above_999() {}` | ✅ green — `999` is preceded by `_`, an identifier character | +| `"architecture.md:1004-1007"` · `":988-993"` · `":1246-1252"` · `"epics.md:1317"` · `"l1.rs:16-18"` | ✅ green **even on a code line** — a hyphenated range is no numeric literal | +| `"n*(n-1)/2"` · `"2 of 3 is 666"` · `"v0.1.1"` | ✅ green | + +The first probe reported exactly three offenders, all three the predicted ones; the rows added above +were measured by a second probe during validation, which wrote AC6's required module doc in full and +found it **green** — every citation it must carry survives, and the prose lives in `//!` anyway. + +⚠️ **`"v0.1"` — one dot, no third segment — was NOT measured.** Do not infer it from `"v0.1.1"`. + +⇒ **The floor is expressed as an INTEGER in per-mille**, and the gate is not touched, not weakened, +not skipped and not given an escape hatch. Two independent grounds, neither invented here: + +1. **D13's own corollary** [architecture.md:988-993]: *"`confidence` is an **INTEGER in milli-units + (0..1000)**, never `REAL`/`DOUBLE` — a threshold at 0.85 compared as a float on two engines = two + different identity decisions for the same input."* Story 5.4b registered this as the corollary + that *"binds the day a float would otherwise appear"*. **This is that day.** +2. **The architecture already named the test**, and its name carries no float: + `blocking_recall_above_999` [architecture.md:2954], listed among the ratified test names with the + rule *"an invariant reads as a claim, so a red test names a broken claim."* **Use that exact + name.** It is the architecture's, not this story's. + +⚠️ **Do not weaken, disable, `#[allow]` or narrow the gate to make room for the assertion.** Story +5.4b's review named this exact failure mode: *"the wrong choice here is the one that gets the gate +weakened or deleted in its second week."* The gate walks **3** `.rs` files under `identity/` today +and must walk **4** after this story. + +### 2. 🔴 D18 refuses a "recall gate" by name — and this assertion is not one. The story owes the distinction. + +A reviewer will find this and it must be answered in the doc, not discovered in review. +[architecture.md:1246-1253] puts **pairwise recall** in Tier 2, *"published per release with +confidence intervals, trended — **blocking nothing**"*, with the reason spelled out: + +> pairwise recall — *"false-split is benign — so why would it block a release? **A loose threshold +> on a benign defect is a gate that can never fall, and a gate that cannot fall is decoration.**"* + +And NFR4 [prd.md:1179-1207] generalises it: at n=300, *"**any fraction is theatre**"* (`:1182`), so the +release gate is **truth-table failures = 0**, binary, at the **device** level. + +**Three differences make D13's `blocking_recall` legitimate where D18's pairwise recall is not, and +all three must be stated at the function:** + +- **Different subject.** D18 measures the ENGINE'S OUTPUT (did it group what should group). This + measures the CANDIDATE GENERATOR'S INPUT COVERAGE (did the pair even reach a rule). D13 names the + gap: *"if the candidate generator does not propose the pair, no downstream logic can ever group. + That is where false-splits are born silently, and **nobody tests blockers**"* + [architecture.md:1004-1007]. +- **Different venue.** D18 refuses a *release gate over bulk statistics*. This is an assertion in a + unit test over the frozen corpus — D13 says so in the same breath: *"a dedicated assertion: + `blocking_recall >= 0.999`, **measured in unit tests, before the scoring exists**."* +- **Different arithmetic, and this is the honest half.** At the committed corpus's denominator the + floor is **not** a tolerance: with 10 required pairs, one miss gives 900‰ and the floor reds. + **`>= 999‰` IS zero-tolerance at this scale**, which is exactly the binary form NFR4 demands. It + only becomes a 0.1% tolerance if the required set ever exceeds 1000 pairs — **and on that day + NFR4's *"any fraction is theatre"* bites and the floor must be revisited rather than inherited.** + Say that; do not let the per-mille dress imply a statistical tolerance the corpus cannot support. + +⚠️ **Do not claim this advances NFR4.** NFR4 is at the **device** level and story 5.8 reports it NOT +MET for this epic. This story adds no truth-table column and gates no release. + +### 3. The corpus is the truth set, and it was measured — 24 traps, 23 pairs, 10 must-merge, 7 sharing a MAC + +Counted at contexting by parsing `fixtures/scenario/traps/*.toml` against the streams they name: + +| quantity | measured | +|---|---| +| committed traps | **24** across 10 files | +| traps naming exactly **2** observations | **23** | +| traps naming **1** observation | **1** — `example-must-abstain` (`example.toml:40`, `replay = "scenario/replay/minimal.jsonl"`) | +| `must-merge` traps | **10** — one per family for the **nine** named families, plus `example-must-merge`, whose `family` is `None` (`trap.rs:133-141`: *"a format/example trap that is part of no family"*) | +| `must-merge` pairs whose two observations **share a MAC** | **7** | +| `must-merge` pairs that share **no** MAC | **3** — `multi-nic`, `shared-hardware-vm`, `docker-veth`, exactly the three whose expected rule is `l2-*` | +| trap pairs whose two observations are in the **same** `Scope` | **23 of 23** | +| replay streams named by a trap that carry a control record (`failure`/`capability`) | **0** — so `read_jsonl` is the right reader | +| trap pairs listing their two ids in an order the stream does not | **0 of 23** — and 0 out of ascending-UUID order either; **this is what makes M4 corpus-invisible** (AC7) | +| distinct `obs_id`s across the **11** streams a trap names | **39, zero collision** — the property AC4's union depends on | + +Two consequences the dev must carry: + +- **The 3 `l2-*` pairs belong in the recall denominator.** *Proposing* is not *answering*: story 5.8 + buckets the `l2-*` traps as unanswerable at this level, but a pair the blocker never proposes can + never be answered by Epic 6 either — that is precisely the false split D13 names. **The recall + truth set is all 10 `must-merge` pairs, not the 7 an L1 engine can answer.** +- **`sameScope` = 23/23 means the corpus cannot judge the scope question at all.** See §5. + +### 4. 🔴 The corpus can only be read from `opencmdb-bin` — the frontier decides where each test lives + +D47 is a gate: `opencmdb-core` may not touch the filesystem, and `read_traps`/`read_jsonl` live in +`crates/opencmdb-bin/src/fixtures.rs` (`:665`, `:647`). The blocker itself belongs in **core** — the +architecture names the file [architecture.md:3368]. So the story splits, and the split is forced: + +| where | what | why | +|---|---|---| +| `crates/opencmdb-core/src/identity/blocking.rs` | the pair type, the generator, the recall function, the floor constant, and **synthetic** tests | the architecture's own file; core cannot read `fixtures/` | +| `crates/opencmdb-bin/src/fixtures.rs`, **test module only** | `blocking_recall_above_999` and the universe-coverage test, over the committed corpus | the corpus-wide walks already live there (`walk_trap_files`, `#[cfg(test)] pub(crate)`, `:834`), added by stories 5.1/5.2/5.2b | + +⚠️ **Not `trap_gate.rs`.** Its `score_corpus` is story 5.7's seam and its code is off limits. +⚠️ `fixtures.rs` is the largest file in the tree, but the `file-size` gate counts only the lines +**before the first `#[cfg(test)]`** (`:729`), and this story adds none of those. + +**Why the truth set is the corpus and not synthetic pairs**, in one sentence the doc should carry: +Epic 4 froze the corpus **before** the engine on purpose — *"a metric written after the engine is +bent to fit the engine"* — so a recall floor measured against a truth set the engine's own author +writes today is the mirror D13 refuses for weights, applied to blocking. + +### 5. 🔴 The wrong blocker that passes the entire corpus + +Story 5.5's equivalent was the bare-MAC key. This story's is **blocking on `l2_domain`**: every +committed trap pair is in one scope (23/23, measured), so a generator that proposes only same-domain +pairs scores **1000‰** on the corpus and is invisible to every corpus test. + +It is wrong because a device's interfaces are not confined to one L2 domain — a router, a firewall +or a dual-homed server has NICs in several VLANs, and D12 makes the device the level where the +product keeps its promise [architecture.md:919-928]. Excluding cross-domain pairs would **build a +false split into the universe**, which is the one thing the blocker exists to prevent. + +⇒ **The universe must contain cross-domain pairs, and the only thing standing between that and green +is a synthetic two-domain test. Write it first.** + +⚠️ **Do not resolve this by consulting L1.** L1 will answer such a pair `l1-distinct-mac` -> +`Disqualifying` -> `NoMatch`, and that is correct **at L1**, about *interface* identity. The blocker +proposes; it does not judge. + +### 6. 🔴 L1 already contradicts a committed `must-merge` trap, and that is NOT a bug to fix here + +`multi-nic-must-merge` expects a merge via `l2-uplink-agrees`, and the two observations carry +different MACs — so `decide_pair` on that pair yields `l1-distinct-mac` -> `Disqualifying` -> +`NoMatch`. The same holds for `shared-hardware-vm-must-merge` and `docker-veth-must-merge`. + +This is **by design**: D12 splits the levels — *"multi-NIC false-split = L1 correct, L2 failed to +group"* [architecture.md:893] — and `multi-nic.toml`'s own header says it, in the committed bytes: +*"This family lives at L2 (device grouping) … never at L1, which is right to keep two distinct MACs +apart."* Story **5.8** owns the bucket that counts an `l2-*` trap as NOT PASSING. + +⇒ **If you find yourself changing an L1 verdict, widening `decide`, or reconciling a level, STOP.** +This story neither calls `decide_pair` nor compares a verdict to a trap. + +### 7. `grep '5\.6'` over the code returns **ZERO** — the doc worklist is by MEANING, not by grep + +Measured: `grep -rn '5\.6' crates/ xtask/ --include=*.rs` returns **one** hit and it is unrelated +(`arp_ping.rs:272`, *"128 * 200ms = 25.6s"*). The idiom story 5.5 used — start from the grep — yields +an **empty worklist here** and would ship four falsified claims. The sites were found by reading: + +| site | the claim this story falsifies | +|---|---| +| `identity/mod.rs:15-16` | *"**There is still no candidate pair generator**… the blocker that would propose pairs is the next story's"* | +| `identity/cascade.rs:292-293` | *"There is still **no blocker**: the pair it answers arrives from its caller, and candidate generation is the next story's"* | +| `identity/l1.rs:16-18` | *"the blocker is an L2 organ… and it **is the next story's**"* — the first half stays TRUE (l1 still generates nothing); only the ownership clause moves | +| `identity/l1.rs:287` | *"which is the **next story's** organ"* | +| `lib.rs:46-47` | *"its consumer (the candidate generator) **does not exist yet**"* — the generator exists after this story, **and does not consume [`join`]**. The reason changes; the conclusion (no flat re-export) survives. ⚠️ Getting this one right means saying what is actually true: `join`'s consumer is still story 5.7's harness. | + +⚠️ **This is a floor, not the set.** Re-read the module doc of every file you touch. Story 5.4b's +review measured that a grep-based enumeration of falsified doc sites is not reproducible. + +### 8. Two register entries name this story as owner, and both are real + +Enumerate with `grep -n '5\.6' _bmad-output/implementation-artifacts/deferred-work.md` — measured +**4 lines** (`:1629`, `:1632`, `:1633`, `:1684`), in **two** entries. **Do not use `grep 'story 5\.6'`**: owner strings wrap across newlines +and that is exactly how story 5.4b came to claim eight register entries where ten existed. + +- **(R1) the float** — §1 above. Closed by this story's per-mille decision. +- **(R2) the self-pair** — §*code review of story-5.5*: *"`verdict_for_pair(a, a)` — the self-pair is + answered but undocumented… `decide_pair`'s doc tells a future candidate generator that the pair + 'arrives as an argument' without telling it that excluding `i == j` is the generator's + responsibility. **Owner: story 5.6**, which writes that generator and is the first place the + precondition has a holder."* ⇒ **the exclusion is this story's, and it belongs in the type** + (AC2), not in a comment. + +Two further entries mention the blocker without owning it — read them, do not close them: +`&str` rule-id constants allocate *"on a function a blocker will call O(pairs) times"* (owner: a +condition), and story 5.5's `L1Key` bare-tuple criticism (owner: 5.9). + +### 9. What already exists, so that nothing is re-created + +- `join(&[Observation]) -> BTreeMap>` (`l1.rs:165`) — **the blocker does not + call it**; see AC1. `L1Key = (L2DomainId, MacAddr)` (`:86`). +- `decide_pair(a, b) -> Decision` (`l1.rs:288`), `verdict_for_pair` is `pub(crate)` — **neither is + called by this story**. +- `Observation { obs_id, scope: Scope { l2_domain, vantage }, facts, observed_at, connector_id, raw }` + (`observation/mod.rs`). `ObsId` is `Ord` (the join's `BTreeSet` proves it). +- `read_jsonl(&Path) -> Result, FixtureError>` (`fixtures.rs:647`), + `read_traps(&Path) -> Result` (`:665`, and it already cross-checks that + every `obs_id` a trap names exists in its stream), `walk_trap_files` (`:834`, `#[cfg(test)]`). +- `Trap { id, replay, observations: Vec, reason, expect, family }` and + `Expectation::{MustMerge{rule}, MustNotMerge{rule}, MustAbstain{cause}}` with + `Expectation::column()` (`trap.rs:69-141`). +- **There is no `dormant` anywhere**: `grep -rn 'dormant\|Dormant' crates/ xtask/ --include=*.rs` + returns nothing. So [architecture.md:1205-1206] — *"the blocker excludes `dormant` from automatic + candidate generation"* — **cannot be implemented here**; it is registered with an owner (AC8), not + written from belief (D45). + +## Acceptance Criteria + +### AC1 — The blocker is an explicit component, and its universe is TOTAL by decision + +**Given** a slice of `Observation`s +**When** the blocker runs +**Then** it returns every unordered pair of observations with **distinct** `ObsId`s, deterministically +and reading nothing but its argument: no clock, no I/O, no SQL, no repository, and not `raw`. + +```rust +pub fn candidates(observations: &[Observation]) -> BTreeSet +``` + +- **Total by DECISION, not by omission — and the doc says so.** D13: at 300 hosts *"the blocker is + **not** there for performance (90k pairs is noise on a NAS i5)"* [architecture.md:1009]. Every + exclusion the component makes is named and tested; there are exactly two, both in AC2. **A + narrowing key (MAC, hostname, domain) is NOT added** — §5. +- **It does not call [`join`], `verdict_for_pair` or `decide_pair`.** Proposing is not judging, and a + blocker that consults a rule is the rule's echo. ⚠️ The relation between the two organs is still + pinned, in the other direction: **every pair sharing an L1 key is in the universe** (AC7), which is + a property of the total universe and would red the day someone narrows it. + 🔴 **The prohibition binds `candidates`, NOT the module.** AC7's test 9 imports `join` on purpose — + `use crate::identity::l1::join;` inside `blocking.rs`'s test module, same crate, `join` is already + `pub`, no circular import and no visibility change (measured: clean under both clippy forms). A + superset property is not checkable without the thing it is a superset of. Do not drop test 9, and + do not re-implement the join to avoid the import. +- **`BTreeSet`, so order-independence and de-duplication hold by CONSTRUCTION**, not by a `sort()` a + refactor can drop — the reasoning that made `join`'s value a `BTreeSet` and `l1`'s evidence sorted. +- **The count is exactly `n*(n-1)/2` where `n` is the number of DISTINCT `obs_id`s in the slice** — + not `observations.len()`. The two coincide until a duplicate id appears (AC2), and a test that + asserts from `len()` is green today and wrong that day. Assert it from the distinct count. +- ⚠️ **The caller supplies the universe.** The doc states the growth is quadratic in the slice it is + handed, that D13's 90k figure is for one poll of 300 hosts, and that **the day a caller hands it a + retention window instead of a poll, the universe must be narrowed — and the recall assertion is + what makes that narrowing safe.** Register it (AC8); do not build it. + +### AC2 — `CandidatePair` is unordered by construction, and refuses the self-pair + +**Given** two `ObsId`s +**When** a pair is built +**Then** `CandidatePair::new(a, b) == CandidatePair::new(b, a)`, and `CandidatePair::new(a, a)` is +`None`. + +- **The fields are PRIVATE**, ordered internally, with accessors. This is deliberate and it answers + a live register criticism of `L1Key` — *"a bare tuple alias creates no distinct type, carries no + invariant, hosts no impl"*. A pair whose order is enforced by its constructor cannot be built + wrong; a tuple can. (⚠️ That register entry is **owned by 5.9** and is about `L1Key`. Do not report + it closed — this story neither renames nor wraps `L1Key`.) +- **`Option`, not a panic and not a silent normalisation.** The self-pair is the register's R2, owned + here: `verdict_for_pair(a, a)` today answers `Decisive` with `evidence = [x, x]` — an observation + is trivially its own interface — and the generator is where the precondition finally has a holder. +- **The ordering carries NO meaning.** `ObsId` is a UUID; low/high is a construction device, not + "first seen". Say so, or a later reader will infer chronology from it. +- **Duplicate ids in the input slice are excluded by the same rule**: two entries carrying the same + `obs_id` produce no pair, because the rule is *distinct id*, not *distinct index*. + +### AC3 — The floor is an integer in per-mille, and the gate is untouched + +**Given** D13's `blocking_recall >= 0.999` and the `float-free` gate +**When** the floor is written +**Then** it is `999` per-mille as an integer constant, the assertion compares integers, and +`cargo xtask ci` reports **six green gates** walking **4** files under `identity/`. + +```rust +pub const BLOCKING_RECALL_FLOOR_PER_MILLE: u32 = 999; +pub fn blocking_recall_per_mille( + proposed: &BTreeSet, + required: &BTreeSet, +) -> Option +``` + +- **`Option`, and `None` for an empty `required` set.** A recall with no denominator is **undefined**, + not perfect: returning `1000` would let the floor pass over nothing, which is the reasoning the + fixture gate already carries (*"reporting 'nothing to check' on the deletion of the thing being + guarded is a guarantee the gate does not have"*) and which D13 states outright — *"without + blocking, abstention has no denominator."* **A test reds if this returns `Some`.** +- **Integer division truncates DOWN**, which is conservative for a floor; state it and pin it with a + case that truncates (e.g. 2 of 3 -> `666`). +- **`required` is a `BTreeSet`**, so a duplicated requirement cannot inflate the denominator. +- 🔴 **The constant is pinned by an INDEPENDENT literal.** A test asserting + `BLOCKING_RECALL_FLOOR_PER_MILLE == 999` written as a literal, citing D13's `0.999`. Without it, + weakening the floor reds nothing — the corpus scores 1000‰, so every assertion that reads the + constant moves with it. **This is story 5.5's M6 lesson, and it is worth quoting accurately**: + non-canonical rule ids left the *validation prototype* green **296/296**, because every expectation + was built from the constant it was checking; the tests 5.5 actually shipped restate the two ids as + independent literals and **red 10** on the same mutation (`5-5-l1-join-pure.md:836`). The prototype + is the warning, the shipped form is the remedy — and the floor takes the remedy. +- ⚠️ **No float anywhere, including in assertion messages and `#[doc = "…"]`.** §1's table is the + measured list. `//` and `///` comments ARE stripped, so D13 may be quoted verbatim in the doc. + +### AC4 — The truth set is the committed corpus, and the assertion carries the architecture's name + +**Given** the 10 committed `must-merge` traps +**When** `blocking_recall_above_999` runs, in `crates/opencmdb-bin/src/fixtures.rs`'s test module +**Then** for each trap, `candidates()` over the observations of the stream that trap names contains +the pair the trap names, and `blocking_recall_per_mille` over the whole truth set is `Some(1000)`, +which is `>= BLOCKING_RECALL_FLOOR_PER_MILLE`. + +- **The test name is `blocking_recall_above_999`** — the architecture's ratified name + [architecture.md:2954], not a paraphrase. +- 🔴 **PAIRS are formed per stream; the recall is computed over a UNION — and these are two separate + tests.** `candidates()` never sees two streams at once (a cross-stream pair is meaningless), but + `required` is 10 pairs drawn from 10 different streams, so the only call that typechecks passes + `proposed` = the union of the per-stream universes. Two obligations follow, and neither is + optional: + - **The per-trap containment assertion is what makes the union honest.** It proves each required + pair is in *its own* stream's universe, so the union can only add pairs, never explain a miss + away. Measured backstop: the **39** `obs_id`s across the **11** trap-named streams are all + distinct (§3), so no coincidental cross-stream hit exists today — but that is a property of the + committed bytes, not of the code, and the per-trap assertion is the one that would still hold if + it changed. + - **Put the containment assertion and the recall value in DIFFERENT tests** — AC7's own rule + (*"split an assertion that pins two properties into two tests"*) applied here, and it is + load-bearing: measured during validation, a single test panics on the first missing pair + (`docker-veth-must-merge: the blocker never proposes the pair this trap requires`) and **never + computes the recall at all**, so M1's 700‰ is never observed. Two tests, or M1's prediction is + unobservable by construction. +- **The counts are ASSERTED, not quoted**: the truth set has **10** pairs; **23** of the **24** traps + name a pair; **exactly one** names fewer than two observations (`example-must-abstain`). Assert + that last count — a skip that can grow silently is how a gate quietly stops testing. +- **A second, separate assertion — do not call it recall:** *every* trap pair (all 23, the + `must-not-merge` and `must-abstain` ones included) is in the universe. Story 5.7 must be able to + answer every trap, and a pair outside the universe can never be answered. D13's recall metric is + about the merge pairs; this is coverage. **Two names, two assertions.** +- **Nothing in `fixtures.rs` above `#[cfg(test)]` (`:729`) changes.** No new `pub` item in bin. +- ⚠️ **`CandidatePair::new` returns `Option`**; a trap naming the same id twice must fail the test + loudly (`.expect("a trap names two distinct observations")`), not vanish. + +### AC5 — What this story must NOT do + +**Given** the seams around it +**When** the work is done +**Then** none of the following has happened, and if any looked necessary it was reported as a +finding rather than absorbed: + +- no verdict is produced, no `Decision` is built, `decide`/`decide_pair`/`verdict_for_pair` are not + called (5.7); +- `score_corpus`, `run_trap`, `Tally`, `Report`, `SourceState`, `Outcome`, `VerdictVectorEntry` and + `trap_gate.rs` are untouched, and no `From for Outcome` appears (5.7); +- no `l2-*` rule, no `InterfaceId`/`EntityId`, no persistence, no `Default` impl anywhere; +- no structural reading is consumed (the U/L bit, the IANA prefixes, the I/G bit) — the blocker + proposes; the group-address gap stays registered with Epic 6 as owner; +- nothing under `fixtures/` is written. + +### AC6 — The doc states WHY the blocker exists, and it is semantics, not performance + +**Given** the blocker +**When** its module doc is read +**Then** it states, with D13's words and its citation: + +> *"If the candidate generator does not propose the pair, no downstream logic can ever group. **That +> is where false-splits are born silently, and nobody tests blockers.**"* [architecture.md:1004-1007] +> +> *"It is there for **SEMANTICS** — it defines the universe of plausible candidates, hence what +> 'ambiguous' MEANS. **Without blocking, abstention has no denominator.**"* [architecture.md:1009-1011] + +- and it names the 90k-pairs-at-300-hosts figure as D13's, with the conclusion D13 draws from it — + the blocker is **not** a performance device at this scale; +- and it carries §2's three-way distinction from D18's refused pairwise-recall gate, including the + honest sentence that **at this denominator the floor is zero-tolerance, not a tolerance**; +- and it says what the component does not do (AC5), in the weaker true form. + +⚠️ **Do not write an inventory of the epic in this doc.** Say what THIS module does and what THIS +test proves; let the register carry what is open. + +### AC7 — Tests, and prove-to-red with the predictions measured at contexting + +**Core (`blocking.rs`), synthetic inputs only, inline trailing `#[cfg(test)] mod tests` (D56b).** +⚠️ **The eleven items below are REQUIREMENTS, not a target `#[test]` count** — applying this AC's own +split rule to them yields more functions than eleven (a validation prototype that did so landed on +19 core + 3 bin, `309 -> 331`; that is an order of magnitude, **not a number to hit**). Writing +literally eleven functions is how a red gets lost. AC8's mechanical re-count is the authority. + +1. 0 and 1 observation -> **empty** universe (not an error). +2. `n = 4` distinct -> **6** pairs; the `n*(n-1)/2` count asserted for at least two values of n. +3. the same observation twice in the slice -> **0** pairs (distinct **id**, not index). +4. `CandidatePair::new(a, a)` -> `None`; `new(a, b) == new(b, a)`; and the **accessors** are + exercised on their own — `new(a, b)` and `new(b, a)` yield the same `low()` and the same `high()`, + with `low() < high()`. (House rule: test every function. The accessors are the only `pub` items + the eleven items would otherwise leave to incidental coverage.) +5. input-order independence: the same observations shuffled -> the same set. +6. the falsifiable half of purity: varying `raw`, `observed_at` and `connector_id` across otherwise + identical observations leaves the set identical (the clock/SQL half is unreachable from + `&[Observation]`, so no test can red on it — say that rather than claiming purity is tested). +7. 🔴 **two observations in DIFFERENT `l2_domain`s are still a candidate pair** (§5) — with an + assertion that the test data actually varied the domain, so it cannot degrade into a + single-domain test unnoticed (story 5.4b's measured hole, story 5.5's AC2 idiom). +8. an observation carrying **no** `Fact::Mac` is still a candidate. ⚠️ **Keep this test, drop the + reason you may have inherited**: the `hostname-absence` family does *not* depend on it — all six + of its observations carry a `Fact::Mac` (it encodes an absent/empty **hostname**). Measured: the + only MAC-less observation any trap names is `minimal.jsonl`'s `…-02`, judged by + `example-must-abstain`. The test is synthetic and stands on the blocker's totality, not on a + family. +9. **superset of the join**: for a set of observations, every pair inside a `join` group is in + `candidates`. The test module imports `join` (AC1) — that import is the point of the test. +10. recall arithmetic: full hit -> `Some(1000)`; 9 of 10 -> `Some(900)`; 2 of 3 -> `Some(666)` + (truncation); empty `required` -> **`None`**; a proposed pair not in `required` does not raise + the value above 1000. +11. the floor constant equals the literal `999`. + +**Bin (`fixtures.rs` test module), over the committed corpus:** AC4's two assertions plus the +one-trap-with-one-observation residue count. + +**Mutations — every red reported, not the first, and each classified compiler-carried vs +assertion-carried.** Predictions marked *(measured)* were computed against the committed corpus at +contexting; the others are estimates and a divergence is expected to be REPORTED, not hidden: + +- **(M1)** narrow the universe to pairs sharing an L1 key (an "exact-MAC blocker") — corpus recall + falls to **700‰** *(measured: 7 of the 10 `must-merge` pairs share a MAC)*, so + `blocking_recall_above_999` reds, and so do the universe-coverage test and several core tests. + ⚠️ **The 700‰ is only observable if AC4's two assertions are two tests.** Measured during + validation: with the containment assertion in the same test, it panics on `docker-veth-must-merge` + before any recall is computed, and the number you would report is the panic, not 700. +- **(M2)** narrow the universe to same-`l2_domain` pairs — 🔴 **the whole corpus stays GREEN** + *(measured: 23/23 trap pairs are same-scope)*; only the synthetic cross-domain test reds. **This is + the mutation that proves test 7 is load-bearing**, exactly as M6 was for story 5.5. +- **(M3)** admit the self-pair -> tests 3 and 4 red. ⚠️ **The `n*(n-1)/2` count test does NOT red** + *(measured)*, and predicting it is a trap: `candidates` only offers index pairs `i < j`, so on test + 2's *distinct* input the `Ordering::Equal` branch is unreachable and the count is unchanged. It + could only red on a fixture carrying a duplicate id — which test 2's own wording excludes. +- **(M4)** order the pair by argument order instead of canonically -> **the core tests red and the + corpus stays entirely GREEN** *(measured: `138 passed; 0 failed` on `opencmdb-bin`)*. 🔴 **The + corpus cannot see this mutation at all** — of the 23 trap pairs, **0** list their two ids in an + order the stream does not, and **0** out of ascending-UUID order (§3). Do not go looking for a + corpus red here; there is none to find, and reporting one would be the defect lesson 9 names. +- **(M5)** `blocking_recall_per_mille` returns `Some(1000)` for an empty `required` set -> test 10's + `None` case reds. +- **(M6)** weaken `BLOCKING_RECALL_FLOOR_PER_MILLE` to `900` -> **only** test 11 reds, because the + corpus scores 1000‰. That single red is the whole reason test 11 exists; report it as such. + +⚠️ **COMMIT the implementation before mutating.** `git checkout ` restores to `HEAD`; story +5.4b lost work to that twice inside one story. Verify each restore with `md5sum` against the +committed baseline **and** `git status`. + +⚠️ **Split an assertion that pins two properties into two tests** — story 5.5 measured a single test +reporting 1 red where the mutation broke 2, because `assert_eq!` aborts on the first mismatch. + +### AC8 — Register, docs, gates + +- **Annotate the register by requirement, appending only, citing entries by TITLE** (never by line + number — a check its own commit falsifies is worse than no check). Dispose of **R1** (the float) + and **R2** (the self-pair); do **not** close what belongs to others: the `L1Key` tuple (5.9), the + `&str` rule-id allocation (a condition), the group-address gap (Epic 6), `RuleId` -> enum (Epic 6). +- **New entries this story owes:** + - **F17/D17 dormancy** — *"the blocker excludes `dormant` from automatic candidate generation"* + [architecture.md:1205-1206] is **not implemented**, because no lifecycle state exists (measured: + zero occurrences of `dormant` in `crates/`). Owner: the lifecycle epic (FR40-42). + - **the quadratic universe** — the day a caller supplies a retention window rather than a poll, a + narrowing key is required; the recall assertion is what makes it safe. Owner: 5.9/5.7, whichever + first hands the blocker something other than one poll. + - **the floor's own arithmetic** — `>= 999‰` is zero-tolerance below 1000 required pairs and + becomes a real tolerance above it, where NFR4's *"any fraction is theatre"* applies. Owner: the + story that first grows the truth set past that size (Tier 2, Epic 11+). +- **Correct the five falsified doc sites of §7 in the same commit**, then re-read the module docs of + every file touched. ⚠️ `grep '5\.6'` gives you **nothing** — this list is by meaning. +- **Re-count mechanically after the last edit and state each number once.** Baseline measured at + contexting: **309 = 135 bin + 128 core + 46 xtask** (`cargo test --workspace --locked`). +- `sprint-status.yaml`, `CLAUDE.md`, `docs/project-context.md` — docs-current-before-push. + `epics.md` — **verify only**; an edit is a finding. +- **Full local gate before push**: `cargo fmt --all` · `cargo clippy --workspace --locked + --all-targets -- -D warnings` · **`cargo clippy --workspace --locked -- -D warnings`** (the CI + form, the only one that catches an import kept alive by a test module or an intra-doc link) · + `cargo test --workspace --locked` · `cargo xtask ci` printing **six** gates — `float-free` over + **4** files — plus `ℹ views-hash STALE` (exit 0 by design). `git status` under `fixtures/` empty. +- **Branch -> PR -> green CI. The story ends at status `review` with the PR open.** `done` is the + merge's business; the `code-review` workflow's step-6 default would set it early and has been + deliberately not followed on any Epic 5 story. + +## Tasks / Subtasks + +- [ ] **Task 1 — Enumerate the obligations before writing code** (AC8) + - [ ] `grep -n '5\.6' _bmad-output/implementation-artifacts/deferred-work.md` — **4 lines, two + entries** (R1 the float, R2 the self-pair). Do **not** use `grep 'story 5\.6'`. + - [ ] Read the two entries in full, plus the two that mention the blocker without owning it. + - [ ] ⚠️ **Check that R1 and R2 have citable TITLES before AC8 asks you to cite them by title** — + this was NOT verified at contexting, and AC8's no-line-numbers rule needs a title to exist. + If one of them has none, say so and cite the smallest stable anchor instead. + - [ ] ⚠️ `grep -rn '5\.6' crates/ xtask/ --include=*.rs` returns **one unrelated hit** + (`arp_ping.rs:272`). The doc worklist is §7's five sites, found by reading. + +- [ ] **Task 2 — Read before writing** (the project's primary named cause of review cycles) + - [ ] `crates/opencmdb-core/src/identity/l1.rs` in full — the join, `decide_pair`, and the module + doc you are about to falsify. + - [ ] `crates/opencmdb-core/src/identity/cascade.rs:280-300` — the `Decision` doc's *"no blocker"* + claim; and `identity/mod.rs`, `lib.rs`. + - [ ] `crates/opencmdb-bin/src/fixtures.rs` — `read_jsonl` (`:647`), `read_traps` (`:665`), + `walk_trap_files` (`:834`), and the test module's helper idiom (`:921`). + - [ ] `crates/opencmdb-core/src/trap.rs:69-150` — `Expectation`, `Trap`. + - [ ] `xtask/src/main.rs`'s `gate_float_free`, `line_has_float`, `float_literal_kind`, + `strip_line_comment` — you are writing under the directory it guards. §1 is the measured + behaviour; re-run the probe yourself if you doubt a line. + +- [ ] **Task 3 — The pair type** (AC2) + - [ ] New file `crates/opencmdb-core/src/identity/blocking.rs`; `pub mod blocking;` in + `identity/mod.rs`. + - [ ] `CandidatePair` with **private** fields, ordered by the constructor; + `new(ObsId, ObsId) -> Option` returning `None` on the self-pair; accessors; derive + `Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash`. + - [ ] Tests 3 and 4 of AC7 — **4 includes the accessors on their own**. + - [ ] Declare the test helpers you need here (`obs_id`, `l2`, `ts`, `mac`, `observation`), copied + from `l1.rs`'s spellings. They are private there; this duplication is sanctioned (Dev Notes). + +- [ ] **Task 4 — The generator** (AC1) + - [ ] `pub fn candidates(observations: &[Observation]) -> BTreeSet`. + - [ ] It calls neither `join` nor `decide_pair` nor `verdict_for_pair`. + - [ ] Tests 1, 2, 5, 6, 7, 8, 9 of AC7 — **write test 7 (two `l2_domain`s) FIRST**; it is the only + thing standing between a domain-blocked universe and green (§5). Test 9's module imports + `join` deliberately (AC1); test 2 asserts from the **distinct-id** count, not `len()`. + +- [ ] **Task 5 — The floor and the recall function** (AC3) + - [ ] `BLOCKING_RECALL_FLOOR_PER_MILLE: u32 = 999` and `blocking_recall_per_mille(...) -> Option`. + - [ ] Tests 10 and 11 — **11 with an independent literal**, not the constant. + - [ ] ⚠️ No float in any assertion message; no story number without a letter suffix inside a string + literal on a code line (§1, measured). + +- [ ] **Task 6 — The corpus assertion** (AC4) + - [ ] In `crates/opencmdb-bin/src/fixtures.rs`'s **test module only**: `blocking_recall_above_999` + over the 10 `must-merge` pairs, plus the universe-coverage assertion over all 23 pairs, plus + the *exactly one trap names fewer than two observations* residue count. + - [ ] 🔴 **The per-trap containment assertion and the recall value go in SEPARATE tests** (AC4). + In one test they cannot both be observed, and M1's 700‰ becomes unmeasurable. + - [ ] Pairs are formed **per stream** via `Trap::replay` + `read_jsonl`; `proposed` is the **union** + of those per-stream universes, which is what lets one recall call cover 10 streams (AC4). + Nothing above `#[cfg(test)]` changes. + - [ ] ⚠️ `trap_gate.rs` is not opened. + +- [ ] **Task 7 — Prove to red** (AC7) + - [ ] **COMMIT first.** Then M1–M6, every red reported and classified. + - [ ] Report divergence from the predictions explicitly — story 5.5's review caught a table that + tabulated six counts and commented only on the one that matched. + +- [ ] **Task 8 — Docs, register, gate, PR** (AC8) + - [ ] The five doc sites of §7; then re-read the module docs of every file touched. + - [ ] Register: dispose R1 and R2; append the three new entries; close nothing that is not yours. + - [ ] `lib.rs` — decide the re-export and **state the reason**: the recommendation is to follow + `join`'s precedent (reach it through `identity::blocking::…`, since `candidates` and + `recall` are very generic root-level names), and to correct `lib.rs:46-47`, whose stated + reason this story falsifies. + - [ ] `sprint-status.yaml`, `CLAUDE.md`, `docs/project-context.md`; `epics.md` verify-only. + - [ ] Full local gate, both clippy forms; six gates, `float-free` over **4** files. + - [ ] Branch `story-5.6-blocker-and-recall-assertion` (from `master`, **after PR #58 merges**) -> + PR -> green CI. **Ends at `review`, PR open.** + +## Dev Notes + +### The float gate, in one paragraph + +It walks `crates/opencmdb-core/src/identity/` **recursively**, strips `//` and `///` comments, and +reds on a word-bounded `f32`/`f64`/`f16`/`f128` or on a numeric literal that tokenises as a float +(one dot with an empty suffix, an exponent, or an `f32`/`f64` suffix). It **fails closed** if the +directory is missing or holds no `.rs` file. There is no `#[allow]`, no allowlist, no `#[cfg(test)]` +skip. **Green**: `999`, `1000`, `"192.168.0.1"` (three dots), `t.0.1`, `0xFF`, `1..32`, +`fn blocking_recall_above_999()`, a float quoted in a `///` comment. **Red**: `0.999`, `1.`, `1e-3`, +`0.85f64`, a one-dot decimal inside a **string literal on a code line** (`"story 5.6"`, +`"blocking_recall >= 0.999"`), a float in a `/* … */` block comment, a decimal in `#[doc = "…"]`. +All of §1's rows were measured on this tree at contexting. + +### Why `identity/blocking.rs`, and why the recall test is not there + +The architecture names the file — `blocking.rs # candidate generator + blocking_recall >= 0.999` +[architecture.md:3368] — so unlike story 5.5, this story does **not** choose its location. What it +must choose is where the *corpus-driven* assertion lives, and D47 settles it: core cannot read +files. §4 is the split. Do not try to route the corpus into core through a feature flag, a +`test-support` reader or an `include_str!` — an `include_str!` of a committed artefact would create a +second copy of a sha256-locked spec, which the fixture gate exists to prevent. + +### The 5.6 / 5.7 boundary + +5.6 **proposes**; 5.7 **answers** and 5.8 **buckets**. The seam is `score_corpus`'s `answers: +&BTreeMap` (`trap_gate.rs:223-226`), which **has no production caller at all**: +measured, all **10** call sites sit at `:410` and below, inside the `#[cfg(test)] mod tests` that +opens at `:385`, and every one of them passes an empty map. +Three things keep it out of reach here: the crate frontier, the deliberate absence of a +`Decision <-> Outcome` mapping, and `VerdictVectorEntry`'s deliberate uninhabitedness (pinned by two +`size_of::>() == 0` tests). **If you find yourself comparing a verdict to a trap, you have +squatted 5.7.** + +### Deliberate redundancy you must not collapse + +- `cascade.rs`'s `expected_conclusion` and `l1.rs`'s `expected_l1_conclusion` — D13's table restated + independently of the code under test. `l1.rs`'s `CORPUS_EXACT_MAC`/`CORPUS_DISTINCT_MAC` — the rule + ids as independent literals. **AC3's floor literal joins that family.** +- `Verdict::all()` / `IdentityAbstentionCause::all()` — the exhaustive-match witnesses. +- `keys_of`'s match over `Fact` is **exhaustive on purpose, no `_` arm**; if the blocker ever reads a + `Fact` (it should not), the same rule applies. +- `fixtures.rs`'s `expected()`; `score.rs`'s `Column::as_str()` vs `Expectation::column()`. + +### House rules that bind this story + +- **`opencmdb-core` is the domain.** No `anyhow`, `axum`, `sqlx`, `askama` (D47, gated). No clock: + `chrono` is built with `default-features = false`, so `Utc::now()` does not compile here. +- **Document every `pub` item** — struct, enum, **field**, **variant**, fn. ⚠️ `opencmdb-core` does + not carry `#![deny(missing_docs)]`; nothing checks you but the review. **A doc comment must be + TRUE**; prefer the weaker true sentence. +- **A comment asserting a checkable property gets checked.** Do not quote a number in a comment when + you can assert the property in a test. +- **`deferred-work.md` is append-only.** Never rewrite a bullet. +- **Test helpers: an idiom to COPY, not items to import.** Measured: `l1.rs`'s helpers are + `obs_id(n: u128)` (`:325`), `l2(n)` (`:329`), `ts()` (`:337`), `mac(last)` (`:343`) and + `observation(…)` (`:358`) — **all private, inside its own `#[cfg(test)] mod tests`**, so none is + reachable from `blocking.rs`. (`fn obs(n)` exists too, but in `trap.rs:417` and `cascade.rs:697` — + a *third* spelling, and not the one in the file this story tells you to read.) Re-declare what you + need in `blocking.rs`'s test module and **name them after `l1.rs`'s spellings**. This duplication + is sanctioned here: the alternative is a `pub(crate)` test-helper surface this story does not want, + and three copies already coexist. Do **not** report it as a DRY violation, and do not extract a + shared helper. +- **The two forms that do compile**, both already used in `l1.rs`, because the obvious ones do not: + `Uuid::from_u128(n)` (⚠️ `Uuid::new_v4()` does not compile — core builds `uuid` with + `features = ["v7","serde"]`, no `v4`) and + `DateTime::parse_from_rfc3339(…).unwrap().with_timezone(&Utc)` (⚠️ `Utc::now()` does not compile — + `chrono` is built `default-features = false, features = ["serde","std"]`, so no `clock`). AC7's + test 6 requires *varying* `observed_at`, so you need the second one. + +### Inherited lessons — read before writing a doc comment or a number + +Cumulative; **ten**, as of story 5.5's code review: + +1. **A check that its own commit falsifies is worse than no check.** Cite register entries by TITLE. +2. **A count in a doc is a claim.** Count mechanically, after the last edit. +3. **A red set is a count too.** Report every red a mutation fires, not the first. +4. **Classify your reds honestly.** A red that fires on `assert_eq!(1, 1)` is the compiler's. +5. **An inventory in a doc comment has no guard behind it.** +6. **Name the test behind every claim**, or write the weaker true sentence. +7. **A mutation pass needs a committed baseline to restore TO.** +8. **Do not quote a number in code — assert the property instead.** +9. 🔴 **The completion record is the most defect-prone artefact you will write.** Story 5.5's review + found 6 HIGH of which **three were about claims rather than code, and all three were the + implementer's — the fourth consecutive story with that defect.** Re-read your own record as if it + were someone else's, and reconcile every number you state against a command you can re-run. +10. **A green-case rationale that names a future story is a claim with an expiry date.** + `xtask/src/main.rs:1821` predicted story 5.5 would write IP literals under `identity/`; it wrote + none, and verifying is what found it false. ⚠️ **The message was corrected during 5.5**, so + grepping `:1821` today shows the replacement rationale, not the prediction — the lesson survives, + the evidence for it is in `deferred-work.md:1622-1628`. Do not write a new prediction about story + 5.7 into an assertion message — and if you must name a story in code, remember §1: **any** story + number without a letter suffix reds the gate, 5.7 included. + +### The self-referential test, one more time + +Story 5.5's validation agent measured it: renaming the two rule-id constants to non-canonical +spellings left its **prototype** suite green 296/296, because every expectation was built from the +constant it was checking. That is why 5.5 shipped the two ids as independent literals — after which +the same mutation reds **10**. **This story's equivalent is the floor**: every assertion that reads +`BLOCKING_RECALL_FLOOR_PER_MILLE` moves with it, so a weakened floor is invisible unless one test +pins the literal. AC7's M6 is the proof, and its expected red count is **one** — confirmed exactly +during validation. + +### What this touches, and what it must not break + +| file | NEW / UPDATE | what | +|---|---|---| +| `crates/opencmdb-core/src/identity/blocking.rs` | **NEW** | `CandidatePair`, `candidates`, `blocking_recall_per_mille`, the floor, and its tests. **Subject to `float-free`.** | +| `crates/opencmdb-core/src/identity/mod.rs` | UPDATE | `pub mod blocking;` + the module doc at `:15-16` (*"There is still no candidate pair generator"*) | +| `crates/opencmdb-core/src/identity/l1.rs` | UPDATE (**docs only**) | `:16-18` and `:287` — the ownership clauses only; the join, the rules and `decide_pair` are **untouched** | +| `crates/opencmdb-core/src/identity/cascade.rs` | UPDATE (**docs only**) | `:292-293` — *"There is still no blocker"* | +| `crates/opencmdb-core/src/lib.rs` | UPDATE | the re-export decision + `:46-47`'s stated reason, which this story falsifies | +| `crates/opencmdb-bin/src/fixtures.rs` | UPDATE (**tests only**) | AC4's assertions, below `#[cfg(test)]` (`:729`). Nothing above it moves | +| `trap_gate.rs`, `score.rs`, `trap.rs`, `observation/mod.rs` | **LEAVE ALONE** | 5.7's seam / no change needed | +| `fixtures/**` | **LEAVE ALONE, READ ONLY** | locked spec; the gate checks both directions | +| `deferred-work.md`, `sprint-status.yaml`, `CLAUDE.md`, `docs/project-context.md` | UPDATE | annotations and docs-current | +| `epics.md`, `architecture.md`, `architecture-views.md` | **NEVER** | verify-only / issue #54 / issue #50 | + +### What STOP means, procedurally + +If a step appears to require editing `fixtures/`, `architecture.md` or `epics.md`; or calling +`decide_pair`; or filling `score_corpus`'s `answers`; or inhabiting `VerdictVectorEntry`; or +weakening the `float-free` gate; or "fixing" L1's `Disqualifying` on the `multi-nic` pair — **stop +and report it as a finding.** Do not absorb it. Every one of those is another story's, and three of +them are load-bearing claims in files this story does not own. + +### Project Structure Notes + +`crates/opencmdb-core/src/identity/` today: `mod.rs`, `cascade.rs`, `l1.rs`. The new file is the +**fourth**, and the `float-free` gate's file count moves from 3 to 4 — a number worth checking in the +gate's own output rather than asserting in prose. + +D54: **the folder is not the frontier — visibility is.** `identity/mod.rs:23-26` currently says +*"nothing yet is"* restricted to this subtree; `verdict_for_pair` became `pub(crate)` in story 5.5, +so **verify that sentence rather than trust it**, and correct it if this story adds a +`pub(in crate::identity)` item. + +### References + +- **D13** the blocker [architecture.md:1004-1011] · decision `:931-932` · float refusal `:956-958` · + the six-row table `:967-974` · **the milli-units corollary `:988-993`** · level split `:984-986` · + structural facts `:995-1002` +- **D12** one engine instantiated twice, *"two rule sets and two blocking keys"* `:917` · a MAC + identifies an INTERFACE `:884` · the L1/L2 table `:888-893` · the device is non-negotiable `:919-928` +- **D14** `AMBIGUOUS` is a LINK, not an absence `:1031-1034` +- **D17** `dormant` excluded from automatic candidate generation `:1205-1206` (**not implementable + today**) · no `presence` level `:1171-1173` +- **D18** the gate is Tier 1, binary, at the device level `:1224-1226` · the three columns + `:1230-1234` · honesty vs cowardice `:1241-1244` · **Tier 2 blocks nothing, and why a loose + threshold is decoration `:1246-1253`** +- **D19** the fixture asserts the RULE `:1307-1310` · ATDD order `:1341-1346` +- **D47** frontier `:2584` · **D56b** identity tests inline, no database `:3302-3306` +- **Ratified test naming**, incl. `blocking_recall_above_999` `:2954` +- **The target source tree**, incl. `identity/blocking.rs` `:3368` +- **NFR4** — read from `prd.md:1179-1207` (it runs to `:1207`; `NFR5` starts at `:1208`), **not** + architecture.md's stale F-tables. *"any fraction is theatre"* is at `:1182`, the **device** level at + `:1183`, and the closing oracle sentence at `:1205-1207` +- Corpus: `fixtures/scenario/traps/*.toml` (24 traps), `fixtures/scenario/replay/*.jsonl` +- GitHub issues: **#54** (D13's table is short one row), **#50** (`architecture-views.md` stale) + +## Dev Agent Record + +### Agent Model Used + +### Debug Log References + +### Completion Notes List + +### File List + +## Change Log + +| Date | Change | +|---|---| +| 2026-08-01 | **Validation pass, two fresh-context agents (fact-check + gap-hunt), MANDATORY per Guy's Epic 4 retrospective decision.** Coverage: **128 factual claims measured (120 true, 6 false, 2 unverifiable)** and the story **implemented end to end** in an isolated worktree — it compiled first try, reached **331 tests (309 + 22)**, six green gates with `float-free` over **4** files and `file-size` unmoved, and all six mutations were run. **16 findings applied: 3 HIGH, 3 MEDIUM, 4 LOW, plus 6 citation corrections.** As on story 5.5, **every HIGH came from the agent that COMPILED the story, none from the agent that checked its claims** — the citations, greps, corpus counts, register enumeration, the 309 baseline and all eight rows of the float probe reproduced exactly. **The three HIGH:** **(H1)** **M4's second half is FALSE by measurement** — *"the corpus test reds wherever a trap lists its two ids in an order the stream does not"*: **0 of 23** trap pairs do, and 0 are out of ascending-UUID order either, so `opencmdb-bin` stays **138/138 green** under M4; the prediction is now corpus-invisible by statement, and the measurement is a new row in §3. **(H2)** **AC4 was self-contradictory** — *"per stream, never across streams"* cannot coexist with *"recall over the whole truth set"* except through a **union** of the per-stream universes, which the story never named; the union is only honest because the per-trap containment assertion proves each required pair is in its OWN stream (backstopped by a new measurement: **39** distinct `obs_id`s across the **11** trap-named streams, zero collision). **(H3)** **AC1 forbade calling `join` while AC7 test 9 requires it** — the prohibition binds `candidates`, not the module; the test module imports `join` deliberately, which is the only way a superset property is checkable. **The three MEDIUM:** M3's predicted red set named a test that **cannot** red (`candidates` offers only `i < j`, so the `Equal` branch is unreachable on distinct input); **M1's 700‰ is unobservable unless AC4's two assertions are two tests** — measured, a single test panics on `docker-veth-must-merge` before any recall is computed, so AC7's own split rule is now imposed on AC4; and **`fn obs(n: u128)` is not importable** — `l1.rs` spells its five helpers `obs_id`/`l2`/`ts`/`mac`/`observation`, all private, so the copy is sanctioned explicitly rather than left to collide with DRY. Also corrected: `n*(n-1)/2` now says **n = distinct ids**; *"eleven core tests"* is restated as eleven **requirements**; the accessors got a test; `Uuid::from_u128` and `parse_from_rfc3339` are named as the forms that compile; **test 8's justification was refuted** (all six `hostname-absence` observations carry a MAC — the family encodes an absent *hostname*); the **296/296** figure was 5.5's *validation prototype*, whose shipped tests red **10**; *"one per family"* omitted `example-must-merge`, which has no family; `xtask:1821`'s prediction **was corrected during 5.5**; `score_corpus` has **no production caller** (all 10 sites are below `:385`); NFR4 runs to `prd.md:1207`; D18's quote to `:1253`. Two rows added to §1's float table, both measured: the citations AC6 demands are **green even on a code line**, and **`"story 5.7"` reds too** — the rule is any story number without a letter suffix. Two things left explicitly unmeasured rather than assumed: `"v0.1"` (one dot), and whether R1/R2 carry citable TITLES, now a task. The prototype implementation was **discarded** (Guy's call) — dev-story rewrites from the corrected story. | +| 2026-08-01 | Story contexted against `master` at `0ebd50f` (5.5 merged; bookkeeping PR #58 still OPEN). **Four findings changed the story rather than decorating it, each measured at contexting.** (1) **The float was resolved by measurement, not by reading the gate**: a probe file under `identity/` run through `cargo xtask ci` reported exactly three offenders — `0.999`, the epic's own assertion text quoted in an assertion message, and `"story 5.6"` in a string on a code line — while `999`, `1000`, `blocking_recall_above_999` and a `///`-quoted `0.999` were green. The floor becomes an INTEGER in per-mille, on D13's own milli-units corollary and on the architecture's ratified test name, which already contains no float. (2) **D18 refuses a pairwise-recall gate by name** (*"a gate that cannot fall is decoration"*), so the story owes the distinction — different subject, different venue, and the honest arithmetic that `>= 999‰` at a 10-pair denominator IS zero tolerance. (3) **The truth set is the committed corpus**: 24 traps, 23 name a pair, 10 `must-merge`, of which **7 share a MAC and 3 do not** — so an exact-MAC blocker scores **700‰** and the recall assertion has a real red. Because core cannot read files (D47), the corpus assertion lives in `fixtures.rs`'s test module and the blocker in `identity/blocking.rs`, the file the architecture already names. (4) **The wrong blocker that passes the whole corpus is the same-`l2_domain` one** — all 23 trap pairs are same-scope, measured — so only a synthetic cross-domain test stands between it and green; it is required to be written first. Also measured: `grep '5\.6'` over the code returns **zero** relevant hits, so the doc worklist is by meaning (five sites, listed); there is no `dormant` anywhere, so D17's blocker clause cannot be implemented and is registered instead; and the baseline is **309 tests = 135 + 128 + 46**. | diff --git a/_bmad-output/implementation-artifacts/sprint-status.yaml b/_bmad-output/implementation-artifacts/sprint-status.yaml index 65665c2..03a12ad 100644 --- a/_bmad-output/implementation-artifacts/sprint-status.yaml +++ b/_bmad-output/implementation-artifacts/sprint-status.yaml @@ -1,5 +1,5 @@ # generated: 2026-07-21 -# last_updated: 2026-07-24 +# last_updated: 2026-08-01 # project: opencmdb # project_key: NOKEY # tracking_system: file-system @@ -50,7 +50,7 @@ # 'optional' and will not be back-filled. generated: 2026-07-21 -last_updated: 2026-07-31 # 5.5 DONE — PR #57 squash-merged as `0ebd50f`. 309 tests on master. +last_updated: 2026-08-01 # 5.6 contexted AND validated (2 fresh agents, 16 findings applied) -> `ready-for-dev`. master at `0ebd50f`, 309 tests. # 6 mutations, all reds assertion-carried; M6 (added by the validation) # reds 10 where the agent's self-referential version red 0. # Code-reviewed (3 layers) and patched: 309 tests. NEXT = merge. @@ -973,7 +973,83 @@ development_status: # deliberately (move the assertion out of the guarded subtree, or express the floor without a # float literal), never a surprise it discovers on its first red CI. 5-5-l1-join-pure: done - 5-6-blocker-and-recall-assertion: backlog + # + # 2026-08-01 — story 5.6 CONTEXTED -> `ready-for-dev`. Four findings, each MEASURED at contexting + # rather than reasoned: + # · THE FLOAT IS SETTLED BY MEASUREMENT. A probe file under `identity/` run through + # `cargo xtask ci` reported exactly three offenders — `0.999`, the epic's own assertion text + # quoted inside an ASSERTION MESSAGE, and `"story 5.6"` in a string on a code line — while + # `999`, `1000`, `fn blocking_recall_above_999()` and a `///`-quoted `0.999` were GREEN. + # The floor becomes an INTEGER IN PER-MILLE (`>= 999`), on D13's own milli-units corollary + # [architecture.md:988-993] and on the architecture's ratified test name + # `blocking_recall_above_999` [:2954], which already carries no float. The gate is NOT + # weakened, skipped or given an escape hatch. + # · D18 REFUSES A PAIRWISE-RECALL GATE BY NAME (*"a gate that cannot fall is decoration"*, + # [:1246-1252]), so the story owes the distinction: different subject (generator input + # coverage, not engine output), different venue (a unit test, not a release gate) — and the + # honest arithmetic that at a 10-pair denominator `>= 999‰` IS zero tolerance, and only + # becomes a real tolerance above 1000 required pairs, where NFR4's "any fraction is theatre" + # would bite. + # · THE TRUTH SET IS THE COMMITTED CORPUS, not synthetic pairs: 24 traps, 23 name a pair, 10 + # `must-merge` — of which **7 share a MAC and 3 do not** (the three whose rule is `l2-*`). + # So an exact-MAC blocker scores **700‰** and the assertion has a real red. Because core + # cannot read files (D47), the corpus assertion lives in `fixtures.rs`'s TEST MODULE and the + # blocker in `identity/blocking.rs` — the file the architecture already names [:3368]. + # · THE WRONG BLOCKER THAT PASSES THE WHOLE CORPUS is the same-`l2_domain` one: all 23 trap + # pairs are same-scope, measured. Only a SYNTHETIC cross-domain test stands between it and + # green, and the story requires that test to be written FIRST. + # Also measured: `grep '5\.6'` over `crates/`+`xtask/` returns ONE unrelated hit, so the doc + # worklist is by MEANING (five sites, listed in the story) — the grep idiom story 5.5 used yields + # an EMPTY worklist here. There is no `dormant` anywhere in the tree, so D17's *"the blocker + # excludes dormant from automatic candidate generation"* [:1205-1206] cannot be implemented and is + # registered instead of written from belief (D45). Baseline: 309 tests = 135 bin + 128 core + 46 + # xtask, re-measured with `cargo test --workspace --locked`. + # ⚠️ Branch from `master` only AFTER PR #58 (5.5 bookkeeping) merges — it is still OPEN. + # + # 2026-08-01 — story 5.6 VALIDATED (two fresh-context agents, MANDATORY per Guy's Epic 4 + # retrospective decision). 128 factual claims measured (120 true, 6 false, 2 unverifiable) and the + # story IMPLEMENTED END TO END in an isolated worktree: it compiled first try, 331 tests + # (309 + 22), six green gates with `float-free` over 4 files, `file-size` unmoved, all six + # mutations run. 16 findings applied (3 HIGH, 3 MEDIUM, 4 LOW + 6 citation fixes). + # 🔑 AGAIN: EVERY HIGH CAME FROM THE AGENT THAT COMPILED THE STORY, none from the one that checked + # its claims — the citations, greps, corpus counts, register enumeration, the 309 baseline and all + # eight rows of the float probe reproduced EXACTLY. What breaks is what a story PRESCRIBES. + # · (H1) M4's second half is FALSE by measurement: 0 of 23 trap pairs list their two ids in an + # order the stream does not (0 out of ascending-UUID order either), so `opencmdb-bin` stays + # 138/138 GREEN under M4. The corpus cannot see that mutation at all. New row in the story's + # corpus table; the prediction now says so, so no dev hunts a red that cannot exist. + # · (H2) AC4 was SELF-CONTRADICTORY — "per stream, never across streams" cannot coexist with + # "recall over the whole truth set" except through a UNION of the per-stream universes, which + # the story never named. The union is honest only because the per-trap containment assertion + # proves each required pair is in its OWN stream; backstopped by a new measurement — 39 + # distinct obs_ids across the 11 trap-named streams, ZERO collision. + # · (H3) AC1 forbade calling `join` while AC7's test 9 requires it. The prohibition binds + # `candidates`, NOT the module: the test module imports `join` deliberately, and that import is + # the only way a superset property is checkable. + # · MEDIUM: M3 predicted a red on a test that CANNOT red (`candidates` offers only i usize { mod tests { use super::*; use opencmdb_core::gap::AbstentionCause; + use opencmdb_core::identity::blocking::{ + BLOCKING_RECALL_FLOOR_PER_MILLE, CandidatePair, blocking_recall_per_mille, candidates, + }; use opencmdb_core::observation::{ ConnectorId, Fact, HostnameSource, L2DomainId, MacAddr, ObsId, Scope, Timestamp, VantageId, }; @@ -4471,4 +4474,197 @@ expect = { must-abstain = { cause = "NoObservedValue" } } "the four last_seen epochs are the four authored instants" ); } + + // ---- The blocker, measured against the committed corpus (story 5.6) ---- + // + // The blocker itself lives in `opencmdb_core::identity::blocking`. These assertions live HERE + // and not beside it because D47 forbids the domain crate to touch the filesystem, and the truth + // set is the corpus: Epic 4 froze it BEFORE the engine on purpose — *"a metric written after + // the engine is bent to fit the engine"* — so a recall floor measured against a truth set the + // engine's own author writes today would be the mirror D13 refuses for weights, applied to + // blocking. + // + // Nothing above `#[cfg(test)]` changes, and no new `pub` item appears in this crate. + + /// What the committed traps say about pairs, read once and reused by the assertions below. + struct CorpusPairs { + /// How many traps the corpus holds, across every trap file. + traps: usize, + /// The traps that name exactly two observations, as `(trap id, replay, the pair)`. + pairs: Vec<(String, String, CandidatePair)>, + /// The traps that name fewer than two observations, by id. + without_a_pair: Vec, + /// The traps that name more than two observations, by id. + beyond_a_pair: Vec, + /// The `must-merge` pairs — the recall truth set. + required: std::collections::BTreeSet, + /// The candidate universe of each replay stream a trap names, keyed by the stream. + universes: std::collections::BTreeMap>, + } + + impl CorpusPairs { + /// The union of the per-stream universes. + /// + /// A cross-stream pair is meaningless — `candidates` never sees two streams at once — but + /// `required` draws its pairs from ten different streams, so the only recall call that + /// typechecks compares against this union. What makes the union HONEST is + /// `the_blocker_proposes_every_required_pair_within_its_own_stream`: it proves each required + /// pair sits in its OWN stream's universe, so the union can only add pairs and can never + /// explain a miss away. + fn union(&self) -> std::collections::BTreeSet { + self.universes.values().flatten().copied().collect() + } + } + + /// Walk every committed trap file, and build the pairs and universes it implies. + fn corpus_pairs() -> CorpusPairs { + let mut found = CorpusPairs { + traps: 0, + pairs: Vec::new(), + without_a_pair: Vec::new(), + beyond_a_pair: Vec::new(), + required: std::collections::BTreeSet::new(), + universes: std::collections::BTreeMap::new(), + }; + walk_trap_files(&mut |path| { + let file = read_traps(path) + .unwrap_or_else(|e| panic!("corpus trap file {} is invalid: {e}", path.display())); + for trap in &file.trap { + found.traps += 1; + if !found.universes.contains_key(&trap.replay) { + let stream = read_jsonl(&fixture_path(&trap.replay).unwrap()) + .unwrap_or_else(|e| panic!("reading {}: {e}", trap.replay)); + found + .universes + .insert(trap.replay.clone(), candidates(&stream)); + } + match trap.observations.as_slice() { + [a, b] => { + // `CandidatePair::new` returns `Option`, and a trap naming one id twice + // must fail LOUDLY rather than vanish out of the truth set. + let pair = CandidatePair::new(*a, *b) + .expect("a trap names two distinct observations"); + found + .pairs + .push((trap.id.0.clone(), trap.replay.clone(), pair)); + if matches!(trap.expect, Expectation::MustMerge { .. }) { + found.required.insert(pair); + } + } + fewer if fewer.len() < 2 => found.without_a_pair.push(trap.id.0.clone()), + _ => found.beyond_a_pair.push(trap.id.0.clone()), + } + } + }); + found + } + + /// The architecture's own ratified test name [architecture.md:2954] — D13's + /// `blocking_recall >= 0.999`, expressed in the milli-units D13's corollary demands. + /// + /// It computes ONLY the recall. The per-trap containment assertion is a separate test on + /// purpose: with both in one function, a missing pair panics inside the loop and the recall is + /// never computed at all, so the value a narrowed blocker would score could not be observed. + #[test] + fn blocking_recall_above_999() { + let corpus = corpus_pairs(); + + assert_eq!( + corpus.required.len(), + 10, + "the truth set is the committed `must-merge` traps; a denominator that shrinks in \ + silence is a gate that quietly stops testing" + ); + + let recall = blocking_recall_per_mille(&corpus.union(), &corpus.required) + .expect("the truth set is not empty, so the recall is defined"); + + assert_eq!( + recall, 1000, + "the blocker proposes every pair the corpus requires" + ); + assert!( + recall >= BLOCKING_RECALL_FLOOR_PER_MILLE, + "recall is {recall} per-mille, below the floor of {BLOCKING_RECALL_FLOOR_PER_MILLE}" + ); + } + + /// Each required pair is in the universe of ITS OWN stream — what makes the union above honest. + /// + /// Backstop, measured on the committed bytes rather than guaranteed by code: the `obs_id`s + /// across the streams the traps name are all distinct, so no coincidental cross-stream hit + /// exists today. That is a property of the corpus and could change; this assertion is the one + /// that would still hold if it did. + #[test] + fn the_blocker_proposes_every_required_pair_within_its_own_stream() { + let corpus = corpus_pairs(); + let mut checked = 0usize; + for (id, replay, pair) in &corpus.pairs { + if !corpus.required.contains(pair) { + continue; + } + assert!( + corpus.universes[replay].contains(pair), + "{id}: the blocker never proposes the pair this trap requires, in {replay}" + ); + checked += 1; + } + assert_eq!( + checked, 10, + "every `must-merge` trap must have been checked, or this test passes by walking past \ + the traps it exists to cover" + ); + } + + /// Coverage, NOT recall — and the two are deliberately not given one name. + /// + /// D13's recall metric is about the merge pairs. This asserts something wider and weaker: every + /// trap pair, `must-not-merge` and `must-abstain` included, is in the universe. A pair outside + /// the universe can never be answered by anything, so the trap runner could never score it. + #[test] + fn every_trap_pair_is_in_the_universe() { + let corpus = corpus_pairs(); + + assert_eq!( + corpus.pairs.len(), + 23, + "23 of the committed traps name a pair; a count that drifts in silence is a scan that \ + has stopped covering the corpus" + ); + for (id, replay, pair) in &corpus.pairs { + assert!( + corpus.universes[replay].contains(pair), + "{id}: the pair it judges is outside the candidate universe of {replay}, so no \ + rule could ever be asked about it" + ); + } + } + + /// Exactly one committed trap names fewer than two observations, and none names more. + /// + /// The residue is asserted rather than quoted: the two pair-based tests above skip these traps, + /// and a skip that can grow silently is how a gate quietly stops testing. + #[test] + fn exactly_one_trap_names_fewer_than_two_observations() { + let corpus = corpus_pairs(); + + assert_eq!(corpus.traps, 24, "the corpus holds 24 traps"); + assert_eq!( + corpus.without_a_pair, + vec!["example-must-abstain".to_string()], + "one trap judges a single observation, and it is named here so the residue cannot grow \ + unnoticed" + ); + assert!( + corpus.beyond_a_pair.is_empty(), + "no trap names more than two observations today; one that did would need a decision \ + about what pair it requires, not a silent skip: {:?}", + corpus.beyond_a_pair + ); + assert_eq!( + corpus.traps, + corpus.pairs.len() + corpus.without_a_pair.len() + corpus.beyond_a_pair.len(), + "every trap lands in exactly one of the three buckets" + ); + } } diff --git a/crates/opencmdb-core/src/identity/blocking.rs b/crates/opencmdb-core/src/identity/blocking.rs new file mode 100644 index 0000000..e87761a --- /dev/null +++ b/crates/opencmdb-core/src/identity/blocking.rs @@ -0,0 +1,608 @@ +//! The blocker — candidate generation, and the recall floor that proves it hides nothing. +//! +//! # Why a blocker exists at all, and it is not performance +//! +//! D13 states the failure this component prevents: *"if the candidate generator does not propose +//! the pair, no downstream logic can ever group. That is where false-splits are born silently, and +//! **nobody tests blockers**"* [architecture.md:1004-1007]. And it names what the component is for: +//! *"It is there for **SEMANTICS** — it defines the universe of plausible candidates, hence what +//! 'ambiguous' MEANS. **Without blocking, abstention has no denominator.**"* +//! [architecture.md:1009-1011]. +//! +//! D13 also disposes of the performance argument with its own arithmetic: at the reference scale of +//! 300 hosts the universe is **90k pairs**, and *"the blocker is **not** there for performance (90k +//! pairs is noise on a NAS i5)"* [architecture.md:1009]. So [`candidates`] is TOTAL by decision: it +//! proposes every unordered pair of distinct observations, and the two exclusions it makes — the +//! self-pair and a repeated `obs_id` — are one rule, named and tested, not a narrowing. +//! +//! ⚠️ **A narrowing key is deliberately absent.** Blocking on the MAC, the hostname or the +//! `l2_domain` would each pass the whole committed corpus and each build a false split into the +//! universe: a device's interfaces are not confined to one L2 domain — a router, a firewall or a +//! dual-homed server has NICs in several VLANs — and D12 makes the device the level where the +//! product keeps its promise [architecture.md:919-928]. +//! +//! # The floor is an INTEGER in per-mille, and that is D13's own corollary +//! +//! D13 gives the assertion as `blocking_recall >= 0.999` and, three paragraphs earlier, refuses the +//! type it is written in: *"`confidence` is an **INTEGER in milli-units (0..1000)**, never +//! `REAL`/`DOUBLE` — a threshold at 0.85 compared as a float on two engines = two different identity +//! decisions for the same input"* [architecture.md:988-993]. So the floor here is +//! [`BLOCKING_RECALL_FLOOR_PER_MILLE`], an integer, and [`blocking_recall_per_mille`] compares +//! integers. `cargo xtask ci`'s `float-free` gate holds that mechanically over this whole directory. +//! +//! # This is NOT the recall gate D18 refuses by name, and the difference is three-fold +//! +//! D18 puts **pairwise recall** in Tier 2, *"published per release with confidence intervals, +//! trended — blocking nothing"*, and says why: *"false-split is benign — so why would it block a +//! release? A loose threshold on a benign defect is a gate that can never fall, and a gate that +//! cannot fall is decoration"* [architecture.md:1246-1253]. Three things separate that from what is +//! asserted here: +//! +//! - **Different subject.** D18 measures the ENGINE'S OUTPUT — did it group what should group. This +//! measures the CANDIDATE GENERATOR'S INPUT COVERAGE — did the pair even reach a rule. +//! - **Different venue.** D18 refuses a release gate over bulk statistics. This is an assertion in a +//! unit test over the frozen corpus, which is where D13 puts it: *"a dedicated assertion: +//! `blocking_recall >= 0.999`, measured in unit tests, before the scoring exists."* +//! - **Different arithmetic, and this is the honest half.** At the committed corpus's denominator +//! the floor is **not** a tolerance: with 10 required pairs, one miss gives 900 per-mille and the +//! floor reds. `>= 999` per-mille **IS zero-tolerance at this scale**, which is the binary form +//! NFR4 demands. It becomes a real tolerance only if the required set ever exceeds 1000 pairs — +//! and on that day NFR4's *"any fraction is theatre"* bites and the floor must be revisited rather +//! than inherited. The per-mille dress must not be read as a statistical tolerance the corpus +//! cannot support. +//! +//! ⚠️ **Nothing here advances NFR4.** NFR4 is at the DEVICE level; this adds no truth-table column +//! and gates no release. +//! +//! # What this module does not do +//! +//! It produces no verdict and builds no [`crate::identity::cascade::Decision`]: [`candidates`] calls +//! neither [`crate::identity::l1::join`] nor [`crate::identity::l1::decide_pair`]. Proposing is not +//! judging, and a blocker that consults a rule is that rule's echo — it would exclude exactly the +//! pairs the rule refuses, which is the false split this component exists to prevent. It consumes no +//! structural reading of a MAC (the U/L bit, the IANA prefixes, the I/G bit) and reads no +//! [`crate::observation::Fact`] at all. It writes nothing and reads nothing but its argument. +//! +//! The relation to the join is still pinned, in the other direction only: every pair sharing an L1 +//! key is in the universe. That is a property of a TOTAL universe, and it is a superset property — +//! checkable only by importing the join, which this module's tests do on purpose. +//! +//! # The universe is quadratic in the slice the CALLER supplies +//! +//! D13's 90k figure is one poll of 300 hosts. The day a caller hands this function a retention +//! window instead of a single poll, the universe must be narrowed — and the recall assertion below +//! is precisely what would make that narrowing safe rather than silent. That day is registered, not +//! built for here. + +use std::cmp::Ordering; +use std::collections::BTreeSet; + +use crate::observation::{ObsId, Observation}; + +/// One unordered candidate pair — two DISTINCT observations the blocker proposes to a rule. +/// +/// # Unordered by construction +/// +/// The two fields are private and ordered by [`Self::new`], so `new(a, b) == new(b, a)` holds +/// because the two calls build the same value, not because a caller remembered to normalise. A bare +/// tuple would carry no invariant and host no impl; this type carries the one property every +/// consumer depends on. +/// +/// ⚠️ **The ordering carries NO meaning.** [`ObsId`] is a UUID, so low/high is a construction device +/// and nothing else — it is not "first seen", not chronology, not precedence. Reading an order into +/// it would be reading identity out of a byte comparison. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct CandidatePair { + /// The smaller of the two ids, by [`ObsId`]'s own ordering. + low: ObsId, + /// The larger of the two ids, by [`ObsId`]'s own ordering. + high: ObsId, +} + +impl CandidatePair { + /// Build the pair of two observation ids, or `None` when they are the same id. + /// + /// # Why the self-pair is refused here + /// + /// [`crate::identity::l1::decide_pair`] answers `(a, a)` with a merge — an observation is + /// trivially its own interface — and its doc says the pair *"arrives as an argument"* without + /// naming who guarantees `a != b`. This constructor is that holder: the generator is the first + /// place in the engine where the precondition has an owner. + /// + /// `None` rather than a panic or a silent normalisation: refusing the pair is an ordinary + /// outcome of asking for it, not a caller's bug, and a normalisation would let a self-pair enter + /// the universe under another shape. + /// + /// # Returns + /// + /// `Some` with the two ids in canonical order, or `None` if `a == b`. + pub fn new(a: ObsId, b: ObsId) -> Option { + match a.cmp(&b) { + Ordering::Less => Some(Self { low: a, high: b }), + Ordering::Greater => Some(Self { low: b, high: a }), + Ordering::Equal => None, + } + } + + /// The smaller of the two ids. See the type's doc: the order is a construction device. + pub fn low(&self) -> ObsId { + self.low + } + + /// The larger of the two ids. See the type's doc: the order is a construction device. + pub fn high(&self) -> ObsId { + self.high + } +} + +/// The universe of candidate pairs over a slice of observations — every unordered pair of +/// DISTINCT observation ids. +/// +/// It reads **nothing** but its argument: no clock, no I/O, no SQL, no repository, and not `raw`. +/// It reads no [`crate::observation::Fact`] either — only [`Observation::obs_id`]. +/// +/// # Total by decision +/// +/// See this module's doc: at the reference scale the universe is noise, and every exclusion a +/// blocker makes is a false split it can never be talked out of. There are exactly two exclusions +/// and they are one rule — **distinct id**, not distinct index — so the self-pair is out and two +/// entries repeating one `obs_id` produce no pair. +/// +/// # The count +/// +/// `n * (n - 1) / 2` where `n` is the number of DISTINCT `obs_id`s in the slice — not +/// `observations.len()`. The two coincide until a duplicate id appears. +/// +/// # Why a [`BTreeSet`] +/// +/// Order-independence and de-duplication hold by CONSTRUCTION rather than through a sort a refactor +/// can drop — the same reasoning that made [`crate::identity::l1::join`]'s value a set. +pub fn candidates(observations: &[Observation]) -> BTreeSet { + let mut universe = BTreeSet::new(); + for (i, left) in observations.iter().enumerate() { + for right in &observations[i + 1..] { + if let Some(pair) = CandidatePair::new(left.obs_id, right.obs_id) { + universe.insert(pair); + } + } + } + universe +} + +/// The scale per-mille is expressed in — 1000 parts, D13's milli-units. +const PER_MILLE: usize = 1000; + +/// D13's `blocking_recall` floor, as an INTEGER in per-mille. +/// +/// D13 writes the assertion as `blocking_recall >= 0.999`. The value is the same; the TYPE is not, +/// and the type is the decision — see this module's doc for the milli-units corollary +/// [architecture.md:988-993] that forbids the float, and for why at the committed corpus's +/// denominator this floor is zero-tolerance rather than a tolerance. +pub const BLOCKING_RECALL_FLOOR_PER_MILLE: u32 = 999; + +/// How much of `required` the blocker actually proposed, in per-mille — or `None` when there is +/// nothing to require. +/// +/// # `None` for an empty requirement set +/// +/// A recall with no denominator is **undefined**, not perfect. Returning the full 1000 would let the +/// floor pass over nothing, which is the guarantee the corpus lock already refuses to give +/// (*"reporting 'nothing to check' on the deletion of the thing being guarded is a guarantee the +/// gate does not have"*) and which D13 states outright: *"without blocking, abstention has no +/// denominator."* +/// +/// # Truncation +/// +/// Integer division truncates DOWN, which is the conservative direction for a floor: 2 of 3 is 666 +/// per-mille, never 667. A borderline set therefore reds rather than rounds up into a pass. +/// +/// # Arguments +/// +/// `required` is a set, so a requirement stated twice cannot inflate the denominator. +/// +/// # Returns +/// +/// `Some(recall)` in `0..=1000`, or `None` if `required` is empty. Pairs proposed but not required +/// do not raise the value: only members of `required` are counted. +pub fn blocking_recall_per_mille( + proposed: &BTreeSet, + required: &BTreeSet, +) -> Option { + if required.is_empty() { + return None; + } + let hits = required + .iter() + .filter(|pair| proposed.contains(pair)) + .count(); + u32::try_from(hits * PER_MILLE / required.len()).ok() +} + +/// Tests for the blocker, over SYNTHETIC inputs only. +/// +/// Nothing here reads `fixtures/` — [`crate::observation`] is a domain type and this crate may not +/// touch the filesystem (D47). The corpus-driven half of the recall assertion lives in +/// `opencmdb-bin`'s `fixtures.rs`, which is where the corpus-wide walks already are. +/// +/// # The corpus cannot see this module's central claim +/// +/// Every committed trap pair sits in ONE scope, so a blocker that proposed only same-`l2_domain` +/// pairs would score a full 1000 per-mille on the corpus and be invisible to every corpus test. +/// [`two_l2_domains_are_still_a_candidate_pair`] is the only thing standing between that +/// implementation and green. +/// +/// # A deliberate duplication, which a DRY pass may not collapse +/// +/// The helpers below re-declare `l1.rs`'s spellings — `obs_id`, `l2`, `ts`, `mac`, `observation`. +/// They are private to that file's own test module and unreachable from here; the alternative is a +/// `pub(crate)` test-helper surface, which is a wider change than this story wants. +#[cfg(test)] +mod tests { + use super::*; + // The superset property below is not checkable without the thing it is a superset of. This + // import is the point of `every_pair_inside_a_join_group_is_a_candidate`; the prohibition in + // this module's doc binds `candidates`, not its tests. + use crate::identity::l1::join; + use crate::observation::{ + ConnectorId, Fact, HostnameSource, L2DomainId, MacAddr, Scope, Timestamp, VantageId, + }; + use uuid::Uuid; + + fn obs_id(n: u128) -> ObsId { + ObsId::from_uuid(Uuid::from_u128(n)) + } + + fn l2(n: u128) -> L2DomainId { + L2DomainId::from_uuid(Uuid::from_u128(n)) + } + + fn vantage(n: u128) -> VantageId { + VantageId::from_uuid(Uuid::from_u128(n)) + } + + fn ts() -> Timestamp { + chrono::DateTime::parse_from_rfc3339("2026-08-01T10:00:00Z") + .unwrap() + .with_timezone(&chrono::Utc) + } + + fn mac(last: u8) -> MacAddr { + MacAddr([0x00, 0x11, 0x22, 0x33, 0x44, last]) + } + + /// An observation carrying the given MACs, in the given L2 domain, seen from the given vantage. + fn observation( + id: u128, + domain: L2DomainId, + seen_by: VantageId, + macs: &[MacAddr], + ) -> Observation { + Observation { + obs_id: obs_id(id), + connector_id: ConnectorId::from_uuid(Uuid::from_u128(7)), + observed_at: ts(), + scope: Scope { + l2_domain: domain, + vantage: seen_by, + }, + facts: macs + .iter() + .map(|addr| Fact::Mac { + addr: *addr, + locally_administered: addr.is_locally_administered(), + }) + .collect(), + raw: None, + } + } + + /// The common shape: one MAC, domain 10, vantage 100. + fn simple(id: u128, addr: MacAddr) -> Observation { + observation(id, l2(10), vantage(100), &[addr]) + } + + /// The pair of two ids, for a test that has already decided they are distinct. + fn pair(a: u128, b: u128) -> CandidatePair { + CandidatePair::new(obs_id(a), obs_id(b)).expect("the two ids are distinct") + } + + // ---- AC2: the pair type is unordered, and refuses the self-pair ---- + + #[test] + fn the_self_pair_is_refused() { + assert_eq!( + CandidatePair::new(obs_id(1), obs_id(1)), + None, + "an observation is not a candidate against itself; the generator is where that \ + precondition has a holder" + ); + } + + #[test] + fn a_pair_is_unordered() { + assert_eq!( + CandidatePair::new(obs_id(1), obs_id(2)), + CandidatePair::new(obs_id(2), obs_id(1)), + "the same two ids build the same pair whichever side they arrive on" + ); + } + + #[test] + fn the_accessors_report_the_canonical_order() { + let forward = pair(1, 2); + let backward = CandidatePair::new(obs_id(2), obs_id(1)).expect("distinct"); + + assert_eq!(forward.low(), backward.low()); + assert_eq!(forward.high(), backward.high()); + assert!( + forward.low() < forward.high(), + "the constructor orders the two ids, so the accessors cannot disagree" + ); + assert_eq!(forward.low(), obs_id(1)); + assert_eq!(forward.high(), obs_id(2)); + } + + // ---- AC1: the universe is total ---- + + #[test] + fn no_observation_yields_no_candidate() { + assert!( + candidates(&[]).is_empty(), + "an empty slice proposes nothing, and that is not an error" + ); + } + + #[test] + fn one_observation_yields_no_candidate() { + assert!( + candidates(&[simple(1, mac(0x01))]).is_empty(), + "one observation has nothing to be paired with" + ); + } + + #[test] + fn four_observations_yield_six_pairs() { + let universe = candidates(&[ + simple(1, mac(0x01)), + simple(2, mac(0x02)), + simple(3, mac(0x03)), + simple(4, mac(0x04)), + ]); + + assert_eq!(universe.len(), 6); + assert_eq!( + universe, + BTreeSet::from([ + pair(1, 2), + pair(1, 3), + pair(1, 4), + pair(2, 3), + pair(2, 4), + pair(3, 4), + ]), + "the universe is every unordered pair, named rather than merely counted" + ); + } + + #[test] + fn the_count_is_quadratic_in_the_number_of_distinct_ids() { + for n in 0u128..=6 { + let observations: Vec = (1..=n).map(|i| simple(i, mac(i as u8))).collect(); + + // The DISTINCT id count, not `observations.len()`. The two coincide here and diverge + // the day a duplicate id appears, which is what `a_repeated_obs_id_yields_no_pair` + // covers — a count asserted from `len()` is green today and wrong that day. + let distinct: BTreeSet = observations.iter().map(|o| o.obs_id).collect(); + let expected = distinct.len() * distinct.len().saturating_sub(1) / 2; + + assert_eq!( + candidates(&observations).len(), + expected, + "with {} distinct ids the universe holds {expected} pair(s)", + distinct.len() + ); + } + } + + #[test] + fn a_repeated_obs_id_yields_no_pair() { + let once = simple(1, mac(0x01)); + let again = once.clone(); + + assert_eq!(once.obs_id, again.obs_id, "the fixture must repeat the id"); + assert!( + candidates(&[once, again]).is_empty(), + "the rule is distinct ID, not distinct index" + ); + } + + #[test] + fn the_universe_is_input_order_independent() { + let a = simple(1, mac(0x01)); + let b = simple(2, mac(0x02)); + let c = simple(3, mac(0x03)); + + assert_eq!( + candidates(&[a.clone(), b.clone(), c.clone()]), + candidates(&[c, b, a]), + "the same observations in any input order propose the same universe" + ); + } + + #[test] + fn the_generator_reads_neither_raw_nor_observed_at_nor_the_connector() { + let plain = [simple(1, mac(0x01)), simple(2, mac(0x02))]; + + let mut decorated = plain.clone(); + decorated[0].raw = Some("{\"whatever\": true}".to_string()); + decorated[1].connector_id = ConnectorId::from_uuid(Uuid::from_u128(999)); + decorated[1].observed_at = chrono::DateTime::parse_from_rfc3339("2019-01-02T03:04:05Z") + .unwrap() + .with_timezone(&chrono::Utc); + + assert_ne!(plain[0].raw, decorated[0].raw, "the test must vary raw"); + assert_ne!(plain[1].connector_id, decorated[1].connector_id); + assert_ne!(plain[1].observed_at, decorated[1].observed_at); + + assert_eq!( + candidates(&plain), + candidates(&decorated), + "provenance and time are not read by the blocker" + ); + // ⚠️ The clock/SQL/repository half of purity is UNREACHABLE from a `&[Observation]`, so no + // test can red on it. This asserts the falsifiable half only. + } + + #[test] + fn two_l2_domains_are_still_a_candidate_pair() { + let here = observation(1, l2(10), vantage(100), &[mac(0x01)]); + let there = observation(2, l2(20), vantage(100), &[mac(0x02)]); + + assert_ne!( + here.scope.l2_domain, there.scope.l2_domain, + "the test must actually vary the domain, or it degrades into a single-domain test and \ + a domain-blocked universe passes it" + ); + + assert_eq!( + candidates(&[here, there]), + BTreeSet::from([pair(1, 2)]), + "a device's interfaces are not confined to one L2 domain; excluding the pair would \ + build a false split into the universe" + ); + } + + #[test] + fn an_observation_with_no_mac_is_still_a_candidate() { + let mut hostname_only = simple(2, mac(0x02)); + hostname_only.facts = vec![Fact::Hostname { + name: "kitchen-pi".to_string(), + source: HostnameSource::Dhcp, + }]; + + assert!( + !hostname_only + .facts + .iter() + .any(|f| matches!(f, Fact::Mac { .. })), + "the fixture must actually carry no MAC" + ); + + assert_eq!( + candidates(&[simple(1, mac(0x01)), hostname_only]), + BTreeSet::from([pair(1, 2)]), + "the blocker proposes; whether a rule can answer is the rule's business" + ); + } + + #[test] + fn every_pair_inside_a_join_group_is_a_candidate() { + let observations = [ + observation(1, l2(10), vantage(100), &[mac(0x01)]), + observation(2, l2(10), vantage(200), &[mac(0x01)]), + observation(3, l2(10), vantage(100), &[mac(0x01), mac(0x02)]), + observation(4, l2(20), vantage(100), &[mac(0x02)]), + ]; + + let universe = candidates(&observations); + let groups = join(&observations); + + let mut checked = 0usize; + for members in groups.values() { + let members: Vec = members.iter().copied().collect(); + for (i, left) in members.iter().enumerate() { + for right in &members[i + 1..] { + let inside = + CandidatePair::new(*left, *right).expect("a group holds distinct ids"); + assert!( + universe.contains(&inside), + "the join grouped {left} with {right}, so the blocker must have proposed \ + that pair" + ); + checked += 1; + } + } + } + assert!( + checked > 0, + "the fixture must produce at least one grouped pair, or this proves nothing" + ); + } + + // ---- AC3: the floor and the recall arithmetic ---- + + #[test] + fn the_floor_is_nine_hundred_and_ninety_nine_per_mille() { + // An INDEPENDENT literal, not a read of the constant: every other assertion in this story + // compares against `BLOCKING_RECALL_FLOOR_PER_MILLE` and would move with it, so weakening + // the floor would otherwise red nothing. The value is D13's, expressed in milli-units. + assert_eq!(BLOCKING_RECALL_FLOOR_PER_MILLE, 999); + } + + #[test] + fn every_required_pair_proposed_is_full_recall() { + let required = BTreeSet::from([pair(1, 2), pair(1, 3), pair(2, 3)]); + let proposed = required.clone(); + + assert_eq!(blocking_recall_per_mille(&proposed, &required), Some(1000)); + } + + #[test] + fn one_miss_in_ten_is_nine_hundred_per_mille() { + let required: BTreeSet = (2..=11).map(|n| pair(1, n)).collect(); + assert_eq!(required.len(), 10, "the denominator must actually be ten"); + + let mut proposed = required.clone(); + let dropped = pair(1, 11); + assert!( + proposed.remove(&dropped), + "the fixture must drop a required pair" + ); + + assert_eq!( + blocking_recall_per_mille(&proposed, &required), + Some(900), + "one miss out of ten reds the floor — at this denominator it is zero-tolerance" + ); + } + + #[test] + fn integer_division_truncates_down() { + let required = BTreeSet::from([pair(1, 2), pair(1, 3), pair(1, 4)]); + let proposed = BTreeSet::from([pair(1, 2), pair(1, 3)]); + + assert_eq!( + blocking_recall_per_mille(&proposed, &required), + Some(666), + "two of three truncates DOWN, which is the conservative direction for a floor" + ); + } + + #[test] + fn an_empty_requirement_set_has_no_recall() { + let proposed = BTreeSet::from([pair(1, 2)]); + + assert_eq!( + blocking_recall_per_mille(&proposed, &BTreeSet::new()), + None, + "a recall with no denominator is undefined, not perfect: a full score here would let \ + the floor pass over nothing" + ); + } + + #[test] + fn a_pair_proposed_but_not_required_does_not_inflate_recall() { + let required = BTreeSet::from([pair(1, 2), pair(1, 3)]); + let proposed = BTreeSet::from([pair(1, 2), pair(1, 3), pair(2, 3), pair(3, 4)]); + + assert!( + proposed.len() > required.len(), + "the fixture must over-propose" + ); + assert_eq!( + blocking_recall_per_mille(&proposed, &required), + Some(1000), + "recall counts the required set, so a larger universe cannot exceed the full value" + ); + } +} diff --git a/crates/opencmdb-core/src/identity/mod.rs b/crates/opencmdb-core/src/identity/mod.rs index 240d4c0..fab7dc2 100644 --- a/crates/opencmdb-core/src/identity/mod.rs +++ b/crates/opencmdb-core/src/identity/mod.rs @@ -26,5 +26,6 @@ //! this subtree, which nothing yet is: [`cascade::IdentityAbstentionCause`] is plain `pub` because //! `score::Outcome`, in another subdomain, names it in a field type. +pub mod blocking; pub mod cascade; pub mod l1; From bb348574952d107cf153b9bdf6b8e48d45a44074 Mon Sep 17 00:00:00 2001 From: Guy Corbaz Date: Sat, 1 Aug 2026 00:49:38 +0200 Subject: [PATCH 2/3] Story 5.6: docs, register, and the six mutations reported MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The five falsified doc sites are corrected — identity/mod.rs, cascade.rs's Decision, l1.rs twice, and lib.rs, whose stated reason for not flat-re-exporting join ('its consumer, the candidate generator, does not exist yet') is exactly what this story falsifies. A sixth site was found by RE-READING rather than by grep: identity/mod.rs's D54 paragraph, which turned out to be TRUE (verdict_for_pair is pub(crate) — the crate, not the subtree), so it was verified and annotated instead of rewritten. Register: R1 (the float) and R2 (the self-pair) disposed of by TITLE, three new entries opened, and four belonging to other stories read and left open — the &str allocation entry's condition is measured NOT met, since candidates calls no rule at all. All six mutations run against a committed baseline, every restore verified by md5sum and git status. Zero compiler-carried reds. Three divergences from the predictions are stated rather than left implicit. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HEE7mz4xufqYW5xKfgJBaF --- CLAUDE.md | 2 +- .../5-6-blocker-and-recall-assertion.md | 203 ++++++++++++++---- .../implementation-artifacts/deferred-work.md | 75 +++++++ .../sprint-status.yaml | 57 ++++- crates/opencmdb-core/src/identity/cascade.rs | 6 +- crates/opencmdb-core/src/identity/l1.rs | 15 +- crates/opencmdb-core/src/identity/mod.rs | 28 ++- crates/opencmdb-core/src/lib.rs | 18 +- docs/project-context.md | 15 +- 9 files changed, 349 insertions(+), 70 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1e1f10d..b93bba3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project status -`opencmdb` is a self-hosted, single-binary **Rust** network reconciliation engine (IPAM + light app CMDB + topology) for home-lab/SMB. Core thesis: continuously compare the **observed** state (auto-discovered) against the **declared** state (documented); the gap is the product. **Planning is COMPLETE** (product brief, PRD, UX spec, architecture — all in `_bmad-output/planning-artifacts/`, decision register D1–D66). **As of 2026-07-22 the code RUNS**: `v0.1.1` is tagged and published to Docker Hub as `gcorbaz/opencmdb`, and it scans a CIDR and shows a real observed-vs-declared gap on one page. **Epics 1–4 of 23 are done** (the walking skeleton shipped as v0.1). **Epic 4 closed on 2026-07-25**: the fixture corpus, the metrics harness and the trap corpus — all written BEFORE the identity engine — are committed and locked (25 artefacts, 24 traps across nine families, plus the wire-format spec). Its story 4.19 was SPLIT at closure: 4.19a shipped, **4.19b (the mutation generator) moved to Epic 11** — recorded in `epic-4-correct-course-2026-07-25.md` and GitHub issue #34, not silently. **Epic 5 — the identity engine — is IN PROGRESS since 2026-07-27** (**16 stories**: 5.4b was INSERTED 2026-07-29 at story 5.4's contexting, splitting the verdict algebra — a function that must be TOTAL over a table D13 leaves one input class short — and its anti-float `xtask` gate out of 5.4). Its **three inherited debt stories come first and are all `done`** — 5.1, 5.2 and 5.2b, PRs #41, #44 and #46, all merged 2026-07-28. The engine proper starts at **5.3, `done`** — PR #48, merged 2026-07-29 — which ships the identity engine's own abstention vocabulary (`IdentityAbstentionCause`, in the new `crates/opencmdb-core/src/identity/`) and **no engine**: no cascade, no rule, no `Decision`, no join. **5.4 follows and is `done`** — PR #52, squash-merged 2026-07-29 as `da87b62`: it adds the engine's RETURN type in the same file — `Verdict` (D13's five verdicts), `RuleVerdict { rule, verdict, evidence }`, `RulesetVersion`, `Conclusion` and `Decision` — and still **no algebra**: nothing combines a verdict set, no rule produces a verdict, nothing produces a decision. **5.4b follows and is `done`** — PR #55 squash-merged 2026-07-30 as `4f4e774` after a green CI run, code-reviewed first (281 tests): it ships the ALGEBRA — `decide(Vec, RulesetVersion) -> Decision`, a TOTAL pure function over D13's six rows plus the one input class the table leaves uncovered (`≥1 Opposes` alone → `Abstained { AbsenceOfProof }`, Guy's arbitration; the D13 correction itself is **GitHub issue #54**, a milestone act) — and a **sixth `cargo xtask ci` gate, `float-free`**, which reds on a float type or a float literal in code under `identity/`, comments stripped so the architecture may be quoted. The review applied 22 patches and registered 5 deferrals: `decide` now matches on `Option` rather than booleans, so the presence test and the rule selection are one act and no unreachable arm has to be justified; and the gate's matcher is a **numeric-literal tokeniser** rather than a digit-dot-digit search, because the search form was measured to pass `1e-3` and `1.` (both `f64`) while reddening `"192.168.0.1"`, `t.0.1` and any identifier containing `f64` — a shape story 5.5 would have tripped on its first MAC/IP rule — although in the event **5.5 wrote no IP literal at all**, and that mis-prediction was itself corrected as a finding. **5.5 follows and is `done`** — PR #57 squash-merged 2026-07-31 as `0ebd50f` after a green CI run, code-reviewed first (three layers): it ships `crates/opencmdb-core/src/identity/l1.rs` — the L1 join `join(&[Observation]) -> BTreeMap<(L2DomainId, MacAddr), BTreeSet>` (the value is a `BTreeSet` so order-independence holds by CONSTRUCTION, not by a `sort()` a refactor can drop), the two rules the committed corpus names (`l1-exact-mac -> Decisive` on **at least one shared key**, `l1-distinct-mac -> Disqualifying` when they share none, `Neutral` when either side carries no MAC), and `decide_pair` — **the first caller of `decide` outside its own tests** — with `CURRENT_RULESET_VERSION = RulesetVersion(1)` and still **no `Default`**. **281 → 309 tests.** Its AC3 was discharged as a NEGATIVE requirement: D13 calls the U/L bit and the IANA prefixes `Disqualifying` *"as GROUPING anchors"* and grouping is L2, so **no IANA predicate was added** and two tests assert `l1-exact-mac` still fires on a locally-administered MAC and on a VRRP MAC — implementing D13's label at L1 would have reddened two committed traps. Prove-to-red: six mutations, **every red assertion-carried, none compiler-carried**. **Still no blocker** (5.6), nothing feeds the corpus harness (5.7), and `Verdict::Supports`/`Opposes` still have **no producer** (Epic 6). **Next is 5.6, the blocker — whose `blocking_recall >= 0.999` is a bare float literal that reds the `float-free` gate, measured and flagged forward.** _(This sentence named only 5.1 until 2026-07-29 — stale for a day, found by story 5.3's code review, not by the two stories that caused the drift.)_ Live status is `_bmad-output/implementation-artifacts/sprint-status.yaml`; grounding is `docs/project-context.md`. +`opencmdb` is a self-hosted, single-binary **Rust** network reconciliation engine (IPAM + light app CMDB + topology) for home-lab/SMB. Core thesis: continuously compare the **observed** state (auto-discovered) against the **declared** state (documented); the gap is the product. **Planning is COMPLETE** (product brief, PRD, UX spec, architecture — all in `_bmad-output/planning-artifacts/`, decision register D1–D66). **As of 2026-07-22 the code RUNS**: `v0.1.1` is tagged and published to Docker Hub as `gcorbaz/opencmdb`, and it scans a CIDR and shows a real observed-vs-declared gap on one page. **Epics 1–4 of 23 are done** (the walking skeleton shipped as v0.1). **Epic 4 closed on 2026-07-25**: the fixture corpus, the metrics harness and the trap corpus — all written BEFORE the identity engine — are committed and locked (25 artefacts, 24 traps across nine families, plus the wire-format spec). Its story 4.19 was SPLIT at closure: 4.19a shipped, **4.19b (the mutation generator) moved to Epic 11** — recorded in `epic-4-correct-course-2026-07-25.md` and GitHub issue #34, not silently. **Epic 5 — the identity engine — is IN PROGRESS since 2026-07-27** (**16 stories**, 7 done and 5.6 in `review`: 5.4b was INSERTED 2026-07-29 at story 5.4's contexting, splitting the verdict algebra — a function that must be TOTAL over a table D13 leaves one input class short — and its anti-float `xtask` gate out of 5.4). Its **three inherited debt stories come first and are all `done`** — 5.1, 5.2 and 5.2b, PRs #41, #44 and #46, all merged 2026-07-28. The engine proper starts at **5.3, `done`** — PR #48, merged 2026-07-29 — which ships the identity engine's own abstention vocabulary (`IdentityAbstentionCause`, in the new `crates/opencmdb-core/src/identity/`) and **no engine**: no cascade, no rule, no `Decision`, no join. **5.4 follows and is `done`** — PR #52, squash-merged 2026-07-29 as `da87b62`: it adds the engine's RETURN type in the same file — `Verdict` (D13's five verdicts), `RuleVerdict { rule, verdict, evidence }`, `RulesetVersion`, `Conclusion` and `Decision` — and still **no algebra**: nothing combines a verdict set, no rule produces a verdict, nothing produces a decision. **5.4b follows and is `done`** — PR #55 squash-merged 2026-07-30 as `4f4e774` after a green CI run, code-reviewed first (281 tests): it ships the ALGEBRA — `decide(Vec, RulesetVersion) -> Decision`, a TOTAL pure function over D13's six rows plus the one input class the table leaves uncovered (`≥1 Opposes` alone → `Abstained { AbsenceOfProof }`, Guy's arbitration; the D13 correction itself is **GitHub issue #54**, a milestone act) — and a **sixth `cargo xtask ci` gate, `float-free`**, which reds on a float type or a float literal in code under `identity/`, comments stripped so the architecture may be quoted. The review applied 22 patches and registered 5 deferrals: `decide` now matches on `Option` rather than booleans, so the presence test and the rule selection are one act and no unreachable arm has to be justified; and the gate's matcher is a **numeric-literal tokeniser** rather than a digit-dot-digit search, because the search form was measured to pass `1e-3` and `1.` (both `f64`) while reddening `"192.168.0.1"`, `t.0.1` and any identifier containing `f64` — a shape story 5.5 would have tripped on its first MAC/IP rule — although in the event **5.5 wrote no IP literal at all**, and that mis-prediction was itself corrected as a finding. **5.5 follows and is `done`** — PR #57 squash-merged 2026-07-31 as `0ebd50f` after a green CI run, code-reviewed first (three layers): it ships `crates/opencmdb-core/src/identity/l1.rs` — the L1 join `join(&[Observation]) -> BTreeMap<(L2DomainId, MacAddr), BTreeSet>` (the value is a `BTreeSet` so order-independence holds by CONSTRUCTION, not by a `sort()` a refactor can drop), the two rules the committed corpus names (`l1-exact-mac -> Decisive` on **at least one shared key**, `l1-distinct-mac -> Disqualifying` when they share none, `Neutral` when either side carries no MAC), and `decide_pair` — **the first caller of `decide` outside its own tests** — with `CURRENT_RULESET_VERSION = RulesetVersion(1)` and still **no `Default`**. **281 → 309 tests.** Its AC3 was discharged as a NEGATIVE requirement: D13 calls the U/L bit and the IANA prefixes `Disqualifying` *"as GROUPING anchors"* and grouping is L2, so **no IANA predicate was added** and two tests assert `l1-exact-mac` still fires on a locally-administered MAC and on a VRRP MAC — implementing D13's label at L1 would have reddened two committed traps. Prove-to-red: six mutations, **every red assertion-carried, none compiler-carried**. **5.6 follows and is in `review`** — the blocker, on branch `story-5.6-blocker-and-recall-assertion` off `master` at `440b30e`: `crates/opencmdb-core/src/identity/blocking.rs` ships `CandidatePair` (private fields ordered by its constructor, `new(a, a) -> None` — which closes the register's self-pair entry **in the type**), `candidates(&[Observation]) -> BTreeSet` — **TOTAL by decision**, every unordered pair of distinct `obs_id`s, calling neither `join` nor `decide_pair` because a blocker that consults a rule is that rule's echo — and the floor. **The float was resolved by TYPING it, not avoiding it**: D13's `blocking_recall >= 0.999` ships as `BLOCKING_RECALL_FLOOR_PER_MILLE: u32 = 999` on D13's own milli-units corollary, and the `float-free` gate was **not** weakened — it walks **4** files under `identity/` where it walked 3. The corpus assertion lives in `opencmdb-bin`'s `fixtures.rs` test module because D47 forbids the domain crate to read files: **24 traps, 23 pairs, 10 `must-merge`, recall 1000‰, all asserted rather than quoted**. `blocking_recall_above_999` computes only the recall and the per-trap containment is a **separate** test — measured to be load-bearing: in one test a missing pair panics before any recall exists. **309 → 332 tests.** Six mutations: **M1 (blocking on an L1 key) scores 700‰ exactly**, and **M2 (blocking on `l2_domain`) leaves the entire corpus GREEN** — only a synthetic cross-domain test reds it, which is why that test was written first. Zero compiler-carried reds. Nothing feeds the corpus harness (5.7), the blocker has **no production caller**, and `Verdict::Supports`/`Opposes` still have **no producer** (Epic 6). **Next is `code-review` 5.6, then merge.** _(This sentence named only 5.1 until 2026-07-29 — stale for a day, found by story 5.3's code review, not by the two stories that caused the drift.)_ Live status is `_bmad-output/implementation-artifacts/sprint-status.yaml`; grounding is `docs/project-context.md`. ### Build / lint / test commands (the stack is chosen and building) diff --git a/_bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md b/_bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md index 2a34530..c077098 100644 --- a/_bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md +++ b/_bmad-output/implementation-artifacts/5-6-blocker-and-recall-assertion.md @@ -1,6 +1,6 @@ # Story 5.6: The blocker, and the recall assertion nobody writes -Status: ready-for-dev +Status: review