From 70dc923948f304d4816dc6c5e38aa056f87a6cf1 Mon Sep 17 00:00:00 2001 From: choiyounggi <74581798+choiyounggi@users.noreply.github.com> Date: Sun, 9 Aug 2026 17:00:34 +0900 Subject: [PATCH] knowledge: ingest 1 verified insight (generated closed-vocabulary reference gates) --- .dev-loop/INGEST_REPORT.md | 167 +++++++++++++----- log.md | 2 + ...nerated-artifacts-as-deliverable-source.md | 4 +- .../generated-reference-drift-gates.md | 111 ++++++++++++ .../spec-document-gates.md | 3 +- wiki/qa/index.md | 1 + 6 files changed, 242 insertions(+), 46 deletions(-) create mode 100644 wiki/qa/document-verification/generated-reference-drift-gates.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..5b8092e 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,134 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 3 insight(s): 1 ingested, 2 dropped as in-flight duplicates -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: 3 pending candidates from 2 sessions (`linkly`, `groundwork`). ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### 1. Generate a closed-vocabulary reference from the owning constant, and gate it — INGESTED + +**Claim.** When a reference document enumerates a closed vocabulary that a model will +emit tokens from (DSL verbs, config keys, diagnostic codes, enum members), generate the +enumerated part from the compiler/runtime constant, gate it with a `--check` mode called +from the test suite, and put a machine-readable generated banner on line 1. A green +`--check` is not sufficient on its own: it compares the committed file against the +generator's own output, so a generator that hardcodes a column regenerates happily and +stays green. Per-member coverage assertions against the constant, plus a +not-a-single-repeated-value negative control, are a separate layer. + +**Sources checked (all fetched this session, quotes verified):** + +| Source | What it establishes | +|--------|---------------------| +| https://pkg.go.dev/cmd/go | "To convey to humans and machine tools that code is generated, generated source should have a line that matches the following regular expression (in Go syntax)": `^// Code generated .* DO NOT EDIT\.$`, before the first non-comment text — the banner is machine-readable by convention, not a polite comment | +| https://prettier.io/docs/en/cli | `--check` "will output a human-friendly message and a list of unformatted files, if any"; returns "exit code `1` in the second case, which is helpful inside the CI pipelines" — the check-mode-as-gate shape | +| https://google.github.io/styleguide/docguide/best_practices.html | "Change your documentation in the same CL as the code change"; and for a fact owned elsewhere, "Do not write your own guide … Link to it instead" | +| https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools | "Provide extremely detailed descriptions. This is by far the most important factor in tool performance" — what the model is given determines what it emits | +| https://go.dev/blog/generate | Shows the convention in practice (`// Code generated by stringer -type Pill pill.go; DO NOT EDIT.`) but states no regex; the normative text is in `cmd/go` above | + +**Local verification (re-run this session, not taken from the harvested block):** + +- `python3 scripts/gen_plugin_references.py --check` in `linkly` → `rc=0` against the + committed references. +- `impl/tests/test_plugin_references.py` runs it inside the suite: + `test_no_drift_between_source_and_committed_files` asserts `returncode == 0`; + `test_check_mode_detects_a_hand_edit` and `test_check_mode_reports_a_missing_file` + both assert `returncode == 1`; + `test_generated_files_carry_the_do_not_edit_banner` asserts the line-1 banner + (confirmed present in `plugins/lnpl/skills/lnpl-authoring/references/declarations.md`). +- The `--check`-is-insufficient nuance is documented in that same suite, in the + docstring of `test_every_diagnostic_code_reaches_the_document_with_its_grade`: + "`--check` alone cannot see this: it compares the committed file against the + generator's own output, so a generator that hardcoded every grade to `warning` would + regenerate happily and stay green" — with + `test_the_grade_column_is_not_a_single_repeated_value` as the negative control. This + refinement was **not** in the harvested candidate; it was found while verifying it and + is the strongest part of the page. +- Counter-example, also asserted in the suite: + `impl/tests/test_cli_diagnostics.py::test_compile_reports_all_six_and_still_succeeds` + pins the hand-written golden `examples/login.lnpl` at `rc == 0` with + `err.count("unknown-verb") == 3` — three steps compile to nothing while the compile + reports success. The harvested block claimed "3 of 6 steps"; the verified form is + "3 unknown-verb diagnostics on a golden example that still exits 0", and the page says + only that. + +**Confidence: `verified`.** The generation/`--check`/banner mechanics are backed by +official docs (Go, Prettier, Google docguide) and by a reproduction run this session; +the agent-consumer framing is backed by Anthropic's own tool-definition guidance. + +### 2. Usage-limit worker pause vs. crash — DROPPED (duplicate, see Open-PR check) + +Claim (workers go quiet simultaneously, every liveness check passes, find the +`You've hit your session limit · resets HH:MM` marker, resume after the reset with a +state-recheck → remaining-DoD → completion-signal prompt) was **not** independently +re-verified, because it is already carried in open PR #47 with the same field evidence. +No new page or edit was made for it. + +### 3. Dispatch immediately after `worker_done` → `runtime_unavailable` — DROPPED (duplicate) + +Same disposition: already carried in open PR #47, including the task-consumption +asymmetry (a failed `worker-start` consumes the Task, so recreate from the spec rather +than retrying it) and the `tui-idle` precondition. ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed via `INDEX.md`. The harvested `domain: platforms` hint was checked and rejected +for candidate 1 — `platforms` is scoped to OS-level differences (shells, BSD-vs-GNU, +filesystems, toolchains), and this is a document-verification case. `qa` owns +"automated verification of document deliverables (spec/RFC gates)". + +Pages read: qa-document-verification-spec-document-gates, qa-document-verification-editing-a-gated-document, qa-deliverables-generated-artifacts-as-deliverable-source, qa-exploratory-lowered-declaration-survival, backend-common-api-design-unenforced-declarations, infrastructure-agent-orchestration-control-signals-vs-primary-artifacts, infrastructure-agent-orchestration-pane-delivery-confirmation, testing-quality-tests-that-cannot-fail + +Overlaps found and how each was resolved: + +| Existing page | Overlap | Resolution | +|---------------|---------|------------| +| `qa-deliverables-generated-artifacts-as-deliverable-source` | Both say "use the generator's output as the body". That page's trigger is a **hand-off deliverable** (ERD, schema reference for a partner/review) and it has no gate: its question is "re-run the generator or hand-write it?" | Kept separate. New page's trigger is a **shipped reference an agent reads back**, and its content is the gating layer (`--check` in the suite, banner assertion, coverage assertions). Cross-linked both directions; step 1 of the new page defers to it for the generated-body-vs-hand-written-narrative split | +| `qa-document-verification-spec-document-gates` | Nearest neighbour. Its "External agreement" axis (added on open PR #66, unmerged) says a gate must resolve the owning constant for a copied cell | Kept separate, and the distinction is stated: that page gates a **hand-written** document with grep/parse checks; the new page covers a **generated** document, where the generator itself becomes the drift source and `--check` is self-consistent. Cross-linked both directions | +| `backend-common-api-design-unenforced-declarations` | Declared-but-not-enforced behavior | Cited from the new page's edge case for "the runtime ignores unknown members" — the reason this gate matters more, not less, in that case | +| `testing-quality-tests-that-cannot-fail` | Red-run proof for gates | `related:` link only; no content duplicated | +| `infrastructure-agent-orchestration-control-signals-vs-primary-artifacts` | Read for candidates 2 and 3 | It already owns the stalled-vs-dead-vs-alive case; PR #47 extends it with exactly these two candidates. No edit made | +| `infrastructure-agent-orchestration-pane-delivery-confirmation` | Read for candidate 2's delivery mechanics | Already covers send/confirm; nothing to add | + +Conflicts flagged: none. No existing directive contradicts the new page. + +Link-integrity note: the new page originally cited +`backend-common-change-impact-widening-a-closed-value-table`, which exists **only** on +open PR #51's branch and not on `main`. Every `related:` id was resolved against this +checkout's `wiki/` and that one was replaced with +`backend-common-api-design-unenforced-declarations` so the page carries no dangling id +if #51 is rejected. + +Edits to existing files: `wiki/qa/index.md` (+1 row), plus `related:` back-links and a +`last_verified` bump on `spec-document-gates.md` and +`generated-artifacts-as-deliverable-source.md`. No existing body text was rewritten. + +## Open-PR check + +Listed with `gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"` +— 14 open heads: #68, #66, #64, #62, #61, #58, #57, #56, #55, #52, #51, #50, #49, #47. +Six with plausibly overlapping triggers were fetched and diffed against `origin/main` +under `wiki/`: #68, #66, #64, #51, #50, #47. + +| Candidate | Overlapping open head | Verdict | +|-----------|----------------------|---------| +| 1 — generated closed-vocabulary reference + `--check` gate | #66 (`knowledge/choiyounggi-20260808-013406`) touches `qa/document-verification/spec-document-gates.md` and adds `qa/deliverables/command-transcripts-in-a-document.md`; #51 adds `backend/common/change-impact/widening-a-closed-value-table.md` | **new** — #66 gates a hand-written table by resolving the owning constant; it does not cover generating the document, `--check` in the suite, the generated banner, or the fact that `--check` cannot detect a hardcoding generator. #51 is about widening a closed value table in code, not its reference document. No open head adds a page under a generated-reference trigger | +| 2 — usage-limit pause vs. crash | #47 (`knowledge/dch0202-20260806-130040`) | **drop** — #47 already adds this verbatim as an edge-case row on `control-signals-vs-primary-artifacts.md` ("search each worker's pane/terminal tail for the CLI's usage-limit marker (e.g. `You've hit your session limit · resets HH:MM`) … re-verify state → remaining definition-of-done → completion signal"), plus the matching `infrastructure/index.md` "load when" extension and the same 2026-08-06 three-worker/`resets 01:10` field-evidence line. The only delta in the queued row was naming the send mechanism (`orca terminal send --enter`, tmux `send-prompt.sh send`), which `pane-delivery-confirmation` already owns — not worth a push to a stale branch | +| 3 — dispatch after `worker_done` | #47 (same head) | **drop** — #47 carries it as the adjacent edge-case row, including `orca terminal wait --for tui-idle` as the precondition, "the done message is the worker's report time, not the substrate's release time", and the consumed-Task rule ("create a new task from the same spec — the consumed one cannot be retried"). Nothing in the queued row is absent | + +No sibling duplicate PR was opened, and nothing was pushed to #47. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Target | New category? | +|---------|--------|---------------| +| 1 | `qa` / `document-verification` / `generated-reference-drift-gates.md` (id `qa-document-verification-generated-reference-drift-gates`) — **new page** | No. `document-verification` already exists and owns automated checks that decide whether a document meets its requirements; this is that, for a generated document. `qa/deliverables` was considered and rejected: its trigger is producing a hand-off deliverable, not gating a shipped one | +| 2 | none — retired as a pending duplicate of #47 | — | +| 3 | none — retired as a pending duplicate of #47 | — | + +The harvested `domain: platforms` hint on insight 1 was overridden: `platforms` covers +OS-level differences across macOS/Linux/Windows, and nothing about this case is +OS-dependent. Insights 2 and 3 also carried `platforms` hints; #47 had already routed the +same content to `infrastructure/agent-orchestration`, which is where it belongs. + +All 3 queued rows are retired from the active queue (1 ingested, 2 dropped) so neither +dropped row re-crosses the auto-flush threshold. diff --git a/log.md b/log.md index c930fc2..df2f0b8 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,5 @@ Append-only. Format: `## [YYYY-MM-DD]