From e8e03c549b6541221a38477d987733e8524c06a6 Mon Sep 17 00:00:00 2001 From: Mike Clay Date: Fri, 14 Aug 2026 09:42:35 +0100 Subject: [PATCH] Split comprehension into a corpus artifact and a session-local log Comprehension wrote everything it produced into the shared corpus: the settled facts about a codebase area, and alongside them the questions the pass asked, the investigations that answered them, and the follow-ups it deferred. A later reader of the corpus got somebody else's working notes mixed into the reference. The corpus artifact now carries what is true of the area, split into structure and behaviour, with rationale marked as inferred. The questions, deep dives, challenge lenses and follow-ups go to the planning folder, where a promotion rule decides what crosses into the corpus. Co-Authored-By: Claude Opus 5 (1M context) --- .../activities/15-codebase-comprehension.yaml | 17 +- work-package/activities/README.md | 2 +- work-package/resources/README.md | 5 +- .../resources/codebase-comprehension.md | 236 +++++++++++++++--- .../techniques/analyse-challenge/run-loop.md | 2 +- .../codebase-comprehension/TECHNIQUE.md | 26 +- .../codebase-comprehension/deep-dive.md | 47 ++-- .../revise-questions.md | 18 +- .../codebase-comprehension/survey.md | 9 +- .../review-assumptions/reconcile.md | 2 +- 10 files changed, 281 insertions(+), 83 deletions(-) diff --git a/work-package/activities/15-codebase-comprehension.yaml b/work-package/activities/15-codebase-comprehension.yaml index 1ed017313..ae2ed664d 100644 --- a/work-package/activities/15-codebase-comprehension.yaml +++ b/work-package/activities/15-codebase-comprehension.yaml @@ -1,5 +1,5 @@ id: codebase-comprehension -version: 1.10.0 +version: 1.11.0 name: Codebase Comprehension description: Build or augment a mental model of the codebase. required: false @@ -33,10 +33,17 @@ steps: technique: name: manage-artifacts::write-artifact inputs: + artifact_content: comprehension_artifact target_dir: comprehension_dir - kind: technique id: revise-initial-questions technique: revise-questions + - kind: technique + id: record-log-initial + technique: + name: manage-artifacts::write-artifact + inputs: + artifact_content: comprehension_log - kind: loop id: deep-dive-iteration name: Deep-Dive Iteration @@ -63,7 +70,14 @@ steps: technique: name: manage-artifacts::write-artifact inputs: + artifact_content: comprehension_artifact target_dir: comprehension_dir + - kind: technique + id: record-log + technique: + name: manage-artifacts::write-artifact + inputs: + artifact_content: comprehension_log - kind: checkpoint id: comprehension-sufficient condition: @@ -126,4 +140,5 @@ outcome: - Key abstractions and design rationale documented - Domain concepts mapped to technical constructs - A reusable comprehension reference lands in the cumulative corpus, where the next work package on the same area reads it instead of re-investigating + - The questions, investigations and follow-ups behind that reference stay in the session's planning folder as a comprehension log - Open questions converged via analyse-challenge before the soft sufficiency gate diff --git a/work-package/activities/README.md b/work-package/activities/README.md index 6b2a16ef4..5ae2a85b4 100644 --- a/work-package/activities/README.md +++ b/work-package/activities/README.md @@ -79,7 +79,7 @@ graph TD ### Codebase Comprehension (optional) -Builds or augments a durable mental model of the codebase sufficient to qualify the design assumptions raised in later activities. Produces persistent knowledge artifacts under `.engineering/artifacts/comprehension/` that grow across successive work packages. Runs after design-philosophy and routes to elicitation, research, analysis, or plan-prepare depending on the chosen path. +Builds or augments a durable mental model of the codebase sufficient to qualify the design assumptions raised in later activities. Produces persistent knowledge artifacts under `.engineering/artifacts/comprehension/` that grow across successive work packages, and a session-local log of the questions and investigations behind them. Runs after design-philosophy and routes to elicitation, research, analysis, or plan-prepare depending on the chosen path. Definition: [`15-codebase-comprehension.yaml`](./15-codebase-comprehension.yaml) diff --git a/work-package/resources/README.md b/work-package/resources/README.md index 454358e2a..45be7b693 100644 --- a/work-package/resources/README.md +++ b/work-package/resources/README.md @@ -34,7 +34,7 @@ Markdown resources for planning-folder templates, elicitation and review guidanc | `follow-ups` | Follow-ups | In-task follow-ups register template (distinct from out-of-scope `deferred-items`) | | `tdd-concepts-rust` | TDD Concepts Rust | TDD best practices for Rust: Red-Green-Refactor, FIRST principles | | `review-mode` | Review Mode | Complete guide for review mode behavior and PR review formats | -| `codebase-comprehension` | Codebase Comprehension | Comprehension techniques, artifact template, and deep-dive guidance from reverse engineering and code forensics literature | +| `codebase-comprehension` | Codebase Comprehension | Comprehension techniques, corpus and log artifact templates, promotion criteria, and deep-dive guidance from reverse engineering and code forensics literature | | `assumption-reconciliation` | Assumption Reconciliation | Assumptions-log integration and scorecard formats | | `research-reconciliation` | Research Reconciliation | Research-candidate inventory shape, reconcilability statuses, and scorecard format | | `pr-review-response` | PR Review Response | Response-format and review-document templates | @@ -66,7 +66,8 @@ Which guide owns each persisted filename's shape. | `architecture-summary.md` | [architecture-summary](architecture-summary.md) | | `strategic-review-{n}.md` | [strategic-review](strategic-review.md) | | `strategic-review-{n}-method.md` | [strategic-review](strategic-review.md#method-record-template) | -| `{codebase_area}.md` | [codebase-comprehension](codebase-comprehension.md) | +| `{codebase_area}.md` | [codebase-comprehension](codebase-comprehension.md#corpus-artifact-template) | +| `codebase-comprehension.md` | [codebase-comprehension](codebase-comprehension.md#comprehension-log-template) | | `{YYYY-MM-DD}-pr{pr_number}-review-analysis.md` | [pr-review-response](pr-review-response.md) | | `kb-research.md` | [knowledge-base-research](knowledge-base-research.md) | | `design-philosophy.md` | [design-framework](design-framework.md) | diff --git a/work-package/resources/codebase-comprehension.md b/work-package/resources/codebase-comprehension.md index 524c371b3..92054f4b7 100644 --- a/work-package/resources/codebase-comprehension.md +++ b/work-package/resources/codebase-comprehension.md @@ -1,8 +1,8 @@ --- name: codebase-comprehension -description: Comprehension techniques, artifact template, and deep-dive guidance from reverse engineering and code forensics literature. +description: Comprehension techniques, corpus and log artifact templates, promotion criteria, and deep-dive guidance from reverse engineering and code forensics literature. metadata: - version: 1.2.0 + version: 1.3.0 order: 25 legacy_id: 25 --- @@ -10,7 +10,7 @@ metadata: # Codebase Comprehension Guide -Systematically build a mental model of an unfamiliar codebase before design decisions are made. Artifacts persist in the comprehension corpus, outside any one session's planning folder, and are augmented across successive work packages into a cumulative knowledge base. +Systematically build a mental model of an unfamiliar codebase before design decisions are made. Comprehension produces a cumulative corpus artifact and a session-local log, each with its own template below. Knowledge-base sources for concept lookups: *Object-Oriented Reengineering Patterns* (Demeyer, Ducasse, Nierstrasz — first contact, reverse engineering lifecycle), *Your Code as a Crime Scene* (Tornhill — hotspots, temporal coupling, knowledge maps), *Software Design X-Rays* (Tornhill — behavioral analysis, complexity trends, change coupling), *Code Reading* (Spinellis — reading strategies, software archaeology, build analysis), *Working Effectively with Legacy Code* (Feathers — seams, characterization tests, dependency breaking). @@ -51,7 +51,7 @@ Use these to decide where to focus comprehension effort: ### 5. Hypothesis-Driven Top-Down Comprehension -Form an initial architecture hypothesis from directory layout and build configuration; verify by sampling entry points, module roots, public APIs; revise as evidence accumulates; document verified understanding and open questions. +Form an initial architecture hypothesis from directory layout and build configuration; verify by sampling entry points, module roots, public APIs; revise as evidence accumulates. Verified understanding lands in the corpus artifact; the hypothesis that produced it, and any question it leaves open, stay in the log. ### 6. Hierarchical Decomposition @@ -94,6 +94,7 @@ Lexicon for the code path under study: - **Execution context**: dispatch class, thread model, execution priority. In Substrate, a `Mandatory` dispatch returning an error rejects the block; in a consensus system where all nodes process the same inputs, that halts the network. Execution context determines whether an error is a local retry, a skipped item, or a system-wide halt. - **Error propagation**: what happens on error — caught and handled? Rolled back at a transaction boundary? Surfaced to the user? Halts processing? For inherent extrinsics, `IsFatalError` — if all variants return `true`, every error is fatal. +- **Resource bounds**: what caps the work an untrusted or unlucky input can induce — declared constants, cache capacities, per-identity budgets — and what each one actually limits, which is often narrower than its name suggests. - **Operational scenarios** beyond the steady-state happy path: - **Startup and genesis**: initial values; a guard assuming "previous value is meaningful" may fail on the first block after genesis when the previous value is zero/default. - **Recovery after downtime**: if external state advanced significantly while offline, a bounded-advance guard may reject the catch-up jump. @@ -103,97 +104,252 @@ Lexicon for the code path under study: These concerns belong in the architecture survey and deep dives, not as a separate end step: key abstractions raise "where does this data come from?"; rationale raises "what happens if this fails?"; domain mapping raises "what is the timing relationship with dependencies?". Open Questions of this kind ("Does the producer enforce the window bound?", "What happens at genesis when the previous position is zero?") prevent guards from becoming halt vectors. -## Artifact Template +## Corpus Artifact Template -Comprehension artifacts follow this structure. When augmenting an existing artifact, add new sections or deepen existing ones — do not replace prior content. +The durable artifact. It states what is true of the codebase area, in the present tense, for a reader who has neither the session nor the code open. Its top-level split is structure (what exists) against behaviour (what happens when it runs). ```markdown -# {Codebase Area Name} — Comprehension Artifact +# {Codebase Area Name} — Comprehension -> YYYY-MM-DD · work packages: [contributing refs] · coverage: [what this artifact covers] · related: [cross-refs to other comprehension artifacts; omit if none] +[One sentence naming what this area does.] -## Architecture Overview +## Structure -### Project Structure -[Directory layout, build system, entry points] +[What exists and how it is arranged. Nothing here says what happens at run time.] + +### Overview + +[The dependency shape in a sentence or two, then a diagram of it.] + +### Project + +[Where the code lives and how a running system reaches it.] + +#### {Build units} +[Table: unit, path, role in this area] + +#### Entry points +[The path from process start to this area, then a diagram of that call chain.] ### Module Map -[Modules, responsibilities, dependency relationships] + +[Table: module, responsibility, depends on] ### Design Patterns -[Overarching architectural patterns observed: layered, event-driven, etc.] -## Key Abstractions +[The structural shapes this arrangement repeats — what is kept apart from what, and what wraps what.] + +#### {Pattern name} +[What the shape is, and where it shows up.] ### Core Types -[Primary types/structs/classes and their roles] + +[The types a reader meets first, and how they group.] + +| Type | Role | +|------|------| +| [type] | [what it is for, in prose] | ### Traits and Interfaces -[Key traits/interfaces, purposes, implementors] -### Data Model -[Core data structures, relationships, state management] +[What this area reaches the wider system through.] -### Error Handling -[Error types, propagation strategy, recovery patterns] +| Interface | Reached for | +|-----------|-------------| +| [interface] | [the capability it supplies] | -## Design Rationale +### Data Model -### {Decision Area} -- **Observation**: [What was observed] -- **Hypothesized rationale**: [Why this choice was likely made] -- **Trade-offs**: [What this optimizes for vs. sacrifices] -- **Implications for changes**: [How this affects modifications] +[The values the design turns on.] -[Repeat per significant design choice] +#### {Data shape} +[One subsection per distinct payload, store, or encoding.] -## Data Flow and Operational Context +## Behaviour + +[What the code does once it is running.] ### Data Flow Map -[Per function the work package modifies: producer → transformations → consumer; which module produces the input, what invariants the producer guarantees] + +[Where trust or authority enters, then a diagram of that at concept level.] + +#### {Path name} +[One subsection per producer→consumer route, each with its own diagram and its own prose.] + +### Design Patterns + +[The runtime shapes — what the area withholds, and how it selects a path from data.] + +#### {Pattern name} +[What the shape is, and when it governs.] ### Invariant Alignment -| Invariant | Producer Enforces? | Consumer Assumes? | Gap? | + +[What a producer/consumer disagreement costs in this area, and why the table below is where the safety argument is checkable.] + +| Invariant | Producer enforces? | Consumer assumes? | Gap? | |-----------|-------------------|-------------------|------| | [invariant] | [yes/no — cite code] | [yes/no] | [gap description if any] | ### Execution Context -[Dispatch class, error propagation path, failure consequences] + +[Dispatch class, thread model, failure consequences, and what an operator can see at default verbosity.] + +### Error Handling + +[How far a failure travels.] + +| Error type | Consumer reaction | +|------------|-------------------| +| [type] | [what the caller does] | + +### Resource Bounds + +[What keeps an untrusted or unlucky input from exhausting the machine.] + +#### {Declared limits} +[Table: constant, value, what it binds] + +#### {Enforcement} +[The caches, budgets or gates that apply those limits, and the scope each one actually covers.] + +#### {Peak cost} +[Table: site, live at peak, bounded by] ### Operational Scenarios -| Scenario | Effect on This Code Path | Risk | + +[How the path behaves in situations a real deployment meets.] + +| Scenario | Effect on this code path | Risk | |----------|------------------------|------| | Genesis / first invocation | [what happens] | [severity] | | Recovery after downtime | [what happens] | [severity] | | External system timing mismatch | [what happens] | [severity] | | External chain reorganization | [what happens] | [severity] | +## Inferred Design Rationale + +[State that rationale here is read out of the code, its comments and its structure; entries say so where the source documents a reason outright.] + +### {The choice, named as a decision taken} + +[What the code does, and the logic that makes it coherent.] + +[What the choice costs, and what it constrains about changing it.] + ## Domain Concept Mapping +[The bridge between the words people use and the constructs that implement them.] + ### Glossary -| Domain Term | Technical Construct | Description | + +| Domain term | Technical construct | Description | |-------------|-------------------|-------------| | [term] | [module/type/function] | [explanation] | ### Domain Model -[How domain concepts map to code structure] + +[How domain concepts map to code structure.] + +## References + +[Coverage: what this artifact covers, and the revision it was read at.] + +| Reference | What it carries | +|-----------|-----------------| +| [the comprehension log] | [the questions, investigations and open items behind this artifact] | +| [related corpus artifact] | [what that area supplies to this one] | + +| Contributing work package | Dates | +|---------------------------|-------| +| [work package] | [range] | +``` + +## Comprehension Log Template + +The session-local artifact. It holds the reasoning that produced the corpus artifact, and everything specific to this work package. + +```markdown +# Codebase Comprehension — {Codebase Area Name} + +> [work package] · {dates} · {status} · coverage: {what was read, at which revision} + +[What this file is: the questions this pass asked, the investigations that answered them, and +the items it left open. Name the corpus artifact that holds the settled facts.] ## Open Questions -[Unresolved questions, including producer-guarantee and operational-scenario questions] +[Which questions remain open, and why each carries forward as an input rather than a gap this +pass is expected to close.] + +| # | Question | Status | Resolution | Deep-Dive Section | +|---|----------|--------|------------|-------------------| +| [n] | [question] | Open / Resolved | [one-line answer, or — while open] | [link to the section that answered it] | ## Deep-Dive Sections -### {Area Name} — [YYYY-MM-DD] -[Targeted exploration findings: data flows, implementation details, edge cases] +### {Area Name} — {date} + +[Targeted exploration findings: traced data flows, implementation detail, edge cases.] + +## Challenge Lenses + +### {Perspective} — {date} + +[What the challenge pass surfaced from this angle.] + +## Follow-up items (out of scope) + +[Items this pass identified and deliberately left for later, each with what it would take to settle.] ``` +## Promotion + +The log is the working; the corpus artifact takes what survives. When a question resolves, its answer is written into the corpus section it belongs to, as a statement about the code, and the reasoning that reached it stays in the log. + +- **Promote** the settled outcome — a measured constant, a resolved gap in an invariant row, a correspondence confirmed against upstream source, a rationale the evidence supports. +- **Keep local** the derivation — the question, the alternatives weighed, the trace that closed it, the challenge-lens output, and anything scoped to this work package. +- A promoted fact reads as a property of the code, carrying no trace of the question that produced it. Where a reader needs the working, the corpus prose links to the log section that holds it. +- Facts persist; prose does not. A later pass may merge, restate, or drop corpus wording that a new fact supersedes, so long as no fact is lost. + ## Cross-Referencing -- Check whether other comprehension artifacts reference the same modules or types; add cross-references in the header line. +- Check whether other corpus artifacts cover the same modules or types; add cross-references as rows in the References section. - Note when understanding of one area depends on another. -- If the work package's problem spans multiple codebase areas, create or update a separate artifact per area and note the relationship. +- If the work package's problem spans multiple codebase areas, create or update a separate corpus artifact per area and note the relationship. One log covers the pass, however many areas it touched. ## Rules -- **Line budget:** ~150 lines per area. An area needing more is two areas. +These govern both the corpus artifact and the comprehension log. + +### explanatory-lead + +Every section and subsection opens with a paragraph saying what the reader is looking at and why it is there. A heading followed directly by a table or a list leaves the reader to infer the frame. + +### prose-over-symbols + +Paragraph prose names things in words and hyperlinks to their definition in the sentence flow. Code identifiers, expressions, field lists and enum variants live in tables, diagrams, fenced blocks and link targets — the surfaces built to carry them. + +### role-columns-in-prose + +Tables carry identifiers generally, under `prose-over-symbols`; a column describing what something is *for* is the exception and carries prose. Parameter lists, field names and variant names belong to the definition the row links to. + +### demonstratives-over-counts + +Introduce a list with a demonstrative rather than a tally — the count goes stale the moment an entry is added. Where a value matters, name the constant that holds it rather than the number it currently is. + +### subdivide-by-topic + +A section covering more than one topic splits into subsections, one per topic, each with its own lead. Subsection names carry their own meaning rather than repeating the parent's. + +### diagram-the-shape + +A section whose subject is a shape carries a diagram of it: module dependencies, the entry-point call chain, and each data-flow path. The diagram carries the shape and the prose carries the detail. + +### present-tense-facts + +The corpus artifact states what is, in the present tense. Narrative of how the code came to be this way, and comparison against what it used to do, belong to the change record. + +### line-budget + +Roughly 250 lines for a corpus artifact covering one area. An area needing materially more is two areas. diff --git a/work-package/techniques/analyse-challenge/run-loop.md b/work-package/techniques/analyse-challenge/run-loop.md index af7ff4e87..a0a84f373 100644 --- a/work-package/techniques/analyse-challenge/run-loop.md +++ b/work-package/techniques/analyse-challenge/run-loop.md @@ -54,7 +54,7 @@ The bound residue variable after combine. - When `{iteration_mode}` is `once`, run one analyse → challenge → combine cycle and proceed to handoff. - When `{iteration_mode}` is `until_converged` (default), repeat while the bag variable named by `{convergence_flag}` is true (or on the first pass when the flag is unset and open concerns exist): - 1. **Analyse** — invoke the bound `{analyse_technique}` technique with forwarded context (`{assumptions_log}`, `{target_path}`, `{comprehension_artifact}` as applicable). When `{concern_kind}` is `open_questions` and `{analyse_technique}` is `codebase-comprehension::deep-dive`, follow with [revise-questions](../codebase-comprehension/revise-questions.md) so Open Questions and `{needs_comprehension}` / `{has_open_questions}` stay current. Analyse updates the concern set and may set the convergence flag true when more agent-resolvable work remains. + 1. **Analyse** — invoke the bound `{analyse_technique}` technique with forwarded context (`{assumptions_log}`, `{target_path}`, `{comprehension_artifact}`, `{comprehension_log}` as applicable). When `{concern_kind}` is `open_questions` and `{analyse_technique}` is `codebase-comprehension::deep-dive`, follow with [revise-questions](../codebase-comprehension/revise-questions.md) so the log's Open Questions and `{needs_comprehension}` / `{has_open_questions}` stay current. Analyse updates the concern set and may set the convergence flag true when more agent-resolvable work remains. 2. **Challenge** — invoke [challenge](./challenge.md) with `{challenge_perspectives}` and the current concern set / log. Challenge fans out adversarially via scatter-gather and returns per-perspective findings without writing shared bag flags itself. 3. **Combine** — invoke [combine](./combine.md) to merge challenge findings into the concern set, resolve or reclassify items, and set `{convergence_flag}` / `{residue_flag}` (and `{residue_collection}` when bound). - Exit when `{convergence_flag}` is false after combine (`until_converged`), or after the single pass (`once`). An empty open set yields `{residue_flag}` false. diff --git a/work-package/techniques/codebase-comprehension/TECHNIQUE.md b/work-package/techniques/codebase-comprehension/TECHNIQUE.md index cc3c89cfc..f246a553c 100644 --- a/work-package/techniques/codebase-comprehension/TECHNIQUE.md +++ b/work-package/techniques/codebase-comprehension/TECHNIQUE.md @@ -1,11 +1,11 @@ --- metadata: - version: 1.2.0 + version: 2.0.0 --- ## Capability -Progressive codebase comprehension with persistent knowledge artifacts. +Progressive codebase comprehension over a cumulative corpus artifact and a session-local log. ## Inputs @@ -21,7 +21,7 @@ Absolute path of the cumulative comprehension corpus — the directory whose art ### comprehension_artifact -Persistent comprehension [artifact](../../resources/codebase-comprehension.md#artifact-template) covering the relevant codebase area +Cumulative [corpus artifact](../../resources/codebase-comprehension.md#corpus-artifact-template) covering the relevant codebase area #### comprehension_artifact_file @@ -43,6 +43,18 @@ Inferred rationale for significant design choices Mapping of domain terms to technical constructs +### comprehension_log + +Session-local [comprehension log](../../resources/codebase-comprehension.md#comprehension-log-template) holding the reasoning behind the corpus artifact + +#### comprehension_log_file + +`codebase-comprehension.md` + +#### open_questions + +Questions the pass opened, each resolved or carried forward + #### deep_dives Targeted exploration sections added during user-driven loop @@ -52,11 +64,7 @@ Targeted exploration sections added during user-driven loop ### persistent-artifacts -Comprehension artifacts persist across work packages — they are cumulative knowledge, not disposable planning documents - -### augment-not-replace - -When existing artifacts cover the same area, augment them with new sections and deeper detail rather than replacing +The corpus artifact persists across work packages as cumulative knowledge; the log belongs to the session that wrote it and carries what is specific to that pass ### progressive-depth @@ -72,4 +80,4 @@ Cross-reference related comprehension artifacts and note dependencies between co ### question-driven-exploration -The Open Questions table is the primary input for selecting deep-dive areas. When open questions exist, present them as the default selection for the next iteration rather than generating new candidate areas from scratch. +The log's Open Questions table is the primary input for selecting deep-dive areas. When open questions exist, present them as the default selection for the next iteration rather than generating new candidate areas from scratch. diff --git a/work-package/techniques/codebase-comprehension/deep-dive.md b/work-package/techniques/codebase-comprehension/deep-dive.md index e7db25edc..6cabb1e82 100644 --- a/work-package/techniques/codebase-comprehension/deep-dive.md +++ b/work-package/techniques/codebase-comprehension/deep-dive.md @@ -1,17 +1,21 @@ --- metadata: - version: 1.1.0 + version: 2.0.0 --- ## Capability -Targeted investigation of a selected codebase area into the comprehension artifact. +Targeted investigation of a selected codebase area, recorded in the comprehension log with its settled outcomes promoted to the corpus artifact. ## Inputs ### comprehension_artifact -The comprehension artifact whose architecture survey and existing Open Questions seed the candidate-area selection, and to which findings are appended. +The corpus artifact for this area, whose architecture survey seeds candidate-area selection. + +### comprehension_log + +*(optional)* The log from earlier passes over this area; its Open Questions are the default candidates for the next investigation. ### gitnexus_indexed @@ -21,7 +25,7 @@ Flag indicating whether the codebase is indexed; selects between gitnexus-operat ### comprehension_artifact -Updated comprehension artifact — written as `{codebase_area}.md` in `{comprehension_dir}`, augmenting prior content rather than replacing it. +The corpus artifact for the area, carrying the outcomes this investigation settled as statements about the code. #### artifact @@ -31,25 +35,38 @@ Updated comprehension artifact — written as `{codebase_area}.md` in `{comprehe `human` +### comprehension_log + +The session-local record of this investigation: the questions it worked, the findings that answered them, and the items it left open. + +#### artifact + +`codebase-comprehension.md` + +#### audience + +`human` + #### deep_dives -Targeted exploration subsections appended for the selected area: traced data flows, implementation details, and edge cases. +Targeted exploration findings for the selected area: traced data flows, implementation detail, and edge cases. ## Protocol ### 1. Deep Dive -- Emit candidate areas based on architecture survey and problem relevance as bindable output for the binding activity to surface. When open questions already exist in the artifact, prefer them as the default selection rather than generating new candidates from scratch (per `question-driven-exploration`). +- Emit candidate areas based on architecture survey and problem relevance as bindable output for the binding activity to surface. When open questions already exist in the log, prefer them as the default selection rather than generating new candidates from scratch (per `question-driven-exploration`). - On the mandatory initial pass, attempt to resolve every open question without a selection gate; only subsequent iterations consume an activity-selected area. -- For selected area: trace data flows, examine implementation details, document edge cases -- When GitNexus is available: apply [gitnexus-operations](../../../meta/techniques/gitnexus-operations/TECHNIQUE.md)::[context](../../../meta/techniques/gitnexus-operations/context.md) to trace callers/callees, read process resources for full execution traces, and [gitnexus-operations](../../../meta/techniques/gitnexus-operations/TECHNIQUE.md)::[cypher](../../../meta/techniques/gitnexus-operations/cypher.md) for custom call chain queries -- Append findings as dedicated subsections in the comprehension artifact +- For selected area: trace data flows, examine implementation details, document edge cases, applying the [Comprehension Techniques](../../resources/codebase-comprehension.md#comprehension-techniques) +- When `{gitnexus_indexed}` is true: apply [gitnexus-operations](../../../meta/techniques/gitnexus-operations/TECHNIQUE.md)::[context](../../../meta/techniques/gitnexus-operations/context.md) to trace callers/callees, read process resources for full execution traces, and [gitnexus-operations](../../../meta/techniques/gitnexus-operations/TECHNIQUE.md)::[cypher](../../../meta/techniques/gitnexus-operations/cypher.md) for custom call chain queries + +### 2. Record the Investigation + +- Write `{comprehension_log}` per the [Comprehension Log Template](../../resources/codebase-comprehension.md#comprehension-log-template) +- Record this investigation alongside the ones earlier passes wrote, rather than in place of them -### 2. Artifact Management +### 3. Promote Settled Outcomes -- Write the `{comprehension_artifact}` following the [Artifact Template](../../resources/codebase-comprehension.md#artifact-template) and the [Comprehension Techniques](../../resources/codebase-comprehension.md#comprehension-techniques) - Derive `{$codebase_area}` from the target project or subsystem name (slugified) -- Artifact naming: `{codebase_area}.md` in `{comprehension_dir}` -- When augmenting: add new sections, update existing sections with deeper detail, preserve prior content -- Include metadata header: date, work-package reference, coverage scope, related artifacts -- Include an 'Open Questions' section (markdown table) between Domain Concept Mapping and Deep-Dive Sections — this section is maintained by the [revise-questions](./revise-questions.md) question-management protocol +- Select which findings cross into `{comprehension_artifact}` per [Promotion](../../resources/codebase-comprehension.md#promotion) +- Write each promoted outcome into the section that owns it, per the [Corpus Artifact Template](../../resources/codebase-comprehension.md#corpus-artifact-template) and the fill [Rules](../../resources/codebase-comprehension.md#rules) diff --git a/work-package/techniques/codebase-comprehension/revise-questions.md b/work-package/techniques/codebase-comprehension/revise-questions.md index 05ba47d61..67150c882 100644 --- a/work-package/techniques/codebase-comprehension/revise-questions.md +++ b/work-package/techniques/codebase-comprehension/revise-questions.md @@ -1,30 +1,30 @@ --- metadata: - version: 1.0.0 + version: 2.0.0 --- ## Capability -Authoritative Open Questions on the comprehension artifact after a deep-dive. +Authoritative Open Questions on the comprehension log after a deep-dive. ## Inputs -### comprehension_artifact +### comprehension_log -The comprehension artifact whose 'Open Questions' section is revised; its existing table and the findings from the latest targeted investigation drive which questions are resolved and which are added. +The log whose Open Questions are revised; its existing table and the findings from the latest targeted investigation drive which questions are resolved and which are added. ## Outputs ### open_questions -The revised 'Open Questions' markdown table (columns: #, Question, Status, Resolution, Deep-Dive Section) in `{comprehension_artifact}` — resolved questions cross-referenced to the section that answered them, newly discovered questions added as 'Open', plus a 'Remaining follow-up items (out of scope)' list. This is the authoritative unresolved-question set; `{has_open_questions}` is true while any remain. +The revised Open Questions set in `{comprehension_log}` — resolved questions cross-referenced to the section that answered them, newly discovered questions added as open, and out-of-scope items listed separately. This is the authoritative unresolved-question set; `{has_open_questions}` is true while any remain. ## Protocol ### 1. Question Management -- The 'Open Questions' section is a markdown table with columns: #, Question, Status, Resolution, Deep-Dive Section -- After each deep-dive iteration, mark resolved questions as 'Resolved' with a one-line summary and cross-reference to the deep-dive section that answered them -- Add new questions discovered during investigation with status 'Open' — questions naturally emerge from tracing data flows, examining edge cases, and reading adjacent code -- Below the table, maintain a 'Remaining follow-up items (out of scope)' list for questions identified but explicitly out of scope for the current work package +- Revise the Open Questions table and the follow-up list in `{comprehension_log}`, in the shape the [Comprehension Log Template](../../resources/codebase-comprehension.md#comprehension-log-template) defines +- After each deep-dive iteration, mark resolved questions as resolved with a one-line summary and a cross-reference to the deep-dive section that answered them +- Add new questions discovered during investigation as open — questions naturally emerge from tracing data flows, examining edge cases, and reading adjacent code +- Record questions identified but out of scope for the current work package as follow-up items diff --git a/work-package/techniques/codebase-comprehension/survey.md b/work-package/techniques/codebase-comprehension/survey.md index b4cbe7112..ecf22282b 100644 --- a/work-package/techniques/codebase-comprehension/survey.md +++ b/work-package/techniques/codebase-comprehension/survey.md @@ -21,7 +21,7 @@ Whether `{host_repo_path}` has a usable GitNexus index; selects between gitnexus ### comprehension_survey -Initial survey of the codebase area, taking the shape and fill rules of the [Artifact Template](../../resources/codebase-comprehension.md#artifact-template). +Initial survey of the codebase area, taking the shape and fill rules of the [Corpus Artifact Template](../../resources/codebase-comprehension.md#corpus-artifact-template). #### architecture_overview @@ -33,7 +33,7 @@ Core types, traits/interfaces, and data structures forming the domain model, wit #### design_rationale -Inferred rationale for significant design choices and their trade-offs, framed as hypotheses for user validation. +Rationale inferred for each significant design choice, with the trade-off it carries and what it constrains in later changes. #### domain_glossary @@ -78,7 +78,8 @@ Mapping of domain-specific terms to the technical modules/constructs that implem - For each significant design choice, infer the likely rationale from context clues: comments, naming, structure, constraints - Identify trade-offs: what does this design optimize for? what does it sacrifice? -- Frame rationale as hypotheses for user validation — not assertions +- Identify what each choice constrains: which later changes it rules out, and which it makes cheap +- State each rationale as a property of the design, under the heading that marks the whole section as read out of the code rather than stated by its authors; where the source documents a reason outright, say so in the entry ### 6. Domain Mapping @@ -88,4 +89,4 @@ Mapping of domain-specific terms to the technical modules/constructs that implem ### 7. Assemble Survey -- Fold the architecture overview, key abstractions, design rationale, and domain glossary into `{comprehension_survey}`, in the section order the [Artifact Template](../../resources/codebase-comprehension.md#artifact-template) defines +- Fold the architecture overview, key abstractions, design rationale, and domain glossary into `{comprehension_survey}`, in the section order the [Corpus Artifact Template](../../resources/codebase-comprehension.md#corpus-artifact-template) defines diff --git a/work-package/techniques/review-assumptions/reconcile.md b/work-package/techniques/review-assumptions/reconcile.md index 31e98c833..2e6577ec8 100644 --- a/work-package/techniques/review-assumptions/reconcile.md +++ b/work-package/techniques/review-assumptions/reconcile.md @@ -11,7 +11,7 @@ Code-analyzable assumptions closed via targeted analysis; callable as the analys ### comprehension_artifact -*(optional)* Existing comprehension [artifact](../../resources/codebase-comprehension.md#artifact-template) to augment with findings. +*(optional)* Existing comprehension [corpus artifact](../../resources/codebase-comprehension.md#corpus-artifact-template) to augment with findings. ## Outputs