feat(constraints): a comment narrates as well as instructs

CHANGELOG.md

@@ -11,6 +11,28 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Changed
+- **A narrative use of `never`/`always` in a comment is no longer a constraint directive
+  (DECISIONS §52).** The constraint gate accounts for **29 of 29** code-bucket fallbacks on the
+  frozen corpus, and roughly twelve of those were sentences like *"the MCP branch has always read
+  these two"* or *"could never have worked"* — a codebase narrating its own history, not
+  instructing anyone. §42 scoped this gate by region (a comment, not an expression); this scopes
+  it by mood within that region.
+
+  Narrowed in the safe direction three ways: only `never`/`always` (so *"must have been called"*
+  is untouched), only provable perfect/past constructions (so *"is always"* keeps firing), and
+  only when *every* keyword in the segment is narrative.
+
+  **Per-row over 576 corpus rows: 572 byte-identical, 4 fallbacks fixed, 0 new, 0 files that
+  stopped reducing, and no change to any row that was already reducing.** TypeScript file route
+  39 → 43 reducing, 16 → 12 fallbacks.
+
+  **The gain is not portable, and the caveat is the point.** All four recovered files are this
+  repository's own source, which M11 measured as 32.8% comment prose written in a
+  what-used-to-be-true style. Python gained nothing. The 6pp on the TypeScript bucket is the
+  corpus-bias trap `CLAUDE.md` warns about, appearing as a favourable number — the harder
+  direction to notice.
+
 ## [v1.4.0] - 2026-08-12
 
 **The target adheres.** v1.3.0 made `--target-reduction-ratio` bind and recorded that adherence

DECISIONS.md

@@ -3613,3 +3613,83 @@ corpus run.**
 The constraint gate is where the fallbacks actually are: 29 of 29 code-bucket fallbacks, and
 §50 measured it as the reason the better sub-region setting costs two files. That is the
 per-item axis with something on it.
+
+---
+
+## 52. A Comment Is Where a Codebase Narrates Itself, Not Only Where It Instructs
+
+**Date:** 2026-08-12 · **Status:** Accepted · **Follows:** §42 (H6), §51
+
+§51 measured where the fallbacks actually are: **29 of 29** code-bucket fallbacks on the frozen
+corpus are `CONSTRAINT_DIRECTIVE_LOST`. Not drift, not AST — the constraint gate, every time.
+
+§42 scoped that gate by **region**: an instruction to a reader lives in a comment or a docstring,
+never in an expression, which stopped it firing on `logger.critical(exc)` and `readonly required?`.
+This scopes it by **mood** within that region, because a comment is also where a codebase explains
+its own history.
+
+### What the gate was refusing
+
+Inspecting the matched text behind those 29 fallbacks, roughly twelve read like this:
+
+> "The MCP branch of `runCli` **has always** read these two"
+> "It **never did**: this branch bypassed pruning entirely"
+> "`HTTP_PROXY` and `HTTPS_PROXY` … could **never have worked**"
+> "a saving that **never reached** the wire (audit C4)"
+
+Losing one of those costs a reader some history. Losing *"never hash items matching
+preserveKinds"* costs a caller a rule. The keyword is identical and the gate refused the whole
+file for either.
+
+### Narrowed in the safe direction, three ways
+
+This gate protects content, so unlike §48 and §50 — where a wrong call costs output size — a
+wrong call here **silently deletes an instruction**. Three deliberate limits:
+
+1. **Only `never` and `always`.** They are the two keywords equally comfortable describing and
+   instructing. `must`, `must not`, `do not`, `required`, `critical`, `only if`, `except when`
+   and `make sure to` are untouched — *"must have been called before"* is a requirement about a
+   past state, and a perfect-tense test applied to `must` would drop it.
+2. **Only perfect or past constructions**, which are provable from the words present: a preceding
+   `have`/`has`/`had`, or a following past-tense verb. *"is always deterministic"* and *"do not
+   support"* are descriptive too, and are **left firing on purpose** — there the line between
+   describing a constraint and stating one is genuinely blurry. Under-narrowing costs reduction;
+   over-narrowing costs content.
+3. **Unanimity before dropping.** A segment is discarded only if *every* keyword in it is
+   narrative-capable and the construction is narrative. *"this has always been true, so you must
+   call it first"* still raises a directive, because of the `must`.
+
+### The negative control is the load-bearing test
+
+`test/unit/narrative-directive-scope.test.ts` asserts the eight narrative sentences above stop
+firing — and, more importantly, that **sixteen real instructions still do**, several taken
+verbatim from this repository. A rule that drops one of those is not a better rule at any
+reduction figure, and the test says so rather than leaving it to a reduction number two layers
+away.
+
+### Measured
+
+Per-row over the frozen corpus, 576 rows, both routes, target 0.3, against v1.4.0:
+
+| | before | after |
+|---|---|---|
+| rows byte-identical | — | **572 of 576** |
+| fallbacks fixed / new | — | **4 / 0** |
+| rows newly reducing / stopped | — | **4 / 0** |
+| rows already reducing that changed at all | — | **0** |
+
+The four are `adapters/mcp/tools.ts`, `cli/main.ts`, `gateway/exec.ts` and `gateway/proxy.ts` —
+`gateway/exec.ts` being the file that contains *"could never have worked"*. TypeScript file
+route: 39 reduced / 16 fallbacks → **43 / 12**, aggregate 18.52% → 24.58%.
+
+### The caveat that matters more than the headline
+
+**Python gained nothing — 0 of the 4.** Every recovered file is this repository's own source,
+and this repository is unusually narrative: M11 measured **32.8%** of `src/` as comment prose,
+written in a style that explains what used to be true. pip's comments describe behaviour in the
+present tense and were never caught by this rule.
+
+So the 6pp on the TypeScript bucket is **not** a portable estimate of what other codebases gain.
+It is the corpus-bias trap `CLAUDE.md` warns about, showing up as a favourable number instead of
+an unfavourable one — which is the harder direction to notice. What is portable is the shape:
+zero regressions, and four files that produced nothing now produce something.

docs/audit-remediation-status.md

@@ -3,9 +3,14 @@
 Working state for the `max_audit.md` remediation. **Read this before picking up audit work**;
 it records what is done, what is measured, and what the next batch actually requires.
 
-Last updated 2026-08-12, after **v1.4.0** — §50 (sub-region elision; the target adheres), on top
-of v1.3.0’s §48 and §49. Suite: **628 passing**, typecheck and lint clean. `npm run format` no
-longer exists; `lint` is the enforced style gate.
+Last updated 2026-08-12, after **v1.4.0** (§50, sub-region elision) **plus two unreleased changes
+on `main`**: §51 (per-item drift, closed without implementing) and §52 (a narrative `never`/
+`always` is not a directive). Suite: **657 passing**, typecheck and lint clean. `npm run format`
+no longer exists; `lint` is the enforced style gate.
+
+**§2's baseline is the v1.4.0 engine and does not include §52.** On the same frozen corpus §52
+takes the TypeScript file row to 43 reduced / 12 fallbacks / 24.58%; python is unchanged. The
+table is left at the released figures on purpose — re-point it when the next release ships.
 
 ---
 
@@ -353,8 +358,19 @@ Ordered by measured value ÷ risk. Preconditions verified as holding — as dist
    `0.4122` in §47 that motivated it came from an engine that elided everything it could. A
    token ceiling and a statement-sized unit cut symbol loss two orders of magnitude below the
    gate. **An open item is a claim about the current build, and it expires like any other.**
-5. **Make the constraint gate finer.** Where the fallbacks actually are: **29 of 29** code-bucket
-   fallbacks are `CONSTRAINT_DIRECTIVE_LOST`, and §50 measured the same gate as the reason the
-   better sub-region setting costs two working files. This is the per-item axis with something
-   on it — the one §51 was looking for on the wrong metric.
+5. ~~**Make the constraint gate finer.**~~ **First pass done — DECISIONS §52.** A narrative use of
+   `never`/`always` in a comment is no longer a directive: **4 fallbacks fixed, 0 new, 572 of 576
+   rows byte-identical**, TypeScript file route 39 → 43 reducing.
+
+   **Read §52's caveat before quoting the 6pp.** All four recovered files are this repository's
+   own source, which M11 measured as 32.8% comment prose written in a what-used-to-be-true style.
+   **Python gained zero.** That is the corpus-bias trap of §4 appearing as a *favourable* number,
+   which is the harder direction to notice.
+
+   **Still open on this axis**, and deliberately not attempted: descriptive present-tense uses
+   (*"is always deterministic"*, *"do not support"*) still raise directives. They are descriptive,
+   but the line between describing a constraint and stating one is blurry there, and this gate
+   protects content — over-narrowing deletes an instruction, which no reduction figure buys back.
+   §50's other open item is the same gate: it is why the better sub-region coverage setting costs
+   two working files.
 6. **M7**, per §6 — small, and the only thing between the audit and an empty list.

src/core/constraints/directives.ts

@@ -94,6 +94,67 @@ export function extractProseRegions(content: string, contentType: ContentType):
  */
 export const IMPERATIVE_DIRECTIVE_REGEX = new RegExp(IMPERATIVE_KEYWORD_SOURCE, 'i');
 
+/**
+ * `never` and `always` in a **narrative** construction — a report about what happened, not an
+ * instruction about what must happen.
+ *
+ * H6 (§42) scoped this check by *region*: an instruction lives in a comment, never in an
+ * expression. This scopes it by *mood* within that region, because a comment is also where a
+ * codebase explains itself. Measured over the code buckets of the frozen corpus, **29 of 29**
+ * fallbacks are `CONSTRAINT_DIRECTIVE_LOST`, and inspecting the matched text, roughly 12 are
+ * sentences like these:
+ *
+ *   "The MCP branch of `runCli` has always read these two"
+ *   "It never did: this branch bypassed pruning entirely"
+ *   "`HTTP_PROXY` ... could never have worked"
+ *   "a saving that never reached the wire (audit C4)"
+ *
+ * Losing one of those costs a reader some history. Losing "never hash items matching
+ * preserveKinds" costs a caller a rule. The gate cannot tell them apart by keyword, and it
+ * currently refuses the whole file for either.
+ *
+ * ### Deliberately narrow, in the safe direction
+ *
+ * Applied **only to `never` and `always`** — the two keywords that are as comfortable describing
+ * as instructing. `must`, `must not`, `do not`, `required`, `critical`, `only if`, `except when`
+ * and `make sure to` are untouched, because "must have been called before" is a requirement
+ * about a past state, not a narrative. Excluding those on a perfect-tense test would drop real
+ * constraints, which is the failure this must not have.
+ *
+ * And only for *perfect or past* constructions, which are provable from the words present:
+ * a preceding `have`/`has`/`had`, or a following past-tense verb. `"is always deterministic"`
+ * and `"do not support"` are descriptive too, and are deliberately **left firing** — the line
+ * between describing a constraint and stating one is blurry there, and a wrong call silently
+ * deletes a real directive. Under-narrowing costs reduction; over-narrowing costs content.
+ */
+const NARRATIVE_KEYWORDS = String.raw`(?:never|always)`;
+
+/** Irregular past tenses that no `-ed` rule will catch. */
+const PAST_TENSE_IRREGULARS = String.raw`(?:did|was|were|had|got|made|took|ran|read|said|meant|came|went|saw|knew|found|left|felt|kept|held|told|became|began|broke|brought|built|chose|drew|drove|fell|flew|gave|grew|hit|kept|knew|lay|led|lost|met|paid|put|sent|set|shot|shut|sold|spent|stood|struck|swore|threw|understood|withdrew|wrote|worked|wanted|needed|existed|reached|shipped|fired|failed|passed|stopped|started|used|meant)`;
+
+/**
+ * Matches a narrative use: `has always read`, `never did`, `could never have worked`,
+ * `never reached the wire`.
+ */
+const NARRATIVE_DIRECTIVE_REGEX = new RegExp(
+  String.raw`(?:\b(?:have|has|had)\s+(?:\w+\s+){0,2}${NARRATIVE_KEYWORDS}\b)` +
+    `|` +
+    String.raw`(?:\b${NARRATIVE_KEYWORDS}\s+(?:have|has|had)\b)` +
+    `|` +
+    String.raw`(?:\b${NARRATIVE_KEYWORDS}\s+(?:\w+ed|${PAST_TENSE_IRREGULARS})\b)`,
+  'i',
+);
+
+/**
+ * Whether this keyword occurrence is narrative rather than imperative.
+ *
+ * Exported so the characterization tests can assert the boundary directly rather than inferring
+ * it from a reduction figure two layers away.
+ */
+export function isNarrativeUse(segment: string): boolean {
+  return NARRATIVE_DIRECTIVE_REGEX.test(segment);
+}
+
 /**
  * Returns whether content contains a natural-language imperative constraint.
  */
@@ -119,6 +180,16 @@ export function extractImperativeDirectives(content: string): {
         continue;
       }
 
+      // A segment is narrative only if **every** keyword in it is one of the two that can
+      // describe as well as instruct, and the construction is perfect or past. One `must`
+      // anywhere keeps the whole segment, so "this has always been true, so you must call it
+      // first" is still a directive. Requiring unanimity is what makes the narrowing safe:
+      // the mixed case resolves toward firing.
+      const allNarrativeCapable = matches.every((match) => /^(?:never|always)$/i.test(match[0]));
+      if (allNarrativeCapable && isNarrativeUse(segment)) {
+        continue;
+      }
+
       directives.push(segment);
       for (const match of matches) {
         const keyword = normalizeKeyword(match[0]);

test/unit/narrative-directive-scope.test.ts

@@ -0,0 +1,102 @@
+import { describe, expect, it } from 'vitest';
+import { extractConstraintDirectives } from '../../src/stages/cleanup/constraint-preservation';
+import { isNarrativeUse } from '../../src/core/constraints/directives';
+
+/**
+ * The constraint gate refuses to lose an imperative, and it is the cause of **29 of 29**
+ * code-bucket fallbacks on the frozen corpus. H6 (§42) scoped it by region — an instruction
+ * lives in a comment, not an expression. This scopes it by *mood* within a comment, because a
+ * comment is also where a codebase narrates its own history.
+ *
+ * **The load-bearing half of this file is the negative control.** Narrowing a gate that protects
+ * content can only be justified if the things it protects still fire, so every sentence below
+ * that carries a real instruction is asserted to survive the change. A rule that drops one of
+ * those is not a better rule at any reduction figure.
+ */
+
+const directivesFor = (line: string): ReadonlyArray<string> =>
+  extractConstraintDirectives(`// ${line}`, 'code').directives;
+
+describe('narrative uses of never/always are not directives', () => {
+  // Verbatim from the corpus fallbacks these were measured on.
+  const narrative = [
+    'The MCP branch of `runCli` has always read these two',
+    'It never did: this branch bypassed pruning entirely whenever only a ratio was set',
+    '`HTTP_PROXY` and `HTTPS_PROXY` used to be set here too, and could never have worked',
+    'a saving that never reached the wire (audit C4)',
+    '`findUnwitnessedItems` has always exempted an item absent from `after`',
+    'That is invariant 10 shape — a clean result from something that never looked',
+    'this had always been the behaviour before the fix landed',
+    'the emitter and the matcher never agreed',
+  ];
+
+  for (const line of narrative) {
+    it(`does not raise a directive for: ${line.slice(0, 48)}…`, () => {
+      expect(isNarrativeUse(line)).toBe(true);
+      expect(directivesFor(line)).toHaveLength(0);
+    });
+  }
+});
+
+describe('the negative control: real instructions still fire', () => {
+  // If any of these stops producing a directive, the narrowing has cost content and must be
+  // reverted rather than tuned. Several are taken verbatim from this repository's own source.
+  const instructions = [
+    'Rule 1: Never hash items matching preserveKinds in OptimizationBudget',
+    'never call this twice',
+    'Installations or downloads using dist restrictions must not combine',
+    'get_not_required must be called firstly in order to find and',
+    'unknown string must not throw here, because this sits inside the fail-open path',
+    'Do not canonicalize this value with e',
+    'This must be done in a second pass, as the pyproject metadata is not yet known',
+    'File must have a valid wheel or sdist name',
+    'The backend must build a fresh instance representing',
+    'Both sides must measure the same kind of thing, or the ratio is meaningless',
+    'always pass the ledger explicitly, or turn 2 falls back',
+    'this must have been called before the stage runs',
+    'never elide an item carrying structured content',
+    'make sure to freeze the metadata before returning it',
+    'only if the candidate re-validates may it be adopted',
+    'except when the item is pinned, in which case it bypasses the knapsack',
+  ];
+
+  for (const line of instructions) {
+    it(`still raises a directive for: ${line.slice(0, 48)}…`, () => {
+      expect(directivesFor(line)).toHaveLength(1);
+    });
+  }
+});
+
+describe('the narrowing is scoped to never/always only', () => {
+  it('leaves `must` in a perfect construction alone — a requirement about a past state', () => {
+    // "must have been" is an instruction, not a narrative. Applying the perfect-tense test to
+    // `must` would drop it, which is exactly the failure this scoping exists to prevent.
+    expect(directivesFor('the ledger must have been created per request')).toHaveLength(1);
+  });
+
+  it('keeps a segment where one keyword is narrative and another instructs', () => {
+    const line = 'this has always been true, so you must call it first';
+    expect(isNarrativeUse(line)).toBe(true);
+    // Unanimity is required before dropping: the `must` keeps it.
+    expect(directivesFor(line)).toHaveLength(1);
+  });
+
+  it('leaves descriptive present-tense uses firing, deliberately', () => {
+    // "is always" and "do not support" describe behaviour rather than instruct, but the line
+    // between describing a constraint and stating one is blurry, and a wrong call here deletes
+    // a real directive. Under-narrowing costs reduction; over-narrowing costs content.
+    expect(directivesFor('the spinner is always non-interactive under logging')).toHaveLength(1);
+    expect(directivesFor('check for VCS schemes that do not support lookup')).toHaveLength(1);
+  });
+});
+
+describe('the scoping from H6 still holds', () => {
+  it('does not raise a directive from an expression, narrative or not', () => {
+    expect(extractConstraintDirectives('logger.critical(exc)', 'code').directives).toHaveLength(0);
+    expect(extractConstraintDirectives('const required = true;', 'code').directives).toHaveLength(0);
+  });
+
+  it('still reads prose content in full', () => {
+    expect(extractConstraintDirectives('You must not delete the manifest.', 'text').directives).toHaveLength(1);
+  });
+});