Skip to content

✨ feat(build): decisions D1-D15, plus cacheable agentic and long-tail rules - #228

Closed
roninjin10 wants to merge 990 commits into
mainfrom
buildsys/bazel-parity
Closed

✨ feat(build): decisions D1-D15, plus cacheable agentic and long-tail rules#228
roninjin10 wants to merge 990 commits into
mainfrom
buildsys/bazel-parity

Conversation

@roninjin10

Copy link
Copy Markdown
Contributor

Fifteen decisions (D1–D15) from a side-by-side comparison of this repository wired with Nx (#226), Turborepo (#225), and Bazel (#227), plus the first two waves of the follow-on work that makes agentic and long-tail tasks expressible as targets.

27 lanes, 47 commits, 142 files, +20,325 / −963.

The number that motivated it

smthrs ci "//..." planned 278 targets, of which 232 (83%) could not replay. DocsParity was the only rule that cached across the workspace; TsBuild, Typecheck, and Vitest declared no cache key at all and fell through to a false default.

Flipping that default alone would have been worse than leaving it. The cache stored a JSON success envelope — {key, target, label, exitOk, output, storedAt} — and captureOutputs digested declared outputs rather than storing them. A hit on a clean tree would have marked TsBuild successful with no dist/ on disk. So the content-addressed store landed first, and the flip came after.

What changed

Configuration split. WORKSPACE.ts declares what exists; BUILD.ts declares targets — the MODULE.bazel / BUILD.bazel shape. The toolchain registers once, so no rule threads packageManager as an attr and no package BUILD.ts imports ../../BUILD.ts.

Execution is confined. Actions run against a projected copy of their declared inputs, with one shared environment allowlist. sandbox: false and --dangerously-no-sandbox are the escape hatches, and neither disables caching — an un-sandboxed target's input list becomes a promise rather than a guarantee, which is how Bazel treats no-sandbox too.

Caching is on by default, opt out by exception. The exceptions keep their stated reasons; the wasm reproducibility gate stays uncached because the rebuild is the check.

Visibility is enforced at plan time, at package and folder level, with tiers derived from the existing smthrs.group manifest field rather than restated.

Outputs are content-addressed. A cache hit restores declared outputs from the store, confined to the workspace, and a missing blob is a miss rather than a hit.

npm packages are targets derived from pnpm-lock.yaml, so an action's inputs name the packages it uses instead of an ambient node_modules.

Consumers key on producer output bytes (Input.Produced), not on the producer's target key. Without this an uncached producer emitting different bytes leaves its consumer replaying a verdict about an artifact that no longer exists.

New rules for the surface both repos actually have: BunTest, BunTestCoverage, NodeTest, ScriptCheck, GeneratedArtifact. ScriptCheck is structurally load-bearing — ToolBuild is kinds: ["build"] and requires declared outputs to exist, so before this there was no way to declare a lint- or test-kind arbitrary command at all.

Bugs found by running the code, not reading it

A mapping pass verified every premise before implementation and found 16 wrong and 15 unbuildable as written (docs/architecture/build-system/corrections.md). Three mattered:

  • Nested BUILD.ts silently pruned its parent's inputs. Adding pkg/src/internal/BUILD.ts removed pkg/src/internal/b.ts from //pkg:lib's declared inputs while the parent's tsc -b still compiled that subtree. Harmless while nothing cached; a stale-green generator the moment D5 flipped.
  • The baseline was already red. GeneratedRootFiles.test.ts hand-copies root BUILD.ts attrs and had drifted.
  • //:ci never ran in CI. GithubCiGen kinds were [build, lint] while ci.yml ran the docs verb and ci //packages/....

Deliberately not done

  • Changesets — there is no .changeset/ and no @changesets/* dependency to build on. Internal range retargeting on bump landed, which is the part that carries the value; adopting the package did not.
  • GithubCiGen write mode — the renderer models no needs, permissions, matrix, or step-level if, and would drop 60+ explanatory comments from ci.yml. Contract mode stays and now covers the full gate surface instead.

Both are recorded in docs/architecture/build-system/corrections.md rather than left as silent omissions.

Measured

smthrs build "//packages/..." --jobs 4, 90 targets:

wall detail
cold, cache cleared, every dist/ removed 58.25s 391s user across 4 jobs
warm, nothing changed 2.19s 90/90 cache hits, 0 re-runs

Per-target digests are byte-identical between the two runs, so the key is stable across processes — the property that makes the cache shareable rather than merely local.

Verification

  • pnpm run check across every workspace: clean.
  • Design record in docs/architecture/build-system/ — the decisions, the corrections, and eight open design questions (six ruled on, two left to the maintainer).

🤖 Generated with Claude Code

roninjin10 and others added 30 commits August 15, 2026 10:17
… refusals

Graph.build recursed twice over author-controlled input: visit recursed per
combinator, so ten thousand chained AndThens overflowed the native stack, and
hydrate/literal/references recursed per payload level, so a cyclic payload
recursed forever and a deep one overflowed. Both walks are now explicit
stacks — an operation stack for topology, frame stacks for payloads — that
preserve the recursive walk's node, edge, diagnostic, and input ordering
exactly. Depth becomes policy instead of a crash: topology past 1000 levels
throws graph_too_deep, a payload past 1000 levels throws payload_too_deep,
and a payload on its own ancestor chain throws cyclic_payload, each naming
the node and the fix. The DECIDED markers on drafts, per-branch and
per-catch subject tokens, and unconstrained-caller placement are untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
…lifetime

Effect.provide forks the enclosing memo map, so a nested provision reuses the
memoized Implementations table instead of building its own — and a nested
toLayer of an already-filed tag permanently replaced the enclosing entry.
add now registers for the registering scope's lifetime: it files the
implementation and restores what it replaced when that scope closes, with
finalizer LIFO unwinding stacked same-tag replacements to exactly what each
one replaced. A sibling scope that already superseded the entry is left in
place; only the currently filed implementation restores its predecessor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Grant patterns compiled to RegExps with every `*` as `.*`, so a
repeated-star pattern such as `a*a*a*a*b` backtracked exponentially
against long non-matching resources — a ReDoS on the authorization
path. Replace the compilation with an iterative two-pointer glob walk
over UTF-16 code units that remembers only the most recent `*`,
bounding every match at O(pattern x resource) with constant memory.

Semantics are preserved exactly: `*` crosses separators and newlines,
`?` matches one code unit, a trailing ` *` also grants the bare
resource, and Windows-path case folding reproduces the ECMA-262
non-Unicode `i`-flag canonicalization per code unit. A new property
test pins agreement with the retired RegExp compilation as the oracle
on non-adversarial inputs, both pinned ReDoS defects are flipped to
passing regressions, and the wall-time pin now covers patterns up to
twelve stars against 4096-character resources.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The sync server's `canFollow` returned true for every non-branch run
and `SyncAuth` was a documented no-op, so any connection that reached
the RPC endpoint could read every engine run in the workspace journal.
Close the launch-list gap fail-closed:

- `SyncPrincipal`: the request's authenticated identity as a
  `Context.Reference` defaulting to anonymous. Non-branch runs and
  workspace listings are refused for anonymous callers; a run-scoped
  read of an engine run without the workspace principal fails
  `unauthorized`, and a branch share link never upgrades a caller to
  workspace listings.
- `WorkspaceShare`: the branch share-capability scheme extended, not
  paralleled — the same length-prefixed HMAC-SHA-256 claims shape with
  a signed `kid` for key rotation and a domain-separation label, over
  a `Redacted` keyring. `layerConfig` provisions the secret from
  `FLOWS_SYNC_SECRET`/`FLOWS_SYNC_KEY_ID`; there is no default secret
  and no plaintext keyring type.
- `SyncAuth.layer`: the production middleware behind
  `SyncRpcs.SyncAuth`. It verifies the workspace capability presented
  in the `flows-sync-workspace` request header and installs the
  principal; a missing header runs the request as anonymous, while a
  malformed, forged, expired, or unknown-kid header refuses the
  request outright. `SyncAuth.layerClient` stamps the header on every
  outgoing client request. Verifier infrastructure faults propagate as
  their own error, never as `unauthorized`.
- `SyncServer.layerHandlers`: the serve seam mirroring
  `BranchServer.layerHandlers`, so the RPC wiring and its middleware
  requirement live in src instead of being re-derived per test.
- Shared HMAC primitives move to `internal/shareSigner.ts`;
  `BranchShare` delegates to them unchanged.

Tests pin the invariant end to end over the RPC machinery: an
unauthenticated workspace read and subscribe are refused, a verified
header grants them, tampered/expired/malformed headers are refused,
and a branch capability still reads exactly its branch anonymously.
The README claim that every operation authorizes through a signed
capability is corrected to describe both authorization boundaries.
Sync coverage stays at the enforced 100%.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The API deep-dive still said `SyncAuth` is middleware a deployment
implements. Point it at the shipped `SyncAuth.layer`, and list the new
`WorkspaceShare` and `SyncPrincipal` namespaces alongside the server's
policy constructors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Grant-event structs silently stripped unknown keys, so a journal payload
mixing envelope and request-only fields decoded as whichever variant its
declared keys satisfied. Replayed payloads become active permission
authority, so decode now runs with onExcessProperty: "error" and treats
any excess key as corruption.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Envelope grants are now a canonical predicate set: duplicate patterns
collapse and the survivors sort by their formatted identity before the
event persists, so repeated or reordered envelopes produce one durable
approval. The store tracks envelope signatures (seeded from journal
replay) and treats a repeated grantEnvelope as a no-op instead of new
durable evidence. JournalGrantStore serializes the construction-envelope
check-then-append through a per-journal critical section and re-replays
inside it, so two concurrent constructors append exactly one envelope.

Fail-closed hardening in the same seam:
- reply refuses a runtime-invalid resolution with a typed
  invalid_resolution error instead of silently succeeding and stranding
  the request's waiter on its Deferred.
- grantEnvelope and the construction envelope refuse a runtime-invalid
  scope with a typed store error before it can reach the event schema
  constructor and die as a defect.
- Replay refuses a journal page that repeats its last sequence with
  hasMore instead of requesting the same page forever; a non-advancing
  cursor is corruption, not progress.

RFC 8785 canonical JSON (@smthrs/canonical-next) was considered for the
signature and does not fit: it canonicalizes object keys but preserves
array order as semantic, and the pattern-array order is exactly what has
to be normalized here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two Node-version-sensitive defects made three suites fail on CI (node
22.19) while passing on newer local Nodes. First, label reconciliation
across tsx module namespaces required implementationDigest equality, but
the digest hashes transpiled function text and the CommonJS and ESM
pipelines annotate their emit differently, so the same declaration in the
same file failed to reconcile; the match is now rule identity plus
canonical attrs in the declaring file, which is the declaration identity a
label actually needs. Second, a BUILD.ts evaluated through the CommonJS
bridge compiles an import of a file: URL to require("file://..."), which
Node 22's CommonJS resolver rejects; _resolveFilename now converts a file:
request to its path first, so URL imports behave identically in both
formats on every supported Node. The shim's format documentation now
states when the ESM override actually applies.

Verified: the three previously red suites (Execute, CacheTrust,
Filegroup) and the full 289-test package suite pass under node 22.19.0
and 24.18.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
worker-e2e + live-check + serve:local follow wrangler.jsonc to apps/server
(spawn cwd set); vite gets --configLoader runner so the TS-source
smithers-shared package resolves from vite.config; deps.test.ts closure guard
re-pointed from the deleted vendor tree to its real invariants (no path
specifiers, single effect instance by realpath); AgentTurnPolicy byte-count
expectation corrected (13, not 14). 465/465 tests, vite build emits dist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Schema.TaggedErrorClass -> Schema.TaggedError<Self>() (journal idiom, cleared
105 of 122 errors); inline visibleText fold (workspace @smthrs/model dropped
the export the vendored copy had — restoring it upstream is the right end
state); restored-test type calibration under the new tsconfig.test.json;
JSDoc conformance incl. prompts.mjs generator so the sync test stays true.
188 tests, 100% coverage, 0 lint errors, no identity-string changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…m scripts

StandardPackage now emits six targets: the new fmt target runs the
package's dprint check through a new Dprint rule, and the new check target
runs tsc over tsconfig.test.json after lib builds, so lint+fmt cover what
the package lint scripts cover and lib+check cover what the check scripts
cover. tsflows ci //packages/... plans 130 targets across all 26 packages
and is now gate-equivalent to the recursive pnpm scripts at the package
level. tsflows-cli and tsflows-rules gain the tsconfig.test.json the
convention assumes; the four hand-written BUILD.ts files export the new
targets, with flow's desugared form kept equivalent to the macro.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The wasm-repro job first executed today, once the vendor/jj checkout landed,
and its byte-compare failed: the committed artifact was built on
aarch64-apple-darwin and CI builds on x86_64-unknown-linux-gnu.

No path escaped the remapping. Cargo compiles build scripts for the host, so
the `-C metadata` hash of every crate that has one — and of everything above
it, which reaches jj-lib and flows-jj — carries the host triple. rustc folds
`-C metadata` into the crate's StableCrateId, which seeds every symbol hash,
and it orders codegen items by symbol name, so a different host reorders the
module and LLVM optimizes it differently. Measured from identical sources on
1.89.0: aarch64-apple-darwin gives 4740406 bytes and 9721 functions,
x86_64-unknown-linux-gnu 4740823 and 9720, aarch64-unknown-linux-gnu 4739906.
The target directory is innocent — in-tree and out-of-tree CARGO_TARGET_DIR
builds are byte-identical on both hosts — so remapping it would have fixed
nothing.

Declare the host instead. build-wasm.mjs reads the `host` line of `rustc -vV`
and refuses to build the committed artifact anywhere but
x86_64-unknown-linux-gnu, printing the container command that produces those
bytes on any machine: the steps CI runs, one layer down. The header, the job
comment, the toolchain pin, and the package README carry the reason.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The committed module was the aarch64-apple-darwin build, which the wasm-repro
job cannot reproduce now that it runs. Replace it with the
x86_64-unknown-linux-gnu build of the same sources, produced by the container
command build-wasm.mjs prints.

Three independent x86_64 Linux builds of this commit agree on
sha256 3a14db0e, across two container images, two mount paths, and both
CARGO_TARGET_DIR layouts. The package's real-artifact contract suite passes
on the new module.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
`flows_step_cache` stays the mutable content-addressed head, but each put
now also lands its entry in `flows_step_cache_recorded` under
`(key_digest, recorded_run_id, recorded_event_seq)`, and nothing ever
deletes a ledger row. `get` grows an optional `recordedBy` fence that
answers from the ledger first and falls back to the head, so a replay
naming the exact recording event reads the bytes that were durable then —
however the head has been evicted or replaced since. Eviction keeps
protecting future executions; it no longer rewrites recorded history.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two replay defects let the derived projection drift from the journal:

- Sealed results were read off the mutable cache head, so evicting or
  replacing an entry between lifetimes changed an old frame's projection.
  Replay now reads through the step cache's `recordedBy` provenance fence:
  the version the replayed record landed answers first, and only an entry
  recorded elsewhere falls back to the shared head.
- The fold consumed journal pages verbatim, so a malformed reader that
  handed back duplicate or out-of-order pages produced a different state.
  The prefix is now normalized before folding — one record per coordinate,
  ordered by seq — because the projection is a function of the run's
  records, never of how a reader paged them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The public rewind claimed ownership, wrote its audit row, and refreshed
anchors before checking what it was asked to do: a non-positive or
non-finite pageSize reached paging after the claim, and a frame that was
not on the run's lineage at all mutated durable state before failing.

The rewind protocol gains a validation phase that runs before anything
durable. A malformed pageSize is refused `invalid` (a new TimeTravelError
code) before the first journal read. A frame is refused `not_found`
unless it addresses the run's history: not past the journal tail, on the
run's current lineage, and — frame zero excepted, the one frame that is
always addressable — backed by a record of that lineage at the exact
coordinate. A refused position leaves no claim, no audit, no anchor
refresh, and no store write behind.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
`migrate` piped its idempotent `ADD COLUMN plan_digest` widening through
`Effect.ignore`, which swallowed every ALTER TABLE failure — a view
squatting on the table name, a locked or corrupt database — not just the
benign column-already-exists error the ignore was written for. The
migration now absorbs exactly the duplicate-column report (SQLite's
`duplicate column name`, Postgres' `column ... already exists`, matched
through the wrapped failure's message chain) and surfaces everything else.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
…vered rewind

Recovery decided "the archive committed" from an absence: an interrupted
rewind whose live suffix rows were gone was declared complete even when
the archive held nothing — a corrupted journal missing rows on both sides
recovered as a successful rewind.

`TimeTravelStore` gains `archivedAt(runId, seq)`, and recovery now
requires the audit's recorded suffix tail to actually be in the archive
before finishing the suspended transition. An empty live suffix with no
archive row — or an audit that recorded a suffix but no tail coordinate —
rolls back instead of completing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Only irreversible dispatches crossed the journaled effect boundary; a
compensable action recorded its pre-image snapshot but no boundary row.
Rewind classifies the doomed suffix by exactly those rows, so rewinding
past a compensable action archived the suffix as "completed" without ever
invoking the tier-2 restore — the tree kept the discarded future's bytes
while the journal said otherwise.

Compensable dispatches now emit the same intended/terminal boundary
records, naming the attempt's anchored `changeId`, so a real rewind
restores the workspace to the frame's recorded jj pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The Unicode well-formedness pass walked the input value, so a lone
surrogate returned by a toJSON during serialization bypassed it and was
emitted instead of refused. Move the refusal into the serializer itself,
at the two points a string is written (values and property names), which
covers every string in the output including toJSON-minted ones, and
delete the now-redundant pre-pass module.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
KeyValue's pattern was anchored to key1_, so a stored key2_ key — which
the module docblock promises remains decodable — was refused. Accept any
key<n>_ version marker on the storage side while fresh derivation stays
pinned to key1_, and pin the refusal of key0_, key_, and zero-padded
markers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
BrowserFileSystem documents every deliberately unsupported operation as
a NotFound failure, but Effect's FileSystem.makeNoop hardcodes the four
makeTemp* operations to a defect. Wire them explicitly to the NotFound
refusal so the documented contract holds for the whole list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
…failure

handleOf cleared the running flag only on a successful provider exit
code, so a handle whose exit observation failed kept reporting itself as
a live process forever. Clear the flag on both settlements of the exit
observation: either way the remote process is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two authoring-surface refusals were untyped:

- Flow.execute validated the caller's payload with payloadSchema.make,
  so invalid input died with a raw constructor throw. It now constructs
  through makeEffect and fails with the schema's typed SchemaError,
  carrying the offending field path. The execute signature declares the
  error.

- poll answered Option.none both for a known, unsettled run and for an
  execution id the runtime never recorded, so callers could not tell
  "still running" from "no such execution". A new typed
  FlowExecutionNotFound failure (declared in @smthrs/flow-next beside
  the other port errors) is raised for an unknown id by every runtime:
  the in-memory engine, the durable RunDriver, and the test fixtures.
  Option.none is now reserved for a known run that has not settled.

The engine's internal poll after ensureRun treats not-found as a broken
store invariant and dies. Tests pinning the old contracts are updated,
and the flows barrel pins for both defects are un-flagged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two Node-version-sensitive defects made three suites fail on CI (node
22.19) while passing on newer local Nodes. First, label reconciliation
across tsx module namespaces required implementationDigest equality, but
the digest hashes transpiled function text and the CommonJS and ESM
pipelines annotate their emit differently, so the same declaration in the
same file failed to reconcile; the match is now rule identity plus
canonical attrs in the declaring file, which is the declaration identity a
label actually needs. Second, a BUILD.ts evaluated through the CommonJS
bridge compiles an import of a file: URL to require("file://..."), which
Node 22's CommonJS resolver rejects; _resolveFilename now converts a file:
request to its path first, so URL imports behave identically in both
formats on every supported Node. The shim's format documentation now
states when the ESM override actually applies.

Verified: the three previously red suites (Execute, CacheTrust,
Filegroup) and the full 289-test package suite pass under node 22.19.0
and 24.18.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
smithers-lane-agent and others added 28 commits August 19, 2026 02:00
…ts root

A handle built in a plain module or a subpackage BUILD.ts could not resolve
file("//pnpm-lock.yaml"): the anchor was read from the nearest
WORKSPACE.ts or BUILD.ts frame, which is absent at module top level and is
the package directory, not the root, in packages/foo/BUILD.ts. The engine
imports WORKSPACE.ts under a marked URL, so a BUILD.ts cannot import the
workspace accessor, and the documented fallback was the case that failed.

- `//` resolves to the nearest ancestor of the declaring module that holds
  a WORKSPACE.ts, with the npm(name) call site as the fallback
- a relative path still anchors at the declaring module's directory
- versions and closures are sorted before digesting
- tests evaluate a BUILD.ts fixture: the refusal names the BUILD.ts line,
  the anchor resolves from a subpackage and from a plain module, and a
  missing WORKSPACE.ts is refused with a named reason

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…the hit-restores contract

A restore removed the declared path and then wrote beneath its parent, which
was checked lexically only. A symbolic link planted at an ancestor would have
carried the write out of the workspace before the re-measure refused the hit.
The restore now resolves the nearest existing ancestor through its links and
refuses one that leaves the canonical root before a directory is created. A
link planted at the declared path itself is unlinked as a path, never followed.

Three tests in CacheTrust and ToolBuild still asserted the pre-store contract,
that a deleted declared output re-executes. Under the artifact store a deleted
output is restored and reported as a hit, and only a swept store re-executes.
The suite was red on those three; they now pin the new contract, including
that a planted link is replaced without writing through it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ports

D14 applies D11's write|check duality to BUILD.ts itself. `PackageDefaults`
passes one static attrs record to every match, so all synthesized packages get
an empty `deps` list and no dependency edges even when their manifests name
workspace siblings. `DepSync` derives the edges from the import graph the
sources actually write.

`plan` is pure and takes the workspace facts as arguments, so it answers in a
browser. It reads imports through `Imports`, not a second scanner, and adds
only what a surgical edit needs: byte offsets into a BUILD.ts. Write mode
inserts one import statement and one array element and leaves every other byte
alone, including the hand-written rationale in the comments. Check mode reports
drift and mutates nothing.

Two declared policies. A type-only import is an edge, because `tsc` cannot
compile a consumer before the producer's declarations exist; the 45
`scripts/circular.mjs` copies set `skipTypeImports: true` for the opposite
question, which is runtime cycles. Edges are only ever added: a declared dep
with no matching import may be a deliberate ordering edge.

Write mode is refused when NODE_ENV is production and when the strict flag is
set, and the refusal is checked before the plan, so a gated run reads nothing.
The write itself goes through `GeneratedFile.writeGeneratedFile`, the same
atomic rename every other generated file uses.

Nothing is wired into a BUILD.ts or into ci.yml. A run against the real
workspace reports 45 of 50 packages missing at least one edge, including all
six hand-written BUILD.ts files, so adoption is a later change.
…n the contract

Seven hand-written `test`-job gates become root ToolBuild genrules: `//:browser`,
`//:circular`, and the five `node --test` script gates. `//:packReleaseTest` and
`//:setReleaseVersionTest` declare both workflow files and all 45 package
manifests, which is the closure their suites read; without it a cacheable gate
replays green over the manifest drift it exists to catch. `//:circular` and
`//:checkTestPinsTest` take `cache: false` and say why: their read set crosses
package boundaries a root glob cannot reach.

`//:ci` grows from one gate and no required jobs to 29 gates across test, rust,
wasm-repro, bun, and browser, plus a `requiredJobs` list. ci.yml gains the step
that plans it. Before that step the contract was inert: the `docs` verb selects
no root target and the shadow lane's `//packages/...` pattern excludes every
`//:` label, so nothing in CI ever planned `//:ci`.

`continue-on-error` stays advisory, as documented. Write mode stays unreachable:
the render model has no needs, permissions, strategy, step `if`, or comments,
and ci.yml carries 60+ load-bearing comment lines.

scripts/pack-release.test.mjs's cross-workflow parity gate now compares TARGET
sets rather than shell strings. Its extractor recognised only `pnpm run
<script>`, `pnpm test`, and `node [--test ]scripts/*.mjs`, so the first gate
converted to a smthrs invocation would have emptied ci.yml's side of the
comparison and passed vacuously.

PnpmWorkspaceFile converts to the shared GeneratedFile check/write pair,
retiring its own `mode` vocabulary. CONTRIBUTING.md separates the contract
gates from the two local-only commands no workflow runs. ci.yml's shadow-lane
comment is corrected to the measured 271 roots over 45 packages, and the
actionlint asymmetry with release.yml is recorded where it happens.
The first `deps: [` array in a BUILD.ts is not a safe edit anchor.
`packages/build/BUILD.ts` writes its first one inside the `PackageDefaults`
macro, a template for every synthesized package, and a dry run of write mode
against the real tree put three cross-package edges there. `plan` now finds
the top-level statement that constructs the `lib` target, following
`export const lib = standard.lib` to the statement that defines `standard`,
and edits the deps array inside that statement only. A lib target with no deps
array blocks the edit and reports the edge, so `packages/build/BUILD.ts` is
reported and left alone.

An inserted import keeps the file's semicolon style.

Tests cover the macro shape, the alias hop, a multi-line lib declaration after
another target, a file with no lib target, and the semicolon case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… close the gate closure

The real-workflow path in GithubWorkflow.test.ts resolved to
packages/.github/workflows/ci.yml, so every case that read the checked-in
pipeline returned early and passed without reading it, including the new
//:ci contract block. The path is corrected, readReal fails instead of
skipping, and the stale job list gains smthrs-shadow.

//:flowsBackupTest drops its cache claim: the script under test imports
@smthrs/database and @smthrs/engine-store from src/, a closure a root
declaration cannot key.

scripts/pack-release.test.mjs gains a case that fails when a
packages/*/package.json on disk is not named in BUILD.ts's manifest list,
which is the key material of the two cacheable manifest gates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`Target.make` defaults `cache` to true, at both flip sites: `cacheableFor`,
which decides what the planner admits, and the `["constant", …]` entry inside
`implementationDigest`, which is what makes the digest identify the cache
decision it claims to identify. Flipping only the first would leave a
now-cacheable rule recording `false`.

Nine rules take the default and become cacheable: BiomeCheck, DepsLint,
DtsBuild, PackageLint, TsBuild, Typecheck, TypedocDocs, Vitest, and
VitestCoverage. Sixteen rules already declare `cache: false` and are untouched;
a decision that omits a rule is not licence to delete its opt-out.

ONE-TIME INVALIDATION: the digest change re-keys every stored entry for those
nine rules. This is the intended, benign consequence of recording the new
default, not a durable-key regression. `//lint:durableIdentityGuard` should
read it as such.

PREREQUISITE, landed here: `StandardPackage` and the `Vitest` path form now
declare `test/**/*` rather than `test/**/*.test.ts`. Vitest imports harness
modules and reads fixtures of any extension, and 163 files under
`packages/**/test/` are neither spec files nor declared inputs of any target.
`check` and `fmt` already declared the whole `.ts` test tree; `test` did not,
and that asymmetry under a cacheable default is a stale green. Measured: with
the narrow glob, editing `packages/plan/test/Crypto.ts` and re-running
`//packages/plan:test` is a hit; with the wide glob it is a miss.

Tests: the default and opt-out assertions invert; a new test proves both flip
sites agree by comparing digests; a catalog scan pins the exact nine defaulted
rules, the fifteen `Target.make` sites declaring `cache: false`, and the
computed and explicitly-true rules, so a future rule cannot silently join or
leave the set; and a workspace fixture proves the widened declaration re-keys
on a harness edit and a `.sse` fixture edit while the old one does not.

Docs record what is now true rather than dropping the caveat: the installed
tool binary is still outside key material, though the declared toolchain and
lockfile digest are in it. They also correct a stale claim: the W3-E artifact
store does restore declared outputs on a hit, so TsBuild, DtsBuild, and
TypedocDocs are not inert. Measured: deleting `packages/plan/dist` and
rebuilding is an 811ms hit that restores the tree, against a 15.7s cold run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…igest

The cache default flip was inert: `Target.make` folded
`Node.functionIdentity` into `implementationDigest`, and an unannotated
closure fails closed there with per-process entropy. Two cold processes
planning an unchanged workspace produced different digests, so every key
changed on every run and no stored entry was ever hit — measured:
`build //packages/plan:check` ran twice with distinct keys, and even the
pre-flip cacheable `DocsParity` never hit.

`Target.make` now keeps the source-plus-captures identity for a captured
function and digests exact source for any other. The planner's ambient
implementation fingerprint already digests every byte of the shipped
implementation trees, so a behavioral change behind one of these closures
still re-keys every target through the ambient half of the key.

ONE-TIME INVALIDATION: the digest algorithm change re-keys every stored
entry again, on top of the flip's. Benign; not a durable-key regression.

Measured after the fix: `build //packages/plan:check` runs once, then
hits (1.6s -> 27ms). Editing the `packages/plan/test/Crypto.ts` harness
re-runs `//packages/plan:test` while `:lib` still hits. Full
`ci //packages/...` runs 179.8s cold, then 57.1s warm with 178 hits; the
two warm failures (`packages/build:fmt`, `packages/flows:test`) reproduce
on committed files no W4 lane touched.

The regression test plans one declaration in two cold processes and
asserts identical digests; an in-process assertion cannot see
per-process entropy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…-written field

D10's manifest half. The generator now reproduces this workspace's 45
checked-in manifests instead of rewriting them.

Three changes make that true.

`preservedFields` widens what a generated manifest carries. `carried` was
exactly `managerOwnedFields`, the dependency blocks. Generating over the
checked-in files would have dropped `smthrs` from all 45, `homepage`,
`repository`, and `bugs` from 43, `tags` and `keywords` from 42, and
`private` and `bin` from 2. Manager-owned blocks are still copied
unconditionally; a preserved field is copied only where the declaration
produced none, so a stated value is still compared and drift is still
reported. The assembly moves out of `sync` into an exported `assemble`.

`publishFields` learns the source-first form this repository publishes.
`exports` names the TypeScript, and `publishConfig.exports` is generated
from it by `distributedExports`, which maps a value under the source
directory onto the compiled triple and passes everything else through: a
sealed subpath stays sealed, `./package.json` stays itself, and an asset
outside the source tree publishes where it sits. The subpath map is
therefore written once rather than twice. `publish.style: "dist"` keeps
the single-map form. `source` is the default, per CONTRIBUTING.md:17.

`repository.directory` is filled in from the declaring package's own
path. A shared template cannot express a per-package value.

The gate is `test/PackageJson.test.ts`, which regenerates all 45 real
manifests in memory and fails on any dropped or changed field. Removing
`smthrs` from `preservedFields` fails all 45; perturbing the compiled
mirror fails 43. Six packages state their own scripts, thirteen their own
exports map, eight their own file list, and two private packages differ on
`publishConfig`; each set is a named constant.

Versioning and release ordering are unchanged and stay where they are.
`scripts/set-release-version.mjs` already implements Nx's
`updateDependents: auto` across all four dependency fields with protocol
ranges excluded, and `scripts/pack-release.mjs` already orders through the
pinned `kernel <-> platform-browser` cycle. The runbook now records both,
and why the retarget set is all 45 while the publish set is 23.
The house prose register forbids em-dash asides. Three appeared in the
manifest generator's JSDoc and one in the release runbook; each is now a
parenthetical or a comma appositive. No behavior changes.
…package edges

D13's missing half. The label layer and glob pruning already shipped; this
builds what did not exist.

- PackageDefaults accepts `marker: null`. A marker-less declaration
  synthesizes targets for any directory matching its glob that directly
  holds a file and has no BUILD.ts, so a folder inside a package is
  addressable (`//pkg/src/internal:lib`) without a manifest or a build file.
  Only a BUILD.ts creates a boundary, so a marker-less unit stays inside the
  parent's globs and keys.
- The workspace derives the edge the subpackage-pruning guard requires:
  every target a nested BUILD.ts prunes gains a dependency on the unit's
  default target, so the subtree re-enters the parent's key through the
  unit's own inputs. A unit with no default gives the edge no endpoint and
  the guard still refuses the plan. SubpackagePruning.test.ts (a landed
  Wave-3 file) is updated to the new semantics: the automatic edge replaces
  the refusal it pinned, and the refusal now covers the no-default case.
- A synthesized PackageJson declaration resolves its scripts against every
  registered target, not only the macro application that produced it, so a
  unit manifest can name its parent's targets. The name-less, version-less
  unit stub manifest remains a cross-lane contract owed by W5-A's
  PackageJson work and is documented as such.
- NewPackage scaffolds tsconfig.test.json, vitest.config.ts,
  eslint.config.js, and dprint.json beside the existing five files; the
  synthesized check/test/lint/fmt targets declare all four as inputs.
- Docs state the honest zero-boilerplate property (zero build files for 39
  of 45 packages, scoped to build-tool config) and record the coverage gap:
  apps/*, examples, and packages/build/infra have package.json and workspace
  membership but no build-system targets.
Review follow-up: the lane's docs, JSDoc, and code comments added six
em-dash asides, which the house prose register forbids. Each is now a
comma appositive, a parenthetical, or a colon. No behavior changes.
captureOutputs required a producer typed `Node<Exec.Result,
Exec.ExecError, R>`, so only a rule that ran a tool through Exec could
declare outputs. The body reads neither the success value nor the error,
so the constraint was a signature accident.

The parameter is now `Node<A, E, R>` and the failure channel is the
producer's own errors plus OutputError. An exec producer still yields
`Exec.ExecError | OutputError`, which is BuildError, so every shipped
caller keeps its type.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add `Input.Produced`, an output-keyed dependency edge, and widen
`Input.Declared` to admit it.

A consumer that keys on a producer's target key alone is unsound whenever the
producer is not cacheable. An agent target invoked with unchanged attrs and
unchanged declared inputs holds its key still while emitting different bytes on
every run, so a consumer keyed on the edge would hit its own cache and replay a
verdict about output that no longer exists.

`Input.produced(target, path)` refuses at the BUILD.ts call site any producer
whose declared outputs are undefined, and any selector that is neither `"."`
nor one of the producer's declared output paths. Rejection carries the typed
`Input.ProducedError`.

`Input.producedDigest` contributes both halves of the edge: the producer's
planned key and the `contentDigest` of every selected manifest entry. The
content half is the digest `ToolBuild.captureOutputs` already computes, which
is also the digest the artifact CAS stores; this combines existing digests and
reads no bytes, so consumer keys and stored blobs agree on what identifies a
subtree.

Selection is at manifest-entry granularity. `"."` selects every declared
output, ordered by path. Any other selector names one declared output exactly.
Naming a file inside an output subtree would require a second digest of the
same bytes and is refused.

`Target.collect` now records the producer of a `Produced` declaration as a
dependency, so an output-keyed edge is also an ordering edge and the producer's
key reaches the consumer's key material.

Widening the `Declared` union changes the JSON-schema identity of every rule
whose attrs carry declared inputs, a one-time invalidation of their stored
cache entries.

Planner-side expansion is not wired here.
`packages/build-cli/src/Workspace.ts:1885` still narrows its final branch to
`GitDiff` and needs an explicit `Produced` case; that file belongs to no lane
in this wave.
…tion

The produced-input lane made Input.ts import Target.ts, but Target.ts
already imports Input.ts for the declared-input union, and the package
circular check rejects the cycle. Input.ts now holds a ProducedTargetRuntime
that Target.ts registers when it initializes, keeping the import graph
one-directional. Every Produced value requires a target that Target.make
created, so the registration always precedes its use.

No behavior changes: the scratch two-target verification produces the same
consumer keys before and after.
… folds

`@smthrs/core`'s KeyMaterial interface omitted `nondeterministic`, which
`@smthrs/plan`'s schema declares and `StepKey.materialBody` folds. Two shapes
for one concept lets a key lose a dimension at the package seam.

The field is optional on both sides, so absence still claims determinism and
no key already computed for a deterministic step moves. Golden keys for a
representative step are byte-identical across the change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Vitest, VitestCoverage, Typecheck, Dprint, EsLint, BiomeCheck, DepsLint,
and PackageLint each gain an `env` attr that defaults to empty and threads
into every tool run they plan, following ToolBuild. The value is attrs, so
it is key material: two targets that differ only in `FC_SEED` now key
apart instead of sharing one cached verdict.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ScriptCheck declares an arbitrary repo script as a lint- or test-kind
target with a declared read set, args, env, expectedExitCodes, and a
timeout. Its verb membership is exactly its kinds attr: a lint gate is
refused under test and a test gate under lint. It is cacheable, keyed
on the script digest, args, srcs digests, dependency outputs, and env.
This is the primitive the hand-written invariant gates need; ToolBuild
cannot express them because its kinds are ["build"] and every declared
output must exist.

NodeTest runs node --test over an explicit ordered file list, with the
read set and a concurrency mapping, as a cacheable test-kind target.

Both rules read the registered toolchain for the interpreter, per D1;
the lane attr lists predate the toolchain registration (they also name
packageManager, which no rule attr may carry).

Target.test.ts hardcodes the catalog's rule count and cache decisions,
so it moves with every rule-adding lane: 30 to 32 here, with NodeTest
and ScriptCheck added to the cache-true list. Sibling lanes must bump
it for their own rules.
BunTest plans bun's test runner as a first-class cacheable test target,
the way Vitest runs vitest. Its tests attr accepts an explicit ordered
file list, and the declared order reaches the command line unchanged:
with --max-concurrency=1 bun runs the named files in the order given,
which the smithers .smithers suite depends on.

BunTestCoverage adds lcov instrumentation plus threshold gating. bun
has no threshold flags, so the gate is a second planned step: an inline
bun program sums the lcov counters and exits non-zero below any
declared threshold. A package coverage does not gate declares no
thresholds and carries a required unsupportedReason instead, copying
the smithers coverage.mjs convention that every unsupported package
names why.

Target.test.ts's catalog pins move from 32 rules to 34.
A bare node --test with no file arguments discovers test files by
scanning the directory tree, and a discovered file is a read nobody
declared. The rule's contract is that the test set is explicit key
material, so the tests attr is now a NonEmptyArray and an empty list
is refused at declaration time.
The produced-input lane widened Input.Declared with the Produced
variant but no expansion branch, so a Produced declaration reaching
Workspace.expandDeclarations fell through to the GitDiff read and
crashed on a missing base, which tsc also flagged as a narrowing
error. The producer's output manifest exists only after the producer
runs, so expansion refuses with a clear error until the planner and
executor wiring that supplies the manifest lands.
A bare `bun test` discovers test files by scanning the directory tree,
and a discovered file is a read nobody declared, so a cacheable target
with an empty tests attr replays stale greens. tests is now a
NonEmptyArray, matching the NodeTest convention; BunTestCoverage
inherits the field through its attrs spread.
Pre-existing at HEAD: dprint check failed the package lint gate on one
continuation line.
The NonEmptyArray tests attr rejects the widened array literal the
shared fixture inferred; pin it to the attr's tuple type.
…triad

Runs an external generator subprocess and either writes its declared
outputs or drift-checks them, generalising the triad from GithubCiGen to
the whole catalog. Contract and check are cacheable and declare the
outputs as inputs, so editing a generated file re-keys the target; write
is never cacheable. Check mode snapshots the declared outputs, runs the
generator through an inline driver under the declared runtime, compares
after applying normalize (strip-timestamps), and with restoreOnFailure
restores the snapshot in a finally, so the tree ends byte-identical pass
or fail. The lint verb maps write to check through attrsForKind.

Target.test.ts counts every catalog rule declaration by scanning src, so
the new rule updates its declaration count and computed-cache list.
…scaffolding

The rework's decision record, its corrections, and its open questions move to
docs/architecture/build-system. They explain why the build system changed and
which premises turned out false, which is worth keeping.

The lane plans (BUILDSYS-PLAN.json, STAGE2-PLAN.json, STAGE2-LANES.json) were
orchestration scaffolding for the agents that did the work. They describe file
ownership and batching, not the product, so they leave the tree.
The generated bun coverage-gate script called console.error. The repository's
console guard (packages/observability/test/NoConsole.test.ts) matches the call
text anywhere under any package's src, including inside the string literals
this script is assembled from, and cannot tell a generated line from a real
call. The guard was red across the whole workspace as a result.

process.stderr.write is the correct call for a script whose only job is to
print one line and exit non-zero, so this satisfies the guard by being right
rather than by being exempted.
Both assertions had drifted from the files they pin, independently of this
branch: pnpm-workspace.yaml carries allowBuilds.playwright (BUILD.ts:65
declares it) and the root package.json carries a dev script, and neither
appeared in the restated literals. The suite was red at the branch point.

This is the third hand-copy of the root configuration found in the tree, after
packages/targets/test/GeneratedRootFiles.test.ts. Each one restates root
BUILD.ts attrs and must be updated in lockstep with any root-config change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant