✨ feat(build): decisions D1-D15, plus cacheable agentic and long-tail rules - #228
Closed
roninjin10 wants to merge 990 commits into
Closed
✨ feat(build): decisions D1-D15, plus cacheable agentic and long-tail rules#228roninjin10 wants to merge 990 commits into
roninjin10 wants to merge 990 commits into
Conversation
… refusals Graph.build recursed twice over author-controlled input: visit recursed per combinator, so ten thousand chained AndThens overflowed the native stack, and hydrate/literal/references recursed per payload level, so a cyclic payload recursed forever and a deep one overflowed. Both walks are now explicit stacks — an operation stack for topology, frame stacks for payloads — that preserve the recursive walk's node, edge, diagnostic, and input ordering exactly. Depth becomes policy instead of a crash: topology past 1000 levels throws graph_too_deep, a payload past 1000 levels throws payload_too_deep, and a payload on its own ancestor chain throws cyclic_payload, each naming the node and the fix. The DECIDED markers on drafts, per-branch and per-catch subject tokens, and unconstrained-caller placement are untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
…lifetime Effect.provide forks the enclosing memo map, so a nested provision reuses the memoized Implementations table instead of building its own — and a nested toLayer of an already-filed tag permanently replaced the enclosing entry. add now registers for the registering scope's lifetime: it files the implementation and restores what it replaced when that scope closes, with finalizer LIFO unwinding stacked same-tag replacements to exactly what each one replaced. A sibling scope that already superseded the entry is left in place; only the currently filed implementation restores its predecessor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Grant patterns compiled to RegExps with every `*` as `.*`, so a repeated-star pattern such as `a*a*a*a*b` backtracked exponentially against long non-matching resources — a ReDoS on the authorization path. Replace the compilation with an iterative two-pointer glob walk over UTF-16 code units that remembers only the most recent `*`, bounding every match at O(pattern x resource) with constant memory. Semantics are preserved exactly: `*` crosses separators and newlines, `?` matches one code unit, a trailing ` *` also grants the bare resource, and Windows-path case folding reproduces the ECMA-262 non-Unicode `i`-flag canonicalization per code unit. A new property test pins agreement with the retired RegExp compilation as the oracle on non-adversarial inputs, both pinned ReDoS defects are flipped to passing regressions, and the wall-time pin now covers patterns up to twelve stars against 4096-character resources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The sync server's `canFollow` returned true for every non-branch run and `SyncAuth` was a documented no-op, so any connection that reached the RPC endpoint could read every engine run in the workspace journal. Close the launch-list gap fail-closed: - `SyncPrincipal`: the request's authenticated identity as a `Context.Reference` defaulting to anonymous. Non-branch runs and workspace listings are refused for anonymous callers; a run-scoped read of an engine run without the workspace principal fails `unauthorized`, and a branch share link never upgrades a caller to workspace listings. - `WorkspaceShare`: the branch share-capability scheme extended, not paralleled — the same length-prefixed HMAC-SHA-256 claims shape with a signed `kid` for key rotation and a domain-separation label, over a `Redacted` keyring. `layerConfig` provisions the secret from `FLOWS_SYNC_SECRET`/`FLOWS_SYNC_KEY_ID`; there is no default secret and no plaintext keyring type. - `SyncAuth.layer`: the production middleware behind `SyncRpcs.SyncAuth`. It verifies the workspace capability presented in the `flows-sync-workspace` request header and installs the principal; a missing header runs the request as anonymous, while a malformed, forged, expired, or unknown-kid header refuses the request outright. `SyncAuth.layerClient` stamps the header on every outgoing client request. Verifier infrastructure faults propagate as their own error, never as `unauthorized`. - `SyncServer.layerHandlers`: the serve seam mirroring `BranchServer.layerHandlers`, so the RPC wiring and its middleware requirement live in src instead of being re-derived per test. - Shared HMAC primitives move to `internal/shareSigner.ts`; `BranchShare` delegates to them unchanged. Tests pin the invariant end to end over the RPC machinery: an unauthenticated workspace read and subscribe are refused, a verified header grants them, tampered/expired/malformed headers are refused, and a branch capability still reads exactly its branch anonymously. The README claim that every operation authorizes through a signed capability is corrected to describe both authorization boundaries. Sync coverage stays at the enforced 100%. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The API deep-dive still said `SyncAuth` is middleware a deployment implements. Point it at the shipped `SyncAuth.layer`, and list the new `WorkspaceShare` and `SyncPrincipal` namespaces alongside the server's policy constructors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Grant-event structs silently stripped unknown keys, so a journal payload mixing envelope and request-only fields decoded as whichever variant its declared keys satisfied. Replayed payloads become active permission authority, so decode now runs with onExcessProperty: "error" and treats any excess key as corruption. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Envelope grants are now a canonical predicate set: duplicate patterns collapse and the survivors sort by their formatted identity before the event persists, so repeated or reordered envelopes produce one durable approval. The store tracks envelope signatures (seeded from journal replay) and treats a repeated grantEnvelope as a no-op instead of new durable evidence. JournalGrantStore serializes the construction-envelope check-then-append through a per-journal critical section and re-replays inside it, so two concurrent constructors append exactly one envelope. Fail-closed hardening in the same seam: - reply refuses a runtime-invalid resolution with a typed invalid_resolution error instead of silently succeeding and stranding the request's waiter on its Deferred. - grantEnvelope and the construction envelope refuse a runtime-invalid scope with a typed store error before it can reach the event schema constructor and die as a defect. - Replay refuses a journal page that repeats its last sequence with hasMore instead of requesting the same page forever; a non-advancing cursor is corruption, not progress. RFC 8785 canonical JSON (@smthrs/canonical-next) was considered for the signature and does not fit: it canonicalizes object keys but preserves array order as semantic, and the pattern-array order is exactly what has to be normalized here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two Node-version-sensitive defects made three suites fail on CI (node
22.19) while passing on newer local Nodes. First, label reconciliation
across tsx module namespaces required implementationDigest equality, but
the digest hashes transpiled function text and the CommonJS and ESM
pipelines annotate their emit differently, so the same declaration in the
same file failed to reconcile; the match is now rule identity plus
canonical attrs in the declaring file, which is the declaration identity a
label actually needs. Second, a BUILD.ts evaluated through the CommonJS
bridge compiles an import of a file: URL to require("file://..."), which
Node 22's CommonJS resolver rejects; _resolveFilename now converts a file:
request to its path first, so URL imports behave identically in both
formats on every supported Node. The shim's format documentation now
states when the ESM override actually applies.
Verified: the three previously red suites (Execute, CacheTrust,
Filegroup) and the full 289-test package suite pass under node 22.19.0
and 24.18.0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
worker-e2e + live-check + serve:local follow wrangler.jsonc to apps/server (spawn cwd set); vite gets --configLoader runner so the TS-source smithers-shared package resolves from vite.config; deps.test.ts closure guard re-pointed from the deleted vendor tree to its real invariants (no path specifiers, single effect instance by realpath); AgentTurnPolicy byte-count expectation corrected (13, not 14). 465/465 tests, vite build emits dist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Schema.TaggedErrorClass -> Schema.TaggedError<Self>() (journal idiom, cleared 105 of 122 errors); inline visibleText fold (workspace @smthrs/model dropped the export the vendored copy had — restoring it upstream is the right end state); restored-test type calibration under the new tsconfig.test.json; JSDoc conformance incl. prompts.mjs generator so the sync test stays true. 188 tests, 100% coverage, 0 lint errors, no identity-string changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…m scripts StandardPackage now emits six targets: the new fmt target runs the package's dprint check through a new Dprint rule, and the new check target runs tsc over tsconfig.test.json after lib builds, so lint+fmt cover what the package lint scripts cover and lib+check cover what the check scripts cover. tsflows ci //packages/... plans 130 targets across all 26 packages and is now gate-equivalent to the recursive pnpm scripts at the package level. tsflows-cli and tsflows-rules gain the tsconfig.test.json the convention assumes; the four hand-written BUILD.ts files export the new targets, with flow's desugared form kept equivalent to the macro. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The wasm-repro job first executed today, once the vendor/jj checkout landed, and its byte-compare failed: the committed artifact was built on aarch64-apple-darwin and CI builds on x86_64-unknown-linux-gnu. No path escaped the remapping. Cargo compiles build scripts for the host, so the `-C metadata` hash of every crate that has one — and of everything above it, which reaches jj-lib and flows-jj — carries the host triple. rustc folds `-C metadata` into the crate's StableCrateId, which seeds every symbol hash, and it orders codegen items by symbol name, so a different host reorders the module and LLVM optimizes it differently. Measured from identical sources on 1.89.0: aarch64-apple-darwin gives 4740406 bytes and 9721 functions, x86_64-unknown-linux-gnu 4740823 and 9720, aarch64-unknown-linux-gnu 4739906. The target directory is innocent — in-tree and out-of-tree CARGO_TARGET_DIR builds are byte-identical on both hosts — so remapping it would have fixed nothing. Declare the host instead. build-wasm.mjs reads the `host` line of `rustc -vV` and refuses to build the committed artifact anywhere but x86_64-unknown-linux-gnu, printing the container command that produces those bytes on any machine: the steps CI runs, one layer down. The header, the job comment, the toolchain pin, and the package README carry the reason. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The committed module was the aarch64-apple-darwin build, which the wasm-repro job cannot reproduce now that it runs. Replace it with the x86_64-unknown-linux-gnu build of the same sources, produced by the container command build-wasm.mjs prints. Three independent x86_64 Linux builds of this commit agree on sha256 3a14db0e, across two container images, two mount paths, and both CARGO_TARGET_DIR layouts. The package's real-artifact contract suite passes on the new module. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
`flows_step_cache` stays the mutable content-addressed head, but each put now also lands its entry in `flows_step_cache_recorded` under `(key_digest, recorded_run_id, recorded_event_seq)`, and nothing ever deletes a ledger row. `get` grows an optional `recordedBy` fence that answers from the ledger first and falls back to the head, so a replay naming the exact recording event reads the bytes that were durable then — however the head has been evicted or replaced since. Eviction keeps protecting future executions; it no longer rewrites recorded history. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two replay defects let the derived projection drift from the journal: - Sealed results were read off the mutable cache head, so evicting or replacing an entry between lifetimes changed an old frame's projection. Replay now reads through the step cache's `recordedBy` provenance fence: the version the replayed record landed answers first, and only an entry recorded elsewhere falls back to the shared head. - The fold consumed journal pages verbatim, so a malformed reader that handed back duplicate or out-of-order pages produced a different state. The prefix is now normalized before folding — one record per coordinate, ordered by seq — because the projection is a function of the run's records, never of how a reader paged them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The public rewind claimed ownership, wrote its audit row, and refreshed anchors before checking what it was asked to do: a non-positive or non-finite pageSize reached paging after the claim, and a frame that was not on the run's lineage at all mutated durable state before failing. The rewind protocol gains a validation phase that runs before anything durable. A malformed pageSize is refused `invalid` (a new TimeTravelError code) before the first journal read. A frame is refused `not_found` unless it addresses the run's history: not past the journal tail, on the run's current lineage, and — frame zero excepted, the one frame that is always addressable — backed by a record of that lineage at the exact coordinate. A refused position leaves no claim, no audit, no anchor refresh, and no store write behind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
`migrate` piped its idempotent `ADD COLUMN plan_digest` widening through `Effect.ignore`, which swallowed every ALTER TABLE failure — a view squatting on the table name, a locked or corrupt database — not just the benign column-already-exists error the ignore was written for. The migration now absorbs exactly the duplicate-column report (SQLite's `duplicate column name`, Postgres' `column ... already exists`, matched through the wrapped failure's message chain) and surfaces everything else. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
…vered rewind Recovery decided "the archive committed" from an absence: an interrupted rewind whose live suffix rows were gone was declared complete even when the archive held nothing — a corrupted journal missing rows on both sides recovered as a successful rewind. `TimeTravelStore` gains `archivedAt(runId, seq)`, and recovery now requires the audit's recorded suffix tail to actually be in the archive before finishing the suspended transition. An empty live suffix with no archive row — or an audit that recorded a suffix but no tail coordinate — rolls back instead of completing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Only irreversible dispatches crossed the journaled effect boundary; a compensable action recorded its pre-image snapshot but no boundary row. Rewind classifies the doomed suffix by exactly those rows, so rewinding past a compensable action archived the suffix as "completed" without ever invoking the tier-2 restore — the tree kept the discarded future's bytes while the journal said otherwise. Compensable dispatches now emit the same intended/terminal boundary records, naming the attempt's anchored `changeId`, so a real rewind restores the workspace to the frame's recorded jj pointer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
The Unicode well-formedness pass walked the input value, so a lone surrogate returned by a toJSON during serialization bypassed it and was emitted instead of refused. Move the refusal into the serializer itself, at the two points a string is written (values and property names), which covers every string in the output including toJSON-minted ones, and delete the now-redundant pre-pass module. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
KeyValue's pattern was anchored to key1_, so a stored key2_ key — which the module docblock promises remains decodable — was refused. Accept any key<n>_ version marker on the storage side while fresh derivation stays pinned to key1_, and pin the refusal of key0_, key_, and zero-padded markers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
BrowserFileSystem documents every deliberately unsupported operation as a NotFound failure, but Effect's FileSystem.makeNoop hardcodes the four makeTemp* operations to a defect. Wire them explicitly to the NotFound refusal so the documented contract holds for the whole list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
…failure handleOf cleared the running flag only on a successful provider exit code, so a handle whose exit observation failed kept reporting itself as a live process forever. Clear the flag on both settlements of the exit observation: either way the remote process is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two authoring-surface refusals were untyped: - Flow.execute validated the caller's payload with payloadSchema.make, so invalid input died with a raw constructor throw. It now constructs through makeEffect and fails with the schema's typed SchemaError, carrying the offending field path. The execute signature declares the error. - poll answered Option.none both for a known, unsettled run and for an execution id the runtime never recorded, so callers could not tell "still running" from "no such execution". A new typed FlowExecutionNotFound failure (declared in @smthrs/flow-next beside the other port errors) is raised for an unknown id by every runtime: the in-memory engine, the durable RunDriver, and the test fixtures. Option.none is now reserved for a known run that has not settled. The engine's internal poll after ensureRun treats not-found as a broken store invariant and dies. Tests pinning the old contracts are updated, and the flows barrel pins for both defects are un-flagged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
Two Node-version-sensitive defects made three suites fail on CI (node
22.19) while passing on newer local Nodes. First, label reconciliation
across tsx module namespaces required implementationDigest equality, but
the digest hashes transpiled function text and the CommonJS and ESM
pipelines annotate their emit differently, so the same declaration in the
same file failed to reconcile; the match is now rule identity plus
canonical attrs in the declaring file, which is the declaration identity a
label actually needs. Second, a BUILD.ts evaluated through the CommonJS
bridge compiles an import of a file: URL to require("file://..."), which
Node 22's CommonJS resolver rejects; _resolveFilename now converts a file:
request to its path first, so URL imports behave identically in both
formats on every supported Node. The shim's format documentation now
states when the ESM override actually applies.
Verified: the three previously red suites (Execute, CacheTrust,
Filegroup) and the full 289-test package suite pass under node 22.19.0
and 24.18.0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DLdDwwRhESjXvJfWFoEop
…ts root
A handle built in a plain module or a subpackage BUILD.ts could not resolve
file("//pnpm-lock.yaml"): the anchor was read from the nearest
WORKSPACE.ts or BUILD.ts frame, which is absent at module top level and is
the package directory, not the root, in packages/foo/BUILD.ts. The engine
imports WORKSPACE.ts under a marked URL, so a BUILD.ts cannot import the
workspace accessor, and the documented fallback was the case that failed.
- `//` resolves to the nearest ancestor of the declaring module that holds
a WORKSPACE.ts, with the npm(name) call site as the fallback
- a relative path still anchors at the declaring module's directory
- versions and closures are sorted before digesting
- tests evaluate a BUILD.ts fixture: the refusal names the BUILD.ts line,
the anchor resolves from a subpackage and from a plain module, and a
missing WORKSPACE.ts is refused with a named reason
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…the hit-restores contract A restore removed the declared path and then wrote beneath its parent, which was checked lexically only. A symbolic link planted at an ancestor would have carried the write out of the workspace before the re-measure refused the hit. The restore now resolves the nearest existing ancestor through its links and refuses one that leaves the canonical root before a directory is created. A link planted at the declared path itself is unlinked as a path, never followed. Three tests in CacheTrust and ToolBuild still asserted the pre-store contract, that a deleted declared output re-executes. Under the artifact store a deleted output is restored and reported as a hit, and only a swept store re-executes. The suite was red on those three; they now pin the new contract, including that a planted link is replaced without writing through it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ports D14 applies D11's write|check duality to BUILD.ts itself. `PackageDefaults` passes one static attrs record to every match, so all synthesized packages get an empty `deps` list and no dependency edges even when their manifests name workspace siblings. `DepSync` derives the edges from the import graph the sources actually write. `plan` is pure and takes the workspace facts as arguments, so it answers in a browser. It reads imports through `Imports`, not a second scanner, and adds only what a surgical edit needs: byte offsets into a BUILD.ts. Write mode inserts one import statement and one array element and leaves every other byte alone, including the hand-written rationale in the comments. Check mode reports drift and mutates nothing. Two declared policies. A type-only import is an edge, because `tsc` cannot compile a consumer before the producer's declarations exist; the 45 `scripts/circular.mjs` copies set `skipTypeImports: true` for the opposite question, which is runtime cycles. Edges are only ever added: a declared dep with no matching import may be a deliberate ordering edge. Write mode is refused when NODE_ENV is production and when the strict flag is set, and the refusal is checked before the plan, so a gated run reads nothing. The write itself goes through `GeneratedFile.writeGeneratedFile`, the same atomic rename every other generated file uses. Nothing is wired into a BUILD.ts or into ci.yml. A run against the real workspace reports 45 of 50 packages missing at least one edge, including all six hand-written BUILD.ts files, so adoption is a later change.
…n the contract Seven hand-written `test`-job gates become root ToolBuild genrules: `//:browser`, `//:circular`, and the five `node --test` script gates. `//:packReleaseTest` and `//:setReleaseVersionTest` declare both workflow files and all 45 package manifests, which is the closure their suites read; without it a cacheable gate replays green over the manifest drift it exists to catch. `//:circular` and `//:checkTestPinsTest` take `cache: false` and say why: their read set crosses package boundaries a root glob cannot reach. `//:ci` grows from one gate and no required jobs to 29 gates across test, rust, wasm-repro, bun, and browser, plus a `requiredJobs` list. ci.yml gains the step that plans it. Before that step the contract was inert: the `docs` verb selects no root target and the shadow lane's `//packages/...` pattern excludes every `//:` label, so nothing in CI ever planned `//:ci`. `continue-on-error` stays advisory, as documented. Write mode stays unreachable: the render model has no needs, permissions, strategy, step `if`, or comments, and ci.yml carries 60+ load-bearing comment lines. scripts/pack-release.test.mjs's cross-workflow parity gate now compares TARGET sets rather than shell strings. Its extractor recognised only `pnpm run <script>`, `pnpm test`, and `node [--test ]scripts/*.mjs`, so the first gate converted to a smthrs invocation would have emptied ci.yml's side of the comparison and passed vacuously. PnpmWorkspaceFile converts to the shared GeneratedFile check/write pair, retiring its own `mode` vocabulary. CONTRIBUTING.md separates the contract gates from the two local-only commands no workflow runs. ci.yml's shadow-lane comment is corrected to the measured 271 roots over 45 packages, and the actionlint asymmetry with release.yml is recorded where it happens.
The first `deps: [` array in a BUILD.ts is not a safe edit anchor. `packages/build/BUILD.ts` writes its first one inside the `PackageDefaults` macro, a template for every synthesized package, and a dry run of write mode against the real tree put three cross-package edges there. `plan` now finds the top-level statement that constructs the `lib` target, following `export const lib = standard.lib` to the statement that defines `standard`, and edits the deps array inside that statement only. A lib target with no deps array blocks the edit and reports the edge, so `packages/build/BUILD.ts` is reported and left alone. An inserted import keeps the file's semicolon style. Tests cover the macro shape, the alias hop, a multi-line lib declaration after another target, a file with no lib target, and the semicolon case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… close the gate closure The real-workflow path in GithubWorkflow.test.ts resolved to packages/.github/workflows/ci.yml, so every case that read the checked-in pipeline returned early and passed without reading it, including the new //:ci contract block. The path is corrected, readReal fails instead of skipping, and the stale job list gains smthrs-shadow. //:flowsBackupTest drops its cache claim: the script under test imports @smthrs/database and @smthrs/engine-store from src/, a closure a root declaration cannot key. scripts/pack-release.test.mjs gains a case that fails when a packages/*/package.json on disk is not named in BUILD.ts's manifest list, which is the key material of the two cacheable manifest gates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`Target.make` defaults `cache` to true, at both flip sites: `cacheableFor`, which decides what the planner admits, and the `["constant", …]` entry inside `implementationDigest`, which is what makes the digest identify the cache decision it claims to identify. Flipping only the first would leave a now-cacheable rule recording `false`. Nine rules take the default and become cacheable: BiomeCheck, DepsLint, DtsBuild, PackageLint, TsBuild, Typecheck, TypedocDocs, Vitest, and VitestCoverage. Sixteen rules already declare `cache: false` and are untouched; a decision that omits a rule is not licence to delete its opt-out. ONE-TIME INVALIDATION: the digest change re-keys every stored entry for those nine rules. This is the intended, benign consequence of recording the new default, not a durable-key regression. `//lint:durableIdentityGuard` should read it as such. PREREQUISITE, landed here: `StandardPackage` and the `Vitest` path form now declare `test/**/*` rather than `test/**/*.test.ts`. Vitest imports harness modules and reads fixtures of any extension, and 163 files under `packages/**/test/` are neither spec files nor declared inputs of any target. `check` and `fmt` already declared the whole `.ts` test tree; `test` did not, and that asymmetry under a cacheable default is a stale green. Measured: with the narrow glob, editing `packages/plan/test/Crypto.ts` and re-running `//packages/plan:test` is a hit; with the wide glob it is a miss. Tests: the default and opt-out assertions invert; a new test proves both flip sites agree by comparing digests; a catalog scan pins the exact nine defaulted rules, the fifteen `Target.make` sites declaring `cache: false`, and the computed and explicitly-true rules, so a future rule cannot silently join or leave the set; and a workspace fixture proves the widened declaration re-keys on a harness edit and a `.sse` fixture edit while the old one does not. Docs record what is now true rather than dropping the caveat: the installed tool binary is still outside key material, though the declared toolchain and lockfile digest are in it. They also correct a stale claim: the W3-E artifact store does restore declared outputs on a hit, so TsBuild, DtsBuild, and TypedocDocs are not inert. Measured: deleting `packages/plan/dist` and rebuilding is an 811ms hit that restores the tree, against a 15.7s cold run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…igest The cache default flip was inert: `Target.make` folded `Node.functionIdentity` into `implementationDigest`, and an unannotated closure fails closed there with per-process entropy. Two cold processes planning an unchanged workspace produced different digests, so every key changed on every run and no stored entry was ever hit — measured: `build //packages/plan:check` ran twice with distinct keys, and even the pre-flip cacheable `DocsParity` never hit. `Target.make` now keeps the source-plus-captures identity for a captured function and digests exact source for any other. The planner's ambient implementation fingerprint already digests every byte of the shipped implementation trees, so a behavioral change behind one of these closures still re-keys every target through the ambient half of the key. ONE-TIME INVALIDATION: the digest algorithm change re-keys every stored entry again, on top of the flip's. Benign; not a durable-key regression. Measured after the fix: `build //packages/plan:check` runs once, then hits (1.6s -> 27ms). Editing the `packages/plan/test/Crypto.ts` harness re-runs `//packages/plan:test` while `:lib` still hits. Full `ci //packages/...` runs 179.8s cold, then 57.1s warm with 178 hits; the two warm failures (`packages/build:fmt`, `packages/flows:test`) reproduce on committed files no W4 lane touched. The regression test plans one declaration in two cold processes and asserts identical digests; an in-process assertion cannot see per-process entropy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…-written field D10's manifest half. The generator now reproduces this workspace's 45 checked-in manifests instead of rewriting them. Three changes make that true. `preservedFields` widens what a generated manifest carries. `carried` was exactly `managerOwnedFields`, the dependency blocks. Generating over the checked-in files would have dropped `smthrs` from all 45, `homepage`, `repository`, and `bugs` from 43, `tags` and `keywords` from 42, and `private` and `bin` from 2. Manager-owned blocks are still copied unconditionally; a preserved field is copied only where the declaration produced none, so a stated value is still compared and drift is still reported. The assembly moves out of `sync` into an exported `assemble`. `publishFields` learns the source-first form this repository publishes. `exports` names the TypeScript, and `publishConfig.exports` is generated from it by `distributedExports`, which maps a value under the source directory onto the compiled triple and passes everything else through: a sealed subpath stays sealed, `./package.json` stays itself, and an asset outside the source tree publishes where it sits. The subpath map is therefore written once rather than twice. `publish.style: "dist"` keeps the single-map form. `source` is the default, per CONTRIBUTING.md:17. `repository.directory` is filled in from the declaring package's own path. A shared template cannot express a per-package value. The gate is `test/PackageJson.test.ts`, which regenerates all 45 real manifests in memory and fails on any dropped or changed field. Removing `smthrs` from `preservedFields` fails all 45; perturbing the compiled mirror fails 43. Six packages state their own scripts, thirteen their own exports map, eight their own file list, and two private packages differ on `publishConfig`; each set is a named constant. Versioning and release ordering are unchanged and stay where they are. `scripts/set-release-version.mjs` already implements Nx's `updateDependents: auto` across all four dependency fields with protocol ranges excluded, and `scripts/pack-release.mjs` already orders through the pinned `kernel <-> platform-browser` cycle. The runbook now records both, and why the retarget set is all 45 while the publish set is 23.
The house prose register forbids em-dash asides. Three appeared in the manifest generator's JSDoc and one in the release runbook; each is now a parenthetical or a comma appositive. No behavior changes.
…package edges D13's missing half. The label layer and glob pruning already shipped; this builds what did not exist. - PackageDefaults accepts `marker: null`. A marker-less declaration synthesizes targets for any directory matching its glob that directly holds a file and has no BUILD.ts, so a folder inside a package is addressable (`//pkg/src/internal:lib`) without a manifest or a build file. Only a BUILD.ts creates a boundary, so a marker-less unit stays inside the parent's globs and keys. - The workspace derives the edge the subpackage-pruning guard requires: every target a nested BUILD.ts prunes gains a dependency on the unit's default target, so the subtree re-enters the parent's key through the unit's own inputs. A unit with no default gives the edge no endpoint and the guard still refuses the plan. SubpackagePruning.test.ts (a landed Wave-3 file) is updated to the new semantics: the automatic edge replaces the refusal it pinned, and the refusal now covers the no-default case. - A synthesized PackageJson declaration resolves its scripts against every registered target, not only the macro application that produced it, so a unit manifest can name its parent's targets. The name-less, version-less unit stub manifest remains a cross-lane contract owed by W5-A's PackageJson work and is documented as such. - NewPackage scaffolds tsconfig.test.json, vitest.config.ts, eslint.config.js, and dprint.json beside the existing five files; the synthesized check/test/lint/fmt targets declare all four as inputs. - Docs state the honest zero-boilerplate property (zero build files for 39 of 45 packages, scoped to build-tool config) and record the coverage gap: apps/*, examples, and packages/build/infra have package.json and workspace membership but no build-system targets.
Review follow-up: the lane's docs, JSDoc, and code comments added six em-dash asides, which the house prose register forbids. Each is now a comma appositive, a parenthetical, or a colon. No behavior changes.
captureOutputs required a producer typed `Node<Exec.Result, Exec.ExecError, R>`, so only a rule that ran a tool through Exec could declare outputs. The body reads neither the success value nor the error, so the constraint was a signature accident. The parameter is now `Node<A, E, R>` and the failure channel is the producer's own errors plus OutputError. An exec producer still yields `Exec.ExecError | OutputError`, which is BuildError, so every shipped caller keeps its type. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add `Input.Produced`, an output-keyed dependency edge, and widen `Input.Declared` to admit it. A consumer that keys on a producer's target key alone is unsound whenever the producer is not cacheable. An agent target invoked with unchanged attrs and unchanged declared inputs holds its key still while emitting different bytes on every run, so a consumer keyed on the edge would hit its own cache and replay a verdict about output that no longer exists. `Input.produced(target, path)` refuses at the BUILD.ts call site any producer whose declared outputs are undefined, and any selector that is neither `"."` nor one of the producer's declared output paths. Rejection carries the typed `Input.ProducedError`. `Input.producedDigest` contributes both halves of the edge: the producer's planned key and the `contentDigest` of every selected manifest entry. The content half is the digest `ToolBuild.captureOutputs` already computes, which is also the digest the artifact CAS stores; this combines existing digests and reads no bytes, so consumer keys and stored blobs agree on what identifies a subtree. Selection is at manifest-entry granularity. `"."` selects every declared output, ordered by path. Any other selector names one declared output exactly. Naming a file inside an output subtree would require a second digest of the same bytes and is refused. `Target.collect` now records the producer of a `Produced` declaration as a dependency, so an output-keyed edge is also an ordering edge and the producer's key reaches the consumer's key material. Widening the `Declared` union changes the JSON-schema identity of every rule whose attrs carry declared inputs, a one-time invalidation of their stored cache entries. Planner-side expansion is not wired here. `packages/build-cli/src/Workspace.ts:1885` still narrows its final branch to `GitDiff` and needs an explicit `Produced` case; that file belongs to no lane in this wave.
…tion The produced-input lane made Input.ts import Target.ts, but Target.ts already imports Input.ts for the declared-input union, and the package circular check rejects the cycle. Input.ts now holds a ProducedTargetRuntime that Target.ts registers when it initializes, keeping the import graph one-directional. Every Produced value requires a target that Target.make created, so the registration always precedes its use. No behavior changes: the scratch two-target verification produces the same consumer keys before and after.
… folds `@smthrs/core`'s KeyMaterial interface omitted `nondeterministic`, which `@smthrs/plan`'s schema declares and `StepKey.materialBody` folds. Two shapes for one concept lets a key lose a dimension at the package seam. The field is optional on both sides, so absence still claims determinism and no key already computed for a deterministic step moves. Golden keys for a representative step are byte-identical across the change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Vitest, VitestCoverage, Typecheck, Dprint, EsLint, BiomeCheck, DepsLint, and PackageLint each gain an `env` attr that defaults to empty and threads into every tool run they plan, following ToolBuild. The value is attrs, so it is key material: two targets that differ only in `FC_SEED` now key apart instead of sharing one cached verdict. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ScriptCheck declares an arbitrary repo script as a lint- or test-kind target with a declared read set, args, env, expectedExitCodes, and a timeout. Its verb membership is exactly its kinds attr: a lint gate is refused under test and a test gate under lint. It is cacheable, keyed on the script digest, args, srcs digests, dependency outputs, and env. This is the primitive the hand-written invariant gates need; ToolBuild cannot express them because its kinds are ["build"] and every declared output must exist. NodeTest runs node --test over an explicit ordered file list, with the read set and a concurrency mapping, as a cacheable test-kind target. Both rules read the registered toolchain for the interpreter, per D1; the lane attr lists predate the toolchain registration (they also name packageManager, which no rule attr may carry). Target.test.ts hardcodes the catalog's rule count and cache decisions, so it moves with every rule-adding lane: 30 to 32 here, with NodeTest and ScriptCheck added to the cache-true list. Sibling lanes must bump it for their own rules.
BunTest plans bun's test runner as a first-class cacheable test target, the way Vitest runs vitest. Its tests attr accepts an explicit ordered file list, and the declared order reaches the command line unchanged: with --max-concurrency=1 bun runs the named files in the order given, which the smithers .smithers suite depends on. BunTestCoverage adds lcov instrumentation plus threshold gating. bun has no threshold flags, so the gate is a second planned step: an inline bun program sums the lcov counters and exits non-zero below any declared threshold. A package coverage does not gate declares no thresholds and carries a required unsupportedReason instead, copying the smithers coverage.mjs convention that every unsupported package names why. Target.test.ts's catalog pins move from 32 rules to 34.
A bare node --test with no file arguments discovers test files by scanning the directory tree, and a discovered file is a read nobody declared. The rule's contract is that the test set is explicit key material, so the tests attr is now a NonEmptyArray and an empty list is refused at declaration time.
The produced-input lane widened Input.Declared with the Produced variant but no expansion branch, so a Produced declaration reaching Workspace.expandDeclarations fell through to the GitDiff read and crashed on a missing base, which tsc also flagged as a narrowing error. The producer's output manifest exists only after the producer runs, so expansion refuses with a clear error until the planner and executor wiring that supplies the manifest lands.
A bare `bun test` discovers test files by scanning the directory tree, and a discovered file is a read nobody declared, so a cacheable target with an empty tests attr replays stale greens. tests is now a NonEmptyArray, matching the NodeTest convention; BunTestCoverage inherits the field through its attrs spread.
Pre-existing at HEAD: dprint check failed the package lint gate on one continuation line.
The NonEmptyArray tests attr rejects the widened array literal the shared fixture inferred; pin it to the attr's tuple type.
…triad Runs an external generator subprocess and either writes its declared outputs or drift-checks them, generalising the triad from GithubCiGen to the whole catalog. Contract and check are cacheable and declare the outputs as inputs, so editing a generated file re-keys the target; write is never cacheable. Check mode snapshots the declared outputs, runs the generator through an inline driver under the declared runtime, compares after applying normalize (strip-timestamps), and with restoreOnFailure restores the snapshot in a finally, so the tree ends byte-identical pass or fail. The lint verb maps write to check through attrsForKind. Target.test.ts counts every catalog rule declaration by scanning src, so the new rule updates its declaration count and computed-cache list.
…scaffolding The rework's decision record, its corrections, and its open questions move to docs/architecture/build-system. They explain why the build system changed and which premises turned out false, which is worth keeping. The lane plans (BUILDSYS-PLAN.json, STAGE2-PLAN.json, STAGE2-LANES.json) were orchestration scaffolding for the agents that did the work. They describe file ownership and batching, not the product, so they leave the tree.
The generated bun coverage-gate script called console.error. The repository's console guard (packages/observability/test/NoConsole.test.ts) matches the call text anywhere under any package's src, including inside the string literals this script is assembled from, and cannot tell a generated line from a real call. The guard was red across the whole workspace as a result. process.stderr.write is the correct call for a script whose only job is to print one line and exit non-zero, so this satisfies the guard by being right rather than by being exempted.
Both assertions had drifted from the files they pin, independently of this branch: pnpm-workspace.yaml carries allowBuilds.playwright (BUILD.ts:65 declares it) and the root package.json carries a dev script, and neither appeared in the restated literals. The suite was red at the branch point. This is the third hand-copy of the root configuration found in the tree, after packages/targets/test/GeneratedRootFiles.test.ts. Each one restates root BUILD.ts attrs and must be updated in lockstep with any root-config change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fifteen decisions (D1–D15) from a side-by-side comparison of this repository wired with Nx (#226), Turborepo (#225), and Bazel (#227), plus the first two waves of the follow-on work that makes agentic and long-tail tasks expressible as targets.
27 lanes, 47 commits, 142 files, +20,325 / −963.
The number that motivated it
smthrs ci "//..."planned 278 targets, of which 232 (83%) could not replay.DocsParitywas the only rule that cached across the workspace;TsBuild,Typecheck, andVitestdeclared nocachekey at all and fell through to afalsedefault.Flipping that default alone would have been worse than leaving it. The cache stored a JSON success envelope —
{key, target, label, exitOk, output, storedAt}— andcaptureOutputsdigested declared outputs rather than storing them. A hit on a clean tree would have markedTsBuildsuccessful with nodist/on disk. So the content-addressed store landed first, and the flip came after.What changed
Configuration split.
WORKSPACE.tsdeclares what exists;BUILD.tsdeclares targets — theMODULE.bazel/BUILD.bazelshape. The toolchain registers once, so no rule threadspackageManageras an attr and no packageBUILD.tsimports../../BUILD.ts.Execution is confined. Actions run against a projected copy of their declared inputs, with one shared environment allowlist.
sandbox: falseand--dangerously-no-sandboxare the escape hatches, and neither disables caching — an un-sandboxed target's input list becomes a promise rather than a guarantee, which is how Bazel treatsno-sandboxtoo.Caching is on by default, opt out by exception. The exceptions keep their stated reasons; the wasm reproducibility gate stays uncached because the rebuild is the check.
Visibility is enforced at plan time, at package and folder level, with tiers derived from the existing
smthrs.groupmanifest field rather than restated.Outputs are content-addressed. A cache hit restores declared outputs from the store, confined to the workspace, and a missing blob is a miss rather than a hit.
npm packages are targets derived from
pnpm-lock.yaml, so an action's inputs name the packages it uses instead of an ambientnode_modules.Consumers key on producer output bytes (
Input.Produced), not on the producer's target key. Without this an uncached producer emitting different bytes leaves its consumer replaying a verdict about an artifact that no longer exists.New rules for the surface both repos actually have:
BunTest,BunTestCoverage,NodeTest,ScriptCheck,GeneratedArtifact.ScriptCheckis structurally load-bearing —ToolBuildiskinds: ["build"]and requires declared outputs to exist, so before this there was no way to declare a lint- or test-kind arbitrary command at all.Bugs found by running the code, not reading it
A mapping pass verified every premise before implementation and found 16 wrong and 15 unbuildable as written (
docs/architecture/build-system/corrections.md). Three mattered:BUILD.tssilently pruned its parent's inputs. Addingpkg/src/internal/BUILD.tsremovedpkg/src/internal/b.tsfrom//pkg:lib's declared inputs while the parent'stsc -bstill compiled that subtree. Harmless while nothing cached; a stale-green generator the moment D5 flipped.GeneratedRootFiles.test.tshand-copies rootBUILD.tsattrs and had drifted.//:cinever ran in CI.GithubCiGenkinds were[build, lint]whileci.ymlran thedocsverb andci //packages/....Deliberately not done
.changeset/and no@changesets/*dependency to build on. Internal range retargeting on bump landed, which is the part that carries the value; adopting the package did not.GithubCiGenwrite mode — the renderer models noneeds,permissions, matrix, or step-levelif, and would drop 60+ explanatory comments fromci.yml. Contract mode stays and now covers the full gate surface instead.Both are recorded in
docs/architecture/build-system/corrections.mdrather than left as silent omissions.Measured
smthrs build "//packages/..." --jobs 4, 90 targets:dist/removedPer-target digests are byte-identical between the two runs, so the key is stable across processes — the property that makes the cache shareable rather than merely local.
Verification
pnpm run checkacross every workspace: clean.docs/architecture/build-system/— the decisions, the corrections, and eight open design questions (six ruled on, two left to the maintainer).🤖 Generated with Claude Code