Skip to content

✨ feat(turbo): automate the workspace with Turborepo - #225

Open
roninjin10 wants to merge 6 commits into
mainfrom
build-compare/turborepo
Open

✨ feat(turbo): automate the workspace with Turborepo#225
roninjin10 wants to merge 6 commits into
mainfrom
build-compare/turborepo

Conversation

@roninjin10

@roninjin10 roninjin10 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Turborepo automation for the flows workspace

One of four build-system lanes being compared (in-repo BUILD.ts, Nx, Bazel, this one). Turborepo 2.10.11 (turbo@latest), Turbo 2 idioms throughout: tasks, //# root tasks, Package Configurations, envMode: "strict", Boundaries, turbo query, turbo prune. Nothing existing was removed — the recursive pnpm scripts and BUILD.ts are untouched, and ci.yml gains one actionlint arg.

The full comparison is in docs/build-systems/turborepo.md. This body is the summary.

What is wired

  • Root turbo.json — the complete task graph over 51 workspaces: build, check, lint, format (uncached mutation), circular, test, coverage, typecheck, plus the root gates as //# tasks: //#browser, //#test:scripts (the five node --test scripts/*.test.mjs gates as one invocation), //#docs (docs parity), and //#cargo:fmt|clippy|test for the Rust half with inputs crates/** + vendor/jj/** + Cargo.lock + rust-toolchain.toml.
  • No ^build edges, deliberately. Workspace manifests resolve exports to ./src/*.ts, so check/lint/test/build never consume dist. Verified: all dists deleted, then tsc -p on @smthrs/flows (19 workspace deps) exits 0. The premise that project references would demand ^build does not hold in this repo — no tsconfig declares references.
  • Package Configurations — 45 packages carry a three-line turbo.json (extends: ["//"] + a Boundaries tag mapped from smthrs.group). Real task config only where packages differ: packages/jj (build:wasm, inputs reaching into ../../crates and ../../vendor/jj, cache: false — see below), apps/server (deploy dependsOn: ["smithers-ui#build"] because wrangler serves ../ui/dist), apps/ui (vite build → dist/**, persistent interactive dev tasks), packages/build/infra, packages/chain, packages/evals, examples.
  • Env done right — envMode: "strict"; test-affecting vars (FC_SEED, FLOWS_SLOW_TESTS, FLOWS_FAKE_JJ, …) hashed via env; secrets (CLOUDFLARE_API_TOKEN, SMITHERS_CACHE_TOKEN, …) available-but-unhashed via passThroughEnv; CI/NODE_ENV in globalEnv.
  • Remote cache — remoteCache.signature: true wired; activates with TURBO_TOKEN, TURBO_TEAM, TURBO_REMOTE_CACHE_SIGNATURE_KEY (absent → local cache, nothing fails). On pointing Turbo at the repo's own cache worker in packages/build/infra: not today — that worker speaks an action-cache/CAS protocol (/ac + /cas, see worker/protocol.ts), Turbo speaks GET/PUT /v8/artifacts/:hash. Same storage model, so an adapter route is a day of work, not a redesign.
  • turbo boundaries — tag rules encode the release train: engine may depend only on engine, agent on engine+agent, tooling on all three.
  • .github/workflows/turbo.yml — --affected on PRs, full graph on main, --dry=json affected graph recorded as an artifact, --summarize with run-summary uploads, advisory boundaries step, root gates, and the query-based assertion.
  • turbo query — scripts/turbo-engine-boundary.mjs asserts the release-train rule through the GraphQL package graph: engine boundary holds: 23 engine packages, 0 agent/tooling dependencies.
  • turbo prune — turbo prune smithers-server --docker → 15 MB subworkspace (out/json pruned lockfile layer + out/full 20-package source closure).
  • Also: ui: "tui", turbo watch (verified), persistent/interactive dev tasks, --graph SVG output.

Measured numbers (16-core aarch64 macOS, controlled cache dir)

Scope Tasks Cold Warm
build 44 41.35s 41ms FULL TURBO
check + lint + circular 137 2m46.6s 105ms FULL TURBO
test (--concurrency=2) 51 3m15.8s 97ms FULL TURBO
Root gates 4 3.0s 52ms FULL TURBO

Whole-graph local cache: 13 MB. Cache restore verified by hand (rm -rf a dist → 33ms hit, outputs return).

--affected proof: a one-file edit in packages/observability/src narrowed 51 packages → 5 (the package + its dependent closure: flows, engine-harness, cli, examples), 232 tasks → 25, measured via --dry=json.

In a git worktree, Turbo silently redirects the local cache into the shared git common dir (.../.git/modules/.turbo/cache) — all linked worktrees share one cache. Nice default; it also means clearing .turbo/cache in the worktree is not a cache clear. Ask me how I know.

What does not work / honest limitations

  • turbo boundaries is red on this compliant repo: 210 findings. 182 are config-file imports Boundaries cannot scope out (BUILD.ts → @smthrs/targets, eslint.config.js → ../../eslint.jsdoc.js; there is no include/exclude). The rest are real bugs it surfaced: 27 imports of undeclared packages (mostly @effect/platform-node in test files, rescued by ancestor node_modules resolution) and the pre-existing @smthrs/kernel ↔ @smthrs/platform-browser package cycle. CI runs it advisory; the query script carries enforcement.
  • The wasm gate must not cache. flows_jj.wasm is a reproducibility contract — a cache hit would skip the rebuild that is the gate, and the bytes are host-locked to x86_64-linux. build:wasm is cache: false; the dedicated wasm-repro CI job stays as-is. "This task must execute even when inputs are unchanged" is inexpressible in Turbo.
  • build and check both emit dist/**. No sandbox, no disjointness check, no same-package serialization guarantee — turbo run build check can interleave build's rmSync("dist") with check's emit. The CI lane runs them as separate steps. (236-task full graph verified green regardless.)
  • Heavy suites need --concurrency=2 — same finding the repo's own shadow job encodes as --jobs 2; an unbounded local run flaked 10 suites that pass in isolation.
  • Root tasks can't depend on package tasks and aren't meaningfully --affected-filterable; they sit beside the graph.
  • Pre-existing main breakage fixed here (own commit): 31 files of dprint drift that have Lint all workspaces red on main. Also found: ci.yml's docs-parity step calls pnpm exec smthrs, a bin the build-cli rename removed — //#docs invokes node packages/build-cli/src/main.js docs '//...' from source, same as the shadow job. Neither is caused by or scoped to this PR beyond those notes.
  • turbo info misreports the package manager as pnpm9 (cosmetic inference bug against pnpm 11.21.0).

Verdict vs BUILD.ts

Turbo is ahead on scheduling ergonomics: --affected, TUI, watch, prune, query, worktree-shared cache, and warm runs that are effectively free. The in-repo system is ahead on everything that requires the graph to own things: typed config and macros without 45 tag files, declared toolchains, generated-and-drift-checked root files (Turbo read pnpm-workspace.yaml; pnpm auto-edited it on a machine with a global policy and nothing in Turbo noticed), secrets by name, node_modules as a target, sub-package granularity, and the wasm contract. Details, file references, and the adoption sequence: docs/build-systems/turborepo.md.

🤖 Generated with Claude Code

smithers-lane-agent added 6 commits August 18, 2026 13:14
smithers-lane-agent
🎨 style: apply the repo's own dprint formatting to drifted files
0040c73 
main is red at the Lint all workspaces step: 31 files (mostly README
markdown tables) drifted from what the pinned dprint plugins produce.
This commit is the output of pnpm --recursive --if-present run format
and nothing else.
smithers-lane-agent
✨ feat(turbo): add the Turborepo 2 task graph and package configurations
2364ad8 
Root turbo.json models build, check, lint, format, circular, test,
coverage, and typecheck over all 51 workspaces, plus the root gates as
//# tasks: browser bundle guard, the five node --test script gates as
one test:scripts invocation, docs parity, and the cargo fmt/clippy/test
gates. envMode is strict; every task declares the env it reads, with
secrets in passThroughEnv so they never enter cache keys.

Per-package turbo.json files extend the root (Package Configurations):
each carries the Boundaries tag mapped from smthrs.group, and the
packages that genuinely differ carry their own task definitions — the
host-locked wasm reproducibility build in @smthrs/jj (uncached on
purpose), smithers-server's wrangler deploy gated on smithers-ui#build,
smithers-ui's vite build and persistent dev tasks, the infra worker's
deploys, and the chain prompts / evals score-gate tasks.

No ^build edges: workspace packages resolve each other through
exports -> src/*.ts, so typecheck, lint, and test never consume dist.
Verified empirically — a package builds clean with every dependency
dist removed.

The vitestCoverageIsolation pin (issue #166) is widened for the five
new root scripts, which is the review path its comment prescribes.
smithers-lane-agent
✨ feat(turbo): add the turbo CI lane and a query-based boundary asser…
c9afd84 
…tion

.github/workflows/turbo.yml runs the affected task graph on pull
requests and the full graph on main pushes: build, check, lint,
circular, and test with --summarize, the recorded --dry=json affected
graph as an artifact, turbo boundaries as an advisory step, the root
gates, and run-summary uploads. Remote caching activates when
TURBO_TOKEN, TURBO_TEAM, and TURBO_REMOTE_CACHE_SIGNATURE_KEY are set;
without them the lane degrades to local caching.

scripts/turbo-engine-boundary.mjs asserts the release-train rule
(engine never depends on agent or tooling) through turbo query's
GraphQL package graph, so the guarantee does not rest on Boundaries'
experimental status alone.

ci.yml gains turbo.yml in its actionlint args; no gate changes.
smithers-lane-agent
📝 docs: compare Turborepo with the in-repo build system
450c0ba 
docs/build-systems/turborepo.md: what was added file by file, how each
BUILD.ts property maps (targets, toolchain, cache keys, macros, drift
checks, secrets, node_modules), where Turbo is ahead with measured
numbers, where it cannot express the repo's invariants (package-task
granularity, glob-only file model, overlapping outputs, root-task
isolation, host-blind caching, unscoped Boundaries), the costs, and a
concrete adoption sequence.
smithers-lane-agent
🔧 chore(turbo): let dist/** cover the check tsbuildinfo
f0e1fb7 
tsc -b writes tsconfig.tsbuildinfo inside dist/, so the extra
*.tsbuildinfo output globs were redundant. Verified by restoring a
deleted dist from cache in 33ms.
smithers-lane-agent
🔧 chore(turbo): pin TURBO_SCM_BASE to the PR base in CI
046dba9 
The comparison doc states the workflow uses TURBO_SCM_BASE for the
--affected diff, but the workflow never set it and relied on merge-base
detection alone. Pin it to the PR base SHA so the affected graph is
exactly the PR's changes regardless of checkout merge history.
@roninjin10 roninjin10 mentioned this pull request Aug 19, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@roninjin10