Branched databases for application isolation

[kriszyp]

Summary

Allow a deployed application to be configured to use a branched copy of one or more databases. A branch is a writable, isolated fork of a base database, invisible to other applications and to replication. The app's code is unchanged — it imports databases.data from harper exactly as it does today, but the binding resolves to the branch.

A branch is durable and deterministically named: it lives at <storage>/`branches`/<app>/<db>, derived only from the application and base database names, and is adopted again on restart rather than re-checkpointed. Nothing process-local appears in the path. That is deliberate — an isolated application deployed onto a cluster is expected to be transparently available across it, so every node must resolve the same application's branch to the same place, and that identity is what a future replicated branch would be addressed by.

Motivation

The driving use case is isolated application versions: deploy multiple variants of an app to the same Harper instance (or cluster), each with its own isolated view of data, so writes don't collide and tests don't pollute the shared database. Today there is no way to give an app a private writable copy of an existing database short of running a separate instance.

UX

Declared on the application's root-config entry, alongside host and urlPath — which databases an application forks is a deployment decision, not something the application checks in. Declaring it in the application's own config.yaml is refused rather than ignored.

my-app:
  package: my-app
  host: api.example.com
  branchedDatabases:
    - data

branchedDatabases: true forks every database on the instance except system. It is a snapshot at load, not a subscription: a database created afterward is not retroactively branched.

The app then imports from harper normally:

import { databases } from 'harper';
await databases.data.MyTable.put({ id: 'x', ... });

The import is load-bearing. A branch is delivered through the module loader's harper exports; the bare databases/tables globals are shared process-wide by the default vm-current-context loader and cannot be scoped, so an application that reaches for them reads and writes the BASE, silently. Per-application globals are a property of the thread-isolation track, not of branching.

Approach

RocksDB checkpoint + hard-linked blob clone + per-application databases binding.

1. On app load, for each declared database, createCheckpoint() into a staging sibling and rename it into place, so a crash mid-copy leaves debris rather than a half-populated directory RocksDB would refuse to open. Concurrent worker threads settle who takes the checkpoint through a claim word in a buffer the base store shares across threads.
2. Open the checkpoint into a caller-owned table graph — never registered in the process-global databases map. That is what keeps a branch out of describe_all, analytics, worker teardown, and replication by construction rather than by a skip list each future enumerator has to remember. The branch's store carries its own identity, length-prefixed so two (app, database) pairs can never compose the same store name.
3. In getHarperExports(scope), a branched application's databases/tables/defineTable resolve through a live view over the real map. An unbranched application gets the process-wide singletons by identity, so the common path is provably unchanged and only branched applications pay for the indirection.
4. Blobs are hard-link cloned alongside the checkpoint, so the OS inode refcount does the reference counting and each branch keeps its own independent directory and ID allocator. No shared allocator, no high-water mark, no gated deletion, no orphan-GC suspension. See Branched databases: blob store sharing, allocator, and GC safety #644 for the full design.
5. A branch is removed deliberately — undeploying the application — not on process exit. There is no startup sweep, because there are no per-process directories to sweep.

Branches must not participate in cluster replication. Verified: replication reaches tables only through the global registry (getDatabases()[databaseName], dbReplicationWorkers.get(databaseName), the wire subscription's sub.database), and a branch is never in that map. Subscriptions are safe by a different mechanism — addSubscription keys on table.primaryStore.path, and a branch has its own path.

Subtasks

• Expose RocksDB Checkpoint API as createCheckpoint(path) rocksdb-js#577 — Expose createCheckpoint() in the native binding
• Branched databases: app config, scope proxy, lifecycle wiring #643 — App config, root-config declaration, scoped bindings, lifecycle (shipped in Give an application a private, durable fork of a database #2352)
• Branched databases: blob store sharing, allocator, and GC safety #644 — Blob hard-link clone (branch reads of pre-existing blobs are broken until this lands)
• Branched databases: scope the table factory through GraphQL @table, ensureTable, and defineTable #2264 — Scope the table factory through GraphQL @table, ensureTable, and defineTable — PR feat(branches): declare tables into a branch through @table, ensureTable and defineTable #2523 (draft)
• Branch removal on undeploy — removeBranches() exists and is tested but has no production caller, so an undeployed application's fork outlives it — PR feat(branches): remove an application's branched databases on drop_component #2517 (merged)
• Integration tests: isolation in both directions, durability across restart, branchedDatabases: true, and the schema gate
• Tier 2, thread isolation (first piece): isolated: true runs an application in a dedicated worker thread — PR feat(threads): run an isolated application in a dedicated worker thread #2524 (draft); then proxy route composition, ops-API host scoping, log isolation

Known gaps

• Schema is frozen at checkpoint. A branch carries its own copy of the schema as of its creation, and the DDL fence refuses dropTable/addAttributes/removeAttributes/subscribe-auditing/@table/ensureTable/defineTable through a branch, so a branched application cannot evolve its schema at all until Branched databases: scope the table factory through GraphQL @table, ensureTable, and defineTable #2264. Tier-1 applications must ship their schema in the base first.
• A killed claim winner is unrecoverable in-process. A worker terminated mid-createCheckpoint leaves the claim at CREATING with nobody to release it, so later loads of that application burn the full 10-minute deadline. A filesystem-level claim is the durable answer.
• operation() is instance-global by decision — the operations API is administrative, so operation('create_table', …) from a branched application reaches the base.
• Deploy pre-flight validates a candidate against the base (pre-existing): validation loads the component without branch bindings, so a branched application's load-time database work hits the shared database during validation.

Explicit non-goals

• Merge / promote branch writes back to base.
• Per-table (rather than per-database) branching.
• Branch sharing across applications.
• LMDB backend — RocksDB only.

---

🤖 Updated by Claude on behalf of Kris (2026-08-31) — brought in line with what shipped in #2352: branches are durable with a deterministic, cluster-addressable identity (not ephemeral/per-process), declared on the root-config entry, kept out of the global map rather than namespaced within it, and blob handling is a hard-link clone rather than a shared store with a high-water mark. Original body preserved in a comment.

[kriszyp]

Superseded by what actually shipped in #2352. The design changed in four ways the body above no longer reflects: branches are durable with a deterministic <storage>/`branches`/<app>/<db> identity rather than ephemeral and per-process; branchedDatabases is declared on the application's root-config entry rather than in its own config.yaml; the branch's table graph is caller-owned and never registered in the global databases map (rather than namespaced within it); and blobs are hard-link cloned rather than sharing the base's directory, allocator and a high-water mark.

Original body, preserved for reference:

---

Summary

Allow a deployed application to be configured to use a branched copy of one or more databases. A branch is a writable, isolated, ephemeral fork of a base database that lives for the lifetime of the process and is invisible to other applications and to replication. The app's code is unchanged — it imports databases.data from harper exactly as it does today, but the binding resolves to the branch.

Motivation

The driving use case is ephemeral application versions on a QA server: deploy multiple variants of an app to the same Harper instance, each with its own isolated view of data, so writes don't collide and tests don't pollute the shared database. Today there is no way to give an app a private writable copy of an existing database short of running a separate instance.

Proposed UX

In an application's config:

package: my-app
branchedDatabases:
  - data

The app then imports from harper normally:

import { databases } from 'harper';
await databases.data.MyTable.put({ id: 'x', ... });

…and databases.data transparently resolves to a branch named my-app:data. Other apps (and the base instance) see the unchanged base data.

Approach

RocksDB Checkpoint + per-application databases proxy + shared blob store.

1. On app load, for each entry in branchedDatabases, call baseDb.createCheckpoint(branchPath) (depends on Expose RocksDB Checkpoint API as createCheckpoint(path) rocksdb-js#577) to hardlink-clone the base DB.
2. Open the checkpoint via the existing database() factory (core/resources/databases.ts:740), registered under a namespaced key <appName>:<baseName>.
3. In getHarperExports(scope) (core/security/jsLoader.ts:678), wrap the global databases object in a Proxy so the app sees the branch where its config requests one. Mirror the same treatment for the tables flat alias when the default database is branched.
4. Blobs share the base's directory and ID allocator. The checkpoint only clones RocksDB; blob references in cloned records point at files under the base's blob dir. Branches must read/write the base's blob path and increment the base's blob ID counter so new blobs never collide. Branch sweep-deletes are gated by a high-water mark recorded at checkpoint time so the branch never removes a blob the base still references. See Branched databases: blob store sharing, allocator, and GC safety #644 for the full design.
5. On app unload / process exit, close the branch DB and fs.rm its directory. Add a startup sweep for orphaned branch directories from prior crashes.

Branches must not participate in cluster replication.

Subtasks

• Expose RocksDB Checkpoint API as createCheckpoint(path) rocksdb-js#577 — Expose createCheckpoint() in the native binding
• Branched databases: app config, scope proxy, lifecycle wiring #643 — App config, scope proxy, lifecycle wiring
• Branched databases: blob store sharing, allocator, and GC safety #644 — Blob store sharing, allocator, and GC safety
• Integration test: two apps with branched data, divergent writes (including blob writes), isolation assertions, base unchanged

Key files

• core/components/Application.ts (config schema lines 28–38, validation lines 72–109, prepareApplication lines 429–480)
• core/components/ApplicationScope.ts (add branches: Map<string, { branchName: string; hwm: bigint }>)
• core/components/componentLoader.ts (lines 307–315 — config plumbing into scope)
• core/security/jsLoader.ts (getHarperExports lines 678–716)
• core/resources/databases.ts (database() factory lines 740–794, cache line 771; startup sweep)
• core/resources/blob.ts (lines 614, 681–699, 775–815, 1218–1337 — shared path, shared allocator, gated delete, GC safety)

Explicit non-goals

• Persistence of branches across process restarts.
• Merge / promote branch writes back to base.
• Per-table (rather than per-database) branching.
• Branch sharing across applications.
• LMDB backend — RocksDB only.

🤖 Filed by Claude on behalf of Kris