Skip to content

feat(database): add db.use() to access multiple column families - #829

Merged
cb1kenobi merged 4 commits into
mainfrom
feat/db-use-column-family
Sep 4, 2026
Merged

feat(database): add db.use() to access multiple column families#829
cb1kenobi merged 4 commits into
mainfrom
feat/db-use-column-family

Conversation

@cb1kenobi

@cb1kenobi cb1kenobi commented Sep 4, 2026

Copy link
Copy Markdown
Member

What

Adds RocksDatabase.use(name, options?) so a single db instance can interact with multiple column
families, instead of constructing a separate new RocksDatabase(path, { name }) per column family.

Closes #828

API

const db = RocksDatabase.open('/path/to/db');
const events = db.use('events');   // RocksDatabase bound to the 'events' CF

await events.put('e1', payload);
await db.put('k', 'v');            // default CF, unaffected

Create-and-open sugar, modeled on useLog. The DBDescriptor/columns machinery already opens
every column family and pins one per DBHandle, so this is TypeScript-only — no native change.

Semantics

  • Returns a RocksDatabase bound to name, opening (and creating, if missing) the CF on first use.
    It shares the same underlying database, so a transaction, backup, or checkpoint still spans every
    column family.
  • Get-or-create, weakly cached: db.use('events') === db.use('events') while the view is
    referenced and open; a closed or garbage-collected view is transparently recreated. The cache
    holds only a WeakRef (plus a FinalizationRegistry that reclaims map entries), so views are
    never pinned for the database's lifetime.
  • Views are independent handles: closing the parent does not close them (and vice versa); the
    underlying database stays open until every handle is closed or collected.
  • The own-column-family name returns this (opened); #name derives from the store, so
    Store-based construction binds the correct CF.
  • Works regardless of the parent's open state — a discarded parent still yields a valid, opened view.

Tests

test/column-families.test.ts covers: data isolation, cache identity, this-return (open + unopened
parent), Store-name derivation, cross-handle data sharing, parent/view close independence,
recreation of an explicitly closed view, per-view transactions, and input validation. GC-collection
is intentionally not asserted (unreliable under the runner per AGENTS.md — verified separately that
use()-created views are collectible).

Review

Ran a cross-model review (Codex + Gemini) on the change; all surfaced findings were resolved before
this PR — notably the own-name shortcut now honors create-and-open (returns this.open()),
Store-based construction derives #name from the store, the WeakRef map is reclaimed via a
FinalizationRegistry, and the docs scope the identity guarantee to referenced-and-open views.

Follow-ups (not in scope)

  • A narrower return type than RocksDatabase that hides whole-database verbs (backup/checkpoint)
    from a single-CF view.
  • Cross-column-family atomic writes through a single Transaction.

🤖 Generated with Claude Code

Review-Coverage: authored=claude; ran=codex; declined=gemini,cursor-grok,cursor-composer,domain; rounds=3 @ 73385e4

Human-Review-Need: 4 @ 73385e4

Add `RocksDatabase.use(name, options?)` — create-and-open sugar (modeled
on `useLog`) that returns a `RocksDatabase` bound to a named column family
of the same database, so a single instance can interact with multiple
column families instead of constructing one `RocksDatabase` per CF.

Views share the process-global, path-keyed DBDescriptor and are weakly
cached by name (WeakRef + FinalizationRegistry), so identity holds while a
view is referenced and open, a closed/collected view is transparently
recreated, and views are never pinned for the database's lifetime. Views
are independent handles: closing the parent does not close them and vice
versa. The own-column-family name returns `this` (opened). `#name` derives
from the store so Store-based construction binds the correct CF.

Closes #828

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cb1kenobi
cb1kenobi requested a review from kriszyp as a code owner September 4, 2026 07:21

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the use(name, options?) method to RocksDatabase, allowing users to open and cache column-family views using a weak cache (WeakRef and FinalizationRegistry). The changes also include updated documentation and comprehensive tests. Feedback on the implementation suggests dynamically preserving custom Store subclasses when instantiating column-family views, and avoiding redundant map deletions in the finalization registry callback.

Comment thread src/database.ts Outdated
Comment thread src/database.ts Outdated
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 24.52K ops/sec 40.79 39.51 480.399 0.112 122,592
🥈 rocksdb 2 10.95K ops/sec 91.36 87.11 24,278.608 0.980 54,728

getSync() > sequential keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 28.95K ops/sec 34.54 33.47 544.224 0.102 144,773
🥈 rocksdb 2 11.54K ops/sec 86.69 83.49 551.002 0.052 57,680

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 24.52K ops/sec 40.79 36.55 2,077.922 0.276 122,591
🥈 rocksdb 2 16.14K ops/sec 61.97 52.12 1,064.219 0.120 80,681

realistic-load.bench.ts

Realistic write load with workers > write variable records with transaction log

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 359.39 ops/sec 2,782.482 76.14 71,430.382 18.46 719
🥈 lmdb 2 26.19 ops/sec 38,181.14 454.982 1,197,803.71 136.603 64.00

transaction-log.bench.ts

Transaction log > read 100 iterators while write log with 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 39.20K ops/sec 25.51 10.47 20,628.503 0.847 195,979
🥈 lmdb 2 437.90 ops/sec 2,283.62 102.383 8,396.097 1.17 2,190

Transaction log > read one entry from random position from log with 1000 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 708.71K ops/sec 1.41 1.21 3,697.448 0.165 3,543,549
🥈 lmdb 2 462.39K ops/sec 2.16 1.14 762.379 0.274 2,311,971

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 10 workers)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 832.35 ops/sec 1,201.425 1,022.51 8,342.149 0.772 1,665
🥈 lmdb 2 1.15 ops/sec 865,997.18 782,928.483 908,225.769 2.99 10.00

worker-transaction-log.bench.ts

Transaction log with workers > write log with 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 19.44K ops/sec 51.45 29.99 21,459.523 2.35 38,876
🥈 lmdb 2 804.86 ops/sec 1,242.452 265.541 13,342.345 5.43 1,610

Results from commit 7ef9e25

Address review feedback: build a column-family view's store from the
parent store's own class (this.store.constructor) rather than the base
Store, so a custom Store subclass's overridden behavior carries over to
views; the merged options are still passed so a nested use() inherits.
Also capture the WeakRef in the FinalizationRegistry callback to skip a
redundant Map.delete on an already-removed/replaced entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@kriszyp kriszyp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat, this is a cool API. A few things to check on from comments, but I like it.
🤖 Reviewed with Codex

Comment thread src/database.ts Outdated
Comment thread src/database.ts Outdated
Address maintainer review (kriszyp): `use()` built the view's store with a
shallow options spread, which shared reference-valued `encoder`/`decoder`
instances between the parent and its supposedly independent view — opening
the view mutated the shared encoder's `name`, and clearing either view
reset its `structures`, corrupting the sibling's codec state. It also
assumed a custom Store could be rebuilt from `(path, options)`.

Move view derivation behind `Store#createColumnFamilyStore(name, options)`:
an overridable factory that builds an independent store of the same class
from a snapshot of the store's construction options. A pre-constructed
encoder/decoder instance (mutable, would be shared) is now rejected with a
clear message pointing to an encoder factory / named encoding / the
override; the base factory also asserts the derived store is bound to the
requested column family (catches a subclass that drops options). Custom
stores with injected dependencies override the hook. Removes the unused
RocksDatabase#options.

Tests: codec-state isolation, rejection of a shared encoder instance, a
custom-constructor subclass overriding the hook (with data isolation), and
the non-forwarding-subclass guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cb1kenobi cb1kenobi left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 73385e4

Comment thread src/store.ts
createColumnFamilyStore(name: string, options?: StoreOptions): Store {
const derived: StoreOptions = { ...this.#columnFamilyOptions, ...options, name };

if (sharesMutableCodec(derived.encoder) || sharesMutableCodec(derived.decoder)) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: per-view encoder instance is rejected

sharesMutableCodec runs on the merged options, so db.use('events', { encoder: { encode, decode } }) throws even when the parent has no encoder (or a factory) and this instance would live only on the new store. Opening then assigns it to the view alone; the parent's codec is a different object.

Suggested fix: reject (or fork) only when derived.encoder === this.#columnFamilyOptions?.encoder (same for decoder). A caller-supplied per-view instance is already independent.

@kriszyp kriszyp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect. Great work.
🤖 Reviewed (poorly) with Codex

@cb1kenobi
cb1kenobi merged commit 7ab102c into main Sep 4, 2026
26 checks passed
@cb1kenobi
cb1kenobi deleted the feat/db-use-column-family branch September 4, 2026 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add db.use(name) to access multiple column families from a single database instance

2 participants