feat(database): add db.use() to access multiple column families - #829
Conversation
Add `RocksDatabase.use(name, options?)` — create-and-open sugar (modeled on `useLog`) that returns a `RocksDatabase` bound to a named column family of the same database, so a single instance can interact with multiple column families instead of constructing one `RocksDatabase` per CF. Views share the process-global, path-keyed DBDescriptor and are weakly cached by name (WeakRef + FinalizationRegistry), so identity holds while a view is referenced and open, a closed/collected view is transparently recreated, and views are never pinned for the database's lifetime. Views are independent handles: closing the parent does not close them and vice versa. The own-column-family name returns `this` (opened). `#name` derives from the store so Store-based construction binds the correct CF. Closes #828 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces the use(name, options?) method to RocksDatabase, allowing users to open and cache column-family views using a weak cache (WeakRef and FinalizationRegistry). The changes also include updated documentation and comprehensive tests. Feedback on the implementation suggests dynamically preserving custom Store subclasses when instantiating column-family views, and avoiding redundant map deletions in the finalization registry callback.
📊 Benchmark Resultsget-sync.bench.tsgetSync() > random keys - small key size (100 records)
getSync() > sequential keys - small key size (100 records)
ranges.bench.tsgetRange() > small range (100 records, 50 range)
realistic-load.bench.tsRealistic write load with workers > write variable records with transaction log
transaction-log.bench.tsTransaction log > read 100 iterators while write log with 100 byte records
Transaction log > read one entry from random position from log with 1000 100 byte records
worker-put-sync.bench.tsputSync() > random keys - small key size (100 records, 10 workers)
worker-transaction-log.bench.tsTransaction log with workers > write log with 100 byte records
Results from commit 7ef9e25 |
Address review feedback: build a column-family view's store from the parent store's own class (this.store.constructor) rather than the base Store, so a custom Store subclass's overridden behavior carries over to views; the merged options are still passed so a nested use() inherits. Also capture the WeakRef in the FinalizationRegistry callback to skip a redundant Map.delete on an already-removed/replaced entry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kriszyp
left a comment
There was a problem hiding this comment.
Neat, this is a cool API. A few things to check on from comments, but I like it.
🤖 Reviewed with Codex
Address maintainer review (kriszyp): `use()` built the view's store with a shallow options spread, which shared reference-valued `encoder`/`decoder` instances between the parent and its supposedly independent view — opening the view mutated the shared encoder's `name`, and clearing either view reset its `structures`, corrupting the sibling's codec state. It also assumed a custom Store could be rebuilt from `(path, options)`. Move view derivation behind `Store#createColumnFamilyStore(name, options)`: an overridable factory that builds an independent store of the same class from a snapshot of the store's construction options. A pre-constructed encoder/decoder instance (mutable, would be shared) is now rejected with a clear message pointing to an encoder factory / named encoding / the override; the base factory also asserts the derived store is bound to the requested column family (catches a subclass that drops options). Custom stores with injected dependencies override the hook. Removes the unused RocksDatabase#options. Tests: codec-state isolation, rejection of a shared encoder instance, a custom-constructor subclass overriding the hook (with data isolation), and the non-forwarding-subclass guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| createColumnFamilyStore(name: string, options?: StoreOptions): Store { | ||
| const derived: StoreOptions = { ...this.#columnFamilyOptions, ...options, name }; | ||
|
|
||
| if (sharesMutableCodec(derived.encoder) || sharesMutableCodec(derived.decoder)) { |
There was a problem hiding this comment.
Medium: per-view encoder instance is rejected
sharesMutableCodec runs on the merged options, so db.use('events', { encoder: { encode, decode } }) throws even when the parent has no encoder (or a factory) and this instance would live only on the new store. Opening then assigns it to the view alone; the parent's codec is a different object.
Suggested fix: reject (or fork) only when derived.encoder === this.#columnFamilyOptions?.encoder (same for decoder). A caller-supplied per-view instance is already independent.
What
Adds
RocksDatabase.use(name, options?)so a singledbinstance can interact with multiple columnfamilies, instead of constructing a separate
new RocksDatabase(path, { name })per column family.Closes #828
API
Create-and-open sugar, modeled on
useLog. TheDBDescriptor/columnsmachinery already opensevery column family and pins one per
DBHandle, so this is TypeScript-only — no native change.Semantics
RocksDatabasebound toname, opening (and creating, if missing) the CF on first use.It shares the same underlying database, so a transaction, backup, or checkpoint still spans every
column family.
db.use('events') === db.use('events')while the view isreferenced and open; a closed or garbage-collected view is transparently recreated. The cache
holds only a
WeakRef(plus aFinalizationRegistrythat reclaims map entries), so views arenever pinned for the database's lifetime.
underlying database stays open until every handle is closed or collected.
this(opened);#namederives from the store, soStore-based construction binds the correct CF.Tests
test/column-families.test.tscovers: data isolation, cache identity,this-return (open + unopenedparent),
Store-name derivation, cross-handle data sharing, parent/view close independence,recreation of an explicitly closed view, per-view transactions, and input validation. GC-collection
is intentionally not asserted (unreliable under the runner per AGENTS.md — verified separately that
use()-created views are collectible).Review
Ran a cross-model review (Codex + Gemini) on the change; all surfaced findings were resolved before
this PR — notably the own-name shortcut now honors create-and-open (returns
this.open()),Store-based construction derives#namefrom the store, theWeakRefmap is reclaimed via aFinalizationRegistry, and the docs scope the identity guarantee to referenced-and-open views.Follow-ups (not in scope)
RocksDatabasethat hides whole-database verbs (backup/checkpoint)from a single-CF view.
Transaction.🤖 Generated with Claude Code
Review-Coverage: authored=claude; ran=codex; declined=gemini,cursor-grok,cursor-composer,domain; rounds=3 @ 73385e4
Human-Review-Need: 4 @ 73385e4