From 49dde1db412350903cd7edc15c30856d09a701f0 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 1 Sep 2026 17:24:13 -0400 Subject: [PATCH 1/3] docs(resources): document oldestRetainedAuditTime and the subscription catch-up horizon `subscribe`'s `startTime` catch-up reads the audit log, so a consumer resuming past `logging.auditRetention` gets a replay that begins after the messages it missed. Nothing in the reference said so, because until now there was no supported way to detect it. Documents `Table.oldestRetainedAuditTime()` (HarperFast/harper#2447) with the resume pattern, and notes the horizon on the `startTime` row so a reader meets the caveat where they meet the option. Covers the parts a consumer can get wrong: the shared time domain, the database-scoped floor, `Infinity` as the fail-closed unknown, the one-way error direction, and that the reading is not a lock. Co-Authored-By: Claude Opus 5 --- reference/resources/resource-api.md | 48 ++++++++++++++++++++++++----- 1 file changed, 40 insertions(+), 8 deletions(-) diff --git a/reference/resources/resource-api.md b/reference/resources/resource-api.md index a5268333..0d3720d7 100644 --- a/reference/resources/resource-api.md +++ b/reference/resources/resource-api.md @@ -238,14 +238,46 @@ Called for MQTT subscribe commands. Returns a `Subscription` — an `AsyncIterab All properties are optional: -| Property | Description | -| -------------------- | ---------------------------------------------------------------------------------------------- | -| `includeDescendants` | Include all updates with an id prefixed by the subscribed id (e.g. `sub/*`) | -| `startTime` | Start from a past time (catch-up of historical messages). Cannot be used with `previousCount`. | -| `previousCount` | Return the last N updates/messages. Cannot be used with `startTime`. | -| `omitCurrent` | Do not send the current/retained record as the first update. | -| `rowFilter` | Synchronous JavaScript predicate applied to authoritative row values. | -| `eventFilter` | Synchronous JavaScript predicate for events that may not carry an authoritative row. | +| Property | Description | +| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `includeDescendants` | Include all updates with an id prefixed by the subscribed id (e.g. `sub/*`) | +| `startTime` | Start from a past time (catch-up of historical messages). Cannot be used with `previousCount`. Catch-up reads the audit log, so a `startTime` older than the retained history replays only what is left — see [`oldestRetainedAuditTime()`](#oldestretainedaudittime-number) below. | +| `previousCount` | Return the last N updates/messages. Cannot be used with `startTime`. | +| `omitCurrent` | Do not send the current/retained record as the first update. | +| `rowFilter` | Synchronous JavaScript predicate applied to authoritative row values. | +| `eventFilter` | Synchronous JavaScript predicate for events that may not carry an authoritative row. | + +--- + +### `oldestRetainedAuditTime(): number` + + + +The oldest point in the audit log from which an incremental catch-up is still complete — the retention floor. + +Subscription catch-up (`startTime`) reads the audit log, and the audit log is pruned on a retention window (`logging.auditRetention`). A consumer that saves a cursor, disconnects, and resumes past that window would otherwise receive a replay that quietly begins after the messages it missed. This method is how a consumer detects that instead: + +```javascript +const floor = tables.Product.oldestRetainedAuditTime(); +if (cursor >= floor) { + // every change after `cursor` is still in the log; resume incrementally + subscription = await tables.Product.subscribe({ startTime: cursor }); +} else { + // history this consumer needs has been pruned; re-read the table instead + await fullResync(); +} +``` + +The cursor is a _last-processed_ position, so a cursor exactly at the floor is safe — everything below it has already been handled. + +Details worth knowing before relying on it: + +- **The time domain is the same one `startTime` uses**, so cursors compare directly with no conversion. Subscription events carry it as `localTime`. +- **The floor is database-scoped, not per-table.** All tables in a database share one audit log, so `cursor >= floor` means no entry of _any_ table in that database was pruned below the cursor. `deleteHistory()` on one table raises the floor for its siblings too. +- **`Infinity` means the floor is unknown**, and no cursor should be treated as safe. This is what a database reports when its retention history cannot be accounted for — most commonly the first time it is opened by a version that records a floor, or after a migration between storage engines, which does not carry the audit log across. Consumers resync once and then get real values. +- **It errs in one direction only.** The floor can ask for a resync that was not strictly necessary; it does not report a cursor as safe when history it needed is gone. +- **It is a reading at a moment in time.** Retention can advance between this call and the `subscribe()` that follows it. The window is milliseconds against a retention window normally measured in days, and losing that race leaves you with the truncated replay you would have had anyway — but it is not a lock. +- Throws if the database has no audit log at all. --- From 90b35e7915243a4717bf09152db867aadbd46fbd Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Wed, 2 Sep 2026 11:14:06 -0400 Subject: [PATCH 2/3] docs(resources): split the floor's safety warnings into their own sentences Per review on #660, and the repo guidance it cites: a warning a reader needs to find while scanning should not be joined to a definition by "and" or hidden behind a semicolon. "Treat no cursor as safe" now stands on its own rather than trailing the definition of `Infinity`, and the one-direction guarantee is two sentences instead of a semicolon. Co-Authored-By: Claude Opus 5 --- reference/resources/resource-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/reference/resources/resource-api.md b/reference/resources/resource-api.md index 0d3720d7..e14c8bf0 100644 --- a/reference/resources/resource-api.md +++ b/reference/resources/resource-api.md @@ -274,8 +274,8 @@ Details worth knowing before relying on it: - **The time domain is the same one `startTime` uses**, so cursors compare directly with no conversion. Subscription events carry it as `localTime`. - **The floor is database-scoped, not per-table.** All tables in a database share one audit log, so `cursor >= floor` means no entry of _any_ table in that database was pruned below the cursor. `deleteHistory()` on one table raises the floor for its siblings too. -- **`Infinity` means the floor is unknown**, and no cursor should be treated as safe. This is what a database reports when its retention history cannot be accounted for — most commonly the first time it is opened by a version that records a floor, or after a migration between storage engines, which does not carry the audit log across. Consumers resync once and then get real values. -- **It errs in one direction only.** The floor can ask for a resync that was not strictly necessary; it does not report a cursor as safe when history it needed is gone. +- **`Infinity` means the floor is unknown.** Treat no cursor as safe. This is what a database reports when its retention history cannot be accounted for — most commonly the first time it is opened by a version that records a floor, or after a migration between storage engines, which does not carry the audit log across. Consumers resync once and then get real values. +- **It errs in one direction only.** The floor can ask for a resync that was not strictly necessary. It does not report a cursor as safe when history it needed is gone. - **It is a reading at a moment in time.** Retention can advance between this call and the `subscribe()` that follows it. The window is milliseconds against a retention window normally measured in days, and losing that race leaves you with the truncated replay you would have had anyway — but it is not a lock. - Throws if the database has no audit log at all. From 839ed7622b5a5015607fec3d3f315ad91a562c5d Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Wed, 2 Sep 2026 11:48:55 -0400 Subject: [PATCH 3/3] docs(resources): warn that getHistory's localTime is not a valid floor cursor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Chris caught a real gap on #660. The `getHistory` caveat was in auditStore.ts's JSDoc, Table.ts's JSDoc and resources/DESIGN.md, but not on the reference page — which is the one place a consumer building this check would actually look. The #660 description claimed it was covered; it was not. It matters because the two values share a name and differ only under backdated or replicated writes: a consumer that persists `getHistory().localTime` can pass `cursor >= floor` while the messages between them are already pruned, which is exactly the silent gap the accessor exists to expose. Co-Authored-By: Claude Opus 5 --- reference/resources/resource-api.md | 1 + 1 file changed, 1 insertion(+) diff --git a/reference/resources/resource-api.md b/reference/resources/resource-api.md index e14c8bf0..42ca6738 100644 --- a/reference/resources/resource-api.md +++ b/reference/resources/resource-api.md @@ -273,6 +273,7 @@ The cursor is a _last-processed_ position, so a cursor exactly at the floor is s Details worth knowing before relying on it: - **The time domain is the same one `startTime` uses**, so cursors compare directly with no conversion. Subscription events carry it as `localTime`. +- **Only a subscription event's `localTime` is a valid cursor here.** `getHistory()` also yields a `localTime`, but that one is the entry's origin version, which a backdated or replicated write makes differ from the audit-log position this floor describes. Do not persist `getHistory().localTime` and compare it against the floor: it can pass `cursor >= floor` while the messages between them have already been pruned, which is the silent gap this method exists to expose. - **The floor is database-scoped, not per-table.** All tables in a database share one audit log, so `cursor >= floor` means no entry of _any_ table in that database was pruned below the cursor. `deleteHistory()` on one table raises the floor for its siblings too. - **`Infinity` means the floor is unknown.** Treat no cursor as safe. This is what a database reports when its retention history cannot be accounted for — most commonly the first time it is opened by a version that records a floor, or after a migration between storage engines, which does not carry the audit log across. Consumers resync once and then get real values. - **It errs in one direction only.** The floor can ask for a resync that was not strictly necessary. It does not report a cursor as safe when history it needed is gone.