Skip to content

Add attachment garbage collection sweep#92

Merged
cuibonobo merged 1 commit into
mainfrom
claude/issue-64-design-review-gc01
Jul 14, 2026
Merged

Add attachment garbage collection sweep#92
cuibonobo merged 1 commit into
mainfrom
claude/issue-64-design-review-gc01

Conversation

@cuibonobo

Copy link
Copy Markdown
Member

Summary

Closes #64.

Attachment bytes were only ever removed by an explicit deleteAttachment(fileId) call — normal app flows delete records, and nothing noticed when the last record referencing a file went away. This adds an explicit, owner-invoked sweep to find and remove those orphans.

  • Stack.collectAttachmentGarbage(opts?) / ScopedStack.collectAttachmentGarbage(opts?) (owner-only): collects attachment bytes no longer reachable from any record — live or soft-deleted — via an attachment Association or a file-ref content field (file-ref field kind: content-held file references are invisible to the reference checks the spec says exist #63). A soft-deleted record is recoverable via undelete() and must find its attachments intact, so it still counts as a reference.
  • _attachment@1 metadata records never themselves count as references (else nothing would ever be garbage), but a file's newest metadata record — or, for bare bytes with none, the blob's own storage timestamp — must be older than graceMs (default 24h) to be collected. This protects the legitimate upload-then-associate window.
  • Deletion goes through deleteAttachment() itself, so its usual conflict check runs once more per file at delete time; a file that races (referenced again, or already gone) is skipped, not treated as a sweep failure — the sweep always completes and reports what it actually collected.
  • dryRun: true computes and returns what would be deleted without deleting anything.
  • New optional StackBlobAdapter.listFiles() capability (same "optional method, checked for truthiness" pattern as deleteUnreferencedAttachmentRecords), implemented in blob-adapter-disk and MemoryAdapter, wired through LocalAdapter. This is what finds bare-bytes orphans — bytes with zero metadata records at all, left by a putAttachment() that stored bytes but crashed before writing metadata — which are otherwise undiscoverable. An adapter without it still gets full protection for the common case.

Two latent bugs fixed along the way

  • deleteAttachment()'s non-atomic fallback path didn't pass includeDeleted on its reference check or its metadata-cleanup scan, so a soft-deleted (recoverable) record's attachment reference didn't block deletion, and a soft-deleted metadata record was never cleaned up. The atomic adapter path already got this right (it doesn't filter on deleted_at at all); the fallback now matches.
  • combineAdapters() silently dropped optional adapter capabilities (deleteUnreferencedAttachmentRecords, and now listFiles) instead of forwarding them when the underlying part actually implements them — meaning a record adapter that supports the atomic delete path lost that capability the moment it was composed via combineAdapters().

Test plan

  • packages/core/tests/stack.test.tscollectAttachmentGarbage: basic collection, live/soft-deleted/file-ref reference protection, default and explicit grace period, reclaimedBytes accounting, dryRun, bare-bytes orphan collection via listFiles(), graceful behavior when listFiles() is absent, and a simulated delete-time race that's skipped without aborting the sweep. Plus regression tests for the two includeDeleted fixes.
  • packages/core/tests/scoped-stack.test.ts — owner-only gating (owner/non-owner/anonymous).
  • packages/core/tests/combine.test.ts (new) — optional-capability forwarding round-trips exactly (present when implemented, absent when not), plus basic composition and lifecycle behavior.
  • packages/blob-adapter-disk/tests/blob.test.tslistFiles() enumeration, sizes, timestamps, post-delete removal, ignoring stray non-fileId files.
  • packages/adapter-local/tests/local.test.tslistFiles() through the real combined adapter.
  • Full monorepo build, typecheck, lint, and test suite (627 tests) pass.

🤖 Generated with Claude Code


Generated by Claude Code

Adds Stack.collectAttachmentGarbage() / ScopedStack.collectAttachmentGarbage()
(owner-only): an explicit sweep that deletes attachment bytes no longer
reachable from any record, live or soft-deleted, via an attachment
Association or a file-ref content field (#63). _attachment@1 metadata
records never themselves count as references, but a grace period (default
24h) protects files whose newest metadata record — or, for bare bytes with
none, the blob's own modifiedAt — is too recent, covering the legitimate
upload-then-associate window. Deletion goes through deleteAttachment()
itself, so a file that races (referenced or already gone by delete time)
is skipped rather than aborting the sweep.

Adds an optional StackBlobAdapter.listFiles() capability (implemented in
blob-adapter-disk and MemoryAdapter, wired through LocalAdapter) so the
sweep can also find bare-bytes orphans left by a putAttachment() that
stored bytes but crashed before writing metadata.

Also fixes two latent gaps in deleteAttachment()'s non-atomic fallback
uncovered while building this: its reference check and metadata-cleanup
scan didn't pass includeDeleted, so a soft-deleted (recoverable) record's
attachment reference didn't block deletion, and a soft-deleted metadata
record wasn't cleaned up — both already handled correctly by the atomic
adapter path, now consistent. Fixes combineAdapters() silently dropping
optional adapter capabilities (deleteUnreferencedAttachmentRecords,
listFiles) instead of forwarding them when the underlying part
implements them.
@cuibonobo cuibonobo merged commit 605ca39 into main Jul 14, 2026
5 checks passed
@cuibonobo cuibonobo deleted the claude/issue-64-design-review-gc01 branch July 14, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Orphaned blob GC: deleting the last referencing record leaves bytes on disk forever

2 participants