Conversation
…e-endian f32 at a fixed dim * 4 stride. Appends are now O(new chunks). Capturing a page writes only its own vectors instead of re-serializing the whole library. That's the quadratic fix — and note that moving a capture between hubs now touches zero vector bytes, since only collectionId changes in the metadata. Crash ordering is deliberate: the sidecar is written and sync_all()'d before the metadata lands. Orphaned vector bytes are harmless; metadata pointing at slots that don't exist would be corruption. That ordering is the whole safety argument. push_chunks is the only slot allocator. The old code did vectors.chunks.extend(...) directly; any caller doing that now would desynchronize the stride index, so allocation lives in one place with a comment saying so. Deletes leave dead slots, reclaimed by compaction once they exceed half the file (with a 512-slot floor, so routine deletes stay cheap). Migration: a v1 store is detected by peeking version before deserializing — necessary because vector is now #[serde(skip)], so the v2 deserializer would have silently discarded every embedding. The v1 file is kept as .bak. Degradation is graceful: a missing or truncated sidecar drops the affected chunks with a warning rather than failing the load, so a damaged sidecar costs re-indexing, not the library. 6 new tests on the risky parts: byte-exact round-trip (plus asserting no vector numbers remain in the JSON), append-only growth (16 → 32 bytes, not a rewrite), v1 migration preserving vectors with .bak retained, missing-sidecar survival, compaction correctly renumbering slots and keeping each vector matched to its chunk, and dimension-mismatch rejection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a major ÆTHER overhaul spanning the native Tauri backend, local AI workflows, browser shell, research surfaces, reliability, accessibility, packaging, and documentation.
Local research and AI
Storage, performance, and resilience
chunks.vecsidecar vectors plus lightweight JSON metadata.Browser and desktop shell
UI, accessibility, and appearance
Security, releases, and developer workflow
Validation