fix(lore): use live SQL coverage gate cached on vector_epoch - #120
Open
syf2211 wants to merge 1 commit into
Open
fix(lore): use live SQL coverage gate cached on vector_epoch#120syf2211 wants to merge 1 commit into
syf2211 wants to merge 1 commit into
Conversation
Appraise's RRF semantic arm was gated on meta.vector_coverage_* counters that drift from live lore_vectors/entries counts. When stale meta reported coverage below 0.90, the semantic arm was silently disabled even though vectors on disk were healthy. - Compute appraise coverage from live COUNT(*) cached on vector_epoch - Add ReconcileNum and extend coverage-reconcile to reset num + den - Add regression tests for drift scenario and epoch cache hot path Fixes mathomhaus#76
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Appraise's RRF semantic arm now computes coverage from live
COUNT(*)queries (lore_vectors rows vs active entries), cached onmeta.vector_epoch, instead of trusting drift-pronemeta.vector_coverage_*counters. Also extendsguild lore coverage-reconcileto reset bothvector_coverage_numandvector_coverage_den.Fixes #76
Motivation
When
meta.vector_coverage_num/dendrift from live state, appraise can report coverage below the 0.90 RRF gate and silently disable the semantic arm — even though vectors on disk are healthy and auto-backfill sees the corpus as fine. This is a silent retrieval-quality regression for paraphrased queries.Changes
readCoverage/liveCoverageCounts: epoch-cached live SQL gate inappraise_rrf.goembed.ReconcileNum: resetvector_coverage_numfrom liveCOUNT(*) FROM lore_vectorscoverage-reconcilecommand: reconcile both num and den, report before/after for bothTests
All lore and embed tests pass.
Notes
vector_epochbumps (vector writes / backfill). Denominator-only inscribe/seal changes may leave a brief stale window; this is documented and bounded, unlike the permanent meta-counter drift this fixes.