feat(timeseries): add time-series analysis extension with embedding similarity and drift detection - #44
Open
chiangchenghsin-hash wants to merge 10 commits into
Conversation
- EMBEDDING_SIMILARITY: generic N-dimension cosine/feature similarity - DETECT_DRIFT_POINTS: generic drift detection on sequential embeddings (generalized from bitemporal's character_similarity + detect_turning_points)
…ne similarity Generalized from character_similarity (hardcoded 4 features) to accept embeddings of any dimension. Returns cosine similarity + mean feature similarity + dimension count.
…n embeddings Generalized from detect_turning_points (hardcoded chapter labels) to accept generic labels (commit hashes, timestamps, version numbers, etc.). Use cases: architecture drift, content drift, behavior drift detection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
timeseriesextension with two table functions for time-series analysis on embedding vectors:EMBEDDING_SIMILARITY
Compute cosine similarity and per-dimension feature similarity between two embedding vectors of any dimension (not hardcoded).
Returns:
cosine_similarity([-1, 1]),feature_similarity([0, 1]),dimension_count.DETECT_DRIFT_POINTS
Detect drift points in a sequence of embeddings by computing pairwise cosine distances between consecutive vectors. Generic labels (commit hashes, timestamps, version numbers) replace hardcoded chapter numbers.
Returns:
label(INT64),drift_magnitude(DOUBLE),significance(DOUBLE),direction("up"/"down").Use Cases
Design
SimpleTableFuncpatterninferInputTypescallback forANY → LIST<DOUBLE>type resolutionCOLLECTat app layer — no table scans neededFiles
timeseries/CMakeLists.txttimeseries/src/function/CMakeLists.txttimeseries/src/main/CMakeLists.txttimeseries/src/include/function/timeseries_function.htimeseries/src/include/main/timeseries_extension.htimeseries/src/main/timeseries_extension.cpptimeseries/src/function/embedding_similarity.cpptimeseries/src/function/detect_drift_points.cpptimeseries/test/test_files/timeseries.testtimeseries/README.mdMotivation
This extension was developed for architecture governance (detecting when code architecture drifts from the intended design via embedding comparison). It generalizes domain-specific bitemporal analysis functions into reusable primitives that work with any embedding space.