Skip to content

perf(query): allow writes during snapshot searches - #23

Open
zhangzhenghao wants to merge 1 commit into
mainfrom
optimize/query-write-concurrency
Open

perf(query): allow writes during snapshot searches#23
zhangzhenghao wants to merge 1 commit into
mainfrom
optimize/query-write-concurrency

Conversation

@zhangzhenghao

Copy link
Copy Markdown
Collaborator

Summary

  • pin immutable collection query snapshots so Query, MultiQuery, and GroupByQuery release the collection read lock before long-running search work
  • keep segment runtimes alive with query/cache ownership until the last lease is released
  • make Close and Destroy wait for active query leases and retry/report delayed runtime close errors
  • capture schema in the same snapshot generation as documents and runtime indexes

This follows Alibaba zvec's separation between short collection-handle locking and stable segment ownership, implemented with pure-Go lifecycle management.

Tests

  • go test ./... -count=1 -timeout=20m
  • go test -race ./... -count=1 -timeout=30m
  • go vet ./...
  • concurrency and runtime lifecycle tests repeated with -count=20/-count=50
  • git diff --check

Scope

This change addresses query/write lock contention and safe runtime reclamation. It does not change the on-disk format, writer serialization, or ANN algorithms.

@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.38710% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.54%. Comparing base (07b981b) to head (f2bfd4f).

Files with missing lines Patch % Lines
collection.go 87.67% 10 Missing and 8 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #23      +/-   ##
==========================================
+ Coverage   61.44%   61.54%   +0.09%     
==========================================
  Files         150      150              
  Lines       40358    40462     +104     
==========================================
+ Hits        24799    24901     +102     
- Misses      12344    12346       +2     
  Partials     3215     3215              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants