feat(bigquery-jdbc): implement non-blocking telemetry batcher and dispatcher#13772
feat(bigquery-jdbc): implement non-blocking telemetry batcher and dispatcher#13772Neenu1995 wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces TelemetryBatcher, a class designed to buffer and periodically flush telemetry events, and increases the default batch size threshold to 5000. The review feedback highlights critical performance bottlenecks in the implementation. Specifically, it recommends replacing ConcurrentLinkedQueue with LinkedBlockingQueue to avoid O(N) size() operations, refactoring helper methods to use the generic Queue interface, and optimizing the payload trimming loop in flush() to remove items in bulk rather than one-by-one to prevent expensive, repetitive protobuf serialization size checks. Additionally, using explicit locks instead of synchronized blocks is recommended for performance-sensitive code.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces TelemetryBatcher to buffer and periodically flush telemetry events, along with increasing the default batch size threshold in TelemetryConfiguration from 100 to 5000. The review feedback focuses on improving robustness and performance, suggesting: avoiding potential IllegalStateException by using queue.offer instead of queue.addAll when requeuing; optimizing queue draining by declaring queues as LinkedBlockingQueue and using drainTo to reduce lock contention; and lazily initializing the ScheduledExecutorService only when telemetry is enabled to prevent resource leaks.
Finalizes the
TelemetryBatcherinfrastructure for high-performance, non-blocking telemetry collection and dispatching in the BigQuery JDBC driver.Key Functionality Added:
LinkedBlockingQueuestorage (max 10,000 items per queue).ReentrantLock(flushLock) for thread-safe flush operations, task rescheduling, and shutdown without locking producer threads.BlockingQueue.drainTo()to drain telemetry batches in a single operation, minimizing lock acquisition overhead.ScheduledExecutorServiceand background dispatcher thread only when telemetry is explicitly enabled.