Skip to content

feat(bigquery-jdbc): implement non-blocking telemetry batcher and dispatcher#13772

Open
Neenu1995 wants to merge 5 commits into
jdbc-telemetry-featurefrom
jdbc-telemetry-pr7
Open

feat(bigquery-jdbc): implement non-blocking telemetry batcher and dispatcher#13772
Neenu1995 wants to merge 5 commits into
jdbc-telemetry-featurefrom
jdbc-telemetry-pr7

Conversation

@Neenu1995

@Neenu1995 Neenu1995 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Finalizes the TelemetryBatcher infrastructure for high-performance, non-blocking telemetry collection and dispatching in the BigQuery JDBC driver.

Key Functionality Added:

  • Non-Blocking Telemetry Ingestion: Buffers connection attempts, statement executions, error metrics, and feature usage events in capacity-bounded LinkedBlockingQueue storage (max 10,000 items per queue).
  • Fine-Grained Concurrency Control: Uses an explicit ReentrantLock (flushLock) for thread-safe flush operations, task rescheduling, and shutdown without locking producer threads.
  • Efficient Payload Sizing: Implements single-pass bulk payload trimming to strictly enforce the 512 KB payload threshold before transport transmission.
  • Optimized Batch Draining: Leverages BlockingQueue.drainTo() to drain telemetry batches in a single operation, minimizing lock acquisition overhead.
  • Conditional Resource Allocation: Lazily allocates the ScheduledExecutorService and background dispatcher thread only when telemetry is explicitly enabled.

@Neenu1995 Neenu1995 requested review from a team as code owners July 14, 2026 20:55

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces TelemetryBatcher, a class designed to buffer and periodically flush telemetry events, and increases the default batch size threshold to 5000. The review feedback highlights critical performance bottlenecks in the implementation. Specifically, it recommends replacing ConcurrentLinkedQueue with LinkedBlockingQueue to avoid O(N) size() operations, refactoring helper methods to use the generic Queue interface, and optimizing the payload trimming loop in flush() to remove items in bulk rather than one-by-one to prevent expensive, repetitive protobuf serialization size checks. Additionally, using explicit locks instead of synchronized blocks is recommended for performance-sensitive code.

@Neenu1995

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces TelemetryBatcher to buffer and periodically flush telemetry events, along with increasing the default batch size threshold in TelemetryConfiguration from 100 to 5000. The review feedback focuses on improving robustness and performance, suggesting: avoiding potential IllegalStateException by using queue.offer instead of queue.addAll when requeuing; optimizing queue draining by declaring queues as LinkedBlockingQueue and using drainTo to reduce lock contention; and lazily initializing the ScheduledExecutorService only when telemetry is enabled to prevent resource leaks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant