Skip to content

fix(queue): preserve durable records across retries - #740

Draft
dustinbyrne wants to merge 3 commits into
mainfrom
fix/durable-queue-lifecycle
Draft

fix(queue): preserve durable records across retries#740
dustinbyrne wants to merge 3 commits into
mainfrom
fix/durable-queue-lifecycle

Conversation

@dustinbyrne

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Durable event and replay queues currently couple retention to flush retry exhaustion: repeated retryable failures can clear every persisted record. Queue filenames also reuse payload UUIDs, so duplicate payload identities do not represent distinct durable entries.

This aligns the shared file-backed queue with PostHog/sdk-specs#53: retryable transport and HTTP failures retain bounded durable records, known-offline periods consume no attempts, and every enqueue has a stable queue-entry identity. HTTP 413 continues shrinking to a singleton poison record.

Cross-SDK implementation: PostHog/posthog-ios#788

💚 How did you test it?

  • ./gradlew :posthog:test --tests 'com.posthog.internal.PostHogQueueTest'
  • make testJava
  • make checkFormat
  • make api — no API snapshot changes
  • git diff --check
  • independent read-only implementation and cross-SDK reviews found no blockers

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with Pi using isolated implementation, scout, and review subagents. The human directed the durability contract and cross-SDK scope. Queue capacity and bounded backoff remain the resource controls; no public option or default was added.

@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "fix(queue): preserve durable records acr..." | Re-trigger Greptile

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

posthog-android Compliance Report

Date: 2026-09-07 20:53:42 UTC
Duration: 118399ms

✅ All Tests Passed!

46/46 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 387ms
Format Validation.Event Has Uuid 32ms
Format Validation.Event Has Lib Properties 28ms
Format Validation.Distinct Id Is String 27ms
Format Validation.Token Is Present 22ms
Format Validation.Custom Properties Preserved 25ms
Format Validation.Event Has Timestamp 24ms
Retry Behavior.Retries On 503 7033ms
Retry Behavior.Does Not Retry On 400 4026ms
Retry Behavior.Does Not Retry On 401 4025ms
Retry Behavior.Respects Retry After Header 7027ms
Retry Behavior.Implements Backoff 17035ms
Retry Behavior.Retries On 500 7014ms
Retry Behavior.Retries On 502 7021ms
Retry Behavior.Retries On 504 7020ms
Retry Behavior.Max Retries Respected 17035ms
Deduplication.Generates Unique Uuids 36ms
Deduplication.Preserves Uuid On Retry 7016ms
Deduplication.Preserves Uuid And Timestamp On Retry 12021ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 7017ms
Deduplication.No Duplicate Events In Batch 37ms
Deduplication.Different Events Have Different Uuids 25ms
Compression.Sends Gzip When Enabled 20ms
Batch Format.Uses Proper Batch Structure 20ms
Batch Format.Flush With No Events Sends Nothing 14ms
Batch Format.Multiple Events Batched Together 34ms
Error Handling.Does Not Retry On 403 4021ms
Error Handling.Does Not Retry On 413 4021ms
Error Handling.Retries On 408 5027ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 32ms
Request Payload.Flags Request Uses V2 Query Param 24ms
Request Payload.Flags Request Hits Flags Path Not Decide 20ms
Request Payload.Flags Request Omits Authorization Header 23ms
Request Payload.Token In Flags Body Matches Init 23ms
Request Payload.Groups Round Trip 33ms
Request Payload.Groups Default To Empty Object 25ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 21ms
Request Payload.Disable Geoip Omitted Defaults To False 21ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 20ms
Request Lifecycle.No Flags Request On Init Alone 12ms
Request Lifecycle.No Flags Request On Normal Capture 20ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 38ms
Request Lifecycle.Mock Response Value Is Returned To Caller 23ms
Retry Behavior.Retries Flags On 502 321ms
Retry Behavior.Retries Flags On 504 321ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 23ms

@dustinbyrne
dustinbyrne force-pushed the fix/durable-queue-lifecycle branch from e99dfe0 to 438a88a Compare September 4, 2026 20:03
@dustinbyrne
dustinbyrne marked this pull request as ready for review September 4, 2026 20:17
@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Reviews (2): Last reviewed commit: "fix(queue): preserve durable records acr..." | Re-trigger Greptile

@dustinbyrne

Copy link
Copy Markdown
Contributor Author

@dustinbyrne
dustinbyrne marked this pull request as draft September 9, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants