ci: cover core, Android and Java SDK harness profiles - #767
Draft
dustinbyrne wants to merge 5 commits into
Draft
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
Author
SDK compliance report — all CI profilesSDK head: These reports are from the completed CI artifacts linked below. All selected results are included; assertion failures remain advisory and are not hidden by successful workflow checks.
Core and Java use genuine JVM entries; Android uses the genuine Android API35/x86_64 app. The SDK owns retries and flag results. Standalone adapter Docker-image validation is separate and was not performed. core: 45/47 passed, 2 failed — full reportposthog-core-integration Compliance ReportDate: 2026-09-08T15:16:46.338902+00:00
|
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 378ms |
| Format Validation.Event Has Uuid | ✅ | 52ms |
| Format Validation.Event Has Lib Properties | ✅ | 47ms |
| Format Validation.Distinct Id Is String | ✅ | 44ms |
| Format Validation.Token Is Present | ✅ | 43ms |
| Format Validation.Custom Properties Preserved | ✅ | 46ms |
| Format Validation.Event Has Timestamp | ✅ | 39ms |
| Format Validation.Non Utc Event Timestamp Is Converted To Utc | ✅ | 45ms |
| Retry Behavior.Retries On 503 | ✅ | 10019ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 12039ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 12042ms |
| Retry Behavior.Respects Retry After Header | ✅ | 9037ms |
| Retry Behavior.Implements Backoff | ✅ | 25043ms |
| Retry Behavior.Retries On 500 | ✅ | 7028ms |
| Retry Behavior.Retries On 502 | ✅ | 7026ms |
| Retry Behavior.Retries On 504 | ✅ | 7023ms |
| Retry Behavior.Max Retries Respected | ✅ | 25049ms |
| Deduplication.Generates Unique Uuids | ✅ | 48ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 7022ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ✅ | 15019ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ✅ | 7032ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 50ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 40ms |
| Compression.Sends Gzip When Enabled | ✅ | 37ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 37ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 10ms |
| Batch Format.Multiple Events Batched Together | ✅ | 47ms |
| Error Handling.Does Not Retry On 403 | ✅ | 12034ms |
| Error Handling.Does Not Retry On 413 | ✅ | 12041ms |
| Error Handling.Retries On 408 | ✅ | 5043ms |
Feature_Flags Tests
View Details
| Test | Status | Duration |
|---|---|---|
| Request Payload.Request With Person Properties Device Id | ❌ | 28ms |
| Request Payload.Flags Request Uses V2 Query Param | ✅ | 17ms |
| Request Payload.Flags Request Hits Flags Path Not Decide | ✅ | 16ms |
| Request Payload.Flags Request Omits Authorization Header | ✅ | 15ms |
| Request Payload.Token In Flags Body Matches Init | ✅ | 17ms |
| Request Payload.Groups Round Trip | ✅ | 17ms |
| Request Payload.Groups Default To Empty Object | ✅ | 15ms |
| Request Payload.Disable Geoip False Propagates As Geoip Disable False | ✅ | 16ms |
| Request Payload.Disable Geoip Omitted Defaults To False | ✅ | 17ms |
| Request Payload.Flag Keys To Evaluate Contains Only Requested Key | ❌ | 15ms |
| Request Lifecycle.No Flags Request On Init Alone | ✅ | 7ms |
| Request Lifecycle.No Flags Request On Normal Capture | ✅ | 40ms |
| Request Lifecycle.Two Flag Calls Produce Two Remote Requests | ✅ | 33ms |
| Request Lifecycle.Mock Response Value Is Returned To Caller | ✅ | 18ms |
| Retry Behavior.Retries Flags On 502 | ✅ | 335ms |
| Retry Behavior.Retries Flags On 504 | ✅ | 325ms |
| Side Effect Events.Get Feature Flag Captures Feature Flag Called Event | ✅ | 45ms |
Failures
request_payload.request_with_person_properties_device_id
Expected geoip_disable=True, got False
request_payload.flag_keys_to_evaluate_contains_only_requested_key
Field 'flag_keys_to_evaluate' not found in /flags request body at path 'flag_keys_to_evaluate'. Available keys: ['$anon_distinct_id', '$device_id', 'geoip_disable', 'api_key', 'group_properties', 'distinct_id', 'timezone', 'person_properties', 'groups']
Android: 45/47 passed, 2 failed — full report
posthog-android-entry Compliance Report
Date: 2026-09-08T15:19:07.449503+00:00
Duration: 182094ms
⚠️ Some Tests Failed
45/47 tests passed, 2 failed
Capture Tests
✅ 30/30 tests passed
View Details
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 3473ms |
| Format Validation.Event Has Uuid | ✅ | 481ms |
| Format Validation.Event Has Lib Properties | ✅ | 442ms |
| Format Validation.Distinct Id Is String | ✅ | 336ms |
| Format Validation.Token Is Present | ✅ | 243ms |
| Format Validation.Custom Properties Preserved | ✅ | 259ms |
| Format Validation.Event Has Timestamp | ✅ | 220ms |
| Format Validation.Non Utc Event Timestamp Is Converted To Utc | ✅ | 287ms |
| Retry Behavior.Retries On 503 | ✅ | 10149ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 12163ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 12330ms |
| Retry Behavior.Respects Retry After Header | ✅ | 9162ms |
| Retry Behavior.Implements Backoff | ✅ | 25078ms |
| Retry Behavior.Retries On 500 | ✅ | 7044ms |
| Retry Behavior.Retries On 502 | ✅ | 7043ms |
| Retry Behavior.Retries On 504 | ✅ | 7052ms |
| Retry Behavior.Max Retries Respected | ✅ | 25057ms |
| Deduplication.Generates Unique Uuids | ✅ | 108ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 7052ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ✅ | 15053ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ✅ | 7047ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 108ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 80ms |
| Compression.Sends Gzip When Enabled | ✅ | 63ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 88ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 43ms |
| Batch Format.Multiple Events Batched Together | ✅ | 111ms |
| Error Handling.Does Not Retry On 403 | ✅ | 12059ms |
| Error Handling.Does Not Retry On 413 | ✅ | 12072ms |
| Error Handling.Retries On 408 | ✅ | 5080ms |
Feature_Flags Tests
View Details
| Test | Status | Duration |
|---|---|---|
| Request Payload.Request With Person Properties Device Id | ❌ | 64ms |
| Request Payload.Flags Request Uses V2 Query Param | ✅ | 68ms |
| Request Payload.Flags Request Hits Flags Path Not Decide | ✅ | 87ms |
| Request Payload.Flags Request Omits Authorization Header | ✅ | 54ms |
| Request Payload.Token In Flags Body Matches Init | ✅ | 67ms |
| Request Payload.Groups Round Trip | ✅ | 52ms |
| Request Payload.Groups Default To Empty Object | ✅ | 70ms |
| Request Payload.Disable Geoip False Propagates As Geoip Disable False | ✅ | 56ms |
| Request Payload.Disable Geoip Omitted Defaults To False | ✅ | 56ms |
| Request Payload.Flag Keys To Evaluate Contains Only Requested Key | ❌ | 54ms |
| Request Lifecycle.No Flags Request On Init Alone | ✅ | 25ms |
| Request Lifecycle.No Flags Request On Normal Capture | ✅ | 85ms |
| Request Lifecycle.Two Flag Calls Produce Two Remote Requests | ✅ | 75ms |
| Request Lifecycle.Mock Response Value Is Returned To Caller | ✅ | 63ms |
| Retry Behavior.Retries Flags On 502 | ✅ | 368ms |
| Retry Behavior.Retries Flags On 504 | ✅ | 370ms |
| Side Effect Events.Get Feature Flag Captures Feature Flag Called Event | ✅ | 117ms |
Failures
request_payload.request_with_person_properties_device_id
Expected geoip_disable=True, got False
request_payload.flag_keys_to_evaluate_contains_only_requested_key
Field 'flag_keys_to_evaluate' not found in /flags request body at path 'flag_keys_to_evaluate'. Available keys: ['$anon_distinct_id', '$device_id', 'geoip_disable', 'api_key', 'group_properties', 'distinct_id', 'timezone', 'person_properties', 'groups']
Java: 40/47 passed, 7 failed — full report
posthog-java-server Compliance Report
Date: 2026-09-08T15:17:12.727413+00:00
Duration: 210457ms
⚠️ Some Tests Failed
40/47 tests passed, 7 failed
Capture Tests
View Details
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 367ms |
| Format Validation.Event Has Uuid | ✅ | 27ms |
| Format Validation.Event Has Lib Properties | ✅ | 27ms |
| Format Validation.Distinct Id Is String | ✅ | 28ms |
| Format Validation.Token Is Present | ✅ | 25ms |
| Format Validation.Custom Properties Preserved | ✅ | 22ms |
| Format Validation.Event Has Timestamp | ✅ | 22ms |
| Format Validation.Non Utc Event Timestamp Is Converted To Utc | ✅ | 44ms |
| Retry Behavior.Retries On 503 | ❌ | 15046ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 12044ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 12039ms |
| Retry Behavior.Respects Retry After Header | ❌ | 15043ms |
| Retry Behavior.Implements Backoff | ✅ | 25048ms |
| Retry Behavior.Retries On 500 | ✅ | 11029ms |
| Retry Behavior.Retries On 502 | ✅ | 11024ms |
| Retry Behavior.Retries On 504 | ✅ | 11021ms |
| Retry Behavior.Max Retries Respected | ❌ | 25049ms |
| Deduplication.Generates Unique Uuids | ✅ | 29ms |
| Deduplication.Preserves Uuid On Retry | ✅ | 11021ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ✅ | 20048ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ✅ | 11029ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 24ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 17ms |
| Compression.Sends Gzip When Enabled | ✅ | 14ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 15ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 7ms |
| Batch Format.Multiple Events Batched Together | ✅ | 24ms |
| Error Handling.Does Not Retry On 403 | ✅ | 12047ms |
| Error Handling.Does Not Retry On 413 | ✅ | 12044ms |
| Error Handling.Retries On 408 | ✅ | 5022ms |
Failures
retry_behavior.retries_on_503
Expected at least 3 requests, got 2
retry_behavior.respects_retry_after_header
Expected at least 2 requests, got 1
retry_behavior.max_retries_respected
Expected 4 requests, got 3
Feature_Flags Tests
View Details
| Test | Status | Duration |
|---|---|---|
| Request Payload.Request With Person Properties Device Id | ✅ | 26ms |
| Request Payload.Flags Request Uses V2 Query Param | ✅ | 16ms |
| Request Payload.Flags Request Hits Flags Path Not Decide | ✅ | 15ms |
| Request Payload.Flags Request Omits Authorization Header | ✅ | 16ms |
| Request Payload.Token In Flags Body Matches Init | ✅ | 16ms |
| Request Payload.Groups Round Trip | ✅ | 15ms |
| Request Payload.Groups Default To Empty Object | ✅ | 19ms |
| Request Payload.Disable Geoip False Propagates As Geoip Disable False | ✅ | 19ms |
| Request Payload.Disable Geoip Omitted Defaults To False | ✅ | 18ms |
| Request Payload.Flag Keys To Evaluate Contains Only Requested Key | ✅ | 16ms |
| Request Lifecycle.No Flags Request On Init Alone | ✅ | 16ms |
| Request Lifecycle.No Flags Request On Normal Capture | ✅ | 18ms |
| Request Lifecycle.Two Flag Calls Produce Two Remote Requests | ✅ | 25ms |
| Request Lifecycle.Mock Response Value Is Returned To Caller | ❌ | 22ms |
| Retry Behavior.Retries Flags On 502 | ❌ | 323ms |
| Retry Behavior.Retries Flags On 504 | ❌ | 318ms |
| Side Effect Events.Get Feature Flag Captures Feature Flag Called Event | ❌ | 20ms |
Failures
request_lifecycle.mock_response_value_is_returned_to_caller
Expected last action result 'variant-a', got None
retry_behavior.retries_flags_on_502
Expected last action result True, got None
retry_behavior.retries_flags_on_504
Expected last action result True, got None
side_effect_events.get_feature_flag_captures_feature_flag_called_event
Event '$feature_flag_called' property '$feature_flag_response': expected True, got False
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 Motivation and Context
Exercise the actual core, Android and Java server SDK entries instead of attributing core-only and synthetic flag behavior to Android.
Shipping SDK APIs, source, defaults and dependency pins are unchanged. An empty changeset is included; no SDK release bump.
💚 How did you test it?
make checkFormatpasses.Known stateful GeoIP/key-scope differences, Java retry-policy disagreements and four legacy-v2 flag-fixture mismatches remain selected and documented. This is coverage work, not an all-green compliance claim.
📝 Checklist
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Implemented with a Pi worker using repository file tools, Git, Gradle, the Android emulator and the pinned SDK test harness. Public SDK entry points own transport and flag evaluation; adapter-only translation and observation are covered separately by regressions. Human review is required before merge.