Skip to content

feat(replay): make screenshot optimizations opt-in - #761

Merged
dustinbyrne merged 4 commits into
perf/replay-pixel-copy-bufferfrom
feat/opt-in-screenshot-optimizations
Sep 9, 2026
Merged

feat(replay): make screenshot optimizations opt-in#761
dustinbyrne merged 4 commits into
perf/replay-pixel-copy-bufferfrom
feat/opt-in-screenshot-optimizations

Conversation

@dustinbyrne

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Stacked on #756. This follow-up must land before the screenshot optimization is released so existing users retain the current capture defaults.

Adds experimental sessionReplayConfig.optimizeScreenshots, defaulting to false:

  • Default: a new full-resolution ARGB_8888 bitmap per screenshot, unchanged mask coordinates, and later capture attempts remain possible while a timed-out copy is pending.
  • Opt-in: perf(replay): reduce PixelCopy screenshot overhead #756's reusable half-width/height RGB_565 bitmap and existing ARGB_8888 fallback. This reduces image detail and removes alpha; transparent window regions appear black. A pending timed-out copy holds the pooled bitmap until its callback arrives.

The option is sampled once per capture, including masking and bitmap release. Both modes safely reclaim late bitmaps and discard captures with non-positive source dimensions. The existing screenshot default and all constructor signatures are unchanged.

The follow-up also removes redundant casts and updates the changeset to describe the opt-in behavior and minor API addition.

💚 How did you test it?

  • make test: 355 Android tests and 9 Compose tests passed; 3 existing skips.
  • make testJava: 942 core tests passed.
  • make checkFormat, make api, ./gradlew apiCheck, and git diff --check passed.
  • Native Robolectric graphics with WebP encode/decode verifies transparency and mask placement in both modes, including an option change during capture. PixelCopy itself is shadowed; no physical-device testing is claimed.
  • Regression tests cover fresh allocation by default, opt-in reuse/fallback, timeout recovery, both option-switch directions while a copy is pending, late callbacks after uninstall, and invalid-dimension recovery.
  • A Java consumer compiled against the old config constructors runs unchanged with the new classes. A new Java consumer verifies the option defaults off and does not enable screenshot capture.

📝 Checklist

  • I reviewed the submitted code and completed a simplify pass.
  • I added tests to verify the changes.
  • I updated the configuration documentation and changeset.
  • Existing constructor signatures and screenshot enablement defaults are preserved.

🤖 Agent context

Autonomy: Human-driven (agent-assisted).

Implemented and self-reviewed with Pi, Git/GitHub CLI, Gradle, Robolectric, and Java consumer checks. Kept one shared capture/completion pipeline with capture-local mode selection rather than duplicating the screenshot implementation. Human review is required; native/OEM PixelCopy behavior remains outside the local test coverage.

@dustinbyrne
dustinbyrne requested a review from a team as a code owner September 4, 2026 21:25
@dustinbyrne dustinbyrne self-assigned this Sep 4, 2026
bitmap = bitmapLease.bitmap
} else {
bitmapLease = null
bitmap = Bitmap.createBitmap(view.width, view.height, Bitmap.Config.ARGB_8888)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: Unbounded bitmap retention after PixelCopy timeouts

When optimizations are disabled, every capture allocates a new full-resolution bitmap. A timed-out request retains that bitmap until its callback arrives, but unlike the lease-backed path, it does not prevent subsequent captures from allocating more; repeated UI redraws during stalled callbacks can therefore exhaust the host app's memory. Keep full-resolution ARGB_8888 capture while applying equivalent single-request backpressure or another explicit bound to outstanding bitmaps.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that this can create memory pressure if PixelCopy requests remain pending: their destination bitmaps stay live while later captures allocate more.

That allocation/capture behavior already exists on main. This PR restores it as the default; adding single-request backpressure would instead make later captures skip while an earlier timed-out request remains pending. That behavior is intentionally opt-in through optimizeScreenshots = true.

We're not changing the default outstanding-request policy in this PR. Leaving this thread unresolved to keep the existing memory-pressure risk visible.

@veria-ai

veria-ai Bot commented Sep 4, 2026

Copy link
Copy Markdown

PR overview

This PR makes screenshot optimizations opt-in for Android session replay, using full-resolution bitmap capture when optimizations are disabled.

One issue remains: stalled PixelCopy callbacks can allow full-resolution bitmaps to accumulate during repeated UI redraws, potentially exhausting the host app’s memory. The impact depends on timeout conditions and sustained redraw activity, and no reported issues have yet been addressed.

Open issues (1)

Fixed/addressed: 0 · PR risk: 4/10

@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
### Issue 1
posthog-android/src/main/java/com/posthog/android/replay/PostHogReplayIntegration.kt:1715
**Pending bitmaps can leak**

If a default-mode PixelCopy times out, its fresh full-resolution bitmap is left for the callback to recycle. If `uninstall()` quits the callback thread before that callback is delivered, the bitmap is not recycled and is not tracked by `pixelCopyBitmapBuffer.close()`. Multiple delayed captures can therefore retain large ARGB_8888 bitmaps without deterministic cleanup. Please retain ownership of pending default-mode bitmaps so uninstall can release them explicitly.

### Issue 2
posthog-android/src/test/java/com/posthog/android/replay/PostHogReplayIntegrationTest.kt:1785
**Cases are not parameterised**

These new tests loop over multiple cases inside ordinary tests, which violates the repository directive to prefer parameterised tests. A failure in one iteration prevents later cases from running and reports all inputs under one result. Please parameterise the optimization modes and dimension pairs before merging. The same pattern appears in the late-callback, transparency, mask-scaling, and invalid-dimension tests.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "feat(replay): make screenshot optimizati..." | Re-trigger Greptile

bitmap = bitmapLease.bitmap
} else {
bitmapLease = null
bitmap = Bitmap.createBitmap(view.width, view.height, Bitmap.Config.ARGB_8888)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Pending bitmaps can leak

If a default-mode PixelCopy times out, its fresh full-resolution bitmap is left for the callback to recycle. If uninstall() quits the callback thread before that callback is delivered, the bitmap is not recycled and is not tracked by pixelCopyBitmapBuffer.close(). Multiple delayed captures can therefore retain large ARGB_8888 bitmaps without deterministic cleanup. Please retain ownership of pending default-mode bitmaps so uninstall can release them explicitly.

Knowledge Base Used: Android session replay

Prompt To Fix With AI
This is a comment left during a code review.
Path: posthog-android/src/main/java/com/posthog/android/replay/PostHogReplayIntegration.kt
Line: 1715

Comment:
**Pending bitmaps can leak**

If a default-mode PixelCopy times out, its fresh full-resolution bitmap is left for the callback to recycle. If `uninstall()` quits the callback thread before that callback is delivered, the bitmap is not recycled and is not tracked by `pixelCopyBitmapBuffer.close()`. Multiple delayed captures can therefore retain large ARGB_8888 bitmaps without deterministic cleanup. Please retain ownership of pending default-mode bitmaps so uninstall can release them explicitly.

**Knowledge Base Used:** [Android session replay](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/posthog-android/-/docs/android-session-replay.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that quitSafely() can reject a callback posted after shutdown, so explicit recycling is not guaranteed in that case. However, missing recycle() does not itself establish a leak: Android normally reclaims bitmap memory through GC once references are gone, and the default path does not add an SDK collection retaining these destinations.

Tracking pending destinations would not make it safe to recycle them during uninstall: PixelCopy could still be using them. The reusable buffer's close() likewise recycles only idle bitmaps and detaches an in-flight lease rather than recycling its bitmap immediately.

The shutdown behavior also exists on main. We're not adding forced recycling here. Leaving this thread unresolved for visibility; the late-callback test verifies cleanup when completion is delivered, not guaranteed callback delivery after real handler shutdown.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s a fair clarification. I agree this does not establish an actionable leak: the SDK does not retain the default-mode bitmap, and recycling it during uninstall() would be unsafe while PixelCopy may still be using it. If the callback is never delivered, the bitmap can still be reclaimed by GC once the platform releases its references. I’ll leave this thread unresolved for visibility rather than requiring pending-bitmap tracking.

@marandaneto

Copy link
Copy Markdown
Member

whats about this instead:

Proposed usage—not available today

      sessionReplayConfig.apply {
          screenshotQuality = PostHogScreenshotQuality.BALANCED

          // Preserve transparency and full color precision.
          screenshotColorMode = PostHogScreenshotColorMode.ARGB_8888
      }

Proposed API

Add these properties inside the existing config class body, preserving its constructor signatures:

  public enum class PostHogScreenshotQuality {
      ORIGINAL,
      BALANCED,
      LOW,
  }

  public enum class PostHogScreenshotColorMode {
      ARGB_8888,
      RGB_565,
  }

  // Inside PostHogSessionReplayConfig:

  @PostHogExperimental
  public var screenshotQuality = PostHogScreenshotQuality.ORIGINAL

  @PostHogExperimental
  public var screenshotColorMode = PostHogScreenshotColorMode.ARGB_8888

Suggested initial behavior

These preset values are starting points to benchmark, not measured recommendations:

  | Setting | Resolution | WebP quality |
  |---|---|---|
  | `ORIGINAL` — default | Full physical resolution | 30, preserving current behavior |
  | `BALANCED` | Half width and height | 30 |
  | `LOW` | Quarter width and height | 20 |

(webp quality could be configurable eg screenshotCompressionQuality = 30 - 0 to 100 but not mandatory)

Separate behavior:

  • ARGB_8888: preserve alpha and full color precision.
  • RGB_565: lower bitmap memory, reduced color precision, no alpha; retain the existing ARGB_8888 fallback.

This allows useful combinations:

  // Lower overhead without losing transparency:
  screenshotQuality = PostHogScreenshotQuality.BALANCED
  screenshotColorMode = PostHogScreenshotColorMode.ARGB_8888

@marandaneto

Copy link
Copy Markdown
Member

@ioannisj wdyt as well? #761 (comment)

@marandaneto

Copy link
Copy Markdown
Member

suggestion: Consider density-aware screenshot sizing as a follow-up

Building on the quality/color-mode proposal above, another possible improvement is to make the reduced-resolution presets density-aware, rather than always dividing physical width and height by a fixed factor. This is a non-blocking design suggestion, not a defect in this PR.

Currently, the optimized capture uses approximately view.width / 2 × view.height / 2 physical pixels. Instead, an opt-in preset could target roughly one bitmap pixel per dp. For example, a 1080-pixel-wide window at density 3 would become 360 pixels wide, rather than 540. This reduces capture-buffer memory and the number of pixels processed during masking/encoding, at the cost of image detail; actual CPU and payload improvements should be benchmarked.

Possible configuration

No additional public setting is necessary: this could refine the proposed screenshotQuality presets, while keeping color mode and capture frequency independent:

sessionReplayConfig.apply {
    screenshot = true
    screenshotQuality = PostHogScreenshotQuality.BALANCED
    screenshotColorMode = PostHogScreenshotColorMode.ARGB_8888
    throttleDelayMs = 1_000
}

These are proposed APIs, not currently available. One possible preset mapping, instead of the fixed half/quarter-resolution mapping above, is:

Preset Capture dimensions Initial WebP quality
ORIGINAL — default Full physical resolution 30, preserving current behavior
BALANCED Approximately 1 bitmap pixel per dp 30
LOW Approximately 0.8 bitmap pixels per dp 20

The reduced preset values are starting points for benchmarks, not measured recommendations. ORIGINAL means original resolution, not lossless compression. Add new config properties in the existing class body to preserve constructor signatures. Bitmap reuse can remain an internal implementation detail, independent of the selected preset/color mode.

Illustrative sizing logic

Assuming a valid, positive source size and display density, sample these values once per capture:

val density = view.resources.displayMetrics.density
val scale = when (quality) {
    PostHogScreenshotQuality.ORIGINAL -> 1f
    PostHogScreenshotQuality.BALANCED -> minOf(1f, 1f / density)
    PostHogScreenshotQuality.LOW -> minOf(1f, 0.8f / density)
}
val captureWidth = maxOf(1, ceil(sourceWidth * scale).toInt())
val captureHeight = maxOf(1, ceil(sourceHeight * scale).toInt())

Clamping the scale to 1 avoids upscaling low-density displays. This applies to the PixelCopy destination bitmap, not just replay layout coordinates.

Tradeoffs / validation

  • Masking correctness is essential: derive mask scale from the actual rounded bitmap dimensions (captureWidth / sourceWidth, independently for Y), not just the nominal preset scale. Preserve outward rounding of mask bounds and scale corner radii consistently. Test both legacy and verified masking paths, Compose, odd sizes, non-integer densities, tiny masked elements, and window resize/orientation changes while capture is pending. Discard captures when mask/pixel alignment cannot be established; don't weaken the existing guards for performance.
  • Fidelity: test small text, thin borders, icons, and zoomed replay playback across densities. Lower resolution can reduce readability; keep full-resolution capture available and default behavior unchanged.
  • Color/alpha: density scaling should not implicitly select RGB_565. ARGB_8888 should still support reduced resolution with transparency preserved.
  • Lifecycle: snapshot preset, source dimensions, and density for each capture; size changes must not reconfigure or recycle a bitmap still owned by PixelCopy.
  • Measurement: compare capture latency, encoding time, allocation/peak memory, dropped frames, and payload size on real devices. No runtime benchmark is claimed for this suggestion.

@marandaneto

marandaneto commented Sep 7, 2026

Copy link
Copy Markdown
Member

a follow up that would spare some memory and cpu usage under heavy redrawns or slow captures
#764

Expose scale, WebP quality, and color-mode settings while preserving default capture fidelity and constructor compatibility.

Reuse compatible destinations without recycling pending copies, and keep recording state and buffer lifecycle atomic across stop/start.
@dustinbyrne
dustinbyrne force-pushed the feat/opt-in-screenshot-optimizations branch from defdd17 to 89644b6 Compare September 8, 2026 21:33
@dustinbyrne
dustinbyrne merged commit ed94770 into main Sep 9, 2026
16 checks passed
@dustinbyrne
dustinbyrne deleted the feat/opt-in-screenshot-optimizations branch September 9, 2026 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants