Skip to content

e2e: wait for dashboard QR content before final screenshot#714

Merged
guysoft merged 1 commit into
develfrom
bugfix/e2e-qr-screenshot
Jul 13, 2026
Merged

e2e: wait for dashboard QR content before final screenshot#714
guysoft merged 1 commit into
develfrom
bugfix/e2e-qr-screenshot

Conversation

@guysoft

@guysoft guysoft commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Problem

The final E2E screenshot captured the desktop wallpaper / logo (stddev ~0.20) instead of the welcome dashboard with the QR code (stddev ~0.31). Tests still reported ALL TESTS PASSED, so the regression was silent.

Root cause (deterministic, not flaky — reproduced twice with identical stddev 0.20752):

  1. Tests run alphabetically, so test_translation_disabled.sh is the last test before screenshot.sh.
  2. Its restore_url() writes the dashboard URL back to fullpageos.txt, killall chromium, then returns without waiting for the kiosk to relaunch. run_onepageos respawns chromium, but the relaunched kiosk first paints the desktop shell/wallpaper and only later loads the welcome iframe (the QR code).
  3. screenshot.sh accepts a candidate as soon as dashboard_is_painted is true, but PAINT_STDDEV_THRESHOLD was 0.08 — and the shell/wallpaper-only state already scores 0.20752, so it passed on the very first candidate before the QR iframe painted.

Stddev measurements (cropped 1280x620+0+80, same as the hook):

  • shell/logo only: 0.20752
  • German test page: 0.123093
  • full dashboard with QR: 0.312033
  • old threshold: 0.08

Fix

testing/hooks/screenshot.sh

  • Raise PAINT_STDDEV_THRESHOLD default 0.080.26 (above the wallpaper/shell baseline 0.20, below the full QR dashboard 0.31).
  • Require two consecutive passing captures (PAINT_STABLE_PASSES=2) so the screenshot is taken only after the welcome iframe has rendered and settled, not on a mid-load transition. On timeout the hook still fails and saves screenshot-not-painted.png.

testing/tests/test_translation_disabled.sh

  • restore_url() now waits for the dashboard kiosk process to relaunch on the original URL (--kiosk --app=http://localhost/FullPageDashboard) before returning, so screenshot.sh starts from a relaunched kiosk rather than the bare desktop.

Evidence

Verified against CustomPiOS PR #274 (pr-274-local image) + FullPageOS devel (7c9b93d), QEMU aarch64.

Before (e2e_results/pr274-devel-rerun-20260713/, unchanged devel):

  • screenshot.png = desktop wallpaper / logo, stddev 0.20752
  • accepted on first candidate: Display screenshot saved after dashboard paint (1x2s)
  • TEST_RESULT=0, exit-code 0 (false pass)

After (e2e_results/pr274-devel-qrfix-20260713/, this branch):

  • restore_url: Dashboard kiosk restored (pid: 8480) after 2x2s
  • paint loop: stddev 00.312033Painted but waiting for stabilization (1/2)0.312033Display screenshot saved after dashboard paint (2 stable passes, 3x2s)
  • screenshot.png = welcome dashboard with QR code + setup speech bubble, stddev 0.312033
  • TEST_RESULT=0, exit-code 0, ALL TESTS PASSED

The fix is on the FullPageOS side only; no CustomPiOS changes.

Test plan

  • Re-ran E2E on unchanged devel to confirm the wallpaper screenshot reproduces (not flaky)
  • Rebuilt E2E container with the fix and re-ran; final screenshot shows QR dashboard (stddev 0.312033)
  • Confirm CI on this PR reproduces the QR-dashboard screenshot

The final E2E screenshot captured the desktop wallpaper (stddev ~0.20)
instead of the welcome dashboard with the QR code (stddev ~0.31) when
test_translation_disabled.sh ran last: its restore_url() killed chromium
right before screenshot.sh, and the paint threshold (0.08) was satisfied
by the wallpaper before the relaunched kiosk's welcome iframe painted.

- screenshot.sh: raise PAINT_STDDEV_THRESHOLD to 0.26 (above the
  wallpaper/shell baseline, below the full QR dashboard) and require
  two consecutive passing captures so the screenshot is taken only
  after the welcome iframe has rendered and settled.
- test_translation_disabled.sh: restore_url() now waits for the dashboard
  kiosk process to relaunch on the original URL before returning, so
  screenshot.sh starts from a relaunched kiosk instead of the bare
  desktop.
@guysoft
guysoft merged commit c2ab09e into devel Jul 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant