Skip to content

Add camera-first, offline-capable GridPuzzle phone app - #18

Draft
senegrom wants to merge 85 commits into
masterfrom
browser-scanner
Draft

Add camera-first, offline-capable GridPuzzle phone app#18
senegrom wants to merge 85 commits into
masterfrom
browser-scanner

Conversation

@senegrom

@senegrom senegrom commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Camera-first GridPuzzle phone app

browser-scanner hosts the installable PWA deployed at https://senegrom.github.io/GridPuzzle/. Nothing in this PR merges automatically into master.

What it includes

  • Rear-camera capture, stable-grid auto capture, photo import, perspective correction and grid-size/type suggestions.
  • Local printed-clue OCR; photographs are not uploaded to a remote recognizer or solver.
  • All twelve GridPuzzle families through a strict row-major JSON adapter and the complete Python 3.14 solver in a cancellable Pyodide worker: Sudoku, Killer Sudoku, Futoshiki, KenKen, Latin square, diagonal/pandiagonal Latin square, Hidato, Numbrix, Kakuro, Slitherlink and Str8ts.
  • Exact native Str8ts street constraints. Solid black cells split streets; numbered black cells still participate in row/column uniqueness without joining a street.
  • Editors for values/blocked or black cells, cages, inequalities and Kakuro directional clues; undo; guided Review highlighted clues → Save & next; JSON import/export.
  • Unique/multiple/no-solution/invalid/unfinished states, time limits and real cancellation.
  • Clean-board and captured-photo solution overlays, including Slitherlink edges.
  • Installable icons, saved puzzle/settings, hash-verified offline preparation and self-hosted Pyodide/Tesseract assets.

Recognition trust and real newspaper regressions

Automatic recognition is a proposal, not proof. Automatically identified rules require confirmation, structural families remain review-gated, and wrong/missed OCR is never silently converted into solver certainty.

Newsprint processing now distinguishes true solid black cells from gray Sudoku shading, suppresses paper/halftone specks with connected-component filtering, and locally Otsu-binarizes digit crops before the single bounded Tesseract atlas call.

The two user-provided newspaper examples from 2026-09-07 are committed under Examples/BrowserScanner/Newspaper/, outside web/ so they are not shipped in the PWA/offline bundle. The Pages gate runs them through the production scanner in Chromium and WebKit.

Current raw results on the tested deployment artifact:

fixture Chromium 153 WebKit 26.6 unsafe discrepancies
shaded Sudoku 24/24 24/24 0 / 0
Str8ts 19/20 19/20 0 / 0

For Str8ts, both engines detect the exact 22-cell black layout. The single missed printed value is explicitly flagged for review in both engines. Generated regressions are also green: Chromium reads every tested generated variant exactly; WebKit reads the perspective/shadow case 29/30 with its miss flagged.

Phone/offline hardening

  • Recognition uncertainty persists atomically; photos do not.
  • Malformed imports and structural inconsistencies are rejected before rendering/solving.
  • Cage/Kakuro solve-ready validation runs before Pyodide loads.
  • Keyboard boundaries and no-op edits use authoritative puzzle state.
  • Large phone-photo decoding is bounded/downscaled.
  • CSP/no-referrer and iPhone safe-area handling are in the core app.
  • Offline runtime assets are content-addressed and digest-verified; startup readiness is cheap while explicit offline preparation performs full verification and requests persistent storage.
  • Query-string root navigation works offline.

CI and deployment

The existing Build and deploy phone scanner workflow is the single expensive Chromium/WebKit deployment gate. It runs the complete native bounded suite, browser unit tests, all twelve solver families through real Pyodide, generated OCR/review tests, the two real newspaper OCR regressions, uploads the tested static artifact, and publishes that exact artifact to Pages. Lightweight PR browser CI avoids duplicating the full gate; Linux/Windows CI and forward compatibility remain separate.

No temporary Str8ts apply/retrigger workflows or staging helpers are present on browser-scanner; the temporary .str8ts-stage payloads were removed before the clean integration commit.

Remaining empirical limit

The strongest remaining validation gap is physical-device testing: actual iPhone autofocus/exposure, installed-PWA camera behaviour, storage eviction and true airplane-mode use. Generated and desktop-engine tests are not presented as substitutes for that hardware testing.

See web/README.md, web/TESTING.md, Examples/BrowserScanner/Newspaper/ground-truth.json, and the uploaded browser test artifacts for the current methodology and measurements.

senegrom commented Sep 6, 2026

Copy link
Copy Markdown
Owner Author

Final verification — 050852f74804

Implementation is pushed to browser-scanner; nothing has been merged into master. Relative to the starting commit, the branch adds 26 files and does not modify any existing solver file, technique hierarchy or search profile.

The build job passed in run 34052179905:

  • 626 native Python tests passed, plus 22 JavaScript tests.
  • 23 recorded acceptance checks passed in each browser: Chromium 153.0.8010.12 and mobile WebKit 26.6. These exercise the real Python/OCR WebAssembly runtimes, all eleven solver families, editing/undo, type overrides, cancellation/restart, persistent uncertainty, camera-denial fallback, photo overlays and offline operation.
  • No uncaught browser errors or external runtime requests were recorded.
  • Offline checks physically stopped the origin server, verified it was unreachable, then reloaded the cached app, started a fresh Python solver and performed fresh photo recognition. Physical iPhone camera/airplane-mode/installation testing remains outstanding.

Recognition is still a review-assisted prototype

On the generated 30-given printed Sudoku fixture, raw recognition correctly read 25/30 clues in Chromium and 24/30 in WebKit. The five/six discrepancies were flagged and corrected through the real cell editor before checking the exact original solution. These correction counts are recorded separately in results.json; they are not presented as automatic recognition success. Cage boundaries, inequalities, path variants and Kakuro clue directions remain experimental and require review.

Deployment blocker: one owner setting

The tested static site and Pages artifact have been uploaded. The workflow's overall failure is solely the configure job: Pages is not enabled. Initial enablement returned HTTP 403 Resource not accessible by integration.

Set Settings → Pages → Source: GitHub Actions, then re-run the failed jobs in the run linked above. If the github-pages environment has branch restrictions, allow browser-scanner. No merge to master is necessary. The intended address is https://senegrom.github.io/GridPuzzle/, but it is not live yet.

Artifacts: scanner-static-build, browser-test-report, and the tested Pages upload. Implementation/limitations are documented in web/README.md and test methodology in web/TESTING.md.

Fix malformed input/autosave freezes, unsafe build output deletion and poisoned offline caches. Isolate task/edit/photo/offline ownership, share grayscale off-thread, own OCR children during initialization and preserve concurrent guided-review/grid-artifact improvements. Include exact stack-safe cage enumeration and uncapped parallel-error observation from master without weakening deductions or changing positive-cap semantics.

Final pre-merge Python 3.14.7 CI: 660 passed Linux, 658 Windows, 32 slow deselected each. Actual installed-wheel solves, forward compatibility (3.14t/3.15), JavaScript regressions and both actual Chromium/mobile-WebKit suites passed. Retained reports cover 27 end-to-end checks per engine, all eleven puzzle families, cancellation during real OCR language loading, poisoned-cache recovery, and origin-offline solving/scanning. Concurrent scanner variation/review suite also passed. Temporary write-enabled validation tooling removed.

Target is browser-scanner only; original app PR #18 remains separate from master.
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 6, 2026 22:08 — with GitHub Actions Inactive

senegrom commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

Starting a complete follow-up pass on the actionable review findings. I’m keeping this on browser-scanner and will preserve solver solution space/technique semantics. Main targets: automatic-scan trust, query-string offline navigation, keyboard movement, no-op edits, earlier structural validation, iPhone safe areas/contrast, offline update continuity/per-request hashing, duplicate browser CI, and stale deployment docs.

@senegrom
senegrom had a problem deploying to gridpuzzle-browser-pages September 7, 2026 16:15 — with GitHub Actions Error
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 7, 2026 16:31 — with GitHub Actions Inactive
Add exact native Str8ts solving, browser/data-boundary support and black-cell editing. Harden digit extraction for shaded and dirty newsprint with solid-block detection, connected-component cleanup and per-cell Otsu binarization. Store the two user-provided newspaper puzzle crops outside the deployed web bundle and gate Pages on real Chromium/WebKit OCR safety regressions.

Remove the temporary .str8ts-stage payloads; no temporary apply/retrigger workflows are added to browser-scanner.
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 7, 2026 23:21 — with GitHub Actions Inactive
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 7, 2026 23:26 — with GitHub Actions Inactive
Keep neighbouring number glyphs while filtering newsprint speckles, include
threshold-boundary ink in binary crops, and retain malformed structural
readings as incomplete review states. Add unit regressions and exact binary
Sudoku/multi-digit Numbrix acceptance checks in both browser engines.
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 8, 2026 01:22 — with GitHub Actions Inactive
Keep numeric clue and cage review flags independent through editing, Undo
and autosave. Preserve unapplied JSON during view and solver refreshes,
expose solved values and Kakuro targets in accessible cell labels, and
ignore photo/JSON import errors after a newer task supersedes them.

Add unit regressions and Chromium/WebKit editor acceptance coverage.
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 8, 2026 04:09 — with GitHub Actions Inactive
…ates

Keep detected and manually edited scan dimensions through view refreshes
and Undo, composite transparent uploads onto white in both decode paths,
and offer an explicit reload when another tab activates an app update.

Add layout and transparent-image browser regressions plus multi-tab
service-worker tests.
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 8, 2026 09:08 — with GitHub Actions Inactive
Show box controls for the selected scan type, including Automatic,
without changing the current board or discarding layout drafts.
Restore focus after selecting or deselecting a cage/inequality cell
so keyboard navigation continues across board redraws.

Add Chromium and WebKit regressions for family changes, applying layouts,
Undo, and keyboard selection through saving both constraint types.
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 8, 2026 11:05 — with GitHub Actions Inactive
…dates

Close live camera streams whenever editing or solving takes task ownership,
and ignore playback that completes after cancellation.

Retain verified versioned solver archives for dedicated workers already
alive when another tab activates an update. Preserve their original URLs
across later updates and prune archives once those worker clients close.

Add camera lifecycle and cache migration regressions, plus Chromium and
WebKit checks for application cancellation wiring and an initializing
worker surviving a real two-tab update online and offline.
Use a worker-local initialization promise instead of a pending fetch
through the old service worker, whose lifetime correctly delays activation.
Run the focused update check before the longer browser suites so failures
are reported separately.
Include original window clients in archive ownership so an unenumerated
worker remains protected. Keep the associated old manifests for requests
still handled by a previous controller, and release both metadata and
archive bytes after the original clients close.

Extend migration regressions and verify real offline behavior by stopping
the origin server, matching the existing browser acceptance methodology.
Keep the solver request racing controllerchange, but wait for the new
worker to finish activating before inspecting its migration metadata.
Chromium delivers controllerchange before activate's waitUntil work has
completed, so the earlier diagnostic read could race cache creation.
Look up the original verified manifest directly, including before the new
controller has finished writing its retention index. Add a regression for
that ordering.

Have the browser fixture load an already-installed document and prove its
worker reads from verified storage before starting the two-tab update.
This matches the initialized app and runtime fetches in the scanner.
@senegrom
senegrom temporarily deployed to gridpuzzle-browser-pages September 8, 2026 12:13 — with GitHub Actions Inactive
senegrom and others added 2 commits September 8, 2026 18:08
master now pins ruff's rule set and lints in CI, and the preceding merge
brings that configuration here. The branch-only files are brought in line:
the build script and the review-three web tests are formatted so their
one-line compound statements become ordinary blocks, the web adapter tests
use dict literals, nested with-blocks are combined, and the adapter's cell
coordinate helper is a named function. No behaviour changes; the adapter and
build tests pass and the site builds from the reformatted script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gf5JZFCF147pbAVQ6PVYCr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant