Skip to content

feat: bounded feature, provider and model usage insights - #259

Merged
yaacovcorcos merged 2 commits into
mainfrom
feat/analytics-insights-20260906
Sep 6, 2026
Merged

feat: bounded feature, provider and model usage insights#259
yaacovcorcos merged 2 commits into
mainfrom
feat/analytics-insights-20260906

Conversation

@yaacovcorcos

Copy link
Copy Markdown
Contributor

What Changed

Add bounded product insight events for visible panel categories, Settings sections, Usage tools and project-picker/search entry points. Export native per-turn token counts with provider and allowlisted public model categories, separate from provider readiness and terminal outcome signals.

Reuse the existing worker/outbox and ProviderService correlation. No transcript scan, provider polling, raw model labels, content, paths or account identities. Token subsets are not added twice, missing counts stay unavailable, and consent epoch changes fence turn usage.

Why

Distinguish what people can use from what they actually use, with explicit coverage rather than misleading zero totals. Existing queue limits remain intentional; this is bounded best-effort delivery, not a lossless guarantee.

Documentation Impact

Updated internal analytics contract and user telemetry explanation. Coordinated website gateway/reporting and Scient measurement-plan PRs will be linked below. Deploy the compatible gateway before releasing revision-3 producers.

Validation and review

  • Repository typecheck, lint (existing warnings), build and desktop smoke passed before main's unrelated release-signing merge.
  • Targeted provider, contract, persistence and renderer tests passed, including consent epoch changes and retry identity after reopening SQLite.
  • Synthetic built desktop-worker to gateway to ledger to export proof passed; no production traffic.
  • Website complete check passed. Generated wire/fixture parity checked.
  • Full desktop test run is in progress; final result will be recorded separately.
  • Independent read-only review found model attribution, hidden-sheet observation, numeric persistence and query conversion issues; corrected and regression-tested.

UI Changes / readiness

Only the existing analytics explanation gains provider/model categories and token-count wording. No visual review performed; human copy/preview acceptance remains pending, so this is a draft. No production configuration, dashboard, consent default or notification activation is changed.

Prepared with Codex API tooling and an independent read-only review agent.

@github-actions github-actions Bot added size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Sep 6, 2026
@yaacovcorcos

Copy link
Copy Markdown
Contributor Author

Coordinated gateway/reporting: ScientFactory/ScientFactory-website#34 . Canonical measurement definitions: ScientFactory/Scient#109 . Deploy the compatible gateway before releasing this desktop producer. No production activation performed.

@yaacovcorcos

Copy link
Copy Markdown
Contributor Author

Final-head qualification update (afbb82a): aligned with main including #258. Repository typecheck and 134 focused provider/contract/release-machinery tests pass. Prior full web run: 6,396 passing tests; full repository server test phase remains running locally. Desktop build and smoke passed; numeric restart/retry and synthetic end-to-end export tests passed. CI is running. No visual acceptance or production activation claimed.

@yaacovcorcos
yaacovcorcos marked this pull request as ready for review September 6, 2026 19:23
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.5 KiB 13.6 KiB +108 B (+0.8%) 15.1 KiB
Codex Thread snapshot wire 7.1 KiB 7.1 KiB −2 B (−0.0%) 7.3 KiB
Codex Live turn WebSocket wire 6.4 KiB 6.5 KiB +110 B (+1.7%) 7.8 KiB
Codex Live turn WebSocket decoded 56.2 KiB 57.0 KiB +822 B (+1.4%) 66.4 KiB
Codex Live turn messages 8 8 0 (0.0%) 21
Claude Total thread wire 13.5 KiB 13.6 KiB +122 B (+0.9%) 15.1 KiB
Claude Thread snapshot wire 7.1 KiB 7.1 KiB +3 B (+0.0%) 7.3 KiB
Claude Live turn WebSocket wire 6.4 KiB 6.5 KiB +119 B (+1.8%) 7.8 KiB
Claude Live turn WebSocket decoded 57.0 KiB 57.8 KiB +884 B (+1.5%) 66.4 KiB
Claude Live turn messages 8 9 +1 (+12.5%) 21

Baseline: 4a1924b · PR result: afbb82a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.9 KiB
  • Claude decoded thread snapshot: 114.6 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@yaacovcorcos
yaacovcorcos merged commit 0daa25f into main Sep 6, 2026
27 checks passed
@yaacovcorcos
yaacovcorcos deleted the feat/analytics-insights-20260906 branch September 6, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant