Fraud detection for leaderboard & token abuse - #78
Open
QuarkOS wants to merge 3 commits into
Open
Conversation
Detects behavioural signatures that the ingest hard caps cannot express: activity-ceiling pinning, physically impossible concurrency, visit-bonus flooding, scripted uniform durations, impossible token throughput, backfilled token spikes, and cost/token mismatch. Every detector is a pure, deterministic function over events_raw + agent_usage_daily rows, biased for precision (each carries its raw evidence). assessUserFraud rolls signals up per category with a capped 0-100 risk score, level band, and a stable dedupe fingerprint. Covered by 22 unit tests. Co-authored-by: Emilio Schwaiger <QuarkOS@users.noreply.github.com>
Migration 061 creates the service-role-only fraud_flags table (one row per user+category, deduped by a signal fingerprint, with open/confirmed/dismissed triage state and an audit-friendly signals snapshot). fraudDetectionServer fetches a user's recent activity + token history, runs the pure assessor, upserts flags (refreshing recurring patterns without resurrecting a staff decision), and alerts staff once per newly-opened flag. sweepFraudSignals assesses the competitive-board candidate set for the cron. All best-effort and never-throwing; covered by 9 unit tests with a mock Supabase client. Co-authored-by: Emilio Schwaiger <QuarkOS@users.noreply.github.com>
The leaderboard-integrity cron now piggybacks sweepFraudSignals (same Vercel cron-budget reason as the sponsor sweep) and reports the result. Adds an abuse.review staff action at the moderator floor, the /api/admin/abuse list (worst-first, with per-status counts and the flagged account's identity) and /api/admin/abuse/[id]/review confirm/dismiss endpoint (audit-first, target- guarded, status-guarded, with optional suspend on confirm via the shared moderation path). Surfaces a new /admin/abuse review-queue page mirroring the teams queue, with a nav entry under Review. Co-authored-by: Emilio Schwaiger <QuarkOS@users.noreply.github.com>
|
@cursoragent is attempting to deploy a commit to the BIRDABO Team on Vercel. A member of the Team first needs to authorize it. |
QuarkOS
marked this pull request as ready for review
August 28, 2026 12:35
Birdabo404
added a commit
that referenced
this pull request
Aug 29, 2026
…ing: scoring v3 paid the flat 40-point bonus for every visit row inside a session, so re-focusing/reloading one tool tab every few seconds minted a bonus per flap while sailing under the 600/day ceiling — observed in production (Aug 2026) as a reload loop banking ~250 same-domain visits a day (~10k pts/day, 69% of the account's total), enough to reach #2 on a five-day-old account. A session already models one continuous same-domain engagement, so v4 pays it at most one visit bonus (sessionVisitCap) — returning after the 5-minute gap opens a new session and earns normally — and the ingest ceiling drops 600 -> 150, calibrated against the heaviest organic day on record (84 visits): honest use is untouched while a maxed-out visit farm now earns less than a genuine heavy day of active time. Repricing is retroactive: scores rebuild from events_raw on the next sync or a verify-score-parity --fix pass, so farmed history deflates without touching accounts or rows. Credit to @QuarkOS (PR #78) for flagging that the ingest hard caps bound volume but never verify authenticity; this lands the structural fix on the scoring side — depricing the exploit beats detecting it after it pays.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The ingest routes already enforce hard bounds (per-event duration caps, a rolling 24h active-time/visit ceiling, a domain allowlist, per-record token caps, freshness windows). Those stop the crudest inflation, but a patient scripted extension can sit just under every cap and climb the board on fabricated activity, and a client can backfill impossible token totals onto the Burn Board.
This PR adds the missing detection layer: it runs over what actually landed in the database, looks for the behavioural signatures the hard caps can't express, raises deduped flags, alerts staff, and gives staff a review queue to confirm (optionally suspending) or dismiss.
How it works
Detection engine —
src/lib/fraudDetection.ts(pure, deterministic)Seven evidence-based detectors, biased for precision (each carries the raw numbers it fired on, for audit):
activity_ceiling_pinningimpossible_concurrencyvisit_floodinguniform_duration_paddingtoken_impossible_ratetoken_spiketoken_cost_mismatchassessUserFraudrolls signals up per category into a capped 0–100 risk score, a level band, and a stable dedupe fingerprint.Persistence & orchestration —
src/lib/fraudDetectionServer.ts+ migration 061fraud_flagstable (service-role only): one row per(user, category), deduped by fingerprint, withopen/confirmed/dismissedtriage state and an audit-friendly signal snapshot.detection_countwithout resurrecting a staff decision), and alerts every staff member once per newly-opened flag. Everything is best-effort and never throws.sweepFraudSignalsscans the competitive-board candidate set (top scorers + Burn Board opt-ins) and piggybacks the existingleaderboard-integritycron (same Vercel cron-budget reason as the sponsor sweep).Staff surface
abuse.reviewstaff action at the moderator floor.GET /api/admin/abuse— worst-first queue with per-status counts and the flagged account's identity.POST /api/admin/abuse/[id]/review— confirm/dismiss (audit-first, target-guarded, status-guarded, reason required), with optional suspend on confirm via the shared moderation path./admin/abusereview-queue page (mirrors the Teams queue) + a nav entry under Review.Testing
typecheck,lint,audit:prod(0 vulns), andnext buildall green; new routes compile (/admin/abuse,/api/admin/abuse,/api/admin/abuse/[id]/review).Deploy notes
migrations/061_fraud_flags.sql(creates the table; ends withNOTIFY pgrst). Thresholds live inDEFAULT_FRAUD_POLICYand can be tuned without schema changes./api/cron/leaderboard-integrityschedule.Possible follow-ups (not in this PR)
device_uuid).confirmedaccounts from the canonical leaderboard ranker.Opened from fork
QuarkOS/Cribble(branchcursor/fraud-detection-leaderboard-e229).