Skip to content

feat: anti-gaming — timer, paste blocking, adaptive follow-up - #3

Merged
krishhimself merged 1 commit into
mainfrom
feat/anti-gaming
Aug 22, 2026
Merged

feat: anti-gaming — timer, paste blocking, adaptive follow-up#3
krishhimself merged 1 commit into
mainfrom
feat/anti-gaming

Conversation

@krishhimself

Copy link
Copy Markdown
Owner

Summary

The obvious attack on a comprehension quiz is to paste the question into a chatbot and
paste the answer back. Three measures raise the cost of that, and the flow is
restructured to generate → submit → followup.

Measured result

Same repo (psf/requests), same time budget, same flow:

Profile Behaviour Score
Pasted AI answer committed with 63s of 75s left, could not defend its own wording 0/100
Genuine author typed distinct answers, defended the follow-up 100/100

The follow-up quotes the candidate's wording back, which is what makes it un-fakeable:

"You stated that 'error states are surfaced through the standard exception mechanism'
without 'leaking implementation details' — when the pool experiences a connection
timeout versus pool exhaustion, what specific exceptions does your 'boundary' raise?"

The genuine run scored 10/10 — "Exceptional defense under pressure" on the follow-up row.

1. Per-question 75s clock

Starts when the question renders. Computed from wall-clock time, not accumulated
ticks
— a backgrounded tab can't buy seconds by having its interval throttled. On
expiry the answer commits as-is (blank included) and the field locks; when every
question has expired the quiz submits itself. Time remaining at commit rides along with
each answer.

2. Paste blocked

paste and drop are both prevented. Drag-and-drop text walks straight past a
paste-only guard.

3. Adaptive follow-up, before grading

The answer least likely to have been typed by its author is selected by weighting typing
rate by length — (length/elapsed) × length — so a suspiciously fast essay outranks a
fast one-liner. Falls back to the first answered question, then the first question, so
the round always happens even with no timing data.

Grading is deferred until after this round on purpose. Grading at submit would let a
candidate bank a score and walk away from the round they can't pass.

API change

/submit no longer returns a score — it returns the follow-up question. /followup
returns the final result in the shape /submit previously returned, so ScoreResult
needed no changes. Attempt documents gain answers, followup, and an
awaiting_followup status between generated and graded.

What this does NOT do

The timer and paste block are client-side. They raise the cost of casual cheating;
they do not stop anyone calling the API directly with a forged seconds_left. The
follow-up round is the measure that actually holds, because it demands understanding at
response time regardless of transport. Server-issued timestamps at generation, with
elapsed time computed server-side, are the real fix and are not built here. This is
stated plainly in the README rather than footnoted — overstating it would be a security
claim that isn't true.

Operational note

The follow-up adds a third Gemini call per quiz. On the free tier's 20 requests/day/model
cap that means roughly 6 complete quizzes per day, down from 10.

Verification

pytest passes, vite build clean, documents persist through
generated → awaiting_followup → graded with answers and follow-up recorded, test data
cleared. Suspect-selection heuristic verified against four cases including no-timing-data
and all-blank fallbacks.

Not verified: the interactive browser flow. The Chrome extension would not connect,
so frontend checks were server-rendered. This PR adds the most timing-sensitive UI in the
project — countdowns, auto-submit on expiry, a two-phase form — and none of it has been
exercised by an actual click. Worth a manual pass before merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PET9qKZXhgjEbZK7MReYQj

The obvious attack on a comprehension quiz is to paste the question into a chatbot
and paste the answer back. Three measures raise the cost of that.

Per-question 75s clock. It starts when the question renders and is computed from
wall-clock time rather than accumulated ticks, so a backgrounded tab cannot buy
extra seconds by having its interval throttled. On expiry the answer commits as-is,
blank included, and the field locks; when every question has expired the quiz
submits itself. Time remaining at commit is sent with each answer.

Paste is blocked in the answer field. Both paste and drop are prevented, since
drag-and-drop text would otherwise walk straight past a paste-only guard.

One adaptive follow-up, asked before anything is graded. The answer least likely to
have been typed by its author is selected by weighting typing rate by length -
(length/elapsed) * length - so a suspiciously fast essay outranks a fast one-liner,
with fallbacks to the first answered question and then the first question so the
round always happens even with no timing data. A single follow-up is generated that
quotes that answer's specific wording back and pushes on what it implies. Grading is
deferred until after this round on purpose: grading at submit would let a candidate
bank a score and walk away from the round they cannot pass.

This restructures the flow into generate -> submit -> followup. /submit no longer
returns a score, it returns the follow-up question; /followup returns the final
result in the shape /submit previously returned, so ScoreResult needs no changes.
Answers, timing, and the follow-up exchange are all persisted on the attempt, and
status gains an awaiting_followup state between generated and graded.

Measured on psf/requests with the same repo and time budget: a pasted AI answer
committed with 63s of 75s left, unable to defend its own wording, scored 0/100; a
genuine author typing distinct answers and defending the follow-up scored 100/100,
with the follow-up appearing as its own row in the breakdown.

Known gap, documented in the README: the timer and paste block are client-side and
do not stop anyone calling the API directly with a forged seconds_left. The
follow-up is the measure that actually holds, since it demands understanding at
response time regardless of transport. Server-issued timestamps are the real fix and
are not built here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PET9qKZXhgjEbZK7MReYQj
@krishhimself
krishhimself merged commit 7f4df3b into main Aug 22, 2026
2 checks passed
@krishhimself
krishhimself deleted the feat/anti-gaming branch August 22, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant