Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 18 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,18 @@ included — and the box locks. Time remaining at commit is recorded, because a
polished answer submitted with most of the clock unspent was not composed in the
box.

**Paste is disabled** in the answer field (`paste` and `drop` are both blocked —
drag-and-drop text would otherwise walk straight past a paste-only guard).
**Paste is detected, not blocked.** Blocking it only teaches a candidate to retype
what they pasted; recording it tells us which answer to interrogate. React fires one
change event per input, so ordinary typing arrives as a stream of single-character
deltas — a single event that adds a whole paragraph did not come from a keyboard.
That answer is silently marked. Nothing is prevented and no warning is shown, so the
paste appears to have worked.

**One adaptive follow-up, before grading.** Once answers are in, the answer least
likely to have been typed by its author is selected — weighting typing rate by
length, so a suspiciously fast essay outranks a fast one-liner — and a single
likely to have been typed by its author is selected. A recorded paste wins outright,
since it is evidence rather than inference, with ties going to the largest single
injection; failing that, typing rate weighted by length, so a suspiciously fast essay
outranks a fast one-liner. A single
follow-up is generated that quotes that answer's specific wording back and pushes
on it. Same clock, same no-paste rule. Grading happens only after this round, so a
candidate cannot bank a score and abandon the round they cannot pass.
Expand All @@ -65,12 +71,15 @@ Measured on `psf/requests`, same repo and same time budget:
| Pasted AI answer | committed with 63s of 75s left, could not defend its wording | **0/100** |
| Genuine author | typed distinct answers, defended the follow-up | **100/100** |

**What this does not do.** The timer and the paste block are client-side. They
**What this does not do.** The timer and the paste detector are client-side. They
raise the cost of casual cheating; they do not stop anyone willing to call the API
directly with a forged `seconds_left`. The follow-up round is the measure that
actually holds, because it demands understanding at response time regardless of how
the request was made. Server-issued timestamps at generation, with elapsed time
computed server-side, are the real fix and are not built yet.
directly with a forged `seconds_left` and `flagged_paste: false`. Detection also has
a seam of its own: pasting after a pause long enough to look like thinking clears the
timing guard, and dictation software can legitimately commit a long phrase in one
event. The follow-up round is the measure that actually holds, because it demands
understanding at response time regardless of how the request was made. Server-issued
timestamps at generation, with elapsed time computed server-side, are the real fix and
are not built yet.

### Known limitation: comprehension is not difficulty

Expand Down
4 changes: 4 additions & 0 deletions backend/app/schemas/quiz.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ class QuizAnswer(BaseModel):
# Countdown remaining when the answer was committed. A long answer submitted with
# most of the clock still left is the signal that it was not typed from scratch.
seconds_left: Optional[float] = None
# Client-observed paste signal: a single input event that added a paragraph.
# Recorded silently — the candidate is never told it was noticed.
flagged_paste: bool = False
paste_delta: int = 0 # largest flagged single-event delta, used to rank flags


class QuizSubmitRequest(BaseModel):
Expand Down
18 changes: 14 additions & 4 deletions backend/app/services/quiz_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,14 @@ def pick_suspect_answer(answers: list[dict], time_limit: int = TIME_LIMIT_SECOND
"""
Choose the answer least likely to have been typed by its author.

Typing pace is the signal. A long, polished answer committed with most of the
clock still unspent was not composed in the box — nobody writes 900 considered
characters in twelve seconds. Weighting rate by length keeps a fast one-liner
from outranking a suspiciously fast essay.
A recorded paste wins outright: the client saw a whole paragraph arrive in one
input event, which is evidence rather than inference. Ties among flagged answers
go to the largest single injection.

Otherwise typing pace is the signal. A long, polished answer committed with most
of the clock still unspent was not composed in the box — nobody writes 900
considered characters in twelve seconds. Weighting rate by length keeps a fast
one-liner from outranking a suspiciously fast essay.

Falls back to the first non-empty answer, then to the first answer, so the
follow-up round always happens even with no timing data at all.
Expand All @@ -54,6 +58,12 @@ def pick_suspect_answer(answers: list[dict], time_limit: int = TIME_LIMIT_SECOND
if not answered:
return answers[0] if answers else None

# A recorded paste outranks every timing heuristic — it is direct evidence rather
# than an inference. Among several, push on the largest single injection.
pasted = [a for a in answered if a.get("flagged_paste")]
if pasted:
return max(pasted, key=lambda a: a.get("paste_delta") or 0)

timed = [a for a in answered if a.get("seconds_left") is not None]
if not timed:
return answered[0]
Expand Down
10 changes: 8 additions & 2 deletions frontend/src/features/quiz/QuizPage.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ export default function QuizPage() {

// Live countdown per question, kept in a ref so ticking never re-renders the page.
const timeLeft = useRef({});
// Paste signals per question. Also a ref — recording must stay invisible, and
// re-rendering on it would risk leaking that something was noticed.
const inputSignal = useRef({});
// Guards against the auto-submit firing twice (expiry racing a manual click).
const sent = useRef({ answers: false, followup: false });

Expand All @@ -34,6 +37,7 @@ export default function QuizPage() {
setFollowupAnswer("");
setExpired(new Set());
timeLeft.current = {};
inputSignal.current = {};
sent.current = { answers: false, followup: false };
try {
const data = await generateQuiz(repoUrl);
Expand All @@ -57,6 +61,8 @@ export default function QuizPage() {
question_id: q.id,
answer: answers[q.id] || "",
seconds_left: timeLeft.current[q.id] ?? null,
flagged_paste: inputSignal.current[q.id]?.flagged_paste ?? false,
paste_delta: inputSignal.current[q.id]?.paste_delta ?? 0,
}));
setFollowup(await submitQuiz(quiz.quiz_id, payload));
} catch (e) {
Expand Down Expand Up @@ -109,8 +115,7 @@ export default function QuizPage() {
{quiz && !followup && !result && (
<div className="quiz">
<p className="rules">
{limit}s per question · typing only, paste is disabled · answers lock when the
timer runs out
{limit}s per question · answers lock when the timer runs out
</p>
{quiz.questions.map((q) => (
<QuestionCard
Expand All @@ -120,6 +125,7 @@ export default function QuizPage() {
timeLimit={limit}
onTick={(id, s) => (timeLeft.current[id] = s)}
onExpire={markExpired}
onInputSignal={(id, sig) => (inputSignal.current[id] = sig)}
onAnswerChange={(id, val) => setAnswers((prev) => ({ ...prev, [id]: val }))}
/>
))}
Expand Down
26 changes: 17 additions & 9 deletions frontend/src/features/quiz/components/QuestionCard.jsx
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { useEffect, useRef, useState } from "react";
import { createInputTracker } from "../pasteDetect";

/**
* One timed question.
Expand All @@ -8,19 +9,23 @@ import { useEffect, useRef, useState } from "react";
* having its timers throttled. On expiry the answer is committed as-is — blank
* included — and the box locks.
*
* Paste is blocked so the answer has to be typed. This is a speed bump, not a
* security control: it stops the casual paste from another tab, and the real
* signal is the typing pace reported via onTick.
* Paste is deliberately NOT blocked. It is recorded instead: a single change event
* that adds a paragraph is reported upward as flagged_paste. The candidate sees no
* warning and nothing is prevented, so a paste still looks to them like it worked.
* The flag decides which answer gets pushed on in the follow-up round.
*/
export default function QuestionCard({
question,
answer,
onAnswerChange,
onExpire,
onTick,
onInputSignal,
timeLimit = 75,
}) {
const [left, setLeft] = useState(timeLimit);
const tracker = useRef(null);
if (tracker.current === null) tracker.current = createInputTracker();
const cbs = useRef({ onExpire, onTick });
cbs.current = { onExpire, onTick };

Expand All @@ -45,12 +50,17 @@ export default function QuestionCard({
return () => clearInterval(handle);
}, [question.id, timeLimit]);

function handleChange(e) {
const next = e.target.value;
tracker.current.record(answer || "", next, Date.now());
onInputSignal?.(question.id, tracker.current.snapshot());
onAnswerChange(question.id, next);
}

const locked = left === 0;
const mins = Math.floor(left / 60);
const secs = String(left % 60).padStart(2, "0");

const block = (e) => e.preventDefault();

return (
<div className={`question${locked ? " locked" : ""}`}>
<p>
Expand All @@ -67,11 +77,9 @@ export default function QuestionCard({

<textarea
value={answer || ""}
onChange={(e) => onAnswerChange(question.id, e.target.value)}
onPaste={block}
onDrop={block}
onChange={handleChange}
disabled={locked}
placeholder={locked ? "Locked — time expired." : "Your answer... (typing only, no paste)"}
placeholder={locked ? "Locked — time expired." : "Your answer..."}
/>
</div>
);
Expand Down
36 changes: 36 additions & 0 deletions frontend/src/features/quiz/pasteDetect.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
// Heuristic for "these characters were not typed here".
//
// React fires one change event per input, so ordinary typing arrives as a stream of
// +1 deltas. A single event that adds a whole paragraph did not come from a keyboard.
// The gap check is the guard against IME composition and autocomplete, which can also
// commit several characters at once but arrive after a normal human pause.
export const PASTE_MIN_DELTA = 40;
export const PASTE_MAX_GAP_MS = 100;

export function createInputTracker() {
let lastAt = null;
let flagged = false;
let maxDelta = 0;

return {
/** Feed one change event. Returns the observed delta/gap, for tests. */
record(prevText, nextText, now) {
const delta = (nextText?.length ?? 0) - (prevText?.length ?? 0);
// No prior event means nothing was typed before this. A paragraph appearing in
// the first input event cannot be typing, so it is treated as instantaneous
// rather than being excused for having no predecessor to compare against.
const gap = lastAt === null ? 0 : now - lastAt;
lastAt = now;

if (delta > PASTE_MIN_DELTA && gap < PASTE_MAX_GAP_MS) {
flagged = true;
if (delta > maxDelta) maxDelta = delta;
}
return { delta, gap, flagged };
},

snapshot() {
return { flagged_paste: flagged, paste_delta: maxDelta };
},
};
}
Loading