TranscriptIndex.lookup: max_tokens has no observable effect on payload size

[laynepenney]

TranscriptIndex.lookup accepts a max_tokens parameter documented as an
"Approximate token budget (~4 chars/token)". Varying it across a 10,000x range
produces a byte-identical payload.

Observed

Measured on a DB-backed index (14,717 chunks), one query, max_chunks held at 5:

max_tokens	payload chars	approx tokens (~4 chars/token)
10	2021	505
500	2021	505
2400	2021	505
100000	2021	505

One distinct payload size across the whole range. At max_tokens=10 the
documented budget permits roughly 40 characters; the call returned 2021, about
50x over.

Control

The same index and query, holding max_tokens=500 and varying max_chunks:

max_chunks	payload chars
1	704
3	704
5	2021
10	3016

Three distinct sizes, so the harness does respond to a parameter that should
move it. Without this the result above would be unreadable: a payload that never
changes could equally mean the measurement never ran.

Reproduction

Runs against any existing index; no fixture or network needed.

from pathlib import Path
from synapt.recall.core import TranscriptIndex

idx = TranscriptIndex.load(Path(".synapt/recall/index"))
q = "deployment"

# subject: 10,000x range in the budget
for mt in (10, 500, 2400, 100000):
    print(mt, len(idx.lookup(q, max_chunks=5, max_tokens=mt)))

# control: a parameter that does bind
for mc in (1, 3, 5, 10):
    print(mc, len(idx.lookup(q, max_chunks=mc, max_tokens=500)))

Absolute sizes depend on your index and query and will not match the tables
above. The finding is the shape: the first loop's sizes are constant across the
whole range, and the second loop's are not.

What this report does and does not claim

max_tokens is threaded through the call rather than dropped at the signature
(core.py:1804 declares it; it is passed onward at core.py:1954 and
core.py:1963), so this is not a case of an unused argument.

The intent also appears to be that it binds. A comment in _global_lookup_fts
(core.py:2282-2283) reads: "Pass extra candidates to _format_results to
compensate for near-duplicate filtering — the token budget is the real limiter."
Extra candidates are supplied downstream on the expectation that the budget will
trim them.

This report does not identify where the behaviour diverges from that intent, and
does not claim to. It is a measured non-effect with a control beside it; a
guessed cause would be worse than the observation.

Why it matters

Callers sizing a retrieval payload will reach for the parameter named after the
budget. On the path measured here, max_chunks binds and threshold_ratio
shapes the result while max_tokens moves nothing, so a caller tuning
max_tokens is adjusting a control that is not connected to the outcome. Any
downstream cost or latency estimate derived from that parameter would be wrong
in the same direction every time.