feat: add Kalshi as a market source

[pythoryn]

Summary

• Add Kalshi as a ForecastBench prediction-market source.
• Add market discovery and question-update Cloud Run entry points.
• Filter and balance sufficiently liquid binary markets across Kalshi categories.
• Generate and maintain daily Kalshi resolution histories.
• Register Kalshi throughout the schemas, source registry, curation, nightly workflow,
  deployment configuration, and website.
• Treat only finalized Kalshi markets as resolved, preserving updates through determination
  and dispute states.
• Handle the first run of a new source when its question file does not exist yet.

Testing

• 104 focused Kalshi, schema, and base-source tests passed.
• Full Windows test suite: 693 passed; 30 pre-existing Windows-only failures.
• Black, isort, flake8, and pydocstyle checks passed.

[nikbpetrov]

A few AI-generated comments. First set is ones I've verified myself that should be considered:

---

I1 — Update downloads the complete resolution backlog

Priority: P2
Location: src/orchestration/func_kalshi_update/main.py:26

load_existing_resolution_files(SOURCE) is called without ids=, which its own docstring warns
will list and download every source resolution file. The resulting frame dictionary is consulted
only in the unresolved-question loop. Resolved rows use a separate set of existing filenames.

Every resolved file is therefore downloaded and parsed nightly but never read. The cost grows
without bound as the source accumulates resolved questions and can eventually dominate the 3-hour,
4-GiB job.

Recommended fix: after loading dfq, pass only unresolved IDs:

ids = dfq.loc[~dfq["resolved"], "id"]
existing_resolution_files = _source_io.load_existing_resolution_files(SOURCE, ids=ids)

This bounds content downloads to the live pool while retaining the cheap all-ID existence listing
for regeneration checks.

T1 — The cap test uses the production limit and dominates the unit suite

Priority: P2
Location: src/tests/test_kalshi.py:751-762

test_caps_new_questions constructs _QUESTION_LIMIT - 1, or 4,999, unresolved rows. update()
then validates and iterates all of them, invoking two mocks per row. Isolated timing was about
35 seconds inside pytest and roughly 39 seconds wall-clock. The entire focused 71-test run took
roughly the same amount of time.

The test is checking boundary behavior, not production-scale performance.

Recommended fix: monkeypatch sources.kalshi._QUESTION_LIMIT to a small value such as two, use
one existing unresolved row and several new rows, and assert the same cap/retention contract.

---

Second set of AI comments I have not verified and looked into as they concern API specifics:

---

K1 — Stored questions omit or contradict the actual binary contract

Priority: P1
Location: src/sources/kalshi.py:172-174

update() stores only:

dfq.at[index, "question"] = market["title"]

Kalshi frequently defines an event-like title shared by many child contracts and puts the actual
Yes outcome in yes_sub_title and rules_primary. A full live pagination audit using the proposed
type, liquidity, open-interest, and date filters found 132 duplicate-title groups covering 789
qualifying markets.

Examples:

• Eight SpaceX thresholds all had the title “How many launches will SpaceX have in 2026?”, while
  their yes_sub_title values represented different thresholds.
• Thirty baseball-team contracts shared “Pro Baseball Playoff Qualifiers”, while the team was only
  identified by the child outcome.

There are also qualifying contracts whose title is factually inconsistent with their own rules.
On 2026-07-24, KXLEAVEPOWELLGOV-27JUN01 cleared both liquidity thresholds, but:

• title asked whether Jerome Powell would leave before August 1, 2026;
• yes_sub_title, rules_primary, ticker, and close_time specified June 1, 2027.

KXLEAVEPOWELLGOV-28JAN31 showed the same stale August 2026 title while its child contract and
rules specified January 2028.

The rules are later appended to an LLM forecaster's background, which partly mitigates omitted
outcome text for that one consumer. It does not fix the contract:

• metadata validation passes only row["question"], not the rules;
• public question identity and human-facing text remain ambiguous;
• duplicate titles represent different targets;
• stale titles and rules can directly contradict one another.

Recommended fix: build a self-contained binary question from market-specific data, including
the child yes_sub_title, and validate it against rules_primary, timing, and ticker semantics.
When Kalshi fields disagree, reject the market rather than guessing. Add tests with two child
markets sharing a title but having different Yes outcomes, plus a contradictory-title fixture.

K2 — Latest close time is not a safe earliest-resolution filter

Priority: P1
Locations:

• discovery: src/sources/kalshi.py:375-382
• stored curation date: src/sources/kalshi.py:178-180
• downstream filter: src/curate_questions/create_question_set/main.py:1137-1140

The code describes close_time as the resolution date and requires it to be at least ten days out.
For Kalshi, close_time can be a hard stop or postponement window substantially later than the
expected event and likely early close. The official lifecycle documentation distinguishes these
concepts.

Two market-specific live records demonstrate the contamination path:

Ticker	Expected event/expiration	Latest/close time	Contract behavior
KXLIGAMXGAME-26JUL24TIJLEO-TIJ	2026-07-25 05:00Z	2026-08-08 02:00Z	Closes after the game winner is declared; two-week close is a postponement/cancellation bound
KXBOXING-26JUL25JOSHUAPRENGA-JOSHUA	2026-07-26 00:00Z	2026-08-08 21:00Z	Closes after the match winner; later close permits rescheduling

July 23, 2026 was an actual ForecastBench curation day, with forecasts due around August 2. Both
markets passed the proposed close_time test even though their outcomes were expected to be known
more than a week before forecasts were produced. They could therefore consume frozen benchmark
slots with already-known outcomes. The base resolver may eventually nullify a question whose
resolution preceded its due date, but that happens after selection and does not prevent
contamination or restore the lost slot.

Blindly replacing close_time with expected_expiration_time is not safe either: the Powell ladder
records in K1 demonstrate that expected-expiration data can be stale across child contracts.

Recommended fix: model both bounds deliberately:

• treat a credible occurrence_datetime/expected_expiration_time or early-close condition as the
  earliest plausible resolution;
• retain close_time/latest expiration as the latest bound;
• reject markets whose credible earliest resolution is inside the freeze window;
• reject internally inconsistent title/rules/time records rather than selecting one stale field;
• persist a curation field representing the earliest plausible resolution, because the shared
  close-only curation filter cannot otherwise enforce the benchmark window.

Tests should cover a sports market with expected resolution before the minimum but a later
postponement close, an ordinary aligned contract, and a stale ladder record with a deliberate
conservative policy.

K3 — ET daily candles violate the UTC forecasting cutoff

Priority: P1
Locations:

• query cutoff: src/sources/kalshi.py:433-435
• candle labeling/fill: src/sources/kalshi.py:473-515
• downstream date joins: src/sources/_market.py:67-89

Kalshi's 1,440-minute candles end at midnight US Eastern time: 04:00 UTC during daylight time and
05:00 UTC during standard time. The implementation subtracts a fixed day and assigns the candle
ending at D+1 04:00/05:00Z to calendar date D.

ForecastBench's nightly job and forecast dates use UTC. This creates two failures from the same
misalignment:

1. The value is stale at the actual nightly run. At 00:00 UTC, the ET daily candle ending four
   or five hours later does not yet exist. The latest returned daily candle is shifted to the
   day-before-yesterday, and ffill() copies it into yesterday.
2. A later rebuild introduces look-ahead. After 04:00/05:00 UTC, that next ET candle exists.
   Backdating it to the previous UTC date assigns the first four or five hours of the next UTC day
   to the benchmark deadline. The same historical date can therefore change retroactively and
   include trades placed after the deadline.

A live comparison for KXLIGAMXGAME-26JUL24TIJLEO-TIJ made the error measurable:

• an hourly query ending exactly 2026-07-24 00:00Z gave the true July 23 UTC-end value, 0.5000;
• the daily query available at that same cutoff ended 2026-07-23 04:00Z at 0.5100;
• the current shift labels that older 0.5100 candle as July 22, then forward-fills 0.5100 into
  July 23 instead of using 0.5000;
• after the next ET boundary, the new daily candle can be mapped backward into July 23 even though
  it contains post-midnight UTC trading.

This directly affects market_value_on_due_date and
market_value_on_due_date_minus_one, used for imputation and naive forecasts. It also makes
missing-file reconstruction dependent on what time of day it is run.

Recommended fix: use hourly or finer candles and select the last observation at or before each
UTC day boundary, with an explicit as-of rule. Merely moving the job after 05:00 UTC fixes the stale
nightly read but not post-deadline look-ahead, so it is not sufficient by itself. Add a regression
proving that no observation after D 23:59:59Z can be stored as D.

This replaces the old report's L7. The old appendix claim that subtracting 24 hours was “safe”
proved only that it recovered an ET calendar label across DST; it did not establish UTC
no-look-ahead and was incorrect for ForecastBench.

K4 — Finalization can permanently preserve a nonterminal probability

Priority: P1
Location: src/sources/kalshi.py:462-466

The early return checks only whether the existing file reaches the resolution date:

if last_date >= cutoff:
    return existing_df

That is insufficient when a market has just transitioned from active/determined to finalized. An
unresolved run may already have written or forward-filled a market probability on the eventual
settlement date. On the first finalized run, last_date >= resolved_date returns that same file
before appending the terminal 0/1 result.

This was reproduced with a finalized Yes market settling on January 13 and an existing January 13
row of 0.63. _build_resolution_df returned the same object, never fetched candles, and retained
0.63. update() marked the question resolved and skipped the file upload because the object was
unchanged. Later runs see an existing resolution file and never enter missing-file regeneration.

MarketSource._resolve reads the last value as the final outcome. Since 0.63 is neither zero nor
one, it logs a warning and sets resolved_to to NaN, permanently discarding an otherwise valid
resolved question.

Recommended fix: the resolved early return must verify the terminal row represents the current
market result, not merely that a date exists. On transition to finalized, always insert/replace the
terminal resolution row (or rebuild deterministically). Add a regression with a same-date stale
price and both Yes and No outcomes.

K5 — Historical routing and 404 lifecycle are conflated

Priority: P2
Locations:

• unresolved handling: src/sources/kalshi.py:164-169, :229-242
• missing-file regeneration: src/sources/kalshi.py:208-227
• live detail/candles only: src/sources/kalshi.py:398-442

Kalshi moves older settled market data behind historical endpoints. On 2026-07-24, the returned
historical cutoff was 2026-05-25. A concrete archived market,
KXHIGHNY-25DEC08-T40, returned:

• 404 from the live market-detail endpoint;
• 404 from the live candlestick endpoint;
• 200 from the historical market endpoint; and
• 200 from the historical candlestick endpoint.

Historical candles also use a different price shape (price.close rather than live
price.close_dollars), so fallback requires normalization.

This breaks the source's explicit missing-resolution-file regeneration path for older resolved
questions. It also means a question that was not updated before crossing the archive cutoff cannot
recover. Depending on which live record remains available during the moving boundary, a candle 404
can abort the update while a detail 404 is mislabeled as permanent delisting and swallowed.

The same MarketNotFoundError policy conflates two more cases:

• Kalshi documents that newly created markets can briefly return 404 because detail propagation is
  asynchronous; the code deliberately makes 404 nonretryable and drops the new row until a later
  nightly fetch.
• A genuinely missing existing unresolved row is kept forever with stale metadata. It remains
  resolved=False, is retried every night, consumes _QUESTION_LIMIT, and remains eligible for
  curation.

Recommended fix: distinguish transient propagation, historical partitioning, and genuine
permanent removal. Route/fallback to the historical detail and candle endpoints when appropriate,
normalize both candle schemas, retry documented transient 404s, and explicitly invalidate or remove
a confirmed permanent orphan rather than keeping it indefinitely.

K6 — Open events can include non-active child markets

Priority: P2
Locations: src/sources/kalshi.py:271-300, :350-383

The events request uses status=open, but Kalshi defines an event as open when at least one child
market is active. Official lifecycle examples and a live pagination snapshot both show mixed child
statuses under open events.

_market_qualifies() checks type, liquidity, open interest, and time, but not the child's own
status. A closed, determined, disputed, inactive, or finalized child can therefore enter the
candidate set if it meets the other conditions. Non-finalized but nontradeable children are
especially problematic because _is_resolved() recognizes only finalized; they can remain
eligible for a new question set even though no new forecast can trade against the market.

No non-active child in the reviewed live snapshot happened to pass every other filter, so this is a
contract-level bug rather than evidence of current stored corruption.

Recommended fix: require market["status"] == "active" during discovery. Also define how an
existing question that closes/determines before curation should be excluded while it awaits final
settlement.

[pythoryn]

Thanks for the detailed review. I have reproduced the core issues and I'm working through them. I'll keep the PR as a single amended commit and follow up with the final verification results.