fix(harness): align the portal web UI with the revamped harness run contract

[jdye64]

Description

The Retriever harness was revamped (#2270 and follow-ups) into a benchmark-registry CLI: runs are launched with retriever harness run <benchmark> --mode {local,batch,service} and dotted --set overrides, and the legacy sweep / nightly / portal / runner / backfill CLI commands were removed. The harness portal web UI was never updated to match, so several "run things" paths are stale or broken. This PR realigns the portal's run-triggering surface with the current harness contract.

Harness ↔ portal differences that were breaking things

1. Execution modes. The harness accepts local / batch / service (contracts.MODE_TO_RUN_MODE, where local → ingest.run_mode=inprocess). The portal only ever emitted run_mode=service and had no way to pick local or batch — even though the harness README requires batch for the large BEIR corpora (BO767, FinanceBench, Earnings, ViDoRe).
2. _resolve_dataset_config signature drift. It now returns a 3-tuple (path, overrides, dataset_meta), but the MCP trigger_benchmark_run tool still unpacked two values, so triggering a run over MCP crashed at call time.
3. Removed CLI commands referenced in the UI. The Runners and Runs empty-states pointed users at retriever harness runner start … and retriever harness backfill, which no longer exist.
4. Invalid dataset options. The dataset form offered input_type=text (not a valid harness input type) and was missing doc / html / video / auto; the harness now supports auto/pdf/doc/txt/html/image/audio/video. It also lacked the pdf_page_modality BEIR doc-id field and defaulted new datasets to pdf_basename instead of the current DatasetSpec default pdf_page.

Changes

• portal/app.py
  • Validate the requested execution mode against MODE_TO_RUN_MODE in /api/runs/trigger and always record it on the job (canonical mode, with run_mode kept as a legacy alias).
  • Add /api/harness-info exposing the current run contract — accepted modes and the code-owned benchmark/runset registry — using the lightweight benchmark_registry import so the UI has a single source of truth.
• portal/mcp_tools.py — fix the 3-tuple unpack crash in trigger_benchmark_run, forward dataset_id/dataset_config_hash, and add an optional validated mode argument.
• portal/static/views/trigger.jsx — add a Local/Batch/Service execution-mode selector (populated from /api/harness-info); service is now an execution mode rather than a pipeline type, and graph pipelines are disabled in service mode.
• portal/static/views/datasets.jsx — correct the input-type list, add pdf_page_modality, default new datasets to pdf_page.
• portal/static/views/runs.jsx, runners.jsx — replace references to removed CLI commands with current guidance.

Known follow-up (out of scope, needs the runner)

The portal still stores managed-dataset/preset config as flat keys (dataset_dir, embed_model_name, ocr_version, beir_loader, …). The current harness only accepts dotted --set namespaces (dataset.*, ingest.*, query.*, evaluation.*) and rejects unknown keys with exit code 2. The flat→dotted translation lives in the runner agent (external to this repo), which must be updated alongside these UI changes for end-to-end runs to succeed.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

  

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.