Skip to content

feat(tui): local models get their own screens, not the admin panel - #225

Open
plombeer31 wants to merge 2 commits into
valeryb/onboarding-choose-copyfrom
valeryb/onboarding-local-models
Open

feat(tui): local models get their own screens, not the admin panel#225
plombeer31 wants to merge 2 commits into
valeryb/onboarding-choose-copyfrom
valeryb/onboarding-local-models

Conversation

@plombeer31

Copy link
Copy Markdown
Collaborator

Stacked on #224#223#222#220.

Why

Picking Local models persisted mode: "managed" and dropped the operator into MANAGE ▸ LLM — tab strip, wrapped cwd line, kv —, tools 0ok/0err, approval L1, and a status: ready header sitting over an install with nothing on disk. The first decision of a first run was being made inside an operator console, with 2.7–22 GB downloads listed and no hint which one this machine should take.

Two screens

Pick a model

  ›  gemma-4-12b         6.7 GB    ★ recommended · 12 GB RAM · Mid-size dense multimodal…
     qwen-3.5-4b         2.7 GB    8 GB RAM · Quality-size sweet spot
     gemma-4-e4b         4.2 GB    8 GB RAM · Compact multimodal reasoning (QAT)
     gemma-4-26b-a4b    14.2 GB    20 GB RAM · Fast MoE with 256K context (QAT)
     ↓ 6 more

RAM comes before the description because it decides whether the model runs here — and because the description is what truncation should eat first. Rows are coloured by fit: green for the recommendation, warn for anything past this machine's RAM (dimmed, not hidden — an operator who knows their swap situation may still pick it).

The recommendation is the largest model that runs comfortably here and stays under an 8 GB download ceiling (FIRST_RUN_MAX_DOWNLOAD_GB). On a 68 GB machine the naive "largest that fits" rule recommended a 22.4 GB download as the first thing a new user does; the ceiling makes it a 6.7 GB one, with the bigger models one row away. Nothing comfortable → the smallest model in the catalog: something that runs beats something that was recommended.

Watch it download

  llama.cpp runtime   ████████████████████████████████████  done
  model weights       ██████████████░░░░░░░░░░░░░░░░░░░░░░  38%   1.6 GB / 4.2 GB
                      12.4 MB/s · about 3 minutes left

Rate and ETA are derived from the same progress events (use-transfer-rate.ts, EMA-smoothed — raw deltas between two HTTP chunks swing wildly). The two phases share one progress slot in state, so the checklist is derived from which one is currently reporting.

Wiring

Both screens drive the existing LocalModelsOrchestrator: onLocalModelsPullRequested starts the pull, local_models_pull_* feeds the bars. The download lifecycle is unchanged — it was already session-scoped and survives leaving the screen.

Reducer order. A finished pull and a saved provider now have two owners, so reduceOnboardingAction moves to the head of the chain and delegates the panel half to the owning slice. It only claims an action while the flow is open, so nothing else changes shape.

Embeddings. The hybrid-recall offer that fires after a first chat pull is suppressed during onboarding — a second download and a second decision, from a screen with no room to explain either. It still appears in the LLM panel, which has both. Pinned by a test in each direction.

Tests

local-model-picks.test.ts (recommendation under/over the ceiling, no-comfortable-fit fallback, empty catalog, fit classification, ordering), onboarding-download-step.test.tsx (pre-progress state, bytes/percent, runtime-done derivation, waiting phase, error surfaced, rate appears on the second sample), and six new reducer cases including both directions of the embedding suppression and the cursor reset between lists.

Full suite: 5186 passed. fs-glob-real and send-message-concurrency fail on main too; llm-health-poller flaked under parallel load and passes in isolation.

Choosing "Local models" wrote `mode: "managed"` and handed the operator
to MANAGE ▸ LLM: an eight-tab strip, a wrapped cwd line, `kv —`,
`tools 0ok/0err`, `approval L1`, and a `status: ready` header over an
install with nothing on disk. The first decision of a first run was
being made inside an operator console.

Two screens instead:

**Pick a model.** Size, RAM, one recommendation, and colour that says
whether this machine can run each row. The recommendation is the largest
model that runs comfortably here *and* stays under an 8 GB download
ceiling — a machine with the RAM for the 22 GB model can still be an
hour from its first answer, and the bigger ones stay one row away. When
nothing fits comfortably the smallest model wins: something that runs
beats something that was recommended.

**Watch it download.** Two phases (llama.cpp runtime, then weights) with
percent, bytes, rate and ETA. The panel's banner reports a percentage,
which cannot answer "how long" — the rate is derived from the same
progress events with an EMA, because raw deltas between HTTP chunks
swing wildly.

Both read the existing orchestrator: `onLocalModelsPullRequested` starts
the pull, `local_models_pull_*` drives the bars. Nothing about the
download lifecycle changes; it was already session-scoped.

Two actions now have two owners — a finished pull and a saved provider —
so `reduceOnboardingAction` moves to the head of the chain and delegates
the panel half. It only ever claims an action while the flow is open.

The hybrid-recall embedding offer is suppressed during the first run: a
second download and a second decision, from a screen with no room to
explain either. It still appears in the LLM panel, which has both.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant