Skip to content

Report why a node's model inventory is empty - #3

Draft
DaveHomeAssist wants to merge 1 commit into
mainfrom
claude/ollama-model-loading-2hf0wu
Draft

Report why a node's model inventory is empty#3
DaveHomeAssist wants to merge 1 commit into
mainfrom
claude/ollama-model-loading-2hf0wu

Conversation

@DaveHomeAssist

Copy link
Copy Markdown
Owner

Problem

"Models aren't loading" was undiagnosable from the router. get_node_models collapsed every outcome into an empty list, so four distinct conditions reached the UI as the same "No models available":

  • Ollama unreachable (connection refused)
  • Ollama bound to loopback only, so reachable from the node but not from the router
  • /api/tags slower than the hardcoded 3s timeout
  • Node reachable with genuinely nothing pulled

GET /nodes/{node_id}/models returned HTTP 200 with models: [] in all four cases, and the only trace of the real cause was a print to the router's stdout.

Changes

Surface the failure reason. fetch_node_models returns {"models", "error"}. GET /nodes/{node_id}/models and GET /nodes/status both carry that error field. describe_node_error names the actual cause and, for a refused connection, points at the usual culprit (OLLAMA_HOST=0.0.0.0:11434). error: null with an empty list now unambiguously means "reachable, nothing pulled".

Malformed DAVE_NODES no longer fails silently. A parse failure registered zero nodes through a bare except Exception, which is indistinguishable from an empty cluster. It now prints the parse error and the expected JSON shape.

Timeout is configurable. DAVE_NODE_TIMEOUT (default 10s) replaces the hardcoded timeout=3 on both /api/tags calls. 3s is tight for a node loading a large model.

Stop clobbering a good inventory. The endpoint overwrote MODEL_INVENTORY[node_id] even when the fetch failed, so one transient timeout made /chat reject models the node really serves with Model '<id>' is not available. The inventory is now only written on a successful read.

Vision heuristic token boundaries. "mm" in lid matched every gemma tag and "vl" in lid was similarly loose, so ordinary text models were labelled vision-capable during load. Both now match as whole tokens, with an added pattern for the qwen2.5vl style.

Frontend. The model dropdown distinguishes "Node unreachable" (with the reason as a tooltip and in the console) from "No models pulled on node". loadModelsBtn.onclick = loadModelsFromNode passed the click MouseEvent in as preferredModelId, which failed every state.modelMeta lookup and silently reset the user's model choice on each manual reload; it now calls with no argument.

Verification

python -m py_compile app.py
python -m pytest -q                      # 31 passed
node --check static/app.js static/prompt-contract.js desktop/main.js desktop/preload.js
bash -n deploy/check-cluster.sh scripts/verify-cluster.sh
npm ci && npm ls --depth=0
git diff --check

Six tests added to tests/test_api_contracts.py covering the unreachable-node reason, inventory preservation across a failed fetch, the reachable-but-empty case, HTTP error surfacing, offline reason in /nodes/status, vision token boundaries, and the loud DAVE_NODES parse failure.

Cluster reachability itself is still not proven by repository tests — these changes make the runtime report which of the four conditions is actually happening, they do not verify a live node.


Generated by Claude Code

Model loading collapsed every failure into an empty list, so an
unreachable node, an HTTP error, a timeout and a node with nothing
pulled all reached the UI as "No models available" with no way to
tell them apart.

- fetch_node_models returns {"models", "error"}; /nodes/{id}/models
  and /nodes/status surface that error, and describe_node_error names
  the actual cause (refused connection, HTTP status, timeout) with the
  OLLAMA_HOST hint for loopback-only daemons.
- A malformed DAVE_NODES silently registered zero nodes. Print the
  parse error and the expected JSON shape instead.
- Raise the /api/tags timeout from a hardcoded 3s to a configurable
  DAVE_NODE_TIMEOUT (default 10s); 3s was tight for a node loading a
  large model.
- Stop overwriting MODEL_INVENTORY after a failed fetch, which made
  chat reject models the node really serves.
- Match "mm" and "vl" as whole tokens in the vision heuristic; the bare
  substring flagged every "gemma" tag as vision-capable.
- Frontend: distinguish "Node unreachable" from "No models pulled on
  node", and stop passing the click MouseEvent in as preferredModelId,
  which reset the user's model choice on every manual reload.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MtgxcGLFACR6q9M8ZYiqRM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants