Skip to content

fix: stop the registry callbacks erasing context_length and local_model - #130

Merged
flyworker merged 1 commit into
mainfrom
fix/models-json-override-erased
Sep 5, 2026
Merged

fix: stop the registry callbacks erasing context_length and local_model#130
flyworker merged 1 commit into
mainfrom
fix/models-json-override-erased

Conversation

@flyworker

Copy link
Copy Markdown
Member

Found while verifying #129 on a live node, and it answers a question #128 asked directly: "models.json supports a manual context_length override. Is it applied?"it was not.

The bug

loadModelMappings unmarshals models.json into modelMappings correctly. Then the registry's added/updated callbacks rebuild each entry from a hand-written field list that omitted ContextLength and LocalModel — and they fire after that load, and again on every hot reload.

So the override survived only until the registry reported the model back, which is immediately.

Observed on this node

models.json says:

"Qwen/Qwen3.8-27B": { "endpoint": "http://localhost:30001", "context_length": 65536 }

and the agent reported:

Qwen/Qwen3.8-27B      -  (not reported)

ModelContext reads the override from modelMappings alone, so for a backend that publishes no max_model_len — llama.cpp, Ollama, an OpenAI-compatible proxy — the agent declared nothing, and the marketplace fell back to the catalog value for a window the operator had stated explicitly. Exactly the failure #128 is concerned with, arriving from the opposite direction: not a guess sent as a measurement, but a measurement discarded.

local_model was dropped too, and does not bite

Every forwarding path asks registry.GetLocalModelName() first and only falls back to mapping.LocalModel, so the name rewrite kept working. It was one refactor away from mattering.

The fix

Both callbacks now go through a single constructor that copies every field the registry holds, so a field added to RegisteredModel cannot be silently lost in the mirror again.

Verified

Two tests: one asserts every field round-trips, the other reproduces the end-to-end failure — an override on a backend that detects nothing, put through the registry callback, must still be declared as override rather than erased to unknown.

8/8 packages pass.

An explicit context_length in models.json was read at startup and then wiped.
loadModelMappings unmarshals models.json into modelMappings correctly, but the
registry's added/updated callbacks rebuild each entry from a field list that
omitted ContextLength and LocalModel — and they fire after that load, and again
on every hot reload. So the override survived only until the registry reported
the model back, which is immediately.

The consequence reaches the marketplace. ModelContext reads the override from
modelMappings alone, so a backend that publishes no max_model_len of its own —
llama.cpp, Ollama, an OpenAI-compatible proxy — declared nothing at all, and
the server fell back to the catalog value for a window the operator had stated
explicitly. On this node Qwen/Qwen3.8-27B carries context_length 65536 in
models.json and was reporting "not reported".

LocalModel was dropped the same way but does not bite: every forwarding path
asks the registry first and only falls back to the mapping, so the name rewrite
kept working. It was one refactor away from mattering.

Both callbacks now go through one constructor that copies every field the
registry holds, so a field added to RegisteredModel cannot be silently lost in
the mirror again.
@flyworker
flyworker merged commit b74da56 into main Sep 5, 2026
1 check passed
@flyworker
flyworker deleted the fix/models-json-override-erased branch September 5, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant