fix: stop the registry callbacks erasing context_length and local_model - #130
Merged
Conversation
An explicit context_length in models.json was read at startup and then wiped. loadModelMappings unmarshals models.json into modelMappings correctly, but the registry's added/updated callbacks rebuild each entry from a field list that omitted ContextLength and LocalModel — and they fire after that load, and again on every hot reload. So the override survived only until the registry reported the model back, which is immediately. The consequence reaches the marketplace. ModelContext reads the override from modelMappings alone, so a backend that publishes no max_model_len of its own — llama.cpp, Ollama, an OpenAI-compatible proxy — declared nothing at all, and the server fell back to the catalog value for a window the operator had stated explicitly. On this node Qwen/Qwen3.8-27B carries context_length 65536 in models.json and was reporting "not reported". LocalModel was dropped the same way but does not bite: every forwarding path asks the registry first and only falls back to the mapping, so the name rewrite kept working. It was one refactor away from mattering. Both callbacks now go through one constructor that copies every field the registry holds, so a field added to RegisteredModel cannot be silently lost in the mirror again.
This was referenced Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while verifying #129 on a live node, and it answers a question #128 asked directly: "models.json supports a manual
context_lengthoverride. Is it applied?" — it was not.The bug
loadModelMappingsunmarshals models.json intomodelMappingscorrectly. Then the registry's added/updated callbacks rebuild each entry from a hand-written field list that omittedContextLengthandLocalModel— and they fire after that load, and again on every hot reload.So the override survived only until the registry reported the model back, which is immediately.
Observed on this node
models.jsonsays:and the agent reported:
ModelContextreads the override frommodelMappingsalone, so for a backend that publishes nomax_model_len— llama.cpp, Ollama, an OpenAI-compatible proxy — the agent declared nothing, and the marketplace fell back to the catalog value for a window the operator had stated explicitly. Exactly the failure #128 is concerned with, arriving from the opposite direction: not a guess sent as a measurement, but a measurement discarded.local_modelwas dropped too, and does not biteEvery forwarding path asks
registry.GetLocalModelName()first and only falls back tomapping.LocalModel, so the name rewrite kept working. It was one refactor away from mattering.The fix
Both callbacks now go through a single constructor that copies every field the registry holds, so a field added to
RegisteredModelcannot be silently lost in the mirror again.Verified
Two tests: one asserts every field round-trips, the other reproduces the end-to-end failure — an override on a backend that detects nothing, put through the registry callback, must still be declared as
overriderather than erased to unknown.8/8 packages pass.