Skip to content

Discover provider models on startup with a local cache - #2

Open
chrisns wants to merge 2 commits into
fcakyon:mainfrom
chrisns:feat/dynamic-model-discovery
Open

Discover provider models on startup with a local cache#2
chrisns wants to merge 2 commits into
fcakyon:mainfrom
chrisns:feat/dynamic-model-discovery

Conversation

@chrisns

@chrisns chrisns commented Sep 10, 2026

Copy link
Copy Markdown

New provider models previously required changes to static lists in both routing and request validation. Discover Codex, Kimi, and Grok models at startup and when running claude-codex models, then share the resulting catalogue across the server, listings, monitor, and validation paths. Newly discovered models such as gpt-6-astra are available without updating the model lists in code.

  • Fetch native catalogues concurrently using existing subscription credentials, with a five-second timeout. Persist successful results in an atomic, account- and endpoint-scoped local cache; fall back to matching cached data or bundled models on failure.
  • Generate Codex -fast aliases dynamically and preserve discovered Kimi IDs through translation. Keep existing alias targets, Claude-to-Anthropic routing, and explicit GPT-to-Codex routing.
  • Preserve Cursor's explicit model prefixes and bundled aliases, and Anthropic's request-authenticated passthrough. Startup discovery does not introduce provider CLI dependencies or a new authentication flow; existing request-time credential refresh remains unchanged.

Validation: formatting, locked clippy, and all 932 tests pass. New tests use toy credentials and mock upstreams to cover discovery, cache replacement, offline reuse, malformed responses, timeouts, account isolation, partial provider failures, and newly discovered model translation. The optimized release binary builds successfully; a manual live check confirmed gpt-6-astra and gpt-6-astra-fast in /v1/models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant