refactor(llm): consolidate the triplicated Gemini client - #17
Merged
Conversation
One place owns the Gemini HTTP call, the per-attempt timeout + optional retry on transient errors (429/5xx/network), the cost-controlled generationConfig (thinkingBudget 0, bounded maxOutputTokens), and token/cost accounting. Callers pass prompts, a response schema, and per-call overrides (temperature, timeoutMs, maxAttempts) and parse the returned text themselves.
Drop the duplicated PRICING map, estimateCost, GeminiResponse type, retry loop, and inline fetch; reviewDiff now calls callGemini() and parses the result. Same span attributes, return shape, model default, and 8192 output-token cap.
Remove the duplicated PRICING/estimateCost/GeminiResponse and inline fetch; scanCodebase now calls callGemini(). Behavior unchanged except scans now inherit the client's per-attempt timeout and single retry, matching reviews.
Replace the inline fetch with callGemini(temperature 0, 16384 output tokens). Keeps autofix's single-attempt, no-retry behavior and sets a 45s per-attempt timeout matching openFixPr's outer cap so a slow-but-valid fix isn't aborted early or retried past budget.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What — Pulls the three copies of the Gemini plumbing into one shared client at src/server/llm/gemini.ts; adapters keep their own prompts/schema/parser.
Why — Three copies had already drifted (only review had retry/timeout); the earlier "reviews silently failing on a throttled model" incident traces to exactly this divergence.
Behavior — Review unchanged; scan gains timeout+retry; autofix keeps single-attempt with a 45s cap matching its outer budget.
Verified — tsc / lint / prettier all clean.