Skip to content

feat: add AI Code Review GitHub App with multi-LLM review pipeline (#862) [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT] - #1458

Open
waterWang wants to merge 21 commits into
SolFoundry:mainfrom
waterWang:feat/ai-code-review-862
Open

feat: add AI Code Review GitHub App with multi-LLM review pipeline (#862) [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT]#1458
waterWang wants to merge 21 commits into
SolFoundry:mainfrom
waterWang:feat/ai-code-review-862

Conversation

@waterWang

Copy link
Copy Markdown

AI Code Review GitHub App

Creates an installable GitHub App that provides automated multi-LLM code reviews on every PR, with security checks, performance analysis, and best practices verification.

Features

  • Multi-LLM Reviews: Claude, OpenAI/Codex, and Gemini review every PR
  • 3 Review Modes: Quick (1 model), Standard (3 models), Thorough (5 models)
  • Security Checks: 7 patterns (hardcoded secrets, SQL injection, eval, unsafe deserialize, HTTP URLs, debug mode, command injection)
  • Performance Analysis: N+1 queries, sync-in-async, large list comprehensions, string concatenation in loops
  • Best Practices: Type hints, console.log, TODO/FIXME, magic numbers, bare except clauses
  • Configurable: Per-repository .github/ai-code-review.yml configuration
  • Check Runs: Results appear as check runs on every PR
  • Inline Comments: Findings are posted as inline review comments
  • GitHub App Manifest: One-click installation with proper permissions

Architecture

integrations/github-app/

  • manifest.json - GitHub App manifest for one-click installation
  • app.py - FastAPI webhook handler for pull_request events
  • config.py - Review configuration models (mode, strictness, comment style)
  • github_client.py - GitHub API client with JWT auth and installation tokens
  • models.py - Shared data models (Finding, DiffFile, ReviewResult)
  • analyzers/security.py - 7 security pattern detectors
  • analyzers/performance.py - 4 performance anti-pattern detectors
  • analyzers/best_practices.py - 5 best-practice checkers
  • reviewers/base.py - Abstract LLM reviewer interface
  • reviewers/claude.py - Claude API reviewer (with heuristic fallback)
  • reviewers/openai.py - OpenAI/Codex API reviewer (with heuristic fallback)
  • reviewers/gemini.py - Gemini API reviewer (with heuristic fallback)
  • reviewers/orchestrator.py - Multi-model orchestration with trimmed-mean scoring
  • tests/test_app.py - 10+ unit tests for models, analyzers, config, and orchestrator

How it works

  1. GitHub App webhook receives pull_request (opened/synchronize/reopened) events
  2. Fetches the unified diff and list of changed files via GitHub API
  3. Runs static analyzers (security, performance, best practices) on all files
  4. Sends diffs to configured LLM models for AI-powered review
  5. Aggregates results using trimmed-mean scoring (drops highest and lowest)
  6. Posts inline review comments for each finding
  7. Creates a check run with the aggregate score

Scoring

Base: 10.0

  • Critical: -2.0
  • High: -1.0
  • Medium: -0.5
  • Low: -0.2
  • Info: -0.1

Acceptance Criteria

  • GitHub App with one-click installation (manifest.json)
  • Multi-LLM reviews (Claude, Codex, Gemini) on PRs
  • Configurable review strictness and comment preferences
  • Security checks (7 patterns)
  • Performance analysis (4 anti-patterns)
  • Best practices verification (5 checkers)
  • Check runs with aggregate scores
  • Inline review comments on PR diffs
  • Heuristic fallback when no API keys are configured
  • Comprehensive test suite

Closes #862

Wallet: fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🏭 Bounty T3: AI Code Review GitHub App (Installable)

1 participant