CodePilot AI combines local LLMs, project-wide RAG, VS Code diagnostics, and guarded code editing to help developers understand, debug, and improve entire codebases directly from their editor.
Most AI coding assistants work well when all the required context fits inside a prompt.
Real projects are different.
Understanding a bug or feature may require information from multiple files, functions, diagnostics, and dependencies.
CodePilot AI was built to explore a different approach:
Give the AI awareness of the project — not just the current prompt.
It runs directly inside VS Code and combines:
- current file context
- cursor and function context
- recent conversation history
- VS Code diagnostics
- project-wide semantic retrieval
- local AI inference
The current version runs using Ollama, so no paid AI API is required.
CodePilot can answer questions using the current workspace and display the source files used to generate the response.
CodePilot reads VS Code diagnostics, generates a proposed correction, and allows the developer to review the result before applying it.
Ask questions directly from the Explorer sidebar without leaving the editor.
Choose between:
- Fast
- Normal
- Detailed
depending on how much explanation you need.
CodePilot can use:
Current File
+
Cursor Position
+
Current Function
+
Recent Conversation
to generate more relevant answers.
Function-level context is detected using VS Code document symbols.
Follow-up questions retain recent conversation context during the current extension session.
Example:
What does this function do?
↓ Follow-up
Can you optimize it?
CodePilot understands that the second question refers to the previous context.
CodePilot can automatically index the current workspace and retrieve relevant code before answering project-level questions.
Example:
Which file contains the button click logic?
CodePilot searches the indexed project, retrieves relevant code chunks, generates the answer, and shows the corresponding source files.
Workspace files are automatically indexed when the extension starts.
When supported files are saved, CodePilot updates the relevant project index rather than rebuilding everything manually.
Project-aware answers show the source files used during retrieval.
These sources can be opened directly inside VS Code.
CodePilot integrates with the VS Code Diagnostics API to detect actual editor errors.
Instead of relying only on raw code, the AI can also understand what VS Code itself is reporting.
For a detected error:
Detected Error
↓
Generate Fix
↓
Review Proposed Fix
↓
Apply Fix
The generated correction is shown before it modifies the code.
For several related diagnostics:
Fix All Errors
↓
Generate Combined Correction
↓
Review Proposed Fix All
↓
Apply All Fixes
CodePilot does not automatically apply the generated Fix All result.
The developer remains in control of the final modification.
Select code inside the editor and use:
- Explain Selected Code
- Fix Selected Code
- Improve Code
These actions send the selected code directly to the configured local model.
Additional actions include:
- Copy Response
- Copy Fix
- Explain Error
- Generate Fix
- Apply Fix
VS Code
│
┌───────────────┼────────────────┐
│ │ │
▼ ▼ ▼
Diagnostics API Active Editor CodePilot Sidebar
│ │ │
│ Current File │
│ Cursor Position │
│ Function Context │
│ │ │
└───────────────┬────────────────┘
│
▼
Context Builder
│
┌────────┴────────┐
│ │
▼ ▼
Current Context RAG Service
│
Workspace Files
│
Text Chunking
│
Local Embeddings
│
Similarity Search
│
┌──────────────────┘
│
▼
Prompt Construction
│
▼
Ollama / Qwen3
│
▼
AI Response / Fix
│
┌──────┴──────┐
▼ ▼
Answer Fix Preview
│
▼
Developer Review
│
▼
Apply Change
CodePilot's project-aware retrieval currently uses:
Workspace Files
↓
RecursiveCharacterTextSplitter
↓
Code Chunks
↓
OllamaEmbeddings
↓
nomic-embed-text
↓
MemoryVectorStore
↓
Similarity Search
↓
Relevant Project Context
↓
User Question + Context
↓
Qwen3
↓
AI Response
The RAG layer is implemented using LangChain.
| Area | Technology |
|---|---|
| Language | TypeScript |
| Platform | VS Code Extension API |
| AI Runtime | Ollama |
| LLM | Qwen3 |
| Embeddings | nomic-embed-text |
| RAG Framework | LangChain |
| Vector Store | MemoryVectorStore |
| Text Splitting | RecursiveCharacterTextSplitter |
| Bundling | esbuild |
Install:
- Visual Studio Code
- Node.js
- npm
- Ollama
git clone https://github.com/Rakshith-028/codepilot-ai.git
cd codepilot-ainpm installPull the language model:
ollama pull qwen3:4bPull the embedding model:
ollama pull nomic-embed-textOllama should be running locally at:
http://localhost:11434
npm run compileOpen the repository in VS Code and press:
F5
A new Extension Development Host window will open.
Open the Explorer sidebar and expand:
CodePilot AI
Place your cursor inside a function and ask:
What does this function do?
CodePilot uses the active file, cursor position, and detected function as context.
Then ask:
Can you optimize it?
Recent conversation context allows CodePilot to understand the follow-up.
Example:
Which file contains the button click logic?
If the workspace has been indexed, CodePilot retrieves relevant chunks and displays the source files used for the answer.
When VS Code reports an error:
- Open CodePilot AI.
- Review the detected diagnostic.
- Click Generate Fix.
- Review the proposed code.
- Click Apply Fix only if the preview is correct.
CodePilot is designed around local inference.
Project code and context are sent to the locally running Ollama server:
http://localhost:11434
The current implementation does not require a paid cloud AI API.
CodePilot AI is still under active development.
Current limitations include:
- RAG data is stored in memory and rebuilt after extension restart.
- Conversation history is session-based.
- Very large repositories may take longer to index.
- AI-generated fixes may still be incorrect and should always be reviewed.
- Function detection depends on VS Code language support and document symbol providers.
- Fix All currently focuses on the active file rather than multi-file refactoring.
Future versions may include:
- Persistent vector storage
- Git-aware context
- Diff-based fix previews
- Inline editor suggestions
- Test generation
- Terminal and runtime error awareness
- Multi-file refactoring
- Configurable local models
- Model and indexing settings
- Persistent project conversations
Initial development release.
Included:
- Local Ollama integration
- AI chat sidebar
- current-file context
- cursor-aware context
- function-aware context
- short conversation memory
- project-wide RAG
- automatic indexing
- incremental re-indexing
- source attribution
- clickable source navigation
- VS Code diagnostics
- Generate Fix
- Apply Fix
- Fix All preview workflow
- selected-code quick actions
- response modes
- loading states
Building CodePilot required combining several systems that normally exist independently:
- VS Code extension development
- local LLM inference
- embedding generation
- semantic retrieval
- project indexing
- RAG pipelines
- editor diagnostics
- conversation context
- guarded code modification
- extension packaging
The core engineering problem was not simply connecting an LLM to VS Code.
It was:
How much project context should the model receive, and which context is actually relevant?
That became the main design problem behind CodePilot.
🌐 https://codepilot-website.vercel.app
Rakshith
B.Tech — Artificial Intelligence & Machine Learning
Building AI systems, developer tools, and full-stack applications.