agentworkshop

An advanced AI agent system that combines language models with tool execution capabilities, featuring a conversational CLI interface and web API.

About

agentworkshop is a comprehensive AI agent framework designed to demonstrate the integration of large language models with practical tool execution capabilities. The project supports multiple LLM providers with automatic fallback, serving as both a functional AI assistant and an educational example of how to build sophisticated agent systems.

Key Differentiators

Unified Interface: Both CLI and Web API provide seamless access to the same agent capabilities
Extensible Tool System: Easy to add new tools for various domains and use cases
Persistent Memory: Conversations are saved and can be resumed with context
Production-Ready: Built with FastAPI, rich logging, and comprehensive error handling
Multi-Provider LLM: Support for OpenRouter, Groq, Gemini, Grok, and Ollama with automatic fallback
Reflection Layer: Built-in step review with automatic retry on failure

Use Cases

Development: Test and prototype AI agent workflows
Research: Experiment with different prompting strategies and tool combinations
Education: Learn about agent architecture and LLM integration patterns
Automation: Build automated assistants for specific tasks
Integration: Embed AI capabilities into existing applications

Features

AI Assistant CLI: Interactive command-line interface for AI conversations
Web API: RESTful API using FastAPI for programmatic access
Tool System: Extensible tool framework including web search and Python execution
Memory Management: Persistent conversation memory with context tracking
Multi-Provider LLM: Support for OpenRouter, Groq, Gemini, Grok, and Ollama with automatic fallback
Stream Processing: Real-time streaming responses for better UX
Structured Logging: Rich logging with colored output and structured logs
Reflection System: Automatic review and retry of failed plan steps

Architecture

graph TD
    A[CLI Interface] --> B[Agent Core]
    C[Web API] --> B
    B --> D[Planner]
    D --> E[StepExecutor]
    E --> F[Reflection]
    F --> G[ActionExecutor]
    E --> H[LLM Runner]
    H --> I[LLM Provider System]
    I --> J[OpenRouterAPI]
    I --> K[GroqAPI]
    I --> L[GeminiAPI]
    I --> M[GrokAPI]
    I --> N[OllamaAPI]
    I --> O[LLMFallback]
    G --> P[Tool Registry]
    P --> Q[Web Search Tool]
    B --> R[Memory Manager]
    R --> S[Conversation Storage]
    D --> T[ExecutionPlan]
    E --> U[Executor Prompt]

The system follows a modular architecture with clear separation of concerns:

CLI Layer: User interaction through command-line interface
Agent Layer: Core agent logic, planning, reflection, and tool orchestration
Planner Layer: Generates ExecutionPlan with high-level steps
Executor Layer: Step execution with LLM calls and tool invocation
Reflection Layer: Reviews each step result and triggers retries if needed
LLM Layer: Language model integration with multi-provider support and fallback
Provider Layer: Pluggable LLM providers (OpenRouter, Groq, Gemini, Grok, Ollama)
Parser Layer: Structured JSON parsing for planner, executor, and reflector outputs
Tool Layer: Extensible tool framework for external capabilities
Memory Layer: Conversation persistence and context management
API Layer: Web service endpoints for programmatic access

Installation

Prerequisites

Python 3.14 or higher
pip or uv package manager

Installation Steps

# Clone the repository
git clone https://github.com/AmirHoseein99/agentworkshop.git

# Navigate to project directory
cd agentworkshop

# Install dependencies using uv (recommended)
uv sync

# Install the package in development mode
uv pip install -e .

Quick Start

# Start the CLI assistant
agentworkshop

# Start the web API
uvicorn src.server:app --reload

Quick Start

Using CLI

# Launch the AI assistant
agentworkshop

# Interactive session features:
/exit - Exit the assistant
/clear - Clear conversation history

Using Web API

# Start the FastAPI server
uvicorn src.server:app --reload

# Test the API endpoints
curl "http://localhost:8000/api/llm/ask?user_input=Hello"
curl "http://localhost:8000/api/llm/stream_llm?user_input=Hello"
curl -X POST "http://localhost:8000/api/agent/call_agent?user_input=Hello"

Example CLI Session

[cyan]AI Assistant ready — /exit to quit, /clear to reset[/cyan]

[bold red]You:[/bold red]
      What is the capital of France?

[blue]⠋ Thinking...[/blue]

[bold green]Assistant:[/bold green]
The capital of France is Paris. It's known for its art, fashion, and culture.

Project Structure

agentworkshop/
├── src/
│   ├── cli.py                   # Command-line interface
│   ├── server.py                # FastAPI web server
│   ├── agent/                   # Agent core system
│   │   ├── agent.py             # Main Agent class
│   │   ├── api.py               # Agent API endpoints
│   │   ├── state.py             # AgentState dataclass
│   │   ├── llm_runner.py        # LLM execution wrapper
│   │   ├── response_handler.py  # Response parsing and state updates
│   │   ├── tool_executer.py     # Tool validation and execution
│   │   ├── action_executer.py   # Action orchestration for tool calls and final responses
│   │   ├── executor/            # Step execution subsystem
│   │   │   ├── step_executor.py # StepExecutor class
│   │   │   └── executor_prompt.py # Executor prompt builder
│   │   ├── parser/              # LLM response parsing
│   │   │   ├── base.py          # Base parsing utilities
│   │   │   ├── executor_parser.py # Executor response parser
│   │   │   ├── planner_parser.py  # Planner response parser
│   │   │   └── reflector_parser.py # Reflection response parser
│   │   ├── planner/             # Planning subsystem
│   │   │   ├── planner.py       # Planner class
│   │   │   ├── planner_prompt.py # Planner prompt builder
│   │   │   └── models.py        # ExecutionPlan, PlanStep, StepStatus models
│   │   ├── reflection/          # Reflection subsystem
│   │   │   ├── reflection.py    # Reflection class
│   │   │   ├── models.py        # StepReflectionResult, ReflectionStatus, IssueType, Severity, CorrectionAction
│   │   │   └── prompt.py        # Reflection prompt builder
│   │   └── tools/               # Tool implementations
│   │       ├── base.py          # Base tool class
│   │       ├── python_executor.py  # Python execution tool
│   │       └── web_search.py      # Web search tool
│   ├── core/                    # Core utilities
│   │   └── config.py            # Configuration management
│   ├── llm/                     # LLM integration
│   │   ├── api.py               # LLM API endpoints
│   │   ├── chat_engine.py       # Chat engine
│   │   ├── factory.py           # LLM provider factory (default_llm)
│   │   ├── openrouter.py        # OpenRouter integration
│   │   ├── parser.py            # OpenRouter stream parsing
│   │   ├── structure.py         # Output schemas for planner, agent, executor, memory, and reflection
│   │   ├── utils.py             # Streaming utilities
│   │   ├── providers/           # LLM provider implementations
│   │   │   ├── base.py          # BaseLLM abstract class
│   │   │   ├── fallback.py      # LLMFallback for provider failover
│   │   │   ├── openai_compatible.py  # OpenAI-compatible API base
│   │   │   ├── openrouter.py    # OpenRouter provider
│   │   │   ├── groq.py          # Groq provider
│   │   │   ├── gemini.py        # Gemini provider
│   │   │   ├── grok.py          # Grok provider
│   │   │   └── ollama.py        # Ollama provider
│   │   └── prompts/             # Prompt templates
│   │       ├── agent_system_prompt.py  # Agent system prompt builder
│   │       ├── memory_prompt.py        # Memory summarizer prompt
│   │       ├── planner.txt             # Planner prompt template
│   │       ├── executor.txt            # Executor prompt template
│   │       └── reflection.txt          # Reflection prompt template
│   ├── memory/                  # Memory system
│   │   ├── json_memory.py       # JSON-based memory storage
│   │   └── memory_manager.py    # Memory management
│   ├── exceptions.py            # Custom exception classes
│   ├── logger.py                # Structured file logging setup
│   └── tests/                   # Test suite
│       ├── test_parser.py       # Parser tests
│       └── test_tools.py        # Tool tests
├── .github/workflows/           # CI workflows
│   └── tests.yml                # Tests, ruff check, and formatting
├── pyproject.toml               # Project configuration
├── README.md                    # This file
└── uv.lock                      # Dependency lock file

Agent System

Core Components

Agent: Main orchestrator that coordinates planning, step execution, reflection, and tool execution
Planner: Generates an ExecutionPlan with high-level steps before running the agent loop
StepExecutor: Executes individual plan steps by calling the LLM with the executor prompt and previous step results
Reflection: Reviews each step result and triggers retries if the output is incomplete or incorrect
ActionExecutor: Handles tool call execution and final response generation
AgentState: Tracks conversation state, steps, tool results, and plan progress
LLMRunner: Wraps LLM API calls for the agent
ResponseHandler: Parses LLM responses and updates agent state
ToolExecutor: Validates arguments and executes registered tools
LLMProviderSystem: Pluggable LLM providers with fallback support

Executor System

The executor system handles the execution of individual plan steps:

StepExecutor: Orchestrates the execution of a single plan step by calling the LLM with the executor prompt and previous step results
Executor Prompt: A system prompt (src/llm/prompts/executor.txt) that guides the LLM to execute exactly one step, use tools when necessary, and produce structured JSON output
Execution Flow: The StepExecutor sends the current step definition and previous results to the LLM, parses the response, and either executes a tool call or returns a final result

Reflection System

The reflection system reviews each executed plan step to ensure quality:

Reflection: Reviews step results using a dedicated LLM call with the reflection prompt
Reflection Prompt: A system prompt (src/llm/prompts/reflection.txt) that evaluates whether the step achieved its expected output
Retry Logic: If reflection status is revise or failed, the step is retried up to AGENT_MAX_RETRY times
Structured Output: Reflection returns StepReflectionResult with status, score, issues, and corrections

Agent Workflow

Plan Generation: The Planner produces an ExecutionPlan from the user's request
Step Execution: The StepExecutor iterates through plan steps, calling the LLM with the executor prompt and previous step results
Reflection: Each step result is reviewed by Reflection.review() and retried up to AGENT_MAX_RETRY times if needed
Response Parsing: ResponseHandler parses the structured LLM response from the executor
Action Execution: ActionExecutor either returns a final answer or executes a tool via ToolExecutor
State Update: Conversation history and tool results are stored in AgentState
Memory Update: Responses are persisted for future context

LLM Provider System

The LLM provider system supports multiple language model providers with automatic fallback:

Architecture

BaseLLM (abstract)
  └── OpenAICompatibleAPI (HTTP client with retry, rate-limit handling)
        ├── OpenRouterAPI
        ├── GroqAPI
        ├── GeminiAPI
        ├── GrokAPI
        └── OllamaAPI

LLMFallback (provider failover)
  └── Tries providers in order until one succeeds

Provider Details

Provider	Class	Base URL	Environment Variables
OpenRouter	`OpenRouterAPI`	Configurable via `OPENROUTER_API_BASE_URL`	`OPENROUTER_API_KEY`, `OPENROUTER_MODEL`
Groq	`GroqAPI`	Configurable via `GROQ_API_BASE_URL`	`GROQ_API_KEY`, `GROQ_MODEL`
Gemini	`GeminiAPI`	`https://generativelanguage.googleapis.com/v1beta/openai`	`GEMINI_API_KEY`
Grok	`GrokAPI`	`https://api.x.ai/v1`	`GROK_API_KEY`
Ollama	`OllamaAPI`	Configurable via `OLLAMA_API_BASE_URL`	`OLLAMA_API_BASE_URL`, `OLLAMA_MODEL`

Fallback Mechanism

The LLMFallback class wraps multiple providers and attempts them in sequence. If one provider fails, it automatically tries the next. This ensures resilience against provider outages or rate limits.

Provider Factory

The default_llm() factory in src/llm/factory.py creates a configured LLMFallback instance with OpenRouter and Groq as providers:

from src.llm.factory import default_llm

llm = default_llm()
response = llm.call(messages, caller="agent")

Tool System

The tool system provides extensible capabilities for the agent to interact with external services. Tools are executed through the ActionExecutor, which validates, executes, and records tool results in the conversation history.

Available Tools

Web Search Tool
- Searches the web for information
- Returns structured results with titles, snippets, and URLs
- Used for research and fact-checking
Python Executor Tool
- Executes Python code safely
- Captures output and errors
- Used for computational tasks and data analysis

Tool Architecture

class BaseTool:
    def __init__(self, name: str, description: str, schema: dict):
        self.name = name
        self.description = description
        self.schema = schema

    def validate(self, args: dict):
        raise NotImplementedError

    async def execute(self, **kwargs) -> str:
        raise NotImplementedError

Tool Registration

Tools are registered with the agent:

from src.agent.agent import agent
from src.agent.tools.web_search import WebSearchTool
from src.agent.tools.python_executor import PythonExecutorTool

agent.register_tool(WebSearchTool())
agent.register_tool(PythonExecutorTool())

The ActionExecutor handles tool execution, error handling, and conversation history updates automatically when the agent processes tool calls.

Planner

The planner generates a structured execution plan before the agent begins working:

from src.agent.planner.planner import Planner
from src.agent.planner.models import ExecutionPlan

planner = Planner()
plan: ExecutionPlan = planner.produce_plan(user_input="Build a todo app")

Execution Plan Structure

goal: Overall objective of the user's request
summary: Brief summary of the planning process
steps: Ordered high-level tasks with expected outputs and dependencies

Reflection

The reflection subsystem reviews each executed plan step to ensure quality and correctness:

from src.agent.reflection.reflection import Reflection
from src.agent.reflection.models import StepReflectionResult, ReflectionStatus

reflection = Reflection()
result: StepReflectionResult = reflection.review(
    user_request=user_input,
    step=step,
    result=step_result,
    previous_results=previous_results,
)

Reflection Status

pass: The step achieved its expected output successfully
revise: The step needs minor corrections before acceptance
failed: The step did not achieve its expected output and should be retried

Reflection Output Structure

{
    "status": "pass | revise | failed",
    "score": 0.0,
    "summary": "Brief explanation of the evaluation",
    "issues": [
        {
            "type": "missing_output | incomplete_output | incorrect_information | unsupported_claim | irrelevant_output | execution_failure",
            "description": "Specific explanation of the problem",
            "severity": "low | medium | high"
        }
    ],
    "corrections": [
        {
            "action": "retry | reexecute | add | modify",
            "description": "Specific description of what should be corrected"
        }
    ]
}

Memory System

The memory system provides persistent storage for conversations and context:

Key Features

Conversation Storage: Persistent conversation history
Context Management: Maintains conversation context across interactions
Memory Manager: Centralized memory operations with automatic summarization
JSON Backend: Simple and portable storage format

Memory Operations

from src.memory.memory_manager import (
    initialize_conversation,
    get_context,
    append_to_conversation,
)

# Initialize a new conversation
initialize_conversation(conversation_id="conv_123")

# Add user message
append_to_conversation(
    role="user",
    content="Hello, how are you?",
    conversation_id="conv_123"
)

# Get conversation context
context = get_context(conversation_id="conv_123")

Memory Files

Memory is stored in the data/ directory:

/data/
└── conversations/
    ├── abc/
    │   ├── conversation.json  # Conversation history
    │   └── memory.json        # Memory context
    ├── conv1/
    │   ├── conversation.json
    │   └── memory.json
    └── ...

Memory Summarization

When the conversation exceeds the configured threshold, the memory manager automatically summarizes older messages into compact facts, tasks, and a high-level summary, allowing long-running sessions without losing context.

Structured Outputs

The project uses strict JSON schemas for LLM responses:

Agent Output: final or tool_call with tool name and arguments
Planner Output: Goal, summary, and ordered steps with dependencies
Executor Output: tool_call, step_result, or final with status, artifacts, and summary
Reflection Output: pass, revise, or failed with score, summary, issues, and corrections
Memory Summarizer Output: Summary, facts, open tasks, and last summarized index

Exception Handling

Custom exceptions provide clear error handling across the system:

ToolNotFoundError - Raised when a requested tool is not registered
ToolValidationError - Raised when tool arguments fail schema validation
ToolExecutionError - Raised when a tool fails during execution
ParserError - Raised when the LLM response cannot be parsed
LLMError - Raised for general LLM API failures
PlanExecutionError - Raised when a plan step fails during execution

Configuration

Configuration is managed through environment variables:

Variable	Description
`OPENROUTER_API_KEY`	API key for OpenRouter
`OPENROUTER_API_BASE_URL`	Base URL for OpenRouter API
`OPENROUTER_MODEL`	Model identifier to use
`GEMINI_API_KEY`	API key for Gemini
`GEMINI_MODEL`	Model identifier for Gemini (default: `gemini-2.5-flash`)
`GROK_API_KEY`	API key for Grok
`GROK_MODEL`	Model identifier for Grok (default: `grok-3-mini`)
`OLLAMA_API_BASE_URL`	Base URL for Ollama API
`OLLAMA_MODEL`	Model identifier for Ollama
`GROQ_API_KEY`	API key for Groq
`GROQ_MODEL`	Model identifier for Groq
`GROQ_API_BASE_URL`	Base URL for Groq API
`TAVILY_API_KEY`	API key for Tavily web search
`PROXY`	Optional HTTP/HTTPS proxy URL

Default agent settings are defined in src/core/config.py:

Setting	Default
`AGENT_MAX_STEP`	5
`AGENT_MAX_RETRY`	3
`CONTEXT_WINDOW_SIZE`	10
`CONVERSATION_MEMORY_THRESHOLD`	20

Default LLM provider settings:

Setting	Default
`GEMINI_MODEL`	`gemini-2.5-flash`
`GROK_MODEL`	`grok-3-mini`
`GROQ_MODEL`	From `GROQ_MODEL` env var
`OLLAMA_MODEL`	From `OLLAMA_MODEL` env var

Running Tests

The project includes a comprehensive test suite:

Test Commands

# Run all tests
uv run pytest

# Run specific test module
uv run pytest src/tests/test_parser.py

# Run tests with verbose output
uv run pytest -v

# Run tests with coverage
uv run pytest --cov=src

Test Categories

Parser Tests: LLM response parsing and validation
Tool Tests: Individual tool functionality

CI Checks

The GitHub Actions workflow runs on every push and pull request:

uv run pytest for tests
uv run ruff check . for linting
uv run ruff format --check . for formatting

License

This project is licensed under the MIT License. See the LICENSE file for details.

Made with ❤️ by the AmirHossein Imani

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
src		src
.gitignore		.gitignore
.python-version		.python-version
README.md		README.md
pyproject.toml		pyproject.toml

Latest commit

History

Folders and files

Repository files navigation

agentworkshop

About

Key Differentiators

Use Cases

Features

Architecture

Installation

Prerequisites

Installation Steps

Quick Start

Quick Start

Using CLI

Using Web API

Example CLI Session

Project Structure

Agent System

Core Components

Executor System

Reflection System

Agent Workflow

LLM Provider System

Architecture

Provider Details

Fallback Mechanism

Provider Factory

Tool System

Available Tools

Tool Architecture

Tool Registration

Planner

Execution Plan Structure

Reflection

Reflection Status

Reflection Output Structure

Memory System

Key Features

Memory Operations

Memory Files

Memory Summarization

Structured Outputs

Exception Handling

Configuration

Running Tests

Test Commands

Test Categories

CI Checks

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages