Intelligent Document Processing with CrewAI

This application provides an intelligent document processing system that leverages CrewAI to orchestrate OCR and AI services for analyzing uploaded documents, automatically classifying them, and extracting relevant information.

Features

• Document upload with drag & drop interface
• Real-time processing pipeline visualization
• CrewAI agent-based orchestration
• Intelligent service selection based on document type
• Detailed results display with confidence scores
• Status indicators showing processing progress

Project Structure

This project consists of two main parts:

1. Frontend: A React application with a user-friendly interface for document upload and visualization
2. Backend: A Python FastAPI service with CrewAI that handles document processing using various OCR and AI services

CrewAI Architecture

The system uses CrewAI to create a team of specialized AI agents that work together to process documents:

1. Document Classifier Agent - Determines document type and structure
2. OCR Specialist Agent - Extracts text using the appropriate OCR service
3. Document Analyzer Agent - Analyzes content and extracts structured information

These agents work together in a sequential process to handle document processing tasks, providing a clear separation of concerns and specialized expertise for each task.

Frontend Setup

The frontend is a React application with Vite and Tailwind CSS.

Environment Variables

Create a .env file in the root directory with the following variables:

VITE_OCR_API_ENDPOINT=http://localhost:8000/api/document-upload

Development

# Install dependencies
npm install

# Run development server
npm run dev

# Build for production
npm run build

Backend Setup

The backend is located in the backend/ directory and is built with Python, FastAPI, and CrewAI.

Prerequisites

• Python 3.8+
• Google Cloud account with Vision API enabled
• Azure account with Computer Vision and OpenAI services

Installation

1. Navigate to the backend directory:

   cd backend

2. Create a virtual environment:

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

   pip install -r requirements.txt

4. Copy .env.example to .env and fill in your API credentials

Running the Backend Server

python app.py

Or with uvicorn directly:

uvicorn app:app --reload --host 0.0.0.0 --port 8000

Connecting OCR Services

Google Cloud Vision OCR

1. Create a Google Cloud account and project at https://cloud.google.com/
2. Enable the Vision API for your project
3. Create an API key with appropriate permissions
4. Add the API key to your backend .env file as GOOGLE_CLOUD_API_KEY

Alternatively, you can use a service account:

1. Create a service account in the Google Cloud Console
2. Download the JSON credentials file
3. Either set the path to this file in your .env as GOOGLE_APPLICATION_CREDENTIALS or include the JSON content directly as GOOGLE_CLOUD_CREDENTIALS_JSON

Azure OCR

1. Create an Azure account at https://azure.microsoft.com/
2. Create a Computer Vision resource in the Azure portal
3. Get your API key and endpoint URL from the resource overview
4. Add these credentials to your backend .env file as AZURE_OCR_ENDPOINT and AZURE_OCR_KEY

Azure OpenAI

1. Apply for access to Azure OpenAI Service
2. Create an Azure OpenAI resource in the Azure portal
3. Deploy a model in the Azure OpenAI Studio
4. Get your API key, endpoint URL, and deployment name
5. Add these credentials to your backend .env file as:
   • AZURE_OPENAI_ENDPOINT
   • AZURE_OPENAI_KEY
   • AZURE_OPENAI_DEPLOYMENT_NAME
   • AZURE_OPENAI_API_VERSION (default is 2023-05-15)

API Endpoints

POST /api/document-upload

Uploads and processes documents using the CrewAI orchestrator.

Request:

• Form data with files under the key "files"

Response:

{
  "success": true,
  "results": [
    {
      "fileName": "invoice.pdf",
      "fileType": "application/pdf",
      "classification": {
        "documentType": "Invoice",
        "confidence": 95.5,
        "service": "CrewAI + Azure OpenAI"
      },
      "extractedText": "...",
      "analysis": {
        "extractedData": {
          "invoiceNumber": {
            "value": "INV-2023-0042",
            "confidence": 98
          },
          "date": {
            "value": "2023-05-15",
            "confidence": 95
          },
          "totalAmount": {
            "value": "$1,245.00",
            "confidence": 92
          }
        },
        "confidence": 95,
        "documentType": "Invoice"
      },
      "confidence": 95
    }
  ]
}

GET /api/health

Health check endpoint.

Response:

{
  "status": "healthy"
}

GET /api/services/status

Check the status of all integrated services.

Response:

{
  "google_cloud_vision": "available",
  "azure_ocr": "available",
  "azure_openai": "available",
  "crewai": "operational"
}

Docker Deployment

You can deploy the backend using Docker:

cd backend
docker build -t document-processing-api .
docker run -p 8000:8000 --env-file .env document-processing-api

Customizing the CrewAI Workflow

To customize how the application processes documents:

1. Edit the backend/services/crewai_orchestrator.py file to modify the agent definitions and tasks
2. Update the document classification logic in backend/utils/document_classifier.py
3. Adjust the service selection logic in the CrewAI orchestrator

License

MIT