🛡️ Sentinel — Intelligent Content Moderation Pipeline

Sentinel is an asynchronous, event-driven Trust & Safety content moderation pipeline combining Computer Vision (CV), LLM-based context reasoning, and a configurable Decision Engine with a real-time Human Moderator Review Dashboard.

Inspired by production Trust & Safety infrastructure at scale (Reddit, Discord, marketplaces), Sentinel automatically classifies user-generated text and images, resolves ambiguous cases with model-generated reasoning, and escalates borderline items to human moderators with WebSocket live updates and an immutable PostgreSQL audit trail.

---

🏗️ Architecture Overview

                      ┌──────────────────────┐
   User Ingestion →   │  FastAPI API Gateway │  (REST submission, auth, file validation)
                      └──────────┬───────────┘
                                 │
                 Writes raw file │ Publishes content.submitted
                                 ▼
             ┌───────────────────────┬────────────────────────┐
             │ Object Storage (MinIO)│ Redis Streams (Broker) │
             └───────────────────────┴───────────┬────────────┘
                                                 │
                        ┌────────────────────────┴────────────────────────┐
                        │ Consumer Groups                                 │
                        ▼                                                 ▼
             ┌─────────────────────┐                           ┌─────────────────────┐
             │      CV Worker      │                           │     LLM Worker      │
             │ (HuggingFace / PyTorch)                          │ (NVIDIA Nemotron LLM) │
             └──────────┬──────────┘                           └──────────┬──────────┘
                        │ Publishes cv.completed                          │ Publishes llm.completed
                        └────────────────────────┬────────────────────────┘
                                                 │
                                                 ▼
                                     ┌───────────────────────┐
                                     │    Decision Engine    │ (Signal buffering, joins CV+LLM,
                                     └───────────┬───────────┘  evaluates category policy thresholds)
                                                 │
                       ┌─────────────────────────┼─────────────────────────┐
                       ▼                         ▼                         ▼
                 [Auto-Approve]            [Auto-Reject]            [Needs Review]
                       │                         │                         │
                       └─────────────────────────┼─────────────────────────┘
                                                 │
                                                 ▼
                                     ┌───────────────────────┐
                                     │  Postgres Audit Log   │ (content, decisions, moderator_actions)
                                     └───────────┬───────────┘
                                                 │ Publishes decision.made
                                                 ▼
                                     ┌───────────────────────┐
                                     │ WebSocket Live Bridge │
                                     └───────────┬───────────┘
                                                 │ WS /v1/ws/review
                                                 ▼
                                     ┌───────────────────────┐
                                     │ Moderator Dashboard   │ (React + Vite + TypeScript)
                                     └───────────────────────┘

---

✨ Key Features

• Asynchronous Ingestion: Ingestion API returns 202 Accepted with a content_id immediately; model execution never blocks client uploads.
• Dual-Modality Classification:
  • Computer Vision: HuggingFace NSFW/nudity image classification (Falconsai/nsfw_image_detection) with GPU acceleration and weighted score normalization.
  • LLM Context Reasoner: Contextual reasoning using NVIDIA NIM / Llama-3.3 Nemotron model for nuanced detection (hate speech, harassment, spam, scams, reclaiming language).
• Configurable Decision Engine: Evaluates category-specific auto_approve and auto_reject thresholds without redeploying models or restarting services.
• Human-in-the-Loop Escalation: Borderline items are routed to a human review queue with complete model scores and reasoning.
• Real-Time Moderator Dashboard: Live queue powered by WebSockets (/v1/ws/review), single-click approval/rejection, and atomic race-condition safeguards.
• Full Auditability: Every automated decision and moderator action is permanently logged to PostgreSQL.

---

📁 Repository Structure

.
├── PRD.md                       # Product Requirements Document
├── ARCHITECTURE.md              # System Architecture & Design
├── AGENTS.md                    # Agent workflows & conventions
├── docker-compose.yml           # Local infrastructure (Postgres, Redis, MinIO)
├── pyproject.toml               # Python package configuration & dependencies
├── migrations/                  # Alembic schema migrations
├── shared/                      # Shared libraries across services
│   ├── config/                  # Pydantic BaseSettings & env configuration
│   ├── models.py                # Declarative SQLAlchemy ORM models
│   ├── schemas/                 # Pydantic schemas (events, verdicts, policy)
│   └── streams.py               # Redis Streams pub/sub & consumer group abstraction
├── services/
│   ├── api-gateway/             # FastAPI REST ingestion, moderator APIs & WS bridge
│   ├── cv-worker/               # Computer Vision classification consumer & adapter
│   ├── llm-worker/              # LLM reasoning consumer & NVIDIA AI adapter
│   ├── decision-engine/         # Signal joining buffer & policy decision consumer
│   └── dashboard/               # React + TypeScript + Vite moderator review UI
└── tests/
    ├── unit/                    # Unit tests for all services, engines, and workers
    └── integration/             # End-to-end integration tests

---

🚀 Getting Started

1. Prerequisites

• Python: 3.11 or higher
• Node.js: 18+ (for Dashboard UI)
• Docker & Docker Compose: for PostgreSQL, Redis, and MinIO

2. Start Infrastructure

Launch the background data stores using Docker Compose:

docker-compose up -d

This starts:

• PostgreSQL: localhost:5432 (sentinel / sentinel)
• Redis: localhost:6379
• MinIO: S3-compatible object storage at localhost:9000 (Console: localhost:9001)

3. Setup Python Virtual Environment

python3 -m venv .venv
source .venv/bin/activate

# Install base dependencies and development tools
pip install -e .

# (Optional) Install Computer Vision extras for local PyTorch/Transformers model inference
pip install -e ".[cv]"

4. Configure Environment Variables

Create a .env file from the example:

cp .env.example .env

Ensure your .env contains your settings and API keys:

DATABASE_URL=postgresql+asyncpg://sentinel:sentinel@127.0.0.1:5432/sentinel
REDIS_URL=redis://127.0.0.1:6379

OBJECT_STORAGE_ENDPOINT=http://localhost:9000
OBJECT_STORAGE_ACCESS_KEY=minioadmin
OBJECT_STORAGE_SECRET_KEY=minioadmin
OBJECT_STORAGE_BUCKET_NAME=sentinel

LLM_PROVIDER=nvidia
LLM_BASE_URL=https://integrate.api.nvidia.com/v1
LLM_API_KEY=your_nvidia_api_key_here
LLM_MODEL=nvidia/llama-3.3-nemotron-super-49b-v1.5

CV_MODEL=dummy  # Use 'dummy' for lightweight testing or 'nsfw' for full HuggingFace model

5. Run Database Migrations

Apply Alembic migrations to initialize tables in PostgreSQL:

alembic upgrade head

---

💻 Running the Services

You can run the microservices in separate terminal windows:

1. API Gateway

PYTHONPATH=services/api-gateway:. uvicorn app.main:create_app --factory --host 0.0.0.0 --port 8000 --reload

2. Computer Vision Worker

PYTHONPATH=services/cv-worker:. python -m cv_worker.consumer

3. LLM Worker

PYTHONPATH=services/llm-worker:. python -m llm_worker.consumer

4. Decision Engine

PYTHONPATH=services/decision-engine:. python -m decision_engine.consumer

5. Moderator Dashboard (React UI)

cd services/dashboard
npm install
npm run dev

---

📡 API Reference

Ingestion

POST /v1/content

Submit text, an image, or mixed content for moderation.

• Request (Multipart Form Data):
  • text (optional string): Post text.
  • image (optional file): JPEG, PNG, or WebP image.
• Response (202 Accepted):

  {
    "content_id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d",
    "status": "pending",
    "type": "mixed",
    "storage_path": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d/original"
  }

---

Moderator Review & Audit

GET /v1/review/queue

Retrieve pending content escalated for human review (FIFO order).

• Query Params: limit=20
• Response (200 OK):

  [
    {
      "content_id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d",
      "type": "mixed",
      "storage_path": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d/original",
      "created_at": "2026-08-23T14:38:44.139291Z",
      "cv_scores": {
        "scores": [{"category": "nudity", "confidence": 0.65}]
      },
      "llm_verdict": {
        "category": "hate_speech",
        "confidence": 0.45,
        "reasoning": "Ambiguous usage of colloquial slang; context unclear."
      }
    }
  ]

POST /v1/review/{content_id}/decide

Record a moderator's decision on escalated content with race condition protection.

• Headers: X-Moderator-Id: <uuid>
• Request Body:

  {
    "action": "approve",
    "notes": "Reviewed in context — false positive on slang."
  }

• Response: 200 OK {"status": "approved"} or 409 Conflict if already decided.

GET /v1/content/{content_id}/file

Securely stream stored media assets to the moderator review dashboard.

WS /v1/ws/review

WebSocket endpoint streaming decision.made events to connected moderators in real-time.

---

🧪 Testing & Quality Assurance

All unit tests, integration tests, and linters run locally via pytest and ruff:

# Run all unit and integration tests
pytest

# Run linter checks
ruff check .

# Check formatting
ruff format --check .

---

⚖️ License

Distributed under the MIT License.