Skip to content
BranchesTags

Repository files navigation

🛡️ ZeroFake

Python Version FastAPI React Docker Accuracy License

Autonomous Multi-Agent Fact-Checking Engine with Dual-Flow Adversarial Verification & Bayesian Evidence Synthesis

Key Features • Architecture • Benchmarks • Quickstart • API Reference • Deployment

🌟 Executive Summary

ZeroFake is an enterprise-grade, real-time automated fact-checking and fake news detection platform. Unlike conventional single-pass LLM prompts or naive RAG pipelines that struggle with hallucination and nuanced misinformation, ZeroFake employs a Cognitive Multi-Agent Hierarchy combined with an Adversarial Debate Protocol and Dual-Flow Adaptive Routing.

🚀 Key Performance Indicators

Metric Traditional Pipeline ZeroFake Multi-Agent Engine Improvement
Accuracy 65.03% 94.91% +29.88% (6x fewer errors)
False Negative Rate 30.94% 2.99% 10x better fake news capture
False Positive Rate 39.00% 7.20% 5.4x reduction in false alarms
Zombie News Detection 35.00% 90.00% +55.00% temporal reasoning

⚡ Key Features

  • 🧠 Multi-Agent Cognitive Framework:
    • PLANNER Agent: Dissects ambiguous claims, infers temporal contexts, identifies entities, and generates 5+ targeted multi-lingual queries.
    • FILTER Agent: Semantic deduplication, removes clickbait, tabloid noise, and social media hallucinations.
    • CRITIC Agent: Adversarial counter-evidence investigator designed to actively challenge assumptions and identify edge-case nuances.
    • JUDGE Agent: Final Bayesian arbitrator synthesizing multi-source signals and chain-of-thought rationale into an explainable verdict.
  • 🔀 Dual-Flow Dynamic Routing:
    • Recent News Flow ($\le$ 3 days): Real-time multi-engine search, live news aggregation, and adversarial validation.
    • Historical Knowledge Flow (> 3 days): Instant Google Fact Check API validation with high-confidence fast-path routing.
  • 🌐 Hybrid Multi-Source Search:
    • Parallel queries across Google News, Wikipedia API, Google Web Search + Trafilatura scraping, and DuckDuckGo failover.
    • Built-in anti-blocking resilience, rotating user-agents, and smart rate-limiting.
  • 🛡️ Source Credibility & Domain Whitelisting:
    • Hierarchical trust scoring using 380+ pre-calibrated trusted domains (Government .gov, Tier-1 wire services, national press).
  • 🖼️ Media Authenticity & Provenance Ready:
    • Extensible hooks for C2PA provenance extraction, image forensic signals, and multimodal verification.
  • 🖥️ Full Stack Experience:
    • High-performance FastAPI backend with asynchronous streaming endpoints.
    • Modern, responsive React + Vite dashboard with real-time trace inspection.
    • Standalone PyQt6 Desktop GUI with dark-mode visualization.

🏗️ System Architecture

flowchart TD
    classDef input fill:#1e293b,stroke:#3b82f6,stroke-width:2px,color:#fff;
    classDef agent fill:#0f172a,stroke:#8b5cf6,stroke-width:2px,color:#fff;
    classDef decision fill:#1e293b,stroke:#f59e0b,stroke-width:2px,color:#fff;
    classDef search fill:#064e3b,stroke:#10b981,stroke-width:2px,color:#fff;
    classDef verdict fill:#312e81,stroke:#6366f1,stroke-width:2px,color:#fff;

    CLAIM["📥 Input Claim / Article"]:::input --> PLANNER["🧠 PLANNER AGENT\n• Query Expansion (5+ queries)\n• Temporal & Entity Extraction"]:::agent
    
    PLANNER --> ROUTER{"🔀 Info Age Route?"}:::decision

    %% Fast Path
    ROUTER -->|"Old Knowledge (> 3 days)"| GFC["🔍 Google Fact Check API"]:::search
    GFC --> GFC_CHECK{"Verdict ≥ 70%?"}:::decision
    GFC_CHECK -->|"Yes (Fast-Path)"| JUDGE["⚖️ JUDGE AGENT\nBayesian Evidence Synthesis"]:::verdict
    GFC_CHECK -->|"No / Miss"| SEARCH

    %% Live Path
    ROUTER -->|"Recent (≤ 3 days)"| SEARCH["🌐 Unified Search Retrieval\n• Google News (VN/EN)\n• Wikipedia API\n• Google Web + Trafilatura\n• DuckDuckGo Fallback"]:::search

    SEARCH --> FILTER["🧹 LLM EVIDENCE FILTER\n• Strip Social Spam & Tabloids\n• Semantic Deduplication"]:::agent
    FILTER --> CRITIC["⚔️ ADVERSARIAL CRITIC\n• Challenge Hypothesis\n• Hunt Counter-Evidence"]:::agent
    CRITIC --> JUDGE

    JUDGE --> OUTPUT["📊 Final Verdict: TIN THAT / TIN GIA\n• Confidence Score (0-100%)\n• Chain-of-Thought Rationale\n• Verifiable Citations"]:::verdict
Loading

📊 Benchmark Performance

Benchmarked over 1,001 diverse Vietnamese & Global test claims (500 Verified True, 501 Fabricated/Zombie News):

========================= BENCHMARK SUMMARY =========================
Total Claims Evaluated : 1,001
Overall Accuracy       : 94.91%
Precision (Fake News)  : 93.20%
Recall (Fake News)     : 97.01%
F1-Score               : 95.07%
False Negative Rate    : 2.99%  (Crucial: Minimizes missed fake news)
=====================================================================

Multi-Agent Model Allocation Matrix

Role Primary Engine Fallback Engine Focus Area
PLANNER Qwen 3 32B / Gemini 2.0 Llama 3.1 8B Multi-lingual query formulation & context scoping
FILTER Llama 3.1 8B (Groq) Gemma 2 9B High-throughput noise reduction & duplicate pruning
CRITIC Qwen 3 32B / Gemini 2.0 Llama 3.3 70B Adversarial thinking & falsification discovery
JUDGE Llama 3.3 70B (Cerebras) Llama 3.3 70B (Groq) / Gemini Final Bayesian synthesis & explainable decision

🚀 Quickstart & Installation

Prerequisites

  • Python 3.10+ (Python 3.11 or 3.12 recommended)
  • Node.js 18+ (Optional: for React frontend)
  • Git

1. Clone & Environment Setup

# Clone the repository
git clone https://github.com/Minwsun/ZeroFake.git
cd ZeroFake

# Create and activate virtual environment
python -m venv .venv

# Windows
.venv\Scripts\activate
# Linux / macOS
source .venv/bin/activate

# Install core dependencies
pip install -r requirements.txt

2. Configure API Keys

Copy the sample configuration and configure your API keys:

cp .env.example .env

Edit .env with your credentials:

# ==========================================
# Core LLM Providers
# ==========================================
GEMINI_API_KEY=AIzaSy...

# Multi-Key Load Balancing (Cerebras & Groq)
CEREBRAS_API_KEY_1=csk_...
CEREBRAS_API_KEY_2=csk_...
GROQ_API_KEY_1=gsk_...
GROQ_API_KEY_2=gsk_...

# ==========================================
# Fact-Checking & Knowledge Tools
# ==========================================
GOOGLE_FACT_CHECK_API_KEY=AIzaSy...
OPENWEATHER_API_KEY=...

💻 Running the Application

Option A: Launch FastAPI Server

uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

Interactive Swagger UI available at: http://localhost:8000/docs

Option B: Modern Web Dashboard (React Frontend)

cd frontend
npm install
npm run dev

Access the web console at: http://localhost:5173

Option C: Standalone PyQt6 Desktop GUI

python gui/main_gui.py

🐳 Docker Deployment

Deploy the entire stack with Docker Compose:

docker compose -f docker/docker-compose.yml up -d --build

📡 API Reference

POST /api/verify

Verify a statement or news snippet with full chain-of-thought analysis.

Request

{
  "claim": "UNESCO công nhận Vịnh Hạ Long là kỳ quan thiên nhiên thế giới mới vào năm 2011",
  "language": "vi",
  "include_trace": true
}

Response

{
  "claim": "UNESCO công nhận Vịnh Hạ Long là kỳ quan thiên nhiên thế giới mới vào năm 2011",
  "verdict": "TIN THAT",
  "confidence": 0.96,
  "flow_taken": "OLD_INFO_FACT_CHECK",
  "summary": "Tuyên bố chính xác. Vịnh Hạ Long được tổ chức New7Wonders công bố là một trong 7 Kỳ quan Thiên nhiên Mới của Thế giới vào năm 2011 và được UNESCO nhiều lần công nhận là Di sản Thế giới.",
  "reasoning_steps": [
    "PLANNER generated 5 contextual validation queries.",
    "FILTER pruned 4 irrelevant forum discussions and retained official press records.",
    "CRITIC verified date attribution and New7Wonders vs UNESCO distinction.",
    "JUDGE issued high-confidence TRUE verdict."
  ],
  "evidence": [
    {
      "title": "Ha Long Bay - World Heritage Centre",
      "url": "https://whc.unesco.org/en/list/672",
      "domain": "unesco.org",
      "trust_score": 0.95
    }
  ],
  "execution_time_seconds": 3.42
}

📂 Repository Structure

ZeroFake/
├── app/                      # Core FastAPI Application & Legacy Pipeline
│   ├── main.py               # REST API Orchestrator
│   ├── agent_planner.py      # PLANNER Agent logic
│   ├── agent_synthesizer.py  # CRITIC & JUDGE Adversarial Agents
│   ├── fact_check.py         # Google Fact Check Tools API client
│   ├── search.py             # Multi-source hybrid search engine
│   ├── ranker.py             # Source credibility rating engine
│   └── model_clients.py      # LLM load-balancer (Cerebras, Groq, Gemini)
├── src/zerofake/             # ZeroFake v5 Modular Architecture
│   ├── authenticity/         # C2PA metadata & image integrity
│   ├── decision/             # Bayesian aggregation & reasoning
│   ├── retrieval/            # Hybrid searchers (BM25, GNews, DDGS)
│   └── runtime/              # Pipeline orchestration & worker pools
├── frontend/                 # React + Vite Web Dashboard
├── gui/                      # PyQt6 Dark-Mode Desktop GUI
├── prompts/                  # Multi-Agent System Prompts (CoT, Adversarial)
├── docker/                   # Dockerfile & Docker Compose configurations
├── evals/                    # Evaluation suite & benchmark datasets
├── compare.md                # Quantitative benchmark comparisons
└── requirements.txt          # Production dependencies

🛡️ Security & Privacy

  • No Secret Storage: All credentials and API tokens are dynamically resolved via environment variables (.env).
  • Data Minimization: Query traces and input payloads are sanitized before downstream dispatch.
  • Fail-Safe Fallbacks: Zero-downtime execution through multi-key rotation and multi-provider failovers.

👥 Author & Acknowledgements

Nguyen Nhat Minh

Built with ❤️ for a safer, trustworthy, and transparent digital information ecosystem.

About

An AI-powered system for detecting and analyzing synthetic and manipulated content, exploring how machine learning can distinguish authentic media from AI-generated or altered content.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Contributors

Languages