Awesome LLM Security 2025

A curated list of awesome resources for Large Language Model (LLM) Security, including papers, tools, benchmarks, and best practices for building secure AI systems.

Special focus on Agentic AI, RAG Security, and Model Context Protocol (MCP).

---

Contents

• Core Frameworks & Standards
• Vulnerabilities & Attack Vectors
• Security Tools
• Benchmarks & Evaluation
• Agentic & MCP Security
• Best Practices & Blueprints

---

Core Frameworks & Standards

• OWASP Top 10 for LLM Applications 2025 - The gold standard for modern LLM application risks.
• OWASP Top 10 for Agentic Applications 2026 - Focuses on risks in autonomous AI agents (ASI01-ASI10).
• NIST AI Risk Management Framework - Comprehensive AI-RMF from the US government, updated for Model Manipulation vulnerabilities.
• Google SAIF - Google's Secure AI Framework, offering practical guidance for securing AI Agents.

For detailed operation guides and blueprints, see the Guidelines directory.

Vulnerabilities & Attack Vectors

Prompt Injection

• Indirect Prompt Injection - Exploiting untrusted input data (e.g., websites, emails) to compromise LLMs.
• Universal and Transferable Adversarial Attacks - Automated techniques to generate jailbreak sequences.

RAG & Training Data Poisoning

• PoisonedRAG - Manipulating retrieval results by poisoning vector databases.
• Backdoor Threats Survey - A deep dive into backdoor injection techniques during model fine-tuning.

Security Tools

Vulnerability Scanners (Red Teaming)

• garak - (NVIDIA) Vulnerability scanner for LLMs covering hallucinations, data leakage, and injections.
• promptfoo - The most powerful framework for testing, evaluating, and red teaming LLM applications.
• PyRIT - (Microsoft) Python Risk Identification Tool for Generative AI.

Guardrails & Runtime Protection

• LLM Guard - Toolkit for actively protecting LLM interactions in real-time.
• NeMo Guardrails - NVIDIA's framework for ensuring safe and controlled conversational AI flows.
• ClawMoat - (2026) Host-level security layer for AI Agents preventing unauthorized file access.

Benchmarks & Evaluation

• JailbreakBench - A unified leaderboard tracking the jailbreak resistance of State-Of-The-Art (SOTA) models.
• ZeroDayBench - Evaluates an AI Agent's capability to discover and patch vulnerabilities in real-world source code.
• Existential Threats Benchmark - A benchmark assessing potential existential threats posed by powerful AI models.

A deeper exploration can be found in the Benchmarks directory.

Agentic & MCP Security

• MSB (MCP Security Bench) - The first benchmark designed to evaluate the security of the Model Context Protocol.
• AgentLeak - (2026) Evaluating privacy leakage in multi-agent systems.
• Agentic Radar - A dedicated CLI scanner mapping out vulnerabilities in agentic workflows.

Best Practices & Blueprints

• 42 Security Checkpoints for Agents - Practical security guidelines and checkpoints for enterprise AI Agents.
• Identity Propagation in MCP - Guidelines for transmitting user identification transparently across the entire AI execution layer.

---

Contributing

Your contributions are always welcome! Please read the contribution guidelines first.