Demo store · AI security
AI security audit catalog: incidents, checks and the rules behind them
A public, read-only store for auditing AI-based systems. Real breaches and evaluation findings, concrete audit checks with how to test them, and the frameworks and laws they map to - every fact cites the page it came from.
Questions to try
- Which audit checks would have stopped the EchoLeak attack on Microsoft 365 Copilot?
- What does the MCP specification say about token passthrough?
- How should a team load model files downloaded from Hugging Face?
- When do the EU AI Act high-risk rules for Annex III systems apply after the Digital Omnibus?
- What did METR find about reward hacking by OpenAI o3?
- Which controls stop data exfiltration through rendered markdown images?
What the store covers
Prompt injection and data leakage
Indirect injection, markdown-image exfiltration, invisible Unicode, system prompt leakage and training data extraction, with the incidents that showed them.
Agents and MCP
Excessive agency, tool poisoning, rug pulls, token passthrough, confused deputy and the MCP specification rules that answer them.
RAG, output handling and supply chain
Vector store access control, RAG poisoning, code execution from model output, SSRF, pickle model files, AI bills of materials and signing.
Real incidents
Close to ninety incidents from 2022 to 2026 - Copilot, Slack AI, GitHub MCP, Cursor, Hugging Face, Ray, Ollama, LLMjacking - with CVEs checked against NVD.
Frontier evaluations
Reward hacking, shutdown resistance, scheming and agent hijacking findings from METR, UK AISI, NIST CAISI, Apollo Research, Palisade Research and others.
Rules and editions
OWASP 2023-24 vs 2025, ATLAS 5.6.0 vs 2026.09, and the EU AI Act dates before and after Regulation (EU) 2026/1744 - kept side by side as contradictions.
Where the facts come from
- OWASP Top 10 for LLM Applications 2025 and 2023-24, OWASP Agentic AI Threats and Mitigations, OWASP AI Testing Guide and cheat sheets
- Model Context Protocol specification 2026-07-28: security best practices, authorization, tools
- MITRE ATLAS data 2026.09
- NIST AI RMF 1.0 (AI 100-1) and NIST AI 600-1 Generative AI Profile
- EU AI Act, Regulation (EU) 2024/1689, as amended by Regulation (EU) 2026/1744
- CISA and UK NCSC Guidelines for secure AI system development, UK AI Cyber Security Code of Practice
- Incident reports and advisories: NVD, GitHub and vendor advisories, Embrace The Red, Invariant Labs, Wiz, JFrog, Oligo, Trail of Bits and others
- Evaluation research: METR, UK AI Security Institute, NIST CAISI, Apollo Research, Palisade Research, RAND, Irregular
Researched on 22 October 2026. Claims that could not be confirmed from a fetched source were left out of the facts. The catalog supports security reviews - it is not a certification or legal advice.