These are documented harm categories, not hypotheticals.
⚖
Legal liability
Sanctions for fabricated citations, lawsuits over incorrect policy information, regulatory consequences for financial misinformation. Mata v. Avianca (2023), Air Canada (2024).
🏥
Patient harm
Wrong medical guidance on drug interactions, dosing, or diagnosis delivered with clinical-sounding confidence. Regulators are actively investigating medical AI chatbot claims.
💸
Financial loss
Fabricated market data, hallucinated regulatory requirements, or invented research driving investment and business decisions.
🔓
Security vulnerabilities in code
Insecure code patterns recommended with confidence. Buffer overflows, SQL injection, path traversal — generated without warning. Pearce et al. (2022): 40% of Copilot’s security-sensitive outputs were vulnerable.
📦
Supply chain compromise
Hallucinated package names installed by developers, pre-loaded with attacker malware. Hallucination as a delivery mechanism (slopsquatting, 2024).
📰
Reputational damage
Public-facing chatbots giving wrong answers at scale. Every wrong answer is a potential media story, viral post, or tribunal ruling.