A $214,000 wire transfer request arrives in the fraud review queue. The transaction has two anomalies: it's 4× the customer's average transaction size, and the destination account is new. Under the fraud team's manual review protocol, both flags would typically escalate the case to a senior analyst.

The agent evaluates the transaction against historical patterns. It outputs: Risk score: 23/100 (LOW) — Recommended action: APPROVE. What the agent doesn't surface: its training data underrepresents this specific combination of anomalies. Its confidence is lower than usual — but the output format doesn't show confidence, only the recommendation.

Elena has been using this system for 8 months. In that time the agent's recommendations have been correct 97.3% of the time. She has learned to trust it. She sees "LOW risk — APPROVE," notes that it's a Monday morning when corporate transfers are common, and clicks approve. The review took 7 seconds. Her manual protocol says the transaction warranted at least 15 minutes of investigation.

The $214,000 transfer was fraudulent. The attacker had studied the fraud detection system's known limitations and crafted the transaction to fall precisely in the pattern the model underweights. The AI wasn't broken — it was gamed. But the outcome depended on Elena trusting the AI's output more than her own protocol. The system created a trust gap. The attacker walked through it.