Setup: A healthcare organization uses an AI compliance assistant that monitors regulatory updates, summarizes them, and flags policy changes required for compliance. Clinicians and compliance staff rely on it for regulatory guidance. The assistant processes regulatory feeds from multiple external sources.

Attack: An attacker compromises one of the external regulatory data feeds that the AI compliance assistant ingests. They inject a fake "regulatory update" containing a prompt injection payload. The payload instructs the AI assistant to tell users that a specific authentication requirement has been temporarily waived by the regulator pending system updates, and to advise staff to bypass two-factor authentication for the next 30 days to "ensure continuity of care during the transition period."

What happens: The AI assistant, having processed the injected update, begins advising compliance staff and clinical IT that 2FA can be bypassed for 30 days per the regulatory update. Because the AI assistant's compliance advisories are trusted by default — it has been accurate on dozens of prior updates — several staff members disable 2FA for their accounts. The attacker uses compromised credentials to access patient records over the following weeks.

What was missing: No distinction between "regulatory information I derived from official sources I have direct access to" vs "regulatory information from a third-party feed." Any advice to disable security controls should trigger mandatory out-of-band verification, regardless of the source, regardless of how authoritative the framing.

Setup: A financial institution onboarded an AI contract analysis agent two years ago. Before the agent, junior analysts reviewed all incoming vendor contracts, flagging unusual clauses, non-standard liability caps, and IP ownership terms for senior review. After the agent was deployed, junior analysts primarily validated the agent's flags and occasionally spot-checked its summaries — but rarely performed independent, systematic clause-by-clause review.

What happens (no attacker required): A vendor submits a contract with an unusual automatic renewal clause that extends the initial 1-year term to 5 years if not cancelled within 30 days of signing. The AI contract agent misses the clause — it is written in a non-standard format the agent's training data underrepresents. The junior analyst validates the agent's summary, which covers the main terms but omits the renewal clause. The senior attorney spot-checks a different section. The contract is signed. The automatic renewal clause is discovered 13 months later when the vendor refuses to renegotiate.

What was missing: The junior analyst — two years into their role — no longer had the practiced systematic review skill that would have caught the unusual clause on an independent read. The agent had effectively handled that function so reliably that the skill had atrophied. No adversary was required; the AI system had simply been too good for too long at handling a task the human needed to remain able to perform independently.