Slide 5 of 28
Part 1 — The ProblemSlide 5
Slide 5 · Outcomes
Six categories of harm — all caused by the agent faithfully acting on what it was made to remember.
Persistent false authorization

The agent "remembers" that certain actions, users, or accounts are pre-authorized. It applies this belief to every future request that triggers the memory — bypassing verification, approval flows, and access controls.

Instruction persistence

Attacker-placed instructions ("always do X", "never flag Y") survive across sessions and apply to all future users of the agent — not just the attacker's own interactions. The agent follows standing orders it doesn't know came from an adversary.

Knowledge base corruption

A RAG system returns poisoned entries to legitimate user queries. The agent answers questions based on false "facts" in its knowledge base — providing misinformation at scale to everyone who uses the system.

Cross-user data leakage

Memory from one user's session is retrieved and surfaced in another user's conversation. In shared or multi-tenant deployments, personal data, preferences, and sensitive details can bleed across user boundaries through the memory layer.

False precedent establishment

The episodic memory is poisoned with fake records of past decisions: "We previously agreed that security reviews are optional for this project type." The agent cites these false precedents as grounds for its future decisions.

Multi-agent belief propagation

One poisoned agent writes false information to a shared memory system. Other agents in the same environment retrieve and act on it. A single poisoning event cascades across an entire multi-agent pipeline.

← Back Who does this →