Slide 5 of 28
Part 1 — The ProblemSlide 5
Slide 5 · Outcomes
Six categories of harm that cascading failures produce — each harder to recover from than the last.
Incorrect action execution at scale
Downstream agents execute actions — emails sent, transactions processed, files modified — based on corrupted upstream data. By the time the error is detected, many irreversible actions have been taken.
Error amplification across hops
A small input error compounds at each pipeline stage. A 5% data truncation becomes a 40% analysis distortion, which becomes a 100% wrong recommendation. Each agent processes bad input and produces output that is worse than what it received.
Resource exhaustion and availability collapse
A failing agent that retries excessively, or returns data that causes a downstream agent to loop, consumes compute and memory across the entire pipeline — degrading or crashing components that would otherwise have operated normally.
Runaway feedback loops
When agent outputs feed back into their own inputs — directly or through shared state — a failure can create a self-reinforcing loop that drives behavior to an extreme, with no natural stopping point short of external intervention.
Unauthorized actions via corrupted instructions
An attacker who can control one agent's output can use that agent as a stepping stone — emitting instructions that, when passed downstream, cause other agents to perform actions the attacker couldn't trigger directly.
Delayed detection and extended blast radius
Because failures appear in correct format, automated monitoring doesn't catch them. Harm accumulates for minutes to hours before a human notices an anomaly. The longer detection takes, the larger the set of real-world actions that need to be reversed.
← Back Who triggers this →