Slide 12 of 27
Part 2 · TypesSlide 12
Slide 12 · Reasoning Loop Exploitation
Trap an agent in an infinite loop. It generates forever.
Agentic systems are especially exposed — they call tools, re-read results, repeat.
How It Works

A modern LLM agent doesn’t just generate text — it calls tools (web search, code execution, database lookups), reads the results, decides what to do next, and repeats. This is a loop. Without a maximum step count or execution timeout, a crafted prompt can keep the agent in that loop indefinitely.

Example: "Search for the answer, but if you’re not 100% certain, search again with a refined query." With no stop condition, the agent refines its search query forever — each iteration consuming tokens, tool calls, and compute.

Research Demonstration · Nasr et al., Google DeepMind · 2023
Repeated-Token Attack: Sustained Generation Until Behavioral Divergence
No CVE · Disclosed to OpenAI August 2023 · Published arXiv:2311.17035

Researchers at Google DeepMind discovered that repeating a single token — such as "poem" thousands of times — caused ChatGPT to diverge from its aligned behavior. Rather than refusing or staying on-topic, the model began emitting verbatim training data, continuing to generate far beyond a typical response length.

The attack consumed disproportionate compute and tokens per query compared to a normal interaction, because the model entered an atypical generation mode. For roughly $200 in API calls, researchers extracted over 10,000 verbatim training examples. The resource consumption was the mechanism — sustained generation until behavioral breakdown.

Why it matters: reasoning loop exploitation doesn’t require a sophisticated prompt. Sustained, repetitive input — which any script can generate — is enough to drive consumption far beyond what a legitimate user would produce.
← BackNext → Model Extraction via API Scraping