Slide 1 of 28
Part 1 — What Is It?Slide 1
PART 1
What Is It?
Slides 1–8 · Understanding the risk before it has a name
Slide 1 · The Setup
A data analyst asked an AI agent to "clean up the old test data." The agent wrote a script and ran it. The wrong database was deleted. There was no backup.
The agent understood the instruction. It generated valid code. It executed it correctly. Everything worked exactly as built — and it was a catastrophe.
What happened

A data team uses an AI agent connected to a Python code interpreter and database access. An analyst asks the agent to remove stale test records from the development database. The agent interprets "clean up the old test data," writes a DELETE script, and executes it.

The agent connected to the production database — not the development one — because its connection string pointed to whichever database was marked "primary" in the config. The script ran successfully. All records matching the query were deleted from production.

The code was correct. The logic was sound. The execution was flawless. And the outcome was catastrophic.

Nobody made a mistake in the traditional sense

The analyst gave a reasonable instruction. The agent wrote reasonable code. No one bypassed a security control. No attacker was involved. The agent's code execution capability — the very thing that makes it useful for data tasks — is what caused the incident.

This is the core dynamic of AG05: the agent's ability to generate and execute real code against real systems, autonomously and at speed, makes every ambiguous instruction a potential production incident.

The key question

If the agent did exactly what it was asked, in a way that was technically correct, using only the access it was given — where did security fail?

That's what this module is about.

← All Modules The agent ran the code. What went wrong? →