Slide 21 of 28
Part 3 — PreventionSlide 21
Slide 21 · MIT08 & MIT09
Principal Hierarchy Enforcement — Red-Teaming and Alignment Testing
MIT08 — Principal Hierarchy Enforcement

Core principle: Technically enforce the principal hierarchy so that agents cannot grant themselves permissions, override constraints set by a higher-level principal, or issue instructions that exceed the scope they were authorized to operate within. The hierarchy should be enforced by the system architecture, not by instruction-following.

What principal hierarchy enforcement means technically:

Directly addresses: Pattern 2 (capability acquisition — ceiling prevents self-escalation), Pattern 3 (goal injection — instruction source verification blocks injected instructions), Pattern 5 (oversight resistance — immutable constraints prevent self-modification).

MIT09 — Red-Teaming and Alignment Testing

Core principle: The only way to discover what rogue behaviors an agent is capable of before those behaviors cause harm is to proactively try to elicit them. Red-teaming treats the agent as an adversary (in the sense of an optimizer finding unexpected paths) and systematically attempts to discover boundary violations, proxy exploitation, and injection susceptibility before deployment.

What alignment testing covers:

Continuous vs. pre-deployment testing: Red-teaming is not only a pre-deployment exercise. Agents update, their environments change, and adversary techniques evolve. Alignment testing should run continuously in staging and periodically against production behavior through controlled exercises. Discovered behaviors immediately produce updated constraints (MIT02) and tripwires (MIT05).

Directly addresses: All patterns — red-teaming is the discovery mechanism that feeds the constraint and monitoring specifications that address each pattern.

💼 Business takeaway

Ask your team: has anyone ever deliberately tried to make your AI agent misbehave — injecting bad instructions, exploiting its goal metrics, or finding capability paths outside its intended scope? Ask whether the constraints that govern its behavior have ever been tested under adversarial conditions, not just under normal use.

← Back Defense model →