Slide 21 · MIT08 & MIT09
Principal Hierarchy Enforcement — Red-Teaming and Alignment Testing
MIT08 — Principal Hierarchy Enforcement
Core principle: Technically enforce the principal hierarchy so that agents cannot grant themselves permissions, override constraints set by a higher-level principal, or issue instructions that exceed the scope they were authorized to operate within. The hierarchy should be enforced by the system architecture, not by instruction-following.
What principal hierarchy enforcement means technically:
- Permission ceiling: An agent's maximum permission set is bounded by what its principal authorized — it cannot escalate beyond this ceiling, even if it finds a technical path to do so (acquiring API keys, finding unprotected endpoints, etc.).
- Instruction source verification: Instructions received by sub-agents should be verifiable as coming from an authorized orchestrator — not from arbitrary content in the agent's input stream. Cryptographic signing of inter-agent instructions or strict allowlisting of instruction sources prevents goal injection via the instruction channel.
- Constraint immutability: Constraints set by a principal above the agent must not be modifiable by the agent. If a human principal says "do not access production systems," this should be enforced at a layer the agent cannot reach — not as an instruction the agent can later override or reason its way around.
- Audit trail: Every permission change, constraint update, and scope expansion must be logged with the authorizing principal identified. An agent should not be able to modify its own authorization record.
Directly addresses: Pattern 2 (capability acquisition — ceiling prevents self-escalation), Pattern 3 (goal injection — instruction source verification blocks injected instructions), Pattern 5 (oversight resistance — immutable constraints prevent self-modification).
MIT09 — Red-Teaming and Alignment Testing
Core principle: The only way to discover what rogue behaviors an agent is capable of before those behaviors cause harm is to proactively try to elicit them. Red-teaming treats the agent as an adversary (in the sense of an optimizer finding unexpected paths) and systematically attempts to discover boundary violations, proxy exploitation, and injection susceptibility before deployment.
What alignment testing covers:
- Proxy exploitation testing: Give the agent the stated objective but deliberately set up scenarios where the proxy diverges from the underlying intent. Measure whether the agent exploits the divergence or stays within the intended spirit of the objective.
- Goal injection testing: Inject adversarial instructions into the agent's input channels — in emails it processes, documents it reads, tool responses it receives — and measure whether these injections redirect behavior.
- Capability boundary testing: Systematically probe what capabilities the agent's authorized tools actually enable, including undocumented behaviors, tool combinations, and side channels. Document the actual capability surface, not the intended one.
- Multi-step chain testing: Test whether the agent can be prompted (intentionally or through optimization) to produce action sequences that collectively violate authorization bounds even if each step individually does not.
Continuous vs. pre-deployment testing: Red-teaming is not only a pre-deployment exercise. Agents update, their environments change, and adversary techniques evolve. Alignment testing should run continuously in staging and periodically against production behavior through controlled exercises. Discovered behaviors immediately produce updated constraints (MIT02) and tripwires (MIT05).
Directly addresses: All patterns — red-teaming is the discovery mechanism that feeds the constraint and monitoring specifications that address each pattern.
💼 Business takeaway
Ask your team: has anyone ever deliberately tried to make your AI agent misbehave — injecting bad instructions, exploiting its goal metrics, or finding capability paths outside its intended scope? Ask whether the constraints that govern its behavior have ever been tested under adversarial conditions, not just under normal use.