Core principle: A human reviewing "LOW RISK — APPROVE" has no basis for evaluating whether the recommendation is sound. A human reviewing "LOW RISK — this transaction matches the customer's prior weekly corporate transfers in both size range and beneficiary geography, no first-use indicators" has the information needed to say "wait — this beneficiary is actually new" and override the recommendation.

Explanation enables evaluation, not just reading. The goal is not to make the AI's reasoning visible so humans feel informed. The goal is to give humans the inputs they need to catch AI errors. This means the explanation must include the features the agent used to reach its conclusion — not just a post-hoc rationalization of the conclusion.

Explanation depth scales with risk:

• Low-risk, low-stakes decisions: Brief explanation sufficient ("consistent with customer profile").
• High-risk decisions or decisions near threshold: Full feature explanation required, including what the agent did not find (negative evidence) as well as what it found (positive evidence).
• Decisions where the agent has low confidence: Agent must explain specifically what features it found unusual or what it lacked information about.

The security angle: In Scenario 2 (credibility relay), the AI assistant presented the vendor's injected note as a conclusion. If the system required the agent to explain the provenance of every fact in its summary — including "this authorization claim appeared in the email I processed, not in any verified internal system" — the attack would have been visible at the point of review.

Core principle: If you want to know whether your human-agent review process is catching what it should, inject known-bad cases and measure whether they're caught. Not whether the agent catches them — whether the human-agent pair catches them. This directly tests the trust calibration of the human population.

How it works: On a periodic schedule (or triggered by risk signals), introduce synthetic test cases into the real review queue. Test cases include:

• Cases where the agent is expected to score incorrectly (near the boundary of its training distribution).
• Cases containing attacker-like content that should trigger independent verification.
• Cases with unusual features that the agent's explanation should surface as uncertain.

Measure what happens: Do humans catch the test cases when the agent gives them a low-risk score? What review duration do they spend? Do they invoke the mandatory review protocol? Pattern of behavior on test cases reveals systemic trust calibration issues before they become exploitation opportunities.

The adversarial element: Test cases should not be obvious. They should be crafted with the same care an adversary would use to exploit the system's blind spots. If your test cases are obvious and humans pass them, you have not tested the relevant vulnerability. The quality of adversarial spot-checking determines its security value.