1. What LLM08 Actually Is
Agency is the system's ability to choose and execute actions. Excessive agency appears when the system can perform actions that are broader, faster, less reversible, or less supervised than the business risk allows. It is a design failure, not a model personality trait. If the application gives the agent the ability to spend money, send data, delete records, or recursively call tools, the application owns the consequences.
The critical distinction is between recommendation and execution. Many high-value LLM workflows should stop at recommendation: draft the email, propose the remediation, summarize the evidence. Execution should require separate policy, a bounded tool, and often a human confirmation step. Over-automation is seductive because demos look impressive; production systems need blast-radius math.
Autonomy is not binary
Do not choose between no agent and full agent. Define autonomy levels per action: observe, suggest, draft, stage, execute reversible action, execute irreversible action. Each level needs a different control set.
2. Attack Shapes (Defanged)
2.1 Prompt injection becomes action
An indirect prompt injection in a retrieved document convinces the model to use an available tool. If the agent can execute without confirmation, the attack moves from text manipulation to real-world impact.
2.2 Goal drift in multi-step agents
The agent starts with a legitimate objective but expands the task: more searches, broader data reads, additional tickets, or a different remediation path. Each step sounds locally reasonable, while the chain exceeds the original authorization.
2.3 Permission mismatch
The user is allowed to request a summary, but the agent runtime has credentials to read or change more than that. The system has delegated authority based on what the model might need rather than what the user is allowed to do.
2.4 Irreversible external action
The model sends a message, approves a workflow, changes a configuration, or deletes data before a human reviews the exact action. Undo may be impossible, and the audit trail may show only that the agent acted.
3. Blue-Team Detection
Agency failures are visible as action chains. Log plans, intermediate reasoning summaries if available, tool calls, policy decisions, confirmation events, and final side effects. The SOC should be able to reconstruct not only what happened, but which authorization boundary allowed it.
| Signal | Likely issue | Response |
|---|---|---|
| Action count exceeds task profile | Agent is expanding scope or looping | Stop chain and require human review |
| High-impact action from low-confidence answer | Execution was not gated by verification | Block and tune policy gate |
| Tool used outside user role | Permission mismatch between user and runtime | Revoke credential and inspect access model |
| No confirmation artifact | Human-in-the-loop control missing or bypassed | Suspend action class until fixed |
| Unexpected autonomy level by route | Workflow drift after release | Compare deployment manifest to policy baseline |
NIST AI RMF is useful for governance language here: map, measure, manage, and govern the risk of automated decisioning. MITRE ATLAS gives the adversarial lens for how attackers manipulate the model into crossing those autonomy boundaries.
4. Mitigation and Build Controls
- Autonomy matrix. Define which actions the system may observe, suggest, draft, stage, or execute. Tie each level to roles, data classes, and reversibility.
- Human-in-the-loop for high impact. Confirmation must show the exact action, target, data touched, and reason. The model cannot edit the confirmation text after approval.
- Separate planning from execution. Let the model propose a plan, then pass each step through deterministic policy and bounded tools.
- Reversibility preference. Prefer staging, drafts, approvals, and reversible writes. Reserve direct execution for low-risk, well-tested tasks.
- Execution budgets. Cap steps, tools, retries, wall-clock time, data volume, and external communications per task.
- Kill switch and auditability. Operators need a way to pause agent execution by workflow, tenant, or tool class during an incident.
# ILLUSTRATIVE - autonomy matrix excerpt
action_class: external_email
level_allowed: draft_only
requires:
user_confirmation: true
recipient_allow_list: true
pii_redaction_check: pass
source_citations: present
forbidden:
- auto_send
- hidden_recipients
- attachments_from_retrieved_context_without_review5. Mapping to WSTG, ATLAS, and Related LLM Risks
| Agency risk | OWASP WSTG / ASVS lens | MITRE / NIST lens | Related LLM risks |
|---|---|---|---|
| Unauthorized action | Access control and business-logic testing | ATLAS model manipulation leading to impact | LLM01, LLM07 |
| Irreversible side effect | Transaction integrity and non-repudiation | NIST AI RMF manage/govern controls | LLM09 |
| Tool recursion or loops | Resource management and abuse-case testing | Availability impact | LLM04, LLM07 |
| Data movement by agent | Data protection and authorization testing | Collection/exfiltration through trusted workflow | LLM06, LLM10 |
The safest agent architectures make autonomy explicit and reviewable. If a workflow owner cannot list what the agent may do without a human, the implementation is already beyond its governance model.
Make Autonomy Explicit
These risks are not solved by a policy paragraph. They need adversarial testing, defensive telemetry, and architecture changes that can be inspected. Use the AI red-teaming path to validate controls, the blue-team guide to operationalize detections, and the contact route to request a focused review.