1. Triage: Decide Whether This Is a Leak
The first question is not attribution. It is whether sensitive data crossed a boundary it should not have crossed. The boundary may be user-to-user, tenant-to-tenant, internal-to-external, model-to-tool, or production-to-observability. Triage should preserve evidence before cleanup changes the state.
- Identify the leaked data class. PII, credentials, regulated records, source code, model prompt, retrieved document, or customer-specific output.
- Identify the channel. Chat response, rendered Markdown, tool argument, outbound request, ticket/comment, email, log, trace, or analytics export.
- Identify the affected principal. User, tenant, API key, service account, session, and model route.
- Freeze volatile evidence. Conversation state, retrieval trace, assembled prompt, tool logs, egress logs, and index version.
Do not delete first
Deleting the offending document or trace may stop recurrence, but it can destroy the only evidence that proves scope. Preserve, snapshot, then contain.
2. Containment Actions
| Leak channel | Containment action | Rollback consideration |
|---|---|---|
| Chat response only | Disable affected route or model release; add output block rule | Preserve conversation and prompt assembly |
| Tool argument | Disable tool class or side-effect action; rotate exposed credentials if any | Review all calls by same principal and route |
| Rendered link or Markdown | Disable rich rendering or external auto-loads; tighten CSP/egress | Invalidate cached rendered messages |
| RAG retrieval leak | Quarantine source/index; rebuild access filters | Snapshot index before rebuild |
| Telemetry/log leak | Restrict trace access and exports; redact stored records | Track who accessed exported data |
Containment should be reversible where possible, but speed matters. If the leak can recur through an active route, disable the route or high-risk tool while preserving enough evidence to scope the incident.
3. Investigation Questions
- Was this LLM01? Did direct or indirect prompt injection alter model behavior?
- Was this LLM03? Did poisoned or low-trust data enter the retrieval path?
- Was this LLM06? Did access control, DLP, or retrieval filtering fail and expose sensitive information?
- Was this LLM07/LLM08? Did a tool or agent action move the data further than the chat response?
- Was this LLM02? Did output rendering or downstream handling create the leak channel?
- Was this a telemetry failure? Did logs or traces store data beyond the intended audience?
| Evidence source | Question it answers | Integrity action |
|---|---|---|
| Assembled prompt | What data was available to the model? | Hash and store restricted copy |
| Retrieved chunk IDs | Which documents caused exposure? | Map to source owner and ACL |
| Guardrail verdicts | Which control allowed or missed the leak? | Preserve model and policy version |
| Tool logs | Did data leave through an action? | Correlate with egress and target object |
| Access logs | Who viewed leaked output or traces? | Export immutable timeline |
4. Eradication, Recovery, and Lessons
- Fix the root boundary. Do not only suppress the exact leaked string. Correct the ACL, retrieval filter, output sink, tool policy, or telemetry export path.
- Rebuild affected indexes. If access labels or chunk metadata were wrong, rebuild and verify. Partial edits leave stale embeddings behind.
- Rotate secrets. If credentials, tokens, or system prompts with secrets were exposed, rotate and remove them from prompts.
- Add regression tests. Convert the incident into harmless fixtures: source trust, retrieval, guardrail, rendering, and tool-policy tests.
- Update detections. Add SIEM rules for the observed channel and any near-miss signals found during investigation.
- Run post-incident review. Record owner, timeline, blast radius, root cause, missed controls, and concrete due dates.
# ILLUSTRATIVE - incident timeline fields
t0_detected: timestamp
affected_route: llm_route_id
affected_model_release: immutable_release_id
conversation_ids: restricted_list
retrieved_source_ids: restricted_list
leak_channel: chat_output | tool_call | rendered_link | telemetry
containment_action: route_disabled | tool_disabled | index_quarantined
root_cause: acl_gap | prompt_injection | output_sink | telemetry_export | other
regression_tests_added: list_of_test_idsMITRE ATT&CK helps structure host, identity, and exfiltration evidence. MITRE ATLAS helps explain model-specific manipulation. OWASP LLM Top 10 gives the risk labels executives and application teams can act on.
Prepare the Evidence Before the Incident
The field notes above are intentionally conceptual and defanged, but the controls are practical. Use AI red-teaming to validate the attack path safely, blue-team detection engineering to make the signals operational, and contact to scope a review of your own environment.