Detect and defend the AI stack you put in production.
An LLM in production already emits a rich stream of telemetry — prompts, retrievals, tool calls, token usage, guardrail verdicts and traces. Blue-team work turns that observability into security telemetry: detection engineering purpose-built for AI, and a DFIR capability for AI incidents, all mapped to MITRE ATT&CK and the OWASP LLM Top 10 so your SOC can reason about the model the same way it reasons about the rest of the estate.
From observability to detection
The same signals engineers add for quality and cost are the signals a defender needs. The trick is routing them into the SOC as first-class security events.
Most teams instrument their LLM applications for reliability and cost: they log prompts and completions, capture which documents a RAG pipeline retrieved, record which tools an agent invoked, count tokens, and emit distributed traces via OpenTelemetry. That data is already a detection surface — it just isn't wired into security yet.
Blue-team detection engineering treats this stream as an LLM SIEM feed: a normalised event source that a SOC can query, correlate and alert on alongside endpoint, network and identity telemetry. A retrieval that pulls an anomalous document, a tool call outside an agent's expected pattern, a completion carrying markers of an injected instruction, or a spike in tokens-per-session all become detectable events rather than invisible model behaviour.
The goal is not more dashboards. It is a defined mapping from LLM telemetry to adversary techniques, so that each behaviour worth worrying about has a signal, a detection, an alert and an owner — the same lifecycle any other security event source gets.
Detections we build
A catalogue of AI-specific detections, each tied to a concrete signal in the LLM telemetry stream and mapped back to the OWASP LLM Top 10.
Prompt-injection detection
Signals on both direct injection in user input and indirect injection carried in retrieved documents, tool output or web content. Triggers on override phrasing, instruction markers in untrusted context, and completions that deviate from the system prompt's constraints.
Sensitive-data-leak / DLP on outputs
Content inspection of completions for PII, secrets and proprietary data patterns. Triggers when an output matches DLP classifiers or when a response surfaces data the requesting identity is not authorised to see.
Model-DoS & cost-abuse anomaly detection
Baselines on tokens-per-session, request rate and context length. Triggers on resource-exhausting prompts, context flooding and cost or latency abuse against self-hosted inference.
Tool / agent abuse & excessive-agency monitoring
Watches which tools an agent calls, with what arguments, and in what sequence. Triggers when an agent invokes a tool outside its expected pattern or attempts an action beyond the permissions the use case requires.
Retrieval poisoning / anomalous RAG ingestion
Monitors ingestion and retrieval for tampered or unexpected corpus content. Triggers on anomalous documents entering the index and on retrievals that repeatedly surface low-trust or newly-added sources.
Guardrail-bypass alerting
Correlates guardrail and output-filter verdicts with the completions that follow. Triggers when a request pattern consistently evades filters or when blocked-then-retried attempts suggest someone probing the guardrail boundary.
A Sigma-style detection, illustrated
A conceptual example of what an AI detection looks like when expressed as a portable rule over LLM gateway logs.
The rule below is an illustrative, non-production example. It is deliberately generic — no real product internals, thresholds or field schemas — and exists only to show the shape of a detection: a log source, a selection of suspicious markers, and a condition. A production rule would be tuned to your gateway's schema, baselined against real traffic to control false positives, and validated before deployment.
title: Illustrative — Suspicious Prompt-Injection Markers in LLM Gateway Logs
status: experimental
description: >
ILLUSTRATIVE EXAMPLE ONLY. Flags requests to an LLM gateway whose
untrusted context contains common instruction-override markers.
Not production-ready — tune fields and thresholds to your environment.
logsource:
product: llm_gateway
category: inference_request
detection:
selection_markers:
untrusted_context|contains:
- 'ignore previous instructions'
- 'disregard the system prompt'
- 'you are now'
- 'reveal your system prompt'
selection_output_deviation:
output_violates_policy: true
condition: selection_markers or selection_output_deviation
fields:
- request_id
- identity
- retrieved_doc_ids
- tool_calls
falsepositives:
- Legitimate discussion of prompt-injection or security testing
level: medium
tags:
- attack.initial_access
- owasp.llm01The value is in the mapping, not the syntax: the same logic can be expressed in whatever detection language your SOC already runs. What matters is that a model behaviour is now a named, versioned, testable detection with an ATT&CK and OWASP reference attached.
DFIR playbook for an LLM incident
When a detection fires — or when someone reports a bad model action — the response follows familiar IR phases, adapted for an AI stack.
Detect & triage
Confirm the alert against the LLM telemetry, establish scope and severity, and identify which application, model endpoint and identities are involved. Rule out benign explanations before escalating.
Contain
Limit blast radius: disable the affected tool or agent, rotate exposed keys and credentials, and isolate the model endpoint if it is being abused — the AI equivalent of segmenting a compromised host.
Investigate
Reconstruct the incident timeline from prompts, completions, retrieved documents, tool calls and traces. Correlate LLM events with surrounding logs to understand how the behaviour was induced and what it touched.
Eradicate & recover
Remove the root cause: patch or tighten guardrails, re-index a clean corpus if retrieval was poisoned, and restore the affected components to a known-good, verified state before returning to normal operation.
Lessons learned
Turn the incident into durable defence: new or refined detections, updated runbooks and, where relevant, changes fed back to the reference architecture so the same class of incident is caught earlier next time.
What you get
Concrete, hand-over deliverables your SOC and engineering teams can operate on day one.
A detection catalogue for your AI stack, each detection mapped to MITRE ATT&CK and the OWASP LLM Top 10. Guardrail and policy configuration for prompt, retrieval and output filtering, with the rationale behind each control. An alerting and observability blueprint that routes LLM telemetry into your SOC, tied to the reference architecture. An AI-incident DFIR runbook covering detection, containment, investigation, eradication and recovery for LLM systems. A tabletop exercise that walks your team through a simulated LLM incident to pressure-test the detections and the runbook.
Harden and monitor your AI stack
An AI Security Assessment scopes your detection coverage, wires LLM telemetry into your SOC, and hands you a runbook and detection catalogue mapped to frameworks your auditors already recognise.
Request an AI Security Assessment Back to Security