STRIDE for AI systems
STRIDE still works for AI systems — you just have to accept that the prompt is code, retrieved content is untrusted input, and the model is a confused deputy with tools.
By Piotr Zenon Żabrowski · Published · Updated
1Untrusted inputs
- User prompt
- Retrieved documents
- Web / email content
- Tool responses
2Trust boundary
- Input normalisation
- PII redaction
- Data / instruction separation
3Model + tools
- System prompt
- Delegated user tokens
- Confirmation on writes
- Step and budget limits
4Outputs
- Contextual encoding
- Schema validation
- URL / image containment
- Audit log
The elements to model
- User interface and session identity
- Prompt assembly layer (system prompt, history, retrieved context)
- Retrieval store and its ingestion pipeline
- Model / inference provider boundary
- Tool and function-calling layer with its credentials
- Inter-agent communication and shared memory or state, where more than one agent is involved
- Output rendering and downstream consumers
- Logging, telemetry and retention
STRIDE per element
| Category | AI-specific instance | Control |
|---|---|---|
| Spoofing | Injected text claiming to be the system or an admin | Structural separation of roles; never trust in-band claims |
| Tampering | Poisoned documents in the retrieval index | Ingestion provenance, source allowlists, content review |
| Repudiation | No record of which context produced an action | Log prompt hash, retrieved doc IDs, tool calls |
| Information disclosure | Cross-tenant retrieval; system-prompt extraction | Per-tenant filters at query time; treat prompts as non-secret |
| Denial of service | Token-exhaustion loops in agent chains | Step limits, budgets, timeouts |
| Elevation of privilege | Agent tool acting with service-level rights | Per-user delegated tokens, confirmation for writes |
Where STRIDE stops being enough
- A chain of agents fails in ways no single element captures — one agent's flawed output becomes the next agent's trusted input, and the error compounds rather than staying local.
- Shared memory or a shared scratchpad is a new tampering surface: one agent can poison state that another agent later treats as fact.
- A single compromised or misaligned agent with broad tool access is a different failure mode from any of the six STRIDE categories on its own — closer to an insider threat than an external attacker.
- This is the gap that agentic-specific frameworks (the OWASP Top 10 for Agentic Applications, MAESTRO's layered model) were written to close, not a reason to discard STRIDE for the single-agent elements underneath.
Definition of done
Before shipping an AI feature
- Diagram exists with trust boundaries drawn around the model and retrieval layer.
- Every tool call is authorised as the requesting user, not the service.
- Write or destructive actions require explicit confirmation.
- Retrieval queries are scoped by tenant and clearance at query time.
- Model output is encoded for its rendering context.
- Prompt, retrieved sources and tool calls are logged with bounded retention.
- For multi-agent designs: each agent's goal is re-validated before consequential actions, and agent-to-agent messages are authenticated rather than trusted by default.
Frequently asked questions
Is a new framework needed for AI?
No, and this holds even for agentic systems. STRIDE plus the OWASP Top 10 for LLM Applications covers a single model call well; what changes is which mitigations are effective. Once a system plans across multiple steps or coordinates several agents, layer in the OWASP Top 10 for Agentic Applications and a framework built for multi-agent layers, such as MAESTRO, rather than trying to stretch STRIDE's six categories over a shape they were not built to describe.
How does this relate to red-teaming?
Threat modeling is a design review: it tells you where the trust boundaries are and what should be true at each one. Red-teaming is adversarial testing against a running system. Use MITRE ATLAS as the technique catalogue for the second activity — it documents observed adversary tactics against AI systems, from model extraction to data poisoning, so red-team coverage can be tracked against a shared taxonomy instead of an internal list.
Related
Threat modeling
A practical STRIDE-based threat modeling method for cloud, application and AI systems, built to be run by engineering teams.
AI security
How to secure GenAI, LLM and agentic AI systems: current OWASP threat classes, layered controls, agent authority, retrieval isolation and monitoring.
AI security assessment
Security assessment of GenAI and LLM systems: prompt injection, tool and agent abuse, data leakage, model supply chain and monitoring gaps.
Prompt injection defence
Why prompt injection cannot be filtered away, why it has no full fix yet, and how to design AI systems — architecturally, not just with better prompts — so a successful injection has no useful consequence.
Threat modeling
Facilitated STRIDE threat modeling on your real architecture, plus the templates and coaching to keep teams doing it without me.
Confidence
Recorded CONFidence talk: threat modeling at the scale of Allegro, from STRIDE to a process engineering teams actually run.