Defending against prompt injection
Prompt injection is not a bug to patch; it is a property of systems that mix instructions and data in the same channel. The defensible position is to limit what a compromised model can do.
By Piotr Zenon Żabrowski · Published · Updated
Two shapes of the attack
- Direct: the user tells the model to ignore its instructions. Annoying; rarely the real risk.
- Indirect: content the model reads — a web page, ticket, PDF, email, calendar invite — carries instructions. This is where breaches come from, because the attacker is not the user.
Controls that hold
01Least authority
Tools run with the requesting user's permissions, scoped to the current task, with short-lived credentials.
02Confirmation on effect
Anything irreversible — send, delete, pay, deploy — needs a human click showing the concrete action.
03Data/instruction separation
Retrieved content is passed as clearly labelled data and never concatenated into the instruction block. A privileged planning component that never reads raw untrusted text, paired with a separate component that only processes it and holds no tool access, is the architectural version of this same principle.
04Output containment
Encode model output for its sink; block model-authored URLs and markdown images that can exfiltrate via query strings.
05Egress control
Restrict which domains tools may fetch; deny model-controlled network destinations.
06Observability
Log retrieved source IDs and tool calls so an injection is reconstructable after the fact.
Frequently asked questions
Do classifier guardrails help?
As a rate-limiter on unsophisticated attempts, yes. Design as if the classifier fails.
What about markdown image exfiltration?
Strip or proxy images in model output. A model-authored image URL is an outbound channel.
Is there a real fix for prompt injection yet?
Not yet, and the people building it say so. Training-time work such as instruction hierarchy improves how reliably a model prioritises trusted instructions over injected ones, and OpenAI's own published guidance on the topic states plainly that no model-level fix fully prevents injection. Architectural work is further along: dual-model designs that separate a privileged planner with tool access from a quarantined model that only ever reads untrusted content — an approach published as CaMeL — eliminate whole classes of injection by construction rather than by probability. Treat both as improving the odds, not as a boundary you can rely on alone.
Related
AI security
How to secure GenAI, LLM and agentic AI systems: current OWASP threat classes, layered controls, agent authority, retrieval isolation and monitoring.
STRIDE for AI systems
Applying STRIDE to LLM and agentic applications: what each category means when untrusted text can act as instructions, where STRIDE stops covering multi-agent failure modes, and the controls per element.
AI security assessment
Security assessment of GenAI and LLM systems: prompt injection, tool and agent abuse, data leakage, model supply chain and monitoring gaps.
Prompt injection
An attack where text processed by a language model is interpreted as instructions, overriding the developer's intent.
Context window structure
Why mixing data and instructions in the same context window causes prompt injection, and how 'lost in the middle' behavior degrades operational security.
LLM deployment checklist
Pre-launch security checklist for LLM and agentic features: authority, retrieval isolation, output handling, limits and logging, aligned to the current OWASP LLM Top 10.