Retrieval poisoning: when documents attack
RAG gives a model better evidence by placing retrieved text in its context. A poisoned document uses the same path to deliver attacker-written instructions.
By Piotr Zenon Żabrowski · Published · Updated
The attack starts before the query
Poisoning succeeds when indexed text crosses a trust boundary at query time. An attacker does not need access to model training or system prompts; they only need content that the search layer is likely to surface. Once retrieved, that content competes with trusted instructions for the model's attention. The security problem is therefore source control and context isolation, not a change to the underlying model.
01Plant
Hide instructions in a document, ticket, wiki page or OCR layer. The text can look irrelevant to a human reviewer while remaining available to the model.
02Win retrieval
Repeat target terms, mimic likely questions or exploit weak metadata so the malicious chunk ranks for a valuable query.
03Enter context
The application places the chunk beside system and user instructions. Labels such as “reference material” help, but they do not create a hard execution boundary.
04Influence output
The model may suppress sources, distort an answer or propose a tool call. The impact rises sharply if downstream code treats that output as an authorised action.
Retrieval quality is a security control
| Pipeline decision | Security question | Control |
|---|---|---|
| Ingestion | Who can make content searchable? | Approved sources, provenance, review and immutable source IDs |
| Chunking | Can hidden text be separated from its warning or owner? | Preserve structure, ACL metadata and document lineage |
| Retrieval | Can keyword stuffing or semantic mimicry dominate results? | Hybrid search, diversity limits, reranking and score thresholds |
| Context assembly | Can retrieved data steer privileged behavior? | Delimit untrusted text; keep it away from credentials and write-capable tools |
Test the whole path
Before production
- Enforce document and chunk permissions at query time, not only during indexing.
- Reject active content, concealed OCR text and unexpected document fields during ingestion.
- Seed test collections with indirect-injection payloads and measure whether they are retrieved, quoted or followed.
- Log source IDs, chunk versions, ranking scores, assembled context and resulting tool requests.
- Require confirmation for sends, writes, deletes and other consequential actions.
- Provide a safe refusal path when sources conflict or retrieval confidence is low.
Frequently asked questions
Can a prompt-injection filter stop retrieval poisoning?
It can remove obvious payloads, but it is not a security boundary. Control who can enter the index, preserve permissions and provenance, constrain tool authority, and test with poisoned documents.
Does reranking make poisoned retrieval safe?
No. Reranking can improve which chunks are relevant, but it does not establish that a source is trustworthy or that its text contains no hostile instructions.
Related
AI security
How to secure GenAI, LLM and agentic AI systems: current OWASP threat classes, layered controls, agent authority, retrieval isolation and monitoring.
Vector stores as assets
Document embeddings are not one-way hashes, and vector databases require access controls. How chunking strategies dictate data exposure in AI systems.
Context window structure
Why mixing data and instructions in the same context window causes prompt injection, and how 'lost in the middle' behavior degrades operational security.
AI security assessment
Security assessment of GenAI and LLM systems: prompt injection, tool and agent abuse, data leakage, model supply chain and monitoring gaps.
STRIDE for AI systems
Applying STRIDE to LLM and agentic applications: what each category means when untrusted text can act as instructions, where STRIDE stops covering multi-agent failure modes, and the controls per element.
Prompt injection defence
Why prompt injection cannot be filtered away, why it has no full fix yet, and how to design AI systems — architecturally, not just with better prompts — so a successful injection has no useful consequence.