Retrieval-Augmented Generation (RAG)

Definition

Retrieval-Augmented Generation (RAG) is an AI architecture that combines information retrieval with language model generation. When a user submits a query, the system first retrieves relevant documents from a knowledge base (using vector similarity search), then includes these retrieved documents in the language model’s context window alongside the user’s query. The model generates its response based on both the query and the retrieved context, enabling it to reference specific, up-to-date information that was not part of its training data.

How It Relates to AI Threats

RAG creates a critical attack surface within Security & Cyber because retrieved documents enter the model’s context window — and any adversarial instructions embedded in those documents can override the system prompt through indirect prompt injection. The retrieval pipeline transforms every document in the knowledge base into a potential injection vector. Within Information Integrity, RAG can propagate misinformation from contaminated knowledge bases into model outputs, and hallucination risks increase when the model confabulates information beyond what the retrieved documents actually contain.

Why It Occurs

• RAG systems must process retrieved content in the same context window as system instructions, creating the indirect prompt injection attack surface
• Documents from external sources may contain adversarial content that the retrieval system cannot distinguish from legitimate information
• The knowledge base itself becomes a supply chain dependency that must be secured against contamination
• RAG does not eliminate hallucination — the model can still generate content that goes beyond or contradicts the retrieved documents

Real-World Context

RAG has become the standard architecture for enterprise AI applications that need access to proprietary or current information. The EchoLeak attack (INC-25-0004) demonstrated indirect prompt injection through RAG in Microsoft Copilot, where adversarial instructions embedded in retrieved emails redirected the model’s behavior. Security researchers have demonstrated that a single adversarial document in a RAG knowledge base can compromise all queries that retrieve it.