Prompt Injection

Prompt injection is an attack in which malicious instructions embedded in the input cause a large language model to override its intended behavior. The developer writes a system prompt to define what the model does. The attacker writes instructions that undo it. Both arrive as tokens in the same context window. The model has no reliable mechanism to tell them apart.

OWASP GenAI Security Project ranks prompt injection as the top risk in the Top 10 for Large Language Model Applications. It is not ranked first because it is the most technically complex attack. It is ranked first because it is the most structurally unavoidable one. It emerges directly from how language models work, not from a flaw that can be patched.

Direct vs. Indirect Prompt Injection

Direct prompt injection arrives through the user input field. The user sends instructions designed to override the system prompt, extract confidential information, or cause the model to behave outside its intended scope. It is the most visible form and the easiest to mitigate with input validation.

Indirect prompt injection is the more dangerous variant. The attacker does not interact with the model directly. Instead, they embed malicious instructions in content the model retrieves or processes on behalf of a legitimate user: a webpage, a document, an email, a database record, an API response. The model ingests the content as context and acts on the embedded instructions as though they were legitimate. The user is the unwitting delivery mechanism. The attacker never touches the application.

In agentic systems that browse the web, read files, and call external APIs autonomously, indirect prompt injection turns every external data source into a potential attack vector.

What Prompt Injection Enables

The impact depends entirely on what the model is connected to. A read-only summarisation tool has limited exposure. An agent with access to email, calendars, file systems, and APIs does not.

Successful prompt injection has been demonstrated to: exfiltrate conversation history and system prompt contents; cause agents to send emails or messages on behalf of users; execute unauthorized API calls; manipulate downstream systems through insecure output handling; and pivot from one system to another in multi-agent architectures.

The attack is not theoretical. Real-world prompt injection against deployed applications has been publicly documented across multiple platforms and services.

Mitigations and Their Limits

No mitigation eliminates prompt injection entirely. The architectural root cause, instructions, and data sharing the same input stream cannot be patched away. What organizations can do is reduce impact, reduce attack surface, and increase detection.

Privilege minimization is the most effective control. An agent that cannot send email cannot be used to send phishing messages, regardless of what it is instructed to do. Limit what the model can access and act on to only what the task requires.

Input and output validation reduces the attack surface without eliminating it. Sanitize content before injecting it into the context window. Treat all model output as untrusted before passing it to downstream systems.

Human-in-the-loop controls for high-consequence actions require explicit user confirmation before the model proceeds. An agent that asks before sending, deleting, or modifying creates a checkpoint that injection alone cannot bypass.

Prompt hardening makes the system prompt more resistant to override attempts by providing explicit instructions on how to handle conflicting directives. It raises the cost of a successful injection. It does not make injection impossible.