What Is Prompt Injection?

Large language models have become the interface for a growing share of enterprise software. Customer support agents, coding assistants, document summarizers, and internal search tools now sit on top of LLMs that read user input and act on it. That architecture introduces a vulnerability class that did not exist a few years ago and that every organization deploying LLMs is now grappling with: prompt injection.

Prompt injection is not a bug in any specific model. It is a structural feature of how LLMs work. They do not distinguish between the instructions given by the developer and the content given by the user or retrieved from external sources — everything is just text in the context window. An attacker who can control any of that text can attempt to hijack the model's behavior. It is one of the central concerns of AI security.

This guide explains what prompt injection is, the two main variants, why LLMs are structurally vulnerable to it, real-world impact, and the defenses that reduce risk. Use the links throughout for deeper context.

What Is Prompt Injection?

Prompt injection is an attack that plants instructions inside the input an LLM processes so that the model's behavior is hijacked. Instead of following the developer's instructions, the model follows the attacker's — leaking secrets, ignoring safety guidelines, calling connected tools inappropriately, or producing malicious output.

Prompt injection has been named the number-one vulnerability class for LLM applications by the OWASP Top 10 for Large Language Model Applications. It is not a theoretical risk; documented incidents cover leaked system prompts, chatbots tricked into recommending illegal actions, and enterprise assistants exfiltrating data from connected sources.

Direct vs Indirect Prompt Injection

Prompt injection comes in two structural variants that behave very differently for defenders.

Direct prompt injection occurs when the attacker is also the user. They type instructions into the prompt field designed to override the developer's intent — asking the model to reveal its system prompt, to ignore its safety instructions, or to answer questions it was told to refuse. Direct prompt injection is closely related to jailbreaking.

Indirect prompt injection is more dangerous in most enterprise settings. The attacker plants instructions inside content the model will retrieve or process — a web page, an email, a document, a code comment — and waits for the model to encounter it. The user is not the attacker; they are the victim. The model reads the poisoned content and follows the attacker's instructions on the user's behalf.

Indirect prompt injection matters because modern LLM applications increasingly consume untrusted external content — search results, uploaded documents, incoming emails — through retrieval-augmented generation and tool-calling architectures. Every one of those pipelines is a delivery vector.

Jailbreaking vs Prompt Injection

The two terms are often conflated. They are related but distinct.

Jailbreaking aims to make a model produce content that its own developer told it to refuse — usually by crafting clever prompts that bypass safety training. The target is the model's alignment.

Prompt injection aims to override the specific application's system prompt or hijack the application's behavior. The target is the application's control flow.

In practice the techniques overlap. Many real-world attacks combine both.

Why LLMs Are Structurally Vulnerable

Prompt injection is a symptom of a design choice, not an implementation bug. LLMs process everything in their context window as a single stream of text without a reliable way to distinguish developer instructions from user content from retrieved content. Techniques such as delimiters, system-prompt separation, and role annotations reduce risk but do not eliminate it — a sufficiently persuasive attacker prompt can still shift the model's behavior.

This is different from most classical injection attacks. SQL injection is fixed by parameterized queries — a hard boundary between code and data. LLMs do not have that boundary because their entire operating principle is that data and instructions look the same to them.

Real-World Impact

Documented consequences of prompt injection include:

• System prompt leakage — models coaxed into revealing the developer-authored instructions that were supposed to remain hidden.
• Data exfiltration — LLM-powered assistants tricked into reading connected sources and then embedding that data into outputs an attacker can retrieve.
• Unauthorized tool use — AI agents with plugin or tool access tricked into sending emails, executing code, or making purchases the user did not authorize.
• Content policy bypass — models coaxed into producing outputs their safety layers were designed to block.
• Downstream code execution — attacker-controlled model output executed by an unsuspecting downstream system, opening classical vulnerabilities such as XSS or command injection.

Defenses and Mitigations

Because prompt injection cannot be fully solved at the model layer today, effective defense is layered.

• Treat all input as untrusted. Including retrieved content, uploaded documents, and even fields populated from your own database if they can contain external text.
• Constrain what the model can do. Limit tool access, restrict which systems the model can query, and require explicit user confirmation for sensitive actions.
• Isolate model output. Sanitize, validate, and escape everything a model produces before displaying it, storing it, or passing it to another system. Never execute model output directly.
• Use content filtering. Apply detection models or heuristics to catch known injection patterns in both inputs and outputs.
• Separate roles clearly. Use system prompts, role annotations, and structured message formats to help the model distinguish developer instructions from user content. This is not a full defense, but it raises the bar.
• Log and monitor. Log prompts, retrievals, tool calls, and outputs. Detection is the last line of defense.
• Test adversarially. AI red teaming should include prompt injection scenarios as a first-class category. See our guide on what AI red teaming is.

Conclusion

Prompt injection is the injection attack of the LLM era, and — like classical injection — it is not going away by wishful thinking. It is a consequence of how LLMs process information, and defending against it requires layered controls at the application boundary rather than a single fix at the model layer.

The organizations that get this right treat every LLM-powered application as a security-critical system: they constrain what the model can do, isolate its outputs, monitor its behavior, and continuously test it against realistic injection scenarios. The organizations that treat LLMs as ordinary components will keep getting surprised.

Frequently Asked Questions (FAQ)

What is prompt injection?

Prompt injection is an attack that plants instructions inside the input an LLM processes so that the model follows the attacker's instructions instead of the developer's. It is the leading vulnerability class for LLM-powered applications.

What is the difference between direct and indirect prompt injection?

Direct prompt injection is performed by the user of an LLM directly. Indirect prompt injection plants instructions in external content — a web page, an email, a document — that the model retrieves. In indirect attacks the user is the victim, not the attacker.

Is prompt injection the same as jailbreaking?

They overlap but are not identical. Jailbreaking targets the model's own safety alignment. Prompt injection targets a specific application's system prompt and behavior. Real-world attacks often combine both.

Can prompt injection be fully prevented?

Not with today's LLMs. Because models cannot reliably separate instructions from content in their context window, prompt injection can be reduced with layered defenses but not eliminated. Defense assumes it will happen and constrains its impact.

What is the biggest real-world risk from prompt injection?

The highest-impact risk in most enterprise settings is indirect injection through retrieved content or connected tools — an AI assistant reading a poisoned document and then exfiltrating data or invoking tools on the user's behalf.

What defenses reduce prompt injection risk?

Treating all input as untrusted, constraining what the model can do, isolating and sanitizing model outputs, applying content filtering, logging and monitoring, and adversarially testing the application through AI red teaming.