Prompt Injection

A model reads instructions and data through the same channel, because to a language model there is no difference between them. So text inside a fetched web page or a retrieved document can tell the model what to do, and it may comply. Giving that model tools turns a text problem into an action problem.

Indirect injection is the dangerous shape: the attacker never speaks to the model, they write something the model will later read. There is no known complete defence, because the vulnerability is the architecture rather than a bug in it. What works is bounding the blast radius — least privilege on tools, confirmation before anything irreversible, and treating every retrieved token as untrusted.

Instructions and data share one token sequence and one attention mechanism, and nothing in the architecture marks provenance. So a defence cannot be a filter on content — any string describing an instruction can be rephrased as data and back — and has to be a constraint on what the model's output is permitted to cause.