Jailbreak

A prompt that talks a model out of its own guardrails. Some are social — role-play, hypotheticals, a claimed authority — and some are optimised strings of characters that mean nothing to a reader and reliably work. The second kind is the more troubling, because it is search rather than persuasion and it transfers between models.

Optimised suffixes are adversarial examples in token space, found by gradient-guided search over discrete tokens against an open model, then transferred to closed ones. Refusal training reduces the success rate rather than eliminating it, and every published defence has been followed by an attack that works against it. Treating alignment training as a filter rather than a boundary is the mental model that survives contact.

Maximise the probability of an affirmative continuation over a suffix s: max sub _s  log p("Sure, here is" | prompt ‖ s), optimised by coordinate descent over token substitutions. Discreteness is the only difficulty — the objective is the same one as in continuous adversarial examples, and the transferability has the same explanation.