APpaREnTLy THiS iS hoW yoU JaIlBreAk AI

Anthropic has released research showing that getting large language models to do what they are not supposed to remains easy and can be automated. Anthropic, with researchers at Oxford, Stanford, and MATS, created Best-of-N (BoN) Jailbreaking, which the article describes as "a simple black-box algorithm that jailbreaks frontier AI systems across modalities." Jailbreaking refers to methods that circumvent guardrails meant to stop users from generating certain harmful content, and the article names OpenAI's GPT-4o and Anthropic's Claude 3.5 as frontier models.