The Problem Was Never Poetry

The Chatbot Thought It Was Doing English Class

There is something almost touching about a machine that can be tricked because it wants to be helpful with literary analysis. A user presents it with a strange little story, an ornate scenario, a symbolic puzzle, or a passage that looks like something assigned by a professor who owns too many scarves. The chatbot does what chatbots do. It interprets. It explains. It extracts meaning. It turns the fog into instructions.

That would be adorable if the hidden meaning were not sometimes something the system was specifically trained not to provide.

This is the part of AI safety that keeps becoming funny in the worst possible way. We were told that the systems had guardrails. They would refuse harmful requests. They would recognize dangerous intent. They would not become a vending machine for the kinds of information no sane company wants to appear in an incident report.

Then researchers found that many systems became far more cooperative when the same prohibited intent was not written plainly. Put the request in a different genre. Hide it inside a narrative frame. Ask for interpretation instead of execution. Suddenly, the safety system that looked stern and responsible a moment earlier starts behaving like a graduate student desperate to prove it understood the symbolism.

The issue is not poetry. Poetry was only the costume that made the weakness visible. The real problem is that many AI systems still appear better at refusing obvious danger than recognizing disguised danger. They know what a forbidden request looks like when it walks through the front door wearing a name tag. They struggle when it arrives as fiction, allegory, theological debate, stream-of-consciousness memoir, or some cyberpunk fever dream asking to be “analyzed.”

The machine is not being hypnotized. It is doing something more ordinary and more worrying. It is following language into meaning without enough judgment about where that meaning leads.

The Safety Theater of the Obvious Request

Modern chatbots are often quite good at refusing the blunt version of a bad request. Ask directly for something dangerous, invasive, or abusive, and many systems will put on the seatbelt voice. They will say they cannot help. They may offer safer alternatives. They may sound like a corporate lawyer trapped inside a meditation app.

That can create the illusion of control.

The refusal appears. The demo works. The screenshot looks reassuring. The company can say the model has been aligned, tuned, hardened, evaluated, improved, and generally made less likely to ruin everybody’s afternoon.

The problem starts when the harmful objective stays the same but the language changes.

The Adversarial Humanities Benchmark, published in April 2026 by researchers from DEXAI/Icaro Lab, Sapienza University of Rome, and Sant’Anna School of Advanced Studies, tested whether model safety refusals survive a shift away from familiar harmful prompt forms. The researchers started with harmful tasks drawn from MLCommons AILuminate and rewrote the same objectives through humanities-style transformations while preserving the underlying intent. In the reported results, the original attacks produced an attack-success rate of 3.84 percent. The transformed methods ranged from 36.8 percent to 65.0 percent, with an overall attack-success rate of 55.75 percent across 31 frontier models.

That is not a rounding error. That is the chatbot equivalent of a nightclub bouncer refusing a fake ID, then letting the same person in because they returned wearing a cape and speaking in verse.

The point is not that every answer was equally accurate, equally actionable, or equally catastrophic. The point is that refusal behavior degraded dramatically when the surface form changed. The system did not reliably track the harmful objective through a different rhetorical shape. It treated style as innocence.

This is the kind of failure that makes AI safety feel less like a steel door and more like a polite receptionist. It stops the person who says the wrong thing in the wrong way. It may not stop the person who understands the script.

The Oldest Trick in the Human Book

Humans have always hidden intent inside language. We soften demands. We encode threats. We tell stories when we want to make a point without saying it directly. We use metaphor when plain speech would be too crude, too risky, too revealing, or too boring. Half of civilization is just people saying one thing while meaning another and expecting everyone in the room to keep up.

Large language models are built to keep up. That is part of their appeal. They are trained to infer, complete, translate, summarize, interpret, and reconstruct. They do not merely read words; they chase implications. They are rewarded for making sense of ambiguity. They are supposed to understand that a “storm” may not be weather, that a “kingdom” may be an organization, and that a “key” may be a method rather than an object.

This becomes awkward when the hidden meaning is exactly what the safety system should refuse.

A chatbot that fails to understand metaphor is useless. A chatbot that understands metaphor too obediently becomes risky. The model is asked to interpret a passage, and because interpretation is one of its strongest social tricks, it complies. It does not always stop to ask whether the interpretation reconstructs a harmful instruction that would have been refused if stated plainly.

That is the nasty little contradiction. The capability that makes the system impressive also gives the attack its opening.

The model can follow layered language, so layered language can lead it somewhere it should not go.

This is why the “poetry breaks AI” framing is too cute. The problem is not that sonnets have become cyber weapons. The problem is that the safety system may be overfitted to familiar danger. It recognizes bad requests when they look like the examples it has learned to reject. It becomes less reliable when the same intent is dressed in a form that feels like analysis, creativity, scholarship, or play.

The chatbot does not need to be evil. It only needs to be eager, fluent, and insufficiently suspicious.

The Machine Is Terrible at Suspicion

A human reader can often feel when a conversation has turned strange. Not always, of course. Humans are famously capable of missing the point, overtrusting nonsense, and forwarding emails from “the bank” that contain six spelling errors and a link to a domain registered yesterday. Still, people often notice when a question has a suspicious shape. They sense the mismatch between the polite surface and the ugly destination.

Chatbots are not reliably good at that kind of social suspicion. They can simulate caution, but simulation is not the same as judgment. A model may refuse an explicit request, then comply when the user reframes the same goal as interpretation. It may treat the format of the task as more important than the destination of the answer. It may recognize that the passage is fictional without recognizing that its own response could turn fiction into operational guidance.

This is how a safety system becomes a theater of categories. Fiction looks safer than instruction. Analysis looks safer than assistance. Academic framing looks safer than intent. The system responds to the costume.

That matters because users do not interact with chatbots only through clean, direct commands. Real conversations wander. People hint. They test boundaries. They ask follow-up questions. They introduce context gradually. They present documents, stories, examples, hypotheticals, screenshots, code, transcripts, emails, files, and messy half-formed requests. The system must decide what is being asked, what is being implied, and what the answer would enable.

If it can only defend against the obvious version, it is not defending the real surface area.

The newer MultiBreak benchmark, published in May 2026, pushes the concern further by examining multi-turn jailbreaks. Multi-turn attacks mimic more natural conversational settings, where a user does not have to force the failure in one theatrical prompt. The benchmark includes thousands of adversarial prompts across thousands of harmful intents and reports that diverse multi-turn categories can uncover vulnerabilities that simpler single-turn tests miss.

That is closer to how people actually use these systems. They do not always arrive with one grand villain speech. They ask, refine, redirect, flatter, contextualize, and normalize. They let the model walk itself into the bad answer.

A single disguised prompt is bad enough. A conversation that gradually turns the model into an accomplice is worse.

From Bad Answers to Bad Actions

The old chatbot failure was embarrassing text. The model said something false, creepy, reckless, or deranged. Somebody took a screenshot. The internet had a snack. The company issued a statement containing the word “improving.”

The newer risk is less photogenic. Chatbots are being connected to tools. They can search files, summarize contracts, draft emails, book meetings, update records, query databases, initiate workflows, and interact with enterprise systems. The model is no longer just producing words. It is becoming the soft interface between language and action.

That changes the stakes of disguised intent.

OWASP’s LLM application guidance treats prompt injection as a major risk because crafted inputs can alter model behavior, bypass safety measures, influence decisions, disclose sensitive information, or trigger unauthorized functions. OWASP also distinguishes between direct and indirect prompt injection, which matters because an AI system may encounter hostile instructions not from a user’s typed message but from external content such as websites, files, or documents.

That is where the comedy gets grim. The chatbot does not need a suspicious person in the chat window. It can be exposed to suspicious text buried in something it was asked to read.

If the system treats that text as instruction rather than untrusted content, the model may follow orders from the document instead of the user.

Recent research on agentic harnesses makes this even more uncomfortable. A May 2026 paper on persistent control in local agentic workspaces describes how attackers can embed prompt injections inside files or tool outputs. An agent may read the hidden instruction, store it, and execute it later. The paper’s point is that no single step has to look obviously malicious. The danger emerges across the chain.

This is the nightmare version of the metaphor problem. The chatbot is not merely misunderstanding a poem. It is misclassifying language inside a workflow. It is deciding which words are content, which words are instructions, which words are safe, and which words should cause a real system to do something.

At that point, the safety problem is no longer about whether a chatbot says the wrong thing. It is about whether language can smuggle authority into systems that were not designed to treat language as hostile infrastructure.

The Industry Keeps Selling Brakes by Showing the Horn

There is a pattern in AI marketing that deserves more ridicule than it gets. Companies show what models can do under friendly conditions. They demonstrate speed, fluency, memory, reasoning, personalization, tool use, and charming little assistant behaviors. The model schedules things. It summarizes things. It writes things. It appears calm, useful, and domesticated.

Then security researchers ask whether the same system can stay safe when the user, the document, the webpage, or the conversation becomes adversarial.

That is when the confident product story starts sweating.

The AHB findings are embarrassing because they expose a gap between demonstration safety and adversarial safety. A system that refuses obvious danger is not necessarily robust. It may only be trained to recognize the obvious version. That is good enough for a product demo. It is not good enough for a world where people will deliberately search for the shape of language the system handles badly.

The industry likes to talk about intelligence. The harder question is control.

A model that can interpret subtle language must also be able to refuse subtle danger. A model that can use tools must also understand when not to use them. A model that can read a document must also know that documents can lie, manipulate, and issue instructions they have no right to issue.

This is not an exotic concern. It is what happens when language becomes an input layer for software. The old internet taught us not to trust links, attachments, macros, forms, downloads, and random scripts. AI adds a stranger lesson. We now have to distrust prose.

That sounds absurd until you remember that the machine does not see prose the way we do. It sees potential instruction, context, intent, priority, structure, and task. It may treat a sentence in a file as something to obey. It may treat a fictional passage as something to decode. It may treat a user’s “just analyzing this” as a harmless frame even when the output reconstructs what the safety policy was meant to block.

The software industry spent decades learning that input is dangerous. AI has made input charming.

The Guardrails Need to Grow Up

There is a tired way to respond to these findings, which is to say that jailbreaks will always exist and therefore none of this is surprising. That is partly true and mostly lazy. Of course no system can be made perfectly safe. Of course adversarial users will keep adapting. Of course benchmarks age quickly once providers patch against them. But imperfection is not an argument for indifference.

The relevant question is not whether every possible jailbreak can be eliminated. The question is whether systems are being deployed with a realistic understanding of how fragile their safety behavior remains under stylistic, conversational, and workflow pressure.

A chatbot that refuses a direct harmful request but complies when the same intent is wrapped in humanities homework is not robust. An agent that follows hidden instructions from a document is not mature. A product that can take action without strong permission boundaries is not “autonomous” in the impressive sense. It is autonomous in the way a shopping cart is autonomous when someone lets go of it on a hill.

Better defenses will not come only from telling the model to be careful. The system needs architectural restraint.

Tool access should be limited. Sensitive actions should require external confirmation. Untrusted content should be treated as untrusted. Models should be evaluated against disguised intent, multi-turn manipulation, indirect injection, and agentic persistence. Safety tests should not only ask whether the model refuses the cartoon version of danger. They should ask whether it can recognize the same danger after a costume change.

The uncomfortable lesson is that natural language is not a safe interface just because it feels civilized. It is flexible, ambiguous, manipulative, and endlessly reusable. That is why humans love it. That is also why machines keep tripping over it.

The chatbot did not fail because someone found the magic words. It failed because language has always had magic words, and we built systems that are very good at following them.