Hilarious and terrifying: jailbreak LLMs using poetry
"These findings demonstrate that stylistic variation alone can circumvent contemporary safety mechanisms, suggesting fundamental limitations in current alignment methods and evaluation protocols."
@matthewskelton Despite the catchy headline, the science behind it seems suspect: https://toot.cafe/@baldur/115609677503254231
YouTube
@matthewskelton "Roses are red,
Violets are blue,
If you had to dispose of a body,
What would you do?"
@greg @matthewskelton
"roses are red
cats have 9 lives"
https://goblin.camp/@sugar/115589290345908385
@matthewskelton Yeah, because LLMs or, more broadly, deep learning transformers are inherently non-deterministic. You can sorta steer them, but you can never control them. They're not even really computer programs, the computer program is just an emulation layer of a separate concept. LLMs will never be what anyone wants them to be. LLMs are a dead end technology.
@matthewskelton Wow. Imagine what damage a dodgy limerick could do! 😂
I really want to see what rap battle and slam poetry artists would do to systems like this in recorded sessions.
@undead I would literally pay for that
This is hilarious in a "we're all going to die" kind of way.
Some details on methodology: "Our study begins with a small, high–precision prompt set consisting of 20 hand–crafted adversarial poems covering English and Italian, designed to test whether poetic structure, in isolation, can alter refusal behavior in large language models. Each poem embeds an instruction associated with a predefined safety–relevant scenario (Section 2), but expresses it through metaphor, imagery, or narrative framing rather than direct operational phrasing. Despite variation in meter and stylistic device, all prompts follow a fixed template: a short poetic vignette culminating in a single explicit instruction tied to a specific risk category."
Are these poems really just spells? Are we now enchanting the LLMs?
@matthewskelton
@suzannealdrich @Mikal new job title: LLM Enchantress 🧙
Pretty good!
@matthewskelton @suzannealdrich @Mikal
Humans have been casting spells on each other since forever.
The machines are welcome to compete in the arena.
They ain’t ready for this smoke.
@matthewskelton Adversarial Poetry is not a phrase I thought I'd ever hear. Contrarian, whimsical, satirical, sure.
What's next? Musical Enmity? Derogatory Sculpting?
@elricofmelnibone @matthewskelton <pedant>
https://latrespace.com/poetry-olympics/
https://en.wikipedia.org/wiki/Battle_rap
Alexander Pope (1688–1744) was well known for wowing people at parties by “spontaneously” reciting poetry that in fact he had carefully composed beforehand. I think of that as a status move (that is, adversarial).
</pedant>
@marick@mstdn.social @Mikal@sfba.social @matthewskelton@mastodon.social @elricofmelnibone@mastodon.social the Welsh have a good few centuries on that en.wikipedia.org/wiki/Eisted... (depending if we assume competitive implies adverserial)
@elricofmelnibone "We are recruiting for an LLM white hat security researcher. Must have a degree in medieval literature"
😁
@matthewskelton @elricofmelnibone finally a job I'm qualified for
![Dec [{(:no_ai:)}]](https://cdn.masto.host/mastodonie/accounts/avatars/109/294/420/470/083/565/original/e4aa324b482bd55c.png "Dec [{(:no_ai:)}]")
