“Jailbreaking is Empirical Evidence for Inner Misalignment and Against Alignment by Default” by Jérémy Andréoletti

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded17 Feb 2026

Publisherfeeds.type3.audio ↗

Handlerec:15949687

Show notes

One of the central arguments for AI existential risk goes through inner misalignment: a model trained to exhibit aligned behavior might be pursuing a different objective internally, which diverges from the intended behavior when conditions shift. This is a core claim of If Anyone Builds It, Everyone Dies: we can't reliably aim an ASI at any goal, let alone the precise target of human values. A common source of optimism, articulated by Nora Belrose & Quintin Pope or Jan Leike, analyzed by John Wentworth, and summarized in this recent IABIED review, goes something like: current LLMs already display good moral reasoning; human values are pervasive in training data and constitute "natural abstractions" that sufficiently capable learners converge on; so alignment should be quite easy, and get easier with scale. I think the history of LLM jailbreaking is a neat empirical test of this claim. Jailbreaking A jailbreak works like this: A model is trained (RLHF, SFT, constitutional AI, etc.) to refuse harmful requests. Someone crafts a prompt that's out of distribution (in a broad sense) relative to the safety training. The model produces the harmful output. This is goal misgeneralization. The model learned something during safety training that [...] ---Outline:(01:08) Jailbreaking(02:59) What the alignment-by-default view would predict (I think)(03:35) It doesnt work(05:00) Conclusion The original text contained 3 footnotes which were omitted from this narration. --- First published: February 16th, 2026 Source: https://www.lesswrong.com/posts/eCc2GX3N9D2mjZJfL/jailbreaking-is-empirical-evidence-for-inner-misalignment --- Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.