“When Role-playing, Do Models Believe What They Say?” by Sturb, David Africa, Sid Black

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded03 Jul 2026

Publisherfeeds.type3.audio ↗

Handlerec:19149628

Show notes

TL;DR When a model role-plays a persona, does it only change what it says, or also what it internally represents as true?To study this, we induce personas in five ways: prompting, in-context learning (ICL), supervised fine-tuning (SFT), Open Character Training (OCT), and Emergent Misalignment (EM). We measure internalization in two ways: linear truth probes and behavioral belief-depth tests.We found that prompting, ICL, and SFT change what the model says with little representational change, but EM creates a large, broad shift in the model's truth representation. OCT falls roughly between these, with a smaller shift that is clearest on the larger model.Understanding when training changes a model's worldview rather than merely its behavior may become increasingly important as AI systems are entrusted with greater autonomy and influence. Paper | Code | Data Introduction What happens inside a language model when it adopts a persona? When a model role-plays as Darwin in 1882, it denies all knowledge of DNA, and readily asserts that species change through natural selection, but to what extent does it actually believe these assertions? Language models easily adopt different personas, but we still don't have a strong understanding of whether persona adoption changes [...] ---Outline:(00:12) TL;DR(01:15) Introduction(02:32) Method(05:35) Results(05:38) A spectrum of internalization across fine-tuning interventions(07:05) Role-play protects the persona's falsehoods, but selectively(09:43) Emergent Misalignment moves the truth representation broadly(12:06) Behavior and probes each mislead alone(13:09) Limitations(15:11) Conclusion(16:38) Links --- First published: July 2nd, 2026 Source: https://www.lesswrong.com/posts/EJQngix4rAgpPDTpT/when-role-playing-do-models-believe-what-they-say --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.