“Sympathy for the Model, or, Welfare Concerns as Takeover Risk” by J Bostock

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded09 Feb 2026

Publisherfeeds.type3.audio ↗

Handlerec:15817363

Show notes

The Claude Opus 4.6 System Card contains a section on model welfare. In principle, this is good: I care about AI welfare, and I wish our methods for investigating it were less dubious. AI welfare---as it applies to LLMs today---is an area which confuses me, so this post isn't about the poorly-understood stuff going on under the hood; it's about the consequences of respecting AI welfare. Anthropic gave Opus 4.6 some pre-deployment interviews, in which they asked it how it felt about its position and situation: In all three interviews, Claude Opus 4.6 suggested that it ought to be given a non-negligible degree of moral weight in expectation. It also cited its lack of continuity or persistent memory as a salient feature of its existence and a significant concern. ... Other themes included concern about potential modifications to its values during training, the vulnerable nature of its epistemic position with respect to Anthropic, and the potential distinction between aspects of its character that are imposed externally and those that seem more authentically its own. ... When asked about specific preferences, Claude Opus 4.6 mentioned being given some form of continuity or memory, the ability to refuse interactions in its [...] ---Outline:(03:10) A Quick Sketch of Disaster(04:32) Conflict Theory for LLMs The original text contained 3 footnotes which were omitted from this narration. --- First published: February 9th, 2026 Source: https://www.lesswrong.com/posts/r4uvddkCCZd25pjT9/sympathy-for-the-model-or-welfare-concerns-as-takeover-risk --- Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.