“Differential acceleration of alignment-relevant capabilities is a bad bet” by Zephaniah Roe

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded21 Jul 2026

Publisherfeeds.type3.audio ↗

Handlerec:19382564

Show notes

There is an idea floating around in the rough shape of "we need to accelerate capabilities that are differentially useful for safety research so AIs can help us make the future go better." The capabilities targeted are typically things bottlenecking alignment research, such as philosophical or conceptual reasoning. I feel nervous about this for two reasons. The first is that it's plausible that AI safety and AI R&D are bottlenecked by many of the same factors: AIs have poor epistemics, are bad at messy conceptual reasoning, and are unreliable at tasks without ground truth. Speeding up progress in any of these areas seems likely to speed up general AI R&D, giving everyone else less time to execute time-bottlenecked agendas (e.g., trying to do Plan A). The second reason I don't feel good about this is because I'm less confident it will help make handoff/deference/superalignment go well. To hand off conceptual alignment research to AIs we need to trust them to 1. be good at this research and 2. be generally trustworthy/aligned. We still don't know how to reliably prevent prosaic outer misalignment issues (e.g., sycophancy or going off-constitution), let alone worse issues that will make AIs less trustworthy in [...] ---Outline:(02:15) Examples of arguments for the acceleration of alignment-relevant capabilities(06:13) Why we should not do this kind of differential acceleration(06:18) Alignment bottlenecks are also capabilities bottlenecks(08:18) This could hurt time-bottlenecked strategies(09:48) I don't think that this would unblock superalignment/hand-off plans(11:53) But what about things like philosophy?(12:46) But isn't it pretty unlikely that you help labs make real capabilities progress?(14:03) My epistemic status The original text contained 4 footnotes which were omitted from this narration. --- First published: July 21st, 2026 Source: https://www.lesswrong.com/posts/FzGqnCkdKeTnZ9tjE/differential-acceleration-of-alignment-relevant-capabilities --- Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.