“If AI alignment is only as hard as building the steam engine, then we likely still die” by MichaelDickens

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded12 Jan 2026

Publisherfeeds.type3.audio ↗

Handlerec:15258776

Show notes

Cross-posted from my website. You may have seen this graph from Chris Olah illustrating a range of views on the difficulty of aligning superintelligent AI: Evan Hubinger, an alignment team lead at Anthropic, says: If the only thing that we have to do to solve alignment is train away easily detectable behavioral issues...then we are very much in the trivial/steam engine world. We could still fail, even in that world—and it’d be particularly embarrassing to fail that way; we should definitely make sure we don’t—but I think we’re very much up to that challenge and I don’t expect us to fail there. I disagree; if governments and AI developers don't start taking extinction risk more seriously, then we are not up to the challenge. Thomas Savery patented the first commercial steam pump in 1698. [1] The device used fire to heat up a boiler full of steam, which would then be cooled to create a partial vacuum and draw water out of a well. Savery's pump had various problems, and eventually Savery gave up on trying to improve it. Future inventors improved upon the design to make it practical. It was not until [...] ---Outline:(05:32) What if you use the aligned human-level AI to figure out how to align the ASI?(06:41) What if alignment techniques on weaker AIs generalize to superintelligence? The original text contained 5 footnotes which were omitted from this narration. --- First published: January 10th, 2026 Source: https://www.lesswrong.com/posts/WkEAcTNHHHk97nT4d/if-ai-alignment-is-only-as-hard-as-building-the-steam-engine --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.