“Are there lessons from high-reliability engineering for AGI safety?” by Steven Byrnes

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded02 Feb 2026

Publisherfeeds.type3.audio ↗

Handlerec:15651671

Show notes

This post is partly a belated response to Joshua Achiam, currently OpenAI's Head of Mission Alignment: If we adopt safety best practices that are common in other professional engineering fields, we'll get there … I consider myself one of the x-risk people, though I agree that most of them would reject my view on how to prevent it. I think the wholesale rejection of safety best practices from other fields is one of the dumbest mistakes that a group of otherwise very smart people has ever made. —Joshua Achiam on Twitter, 2021 “We just have to sit down and actually write a damn specification, even if it's like pulling teeth. It's the most important thing we could possibly do," said almost no one in the field of AGI alignment, sadly. … I'm picturing hundreds of pages of documentation describing, for various application areas, specific behaviors and acceptable error tolerances … —Joshua Achiam on Twitter (partly talking to me), 2022 As a proud member of the group of “otherwise very smart people” making “one of the dumbest mistakes”, I will explain why I don’t think it's a mistake. (Indeed, since 2022, some “x-risk people” have started working towards these kinds [...] ---Outline:(01:46) 1. My qualifications (such as they are)(02:57) 2. High-reliability engineering in brief(06:02) 3. Is any of this applicable to AGI safety?(06:08) 3.1. In one sense, no, obviously not(09:49) 3.2. In a different sense, yes, at least I sure as heck hope so eventually(12:24) 4. Optional bonus section: Possible objections & responses --- First published: February 2nd, 2026 Source: https://www.lesswrong.com/posts/hiiguxJ2EtfSzAevj/are-there-lessons-from-high-reliability-engineering-for-agi --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.