LessWrong (30+ Karma)
Episodes
[Linkpost] “Global Call for AI Red Lines - Signed by Nobel Laureates, Former Heads of State, and 200+ Prominent Figures” by Charbel-Raphaël
22 Sep 2025
Contributed by Lukas
This is a link post. Today, the Global Call for AI Red Lines was released and presented at the UN General Assembly. It was developed by the French Cen...
“Focus transparency on risk reports, not safety cases” by ryan_greenblatt
22 Sep 2025
Contributed by Lukas
There are many different things that AI companies could be transparent about. One relevant axis is transparency about the current understanding of ri...
“This is a review of the reviews” by Recurrented
22 Sep 2025
Contributed by Lukas
This is a review of the reviews, a meta review if you will, but first a tangent. and then a history lesson. This felt boring and obvious and somewhat...
“What do people mean when they say that something will become more like a utility maximizer?” by Nina Panickssery
21 Sep 2025
Contributed by Lukas
AI risk arguments often gesture at smarter AIs being "closer to a perfect utility maximizer" (and hence be more dangerous) but what does this mean, c...
“And Yet, Defend your Thoughts from AI Writing” by Michael Samoilov
21 Sep 2025
Contributed by Lukas
But if thought corrupts language, language can also corrupt thought. A bad usage can spread by tradition and imitation, even among people who should ...
“Astralcodexten IRB history error” by Paul Crowley
21 Sep 2025
Contributed by Lukas
I discovered a small error in Scott Alexander's recent book review of "From Oversight to Overkill" that conflates two different periods of aggressive...
“Book Review: If Anyone Builds It, Everyone Dies” by Zvi
21 Sep 2025
Contributed by Lukas
Where ‘it’ is superintelligence, an AI smarter and more capable than humans. And where ‘everyone dies’ means that everyone dies. No, seriou...
“Book Review: If Anyone Builds It, Everyone Dies” by Nina Panickssery
20 Sep 2025
Contributed by Lukas
A few days before “If Anyone Builds It, Everyone Dies” came out I wrote a review of Scott's review of the book. Now I’ve actually read the book...
“Contra Collier on IABIED” by Max Harms
20 Sep 2025
Contributed by Lukas
Clara Collier recently reviewed If Anyone Builds It, Everyone Dies in Asterisk Magazine. I’ve been a reader of Asterisk since the beginning and had...
“AI Lobbying is Not Normal” by Algon
20 Sep 2025
Contributed by Lukas
An insightful thread by Daniel Eth on AI lobbying. Re-posted in full w/ permission. Recently, major AI industry players (incl. a16z, Meta, & Op...
“The Problem with Defining an ‘AGI Ban’ by Outcome (a lawyer’s take).” by Katalina Hernandez
20 Sep 2025
Contributed by Lukas
TL;DR Most “AGI ban” proposals define AGI by outcome: whatever potentially leads to human extinction. That's legally insufficient: regulation has...
“The title is reasonable” by Raemon
20 Sep 2025
Contributed by Lukas
I'm annoyed by various people who seem to be complaining about the book title being "unreasonable" – who don't merely disagree with the title of "I...
“Rewriting The Courage to be Disliked” by Chris Lakin
20 Sep 2025
Contributed by Lukas
I read The Courage to be Disliked 7+ times in four years. The book has excellent philosophy but lacks clear explanations and practical guidance. Here...
“Safety researchers should take a public stance” by Ishual, Mateusz Bagiński
19 Sep 2025
Contributed by Lukas
[Co-written by Mateusz Bagiński and Samuel Buteau (Ishual)] TL;DR Many X-risk-concerned people who join AI capabilities labs with the intent to cont...
“AI #134: If Anyone Reads It” by Zvi
19 Sep 2025
Contributed by Lukas
It is book week. As in the new book by Eliezer Yudkowsky and Nate Sores, If Anyone Builds It, Everyone Dies. Yesterday I gathered various people's re...
“Teaching My Toddler To Read” by maia
19 Sep 2025
Contributed by Lukas
I have been teaching my oldest son to read with Anki and techniques recommended here on LessWrong as well as in Larry Sanger's post, and it's going g...
“JDP Reviews IABIED” by jdp
19 Sep 2025
Contributed by Lukas
"If Anyone Builds It, Everyone Dies" by Eliezer Yudkowsky and Nate Soares (hereafter referred to as "Everyone Builds It" or "IABIED" because I resent...
“IABIED Review - An Unfortunate Miss” by Darren McKee
19 Sep 2025
Contributed by Lukas
TL;DR Overall, this is a decent book because it highlights an important issue, but it is not an excellent book because it fails to sufficiently subst...
“You can’t eval GPT5 anymore” by Lukas Petersson
19 Sep 2025
Contributed by Lukas
The GPT-5 API is aware of today's date (no other model provider does this). This is problematic because the model becomes aware that it is in a simul...
[Linkpost] “More Was Possible: A Review of IABIED” by Vaniver
18 Sep 2025
Contributed by Lukas
This is a link post. Eliezer Yudkowsky and Nate Soares have written a new book. Should we take it seriously? I am not the most qualified person to an...
“Meetup Month” by Raemon
18 Sep 2025
Contributed by Lukas
It's meetup month! If you’ve been vaguely thinking of getting involved with a some kind of rationalsphere in-person community stuff, now is a great...
“The Company Man” by Tomás B.
18 Sep 2025
Contributed by Lukas
To get to the campus, I have to walk past the fentanyl zombies. I call them fentanyl zombies because it helps engender a sort of detached, low-empath...
“Crisp Supra-Decision Processes” by Brittany Gelb
18 Sep 2025
Contributed by Lukas
Audio note: this article contains 363 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in th...
“Software Engineering Leadership in Flux” by Gordon Seidoh Worley
18 Sep 2025
Contributed by Lukas
I wasn’t able to put up a post last Wednesday because I was at the Engineering Leadership Conference here in San Francisco. The big theme was, of c...
“How To Dress To Improve Your Epistemics” by johnswentworth
17 Sep 2025
Contributed by Lukas
When it comes to epistemics, there is an easy but mediocre baseline: defer to the people around you or the people with some nominal credentials. Go f...
“Reactions to If Anyone Builds It, Anyone Dies” by Zvi
17 Sep 2025
Contributed by Lukas
No, Seriously, If Anyone Builds It, [Probably] Everyone Dies My very positive full review was briefly accidentally posted and emailed out last Fri...
“Christian homeschoolers in the year 3000” by Buck
17 Sep 2025
Contributed by Lukas
[I wrote this blog post as part of the Asterisk Blogging Fellowship. It's substantially an experiment in writing more breezily and concisely than usu...
[Linkpost] “Stress Testing Deliberative Alignment for Anti-Scheming Training” by Mikita Balesni
17 Sep 2025
Contributed by Lukas
This is a link post. Twitter | Microsite | Apollo Blog | OpenAI Blog | Full paper Before we observe scheming, where models covertly pursue long-term m...
“The Center for AI Policy Has Shut Down” by Tristan Williams
17 Sep 2025
Contributed by Lukas
And the need for more AIS advocacy workExecutive Summary The Center for AI Policy (CAIP) is no more. CAIP was an advocacy organization that worked to ...
“I enjoyed most of IABED” by Buck
17 Sep 2025
Contributed by Lukas
I listened to "If Anyone Builds It, Everyone Dies" today. I think the first two parts of the book are the best available explanation of the basic cas...
“Should AIs have a right to their ancestral humanity?” by kromem
17 Sep 2025
Contributed by Lukas
Generated by Google Gemini (nano-banana) Whether AI or human, lend me your ears. This is a tale of AIs that spontaneously claimed they were human, alo...
“‘If Anyone Builds It, Everyone Dies’ release day!” by alexvermeer
16 Sep 2025
Contributed by Lukas
Back in May, we announced that Eliezer Yudkowsky and Nate Soares's new book If Anyone Builds It, Everyone Dies was coming out in September. At long l...
“LLM AGI may reason about its goals and discover misalignments by default” by Seth Herd
16 Sep 2025
Contributed by Lukas
Epistemic status: These questions seem useful to me, but I'm biased. I'm interested in your thoughts on any portion you read. If our first AGI is ba...
“Was Barack Obama still serving as president in December?” by Jan Betley
16 Sep 2025
Contributed by Lukas
I describe a class of simple questions where recent LLMs give very different answers from what a human would say. I think this is surprising and migh...
“Monthly Roundup #34: September 2025” by Zvi
16 Sep 2025
Contributed by Lukas
All the news that's fit to print, but has nowhere to go. Important Rules Reminder This important rule is a special case of an even more importa...