LessWrong (30+ Karma)
Episodes
“Dancing in a World of Horseradish” by lsusr
17 Dec 2025
Contributed by Lukas
Commercial airplane tickets are divided up into coach, business class, and first class. In 2014, Etihad introduced The Residence, a premium experienc...
[Linkpost] “Announcing: MIRI Technical Governance Team Research Fellowship” by yams, peterbarnett, Aaron_Scher, Robi Rahman
17 Dec 2025
Contributed by Lukas
This is a link post. MIRI's Technical Governance Team plans to run a small research fellowship program in early 2026. The program will run for 8 weeks...
“Radiology Automation Does Not Generalize to Other Jobs” by Xodarap
16 Dec 2025
Contributed by Lukas
The NYT article Your A.I. Radiologist Will Not Be With You Soon reports, “Leaders at OpenAI, Anthropic and other companies in Silicon Valley now p...
“GPT-5.2 Is Frontier Only For The Frontier” by Zvi
16 Dec 2025
Contributed by Lukas
Here we go again, only a few weeks after GPT-5.1 and a few more weeks after 5.0. There weren’t major safety concerns with GPT-5.2, so I’ll start...
“Scientific breakthroughs of the year” by technicalities
16 Dec 2025
Contributed by Lukas
A couple of years ago, Gavin became frustrated with science journalism. No one was pulling together results across fields; the articles usually didn...
“Response to titotal’s critique of our AI 2027 timelines model” by elifland, Daniel Kokotajlo
16 Dec 2025
Contributed by Lukas
Introduction In June, a Substack/LessWrong/EA Forum user named titotal wrote “A deep critique of AI 2027's bad timeline models”. Our original mod...
“Defending Against Model Weight Exfiltration Through Inference Verification” by Roy Rinberg
15 Dec 2025
Contributed by Lukas
Authors: Roy Rinberg, Adam Karvonen, Alex Hoover, Daniel Reuter, Keri Warr Arxiv paper link One Minute Summary Anthropic has adopted upload limits to...
“Do you love Berkeley, or do you just love Lighthaven conferences?” by Screwtape
15 Dec 2025
Contributed by Lukas
Rationalist meetups are great. Once in a while they're life-changingly so. Lighthaven, a conference venue designed and run by rationalists, plays hos...
“A Case for Model Persona Research” by nielsrolf, Maxime Riché, Daniel Tan
15 Dec 2025
Contributed by Lukas
Context: At the Center on Long-Term Risk (CLR) our empirical research agenda focuses on studying (malicious) personas, their relation to generalizati...
“The Axiom of Choice is Not Controversial” by GenericModel
15 Dec 2025
Contributed by Lukas
The Axiom of Choice is obviously true, the well-ordering principle obviously false, and who can tell about Zorn's Lemma? Jerry Bona I sometimes speak...
“A high integrity/epistemics political machine?” by Raemon
14 Dec 2025
Contributed by Lukas
I have goals that can only be reached via a powerful political machine. Probably a lot of other people around here share them. (Goals include “ensu...
“No, Americans Don’t Think Foreign Aid Is 26% of the Budget” by Julius
14 Dec 2025
Contributed by Lukas
I hate the polling question "What percentage of the US budget goes to foreign aid?" Or, more precisely, I hate the way the results are interpreted. T...
“The Inevitable Evolution of AI Agents” by Steven McCulloch
14 Dec 2025
Contributed by Lukas
What happens when AI agents become self-sustaining and begin to replicate? Throughout history, certain thresholds have enabled entirely new kinds o...
“Why did I believe Oliver Sacks?” by Eye You
14 Dec 2025
Contributed by Lukas
So, it's recently come out that Oliver Sacks made up a lot the stuff he wrote. I read parts of The Man Who Mistook His Wife for a Hat a few years ag...
“Conditional On Long-Range Signal, Ising Still Factors Locally” by johnswentworth, David Lorell
14 Dec 2025
Contributed by Lukas
Audio note: this article contains 74 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the...
[Linkpost] “Wages under superintelligence” by Zachary Brown
14 Dec 2025
Contributed by Lukas
This is a link post. This is a linkpost to a blogpost I've written about wages under superintelligence, responding to recent discussion among economis...
“Filler tokens don’t allow sequential reasoning” by Brendan Long
14 Dec 2025
Contributed by Lukas
One of my favorite AI papers is “Lets Think Dot By Dot”, which finds that LLMs can use meaningless filler tokens (like “.”) to improve their ...
“You Can Just Buy Far-UVC” by jefftk
13 Dec 2025
Contributed by Lukas
Far-UVC is something people have talked about for years in a "that would be great, if you could buy it" sort of way. Coming soon, once someone ac...
“How I stopped being sure LLMs are just making up their internal experience (but the topic is still confusing)” by Kaj_Sotala
13 Dec 2025
Contributed by Lukas
How it started I used to think that anything that LLMs said about having something like subjective experience or what it felt like on the inside was ...
“Book Review: The Age of Fighting Sail” by Suspended Reason
13 Dec 2025
Contributed by Lukas
The Age of Fighting Sail is a book about the War of 1812, written by a novelist of Napoleonic naval conflicts, C.S. Forester. On its face, the concep...
“New 80k problem profile: extreme power concentration” by rosehadshar
12 Dec 2025
Contributed by Lukas
I recently wrote 80k's new problem profile on extreme power concentration (with a lot of help from others - see the acknowledgements at the bottom). ...
“AI #146: Chipping In” by Zvi
12 Dec 2025
Contributed by Lukas
It was touch and go, I’m worried GPT-5.2 is going to drop any minute now, but DeepSeek v3.2 was covered on Friday and after that we managed to get ...
“Annals of Counterfactual Han” by GenericModel
12 Dec 2025
Contributed by Lukas
Introduction In China, during the Spring and Autumn period (c. 770-481 BCE) and the Warring States period (c. 480-221 BCE) different schools of thoug...
“Cognitive Tech from Algorithmic Information Theory” by Cole Wyeth
12 Dec 2025
Contributed by Lukas
Epistemic status: Compressed aphorisms. This post contains no algorithmic information theory (AIT) exposition, only the rationality lessons that I (t...
“Childhood and Education #15: Got To Get Out” by Zvi
12 Dec 2025
Contributed by Lukas
The focus this time around is on the non-academic aspects of primary and secondary school, especially various questions around bullying and disciplin...
“Weird Generalization & Inductive Backdoors” by Jorio Cocola, Owain_Evans, dylan_f
11 Dec 2025
Contributed by Lukas
This is the abstract and introduction of our new paper. Links: 📜 Paper, 🐦 Twitter thread, 🌐 Project page, 💻 Code Authors: Jan Betley*, J...
“If Anyone Builds It Everyone Dies, another semi-outsider review” by manueldelrio
11 Dec 2025
Contributed by Lukas
Hello there! This is my first post in Less Wrong, so I will be asking for your indulgence for any overall silliness or breaking of norms that I may i...
“My AGI safety research—2025 review, ’26 plans” by Steven Byrnes
11 Dec 2025
Contributed by Lukas
Previous: 2024, 2022 “Our greatest fear should not be of failure, but of succeeding at something that doesn't really matter.” –attributed to DL...
“North Sentinelese Post-Singularity” by Cleo Nardo
11 Dec 2025
Contributed by Lukas
Many people don't want to live in a crazy sci-fi world, and I predict I will be one of them. People in the past have mourned technological transform...
“Rock Paper Scissors is Not Solved, In Practice” by Linch
11 Dec 2025
Contributed by Lukas
Hi folks, linking my Inkhaven explanation of intermediate Rock Paper Scissors strategy, as well as feeling out an alternative way to score rock paper...
“MIRI Comms is hiring” by Duncan Sabien (Inactive)
11 Dec 2025
Contributed by Lukas
See details and apply. In the wake of the success of Nate and Eliezer's book, If Anyone Builds It, Everyone Dies, we have an opportunity to push thro...
“Gradual Disempowerment Monthly Roundup #3” by Raymond Douglas
11 Dec 2025
Contributed by Lukas
Farewell to Friction So sayeth Zvi: “when defection costs drop dramatically, equilibria break”. Even if AI makes individual tasks easier, this ca...
“Follow-through on Bay Solstice” by Raemon
11 Dec 2025
Contributed by Lukas
There is a Bay 2025 Solstice Feedback Form. Please fill it out if you came, and especially fill it out if you felt alienated, or disengaged, or that ...
“Most Algorithmic Progress is Data Progress [Linkpost]” by Noosphere89
10 Dec 2025
Contributed by Lukas
So this post brought to you by Beren today is about how a lot of claims about within-paradigm algorithmic progress is actually mostly about just gett...
“Selling H200s to China Is Unwise and Unpopular” by Zvi
10 Dec 2025
Contributed by Lukas
AI is the most important thing about the future. It is vital to national security. It will be central to economic, military and strategic supremacy. ...
“The funding conversation we left unfinished” by jenn
10 Dec 2025
Contributed by Lukas
People working in the AI industry are making stupid amounts of money, and word on the street is that Anthropic is going to have some sort of liquidit...
“Human Dignity: a review” by owencb
10 Dec 2025
Contributed by Lukas
I have in my possession a short document purporting to be a manifesto from the future. That's obviously absurd, but never mind that. It covers some i...
“Insights into Claude Opus 4.5 from Pokémon” by Julian Bradshaw
10 Dec 2025
Contributed by Lukas
Credit: Nano Banana, with some text provided. You may be surprised to learn that ClaudePlaysPokemon is still running today, and that Claude still hasn...
“My experience running a 100k” by Alexandre Variengien
09 Dec 2025
Contributed by Lukas
The SVP100 route. On the 3rd of August last year, I woke up early. I stood nervously with a hundred other runners in a hall in the city of N...
“[paper] Auditing Games for Sandbagging” by Jordan Taylor, Joseph Bloom
09 Dec 2025
Contributed by Lukas
Jordan Taylor, Sid Black, Dillon Bowen, Thomas Read, Satvik Golechha, Alex Zelenka-Martin, Oliver Makins, Connor Kissane, Kola Ayonrinde, Jacob Meriz...
“Towards Categorization of Adlerian Excuses” by romeostevensit
09 Dec 2025
Contributed by Lukas
[Author's note: LLMs were used to generate and sort examples into their requisite categories, as well as find and summarize relevant papers, and exte...
“Every point of intervention” by TsviBT
09 Dec 2025
Contributed by Lukas
Crosspost from my blog. Events are already set for catastrophe, they must be steered along some course they would not naturally go. [...] Are yo...
“How Stealth Works” by Linch
09 Dec 2025
Contributed by Lukas
Stealth technology is cool. It's what gave the US domination over the skies during the latter half of the Cold War, and the biggest component of the ...
“Reward Function Design: a starter pack” by Steven Byrnes
08 Dec 2025
Contributed by Lukas
In the companion post We need a field of Reward Function Design, I implore researchers to think about what RL reward functions (if any) will lead to ...
“We need a field of Reward Function Design” by Steven Byrnes
08 Dec 2025
Contributed by Lukas
(Brief pitch for a general audience, based on a 5-minute talk I gave.) Let's talk about Reinforcement Learning (RL) agents as a possible path to Arti...
“2025 Unofficial LessWrong Census/Survey” by Screwtape
08 Dec 2025
Contributed by Lukas
The Less Wrong General Census is unofficially here! You can take it at this link. The kinda-sorta-annual-if-you-really-squint tradition of the Less W...
“Little Echo” by Zvi
08 Dec 2025
Contributed by Lukas
I believe that we will win. An echo of an old ad for the 2014 US men's World Cup team. It did not win. I was in Berkeley for the 2025 Secular Solst...
“I said hello and greeted 1,000 people at 5am this morning” by Mr. Keating
08 Dec 2025
Contributed by Lukas
At the ass crack of dawn, in the dark and foggy mist, thousands of people converged on my location, some wearing short shorts, others wearing an elf ...
“AI in 2025: gestalt” by technicalities
07 Dec 2025
Contributed by Lukas
This is the editorial for this year's "Shallow Review of AI Safety". (It got long enough to stand alone.) Epistemic status: subjective impressions p...
“AI in 2025: gestalt” by technicalities
07 Dec 2025
Contributed by Lukas
This is the editorial for this year's "Shallow Review of AI Safety". (It got long enough to stand alone.) Epistemic status: subjective impressions p...
“Eliezer’s Unteachable Methods of Sanity” by Eliezer Yudkowsky
07 Dec 2025
Contributed by Lukas
"How are you coping with the end of the world?" journalists sometimes ask me, and the true answer is something they have no hope of understanding and...
“Answering a child’s questions” by Alex_Altair
06 Dec 2025
Contributed by Lukas
I recently had a conversation with a friend of a friend who has a very curious child around 5 years of age. I offered to answers some of their questi...
“The corrigibility basin of attraction is a misleading gloss” by Jeremy Gillen
06 Dec 2025
Contributed by Lukas
The idea of a “basin of attraction around corrigibility” motivates much of prosaic alignment research. Essentially this is an abstract way of thi...
“why america can’t build ships” by bhauth
06 Dec 2025
Contributed by Lukas
the Constellation-class frigate Last month, the US Navy's Constellation-class frigate program was canceled. The US Navy has repeatedly failed at mak...
“Help us find founders for new AI safety projects” by lukeprog
06 Dec 2025
Contributed by Lukas
In the past 10 years, Coefficient Giving (formerly Open Philanthropy) has funded dozens of projects doing important work related to AI safety / navig...
“Critical Meditation Theory” by lsusr
06 Dec 2025
Contributed by Lukas
[Terminology note: "samatha", "jhana", "insight", "homunculus" and "non-local time" are technical jargon defined in Rationalist Cyberbuddhist Jargon ...
“Announcing: Agent Foundations 2026 at CMU” by David Udell, Alexander Gietelink Oldenziel, windows, Matt Dellago
06 Dec 2025
Contributed by Lukas
Iliad is now opening up applications to attend Agent Foundations 2026 at CMU! Agent Foundations 2026 will be a 5-day conference (of ~35 attendees) o...
“An Ambitious Vision for Interpretability” by leogao
05 Dec 2025
Contributed by Lukas
The goal of ambitious mechanistic interpretability (AMI) is to fully understand how neural networks work. While some have pivoted towards more pragma...
“Journalist’s inquiry into a core organiser breaking his nonviolence commitment and leaving Stop AI” by Remmelt
05 Dec 2025
Contributed by Lukas
Some key events described in the Atlantic article: Kirchner, who’d moved to San Francisco from Seattle and co-founded Stop AI there last year, publ...
“Is Friendly AI an Attractor? Self-Reports from 22 Models Say Probably Not” by Josh Snider
05 Dec 2025
Contributed by Lukas
TL;DR: I tested 22 frontier models from 5 labs on self-modification preferences. All reject clearly harmful changes (deceptive, hostile), but labs di...
“Epistemology of Romance, Part 2” by DaystarEld
05 Dec 2025
Contributed by Lukas
In Part 1, I argued that the four main sources most people learn about romance from—media, family, religion/culture, and friends—are all unreliab...
“Center on Long-Term Risk: Annual Review & Fundraiser 2025” by Tristan Cook
05 Dec 2025
Contributed by Lukas
This is a brief overview of the Center on Long-Term Risk (CLR)'s activities in 2025 and our plans for 2026. We are hoping to fundraise $400,000 to fu...
“AI #145: You’ve Got Soul” by Zvi
05 Dec 2025
Contributed by Lukas
The cycle of language model releases is, one at least hopes, now complete. OpenAI gave us GPT-5.1 and GPT-5.1-Codex-Max. xAI gave us Grok 4.1. Goo...
“The behavioral selection model for predicting AI motivations” by Alex Mallen, Buck
04 Dec 2025
Contributed by Lukas
Highly capable AI systems might end up deciding the future. Understanding what will drive those decisions is therefore one of the most important ques...
[Linkpost] “Embedded Universal Predictive Intelligence” by Cole Wyeth
04 Dec 2025
Contributed by Lukas
This is a link post. A team at Google has substantially advanced the theory of embedded agency with a grain of truth (GOT), including new developments...
“Categorizing Selection Effects” by romeostevensit
04 Dec 2025
Contributed by Lukas
[Author's note: LLMs were used to generate and sort many individual examples into their requisite categories, as well as find and summarize relevant ...
“Front-Load Giving Because of Anthropic Donors?” by jefftk
04 Dec 2025
Contributed by Lukas
Summary: Anthropic has many employees with an EA-ish outlook, who may soon have a lot of money. If you also have that kind of outlook, money donate...
“Beating China to ASI” by PeterMcCluskey
04 Dec 2025
Contributed by Lukas
Who benefits if the US develops artificial superintelligence (ASI) faster than China? One possible answer is that AI kills us all regardless of whic...
“On Dwarkesh Patel’s Second Interview With Ilya Sutskever” by Zvi
04 Dec 2025
Contributed by Lukas
Some podcasts are self-recommending on the ‘yep, I’m going to be breaking this one down’ level. This was very clearly one of those. So here we ...
“Racing For AI Safety™ was always a bad idea, right?” by Wei Dai
03 Dec 2025
Contributed by Lukas
Recently I've been relitigating some of my old debates with Eliezer, to right the historical wrongs. Err, I mean to improve the AI x-risk community's...
“6 reasons why ‘alignment-is-hard’ discourse seems alien to human intuitions, and vice-versa” by Steven Byrnes
03 Dec 2025
Contributed by Lukas
Tl;dr AI alignment has a culture clash. On one side, the “technical-alignment-is-hard” / “rational agents” school-of-thought argues that we s...
“Human art in a post-AI world should be strange” by Abhishaike Mahajan
03 Dec 2025
Contributed by Lukas
Bubble Tanks is a Flash game originally released on Armor Games, a two-decade-old online game aggregator that somehow still exists. In the game, you ...
“Effective Pizzaism” by Screwtape
03 Dec 2025
Contributed by Lukas
I am an effective pizzaist. Sometimes, I want the world to contain more pizza, and when that happens I want as much good pizza as I can get for as li...
“Becoming a Chinese Room” by Raelifin
02 Dec 2025
Contributed by Lukas
[My novel, Red Heart, is on sale for $4 this week. Daniel Kokotaijlo liked it a lot, and the Senior White House Policy Advisor on AI is currently rea...
“Reward Mismatches in RL Cause Emergent Misalignment” by Zvi
02 Dec 2025
Contributed by Lukas
Learning to do misaligned-coded things anywhere teaches an AI (or a human) to do misaligned-coded things everywhere. So be sure you never, ever teach...
“Future Proofing Solstice” by Raemon
02 Dec 2025
Contributed by Lukas
Bay Solstice is this weekend (Dec 6th at 7pm, with a Megameetup at Lighthaven earlier in the day). I wanted to give people a bit more idea of what to...
“MIRI’s 2025 Fundraiser” by alexvermeer
02 Dec 2025
Contributed by Lukas
MIRI is running its first fundraiser in six years, targeting $6M. The first $1.6M raised will be matched 1:1 via an SFF grant. Fundraiser ends at mid...
“The 2024 LessWrong Review” by RobertM
01 Dec 2025
Contributed by Lukas
We have a ritual around these parts. Every year, we have ourselves a little argument about the annual LessWrong Review, and whether it's a good use o...
“A Statistical Analysis of Inkhaven” by Ben Pace
01 Dec 2025
Contributed by Lukas
Okay, we got 41 people to do 30 posts in 30 days. How did it go? How did they like it? Well I just had 36 of them fill out an extensive feedback form...
“Announcing: OpenAI’s Alignment Research Blog” by Naomi Bashkansky
01 Dec 2025
Contributed by Lukas
The OpenAI Alignment Research Blog launched today at 11 am PT! With 1 introductory post, and 2 technical posts. Blog: https://alignment.openai.com/ ...
“Interview: What it’s like to be a bat” by Saul Munn
01 Dec 2025
Contributed by Lukas
For the purposes of this transcript, some high-pitched clicking sounds have been removed. The below is an otherwise unedited transcript of an intervi...
“How Can Interpretability Researchers Help AGI Go Well?” by Neel Nanda
01 Dec 2025
Contributed by Lukas
Executive Summary Over the past year, the Google DeepMind mechanistic interpretability team has pivoted to a pragmatic approach to interpretabilit...
“A Pragmatic Vision for Interpretability” by Neel Nanda
01 Dec 2025
Contributed by Lukas
Executive Summary The Google DeepMind mechanistic interpretability team has made a strategic pivot over the past year, from ambitious reverse-engi...
[Linkpost] “How middle powers may prevent the development of artificial superintelligence” by Alex Amadori, Gabriel Alfour, Andrea_Miotti, Eva_B
01 Dec 2025
Contributed by Lukas
This is a link post. In this paper, we make recommendations for how middle powers may band together through a binding international agreement and achi...
“Claude Opus 4.5 Is The Best Model Available” by Zvi
01 Dec 2025
Contributed by Lukas
Claude Opus 4.5 is the best model currently available. No model since GPT-4 has come close to the level of universal praise that I have seen for Cla...
“Insulin Resistance and Glycemic Index” by lsusr
01 Dec 2025
Contributed by Lukas
In my previous post Traditional Food*, I explained how what we think of as a "traditional" diet is a nationalist propaganda campaign that's making us...
“November Retrospective” by johnswentworth
01 Dec 2025
Contributed by Lukas
Throughout November, I’ve been keeping up with the Inkhaven mandate to write and post a blogpost, of at least 500 words, every day. It's the last d...
“Inkhaven Retrospective” by abramdemski
30 Nov 2025
Contributed by Lukas
This will be the 30th post of at least 500 words I have written this month. (I did somewhat cheat two days ago, by making a 500+ word edit to Legitim...
“Explosive Skill Acquisition” by Ben Goldhaber
30 Nov 2025
Contributed by Lukas
If you’re going to learn a new skill or change in some way, going hard at it for a short intensive period beats spreading a gentler effort across m...
“A Blogger’s Guide To The 21st Century” by Screwtape
30 Nov 2025
Contributed by Lukas
Here's a fun format: get a big white board, and write the years of the 21st century. Write a category; something that has many variations come out ev...
“Ben’s 10 Tips for Event Feedback Forms” by Ben Pace
30 Nov 2025
Contributed by Lukas
I have made many many feedback forms for events I have run or been a part of. Here are some simple heuristics of mine, that I write for others' to le...
“The Moonrise Problem” by johnswentworth
30 Nov 2025
Contributed by Lukas
On October 5, 1960, the American Ballistic Missile Early-Warning System station at Thule, Greenland, indicated a large contingent of Soviet missiles ...
“The Joke” by Ape in the coat
30 Nov 2025
Contributed by Lukas
There is a joke format which I find quite fascinating. Let's call it Philosopher vs Engineer. It goes like this: the Philosopher raises some complica...
College life with short AGI timelines
30 Nov 2025
Contributed by Lukas
When I started my freshman year, my median estimate for AGI was 20 years. In my senior year it was down to 3 years (although it's gone back up to 5 y...
“I wrote a blog post every day for a month, and all I got was this lousy collection of incoherent ramblings” by Dentosal
30 Nov 2025
Contributed by Lukas
It's done. I made it to the end. A Finnish proverb fits the situation perfectly: Paska reissu mutta tulipahan tehtyä Which translates to somethi...
Change My Mind: The Rationalist Community is a Gift Economy
29 Nov 2025
Contributed by Lukas
Anthropologists have several categories for how groups exchange goods and services. The one you're probably most familiar with is called a Market Eco...
Epistemology of Romance, Part 1
29 Nov 2025
Contributed by Lukas
The Notebook is one of the most beloved romance films of the 21st century. When I run this activity, whether it's at a rationality workshop or Vibeca...
“A Harried Meeting” by Ben Pace
29 Nov 2025
Contributed by Lukas
An old pub that nobody much visits. An owner who is always in a drugged-out stupor. Background music that never changes. A pub that has remained thro...
Drugs Aren’t A Moral Category
29 Nov 2025
Contributed by Lukas
Are drugs good? This question doesn't really make sense. Yet Western society answers with a firm "NO". I have ADHD and have a prescription for Meth...
Claude 4.5 Opus’ Soul Document
29 Nov 2025
Contributed by Lukas
Summary As far as I understand and uncovered, a document for the character training for Claude is compressed in Claude's weights. The full document c...