LessWrong (30+ Karma)

“Chicken-Free Egg Whites” by jefftk

04 Apr 2026

Contributed by Lukas

Baking has traditionally made extensive use of egg whites, especially the way they can be beaten into a foam and then set with heat. While I eat eg...

“dark ilan” by ozymandias

04 Apr 2026

Contributed by Lukas

The second time Vellam uncovers the conspiracy underlying all of society, he approaches a Keeper. Some of the difference is convenience. Since Vellam...

“Mean field sequence: an introduction” by Dmitry Vaintrob, Lauren Greenspan

04 Apr 2026

Contributed by Lukas

This is the first post in a planned series about mean field theory that we am writing together with Lauren Greenspan (this post was generated by Dmit...

“Democracy Dies With The Rifleman” by Vaniver

04 Apr 2026

Contributed by Lukas

Political power grows out of the barrel of a gun -- Mao Zedong Halfway thru recorded history, Athens became the first state we're sure was a democrac...

“The bar is lower than you think” by XelaP

04 Apr 2026

Contributed by Lukas

TL;DR: The efficient market hypothesis is a lie, there are no adults, you don't have to be as cool as the Very Cool People to contribute something, y...

“Anthropic Responsible Scaling Policy v3: Dive Into The Details” by Zvi

04 Apr 2026

Contributed by Lukas

Wednesday's post talked about the implications of Anthropic changing from v2.2 to v3.0 of its RSP, including that this broke promises that many peopl...

“Did Anyone Predict the Industrial Revolution?” by Lost Futures

04 Apr 2026

Contributed by Lukas

The Fighting Temeraire. 1839, by Joseph Mallord William Turner. (Source: Wikimedia) Editor's note: Post 2/30 for Inkhaven Why did the philosophers fai...

“Why do I believe preserving structure is enough?” by Aurelia

04 Apr 2026

Contributed by Lukas

There's a lot even our best neuroscientists don't know about the human brain. How can we have any reasonable hope for preservation given those unknow...

“There should be $100M grants to automate AI safety” by Marius Hobbhahn

03 Apr 2026

Contributed by Lukas

This post reflects my personal opinion and not necessarily that of other members of Apollo Research. TLDR: I think funders should heavily incentivize...

“Sadly, The Whispering Earring” by Dentosal

03 Apr 2026

Contributed by Lukas

The Whispering Earring (which you should read first) explores one of the most dystopic-utopic scenarios. Imagine you could achieve all you've ever wa...

“Common research advice #2: say precisely what you want to say” by LawrenceC

03 Apr 2026

Contributed by Lukas

As previously mentioned, research feedback I give to more junior research collaborators tends to fall into one of three categories: Doing quick sanit...

“AI #162: Visions of Mythos” by Zvi

03 Apr 2026

Contributed by Lukas

Anthropic had some problem with leaks this week. We learned that they are sitting on a new larger-than-Opus AI model, Mythos, that they believe offer...

“2026: The year of throwing my agency at my health (now with added cyborgism)” by Ruby

03 Apr 2026

Contributed by Lukas

I have bipolar disorder. I was diagnosed in late 2012 following my one and only severe manic episode. Most psychiatrists would regard me as a resound...

[Linkpost] “Q1 2026 Timelines Update” by Daniel Kokotajlo, elifland, bhalstead

03 Apr 2026

Contributed by Lukas

This is a link post. We’re mostly focused on research and writing for our next big scenario, but we’re also continuing to think about AI timelines...

“How social ideas get corrupt” by Kaj_Sotala

02 Apr 2026

Contributed by Lukas

I’ve noticed that sometimes there is an idea or framework that seems great to me, and I also know plenty of people who use it in a great and sensib...

“The Indestructible Future” by WillPetillo

02 Apr 2026

Contributed by Lukas

Doctor: Mr. Burns, I'm afraid you are the sickest man in the United States. You have everything! [...] Burns: You're sure you just haven't made thous...

“My most common advice for junior researchers” by LawrenceC

02 Apr 2026

Contributed by Lukas

Written quickly as part of the Inkhaven Fellowship. At a high level, research feedback I give to more junior research collaborators often can fall i...

“The Practical Guide to Superbabies” by GeneSmith

02 Apr 2026

Contributed by Lukas

It's Summer of 2025. I’m standing in a grass covered field on the longest day of the year. A friend of mine walks towards me, holding his newborn s...

“The Corner-Stone” by Benquo

02 Apr 2026

Contributed by Lukas

Is the US a ruthless cognitive meritocracy that reliably promotes outlier talent? VB Knives defended that claim in a Twitter argument against Living ...

“Systematically dismantle the AI compute supply chain.” by David Scott Krueger (formerly: capybaralet)

02 Apr 2026

Contributed by Lukas

This is not an April fool's joke, I’m participating in Inkhaven, which means I need to write a blog post every day. I recently watched The AI Doc....

“The quest for general intelligence is hitting a wall” by Sean Herrington

02 Apr 2026

Contributed by Lukas

There has been a lot of talk in the AI community lately about the possibility of achieving general intelligence. Indeed, recent progress in areas suc...

“Intelligence Dissolves Privacy” by Vaniver

02 Apr 2026

Contributed by Lukas

The future is going to be different from the present. Let's think about how. Specifically, our expectations about what's reasonable are downstream of...

“Anthropic’s Pause is the Most Expensive Alarm in Corporate History” by Ruby

02 Apr 2026

Contributed by Lukas

Imagine Apple halting iPhone production because studies linked smartphones to teen suicide rates. Imagine Pfizer proactively pulling Lipitor because ...

“I’m Suing Anthropic for Unauthorized Use of My Personality” by Linch

02 Apr 2026

Contributed by Lukas

Last year, I was sitting in my favorite coffee shop Caffe Strada, sipping on a matcha latte and writing a self-insert fanfic about how our plucky pro...

“Orders of magnitude: use semitones, not decibels” by Oliver Sourbut

02 Apr 2026

Contributed by Lukas

I'm going to teach you a secret. It's a secret known to few, a secret way of using parts of your brain not meant for mathematics... for mathematics. ...

“Dying with Whimsy” by NickyP

02 Apr 2026

Contributed by Lukas

To me it feels pretty emotionally clear we are nearing the end-times with AI. That in 1-4 years[1] things will be radically transformed, that at leas...

“AI for AI for Epistemics” by owencb, Lukas Finnveden

01 Apr 2026

Contributed by Lukas

We feel conscious that rapid AI progress could transform all sorts of cause areas. But we haven’t previously analysed what this means for AI for ep...

“Announcing Doublehaven with Reflections on Humour” by J Bostock

01 Apr 2026

Contributed by Lukas

Inkhaven is a writers’ retreat, well, really it's a bloggers’ retreat. In the Lighthaven campus, Berkeley, a couple dozen bloggers get together t...

“Anthropic Responsible Scaling Policy v3: A Matter of Trust” by Zvi

01 Apr 2026

Contributed by Lukas

Anthropic has revised its Responsible Scaling Policy to v3. The changes involved include abandoning many previous commitments, including one not to ...

“Save the Sun Shrimp!” by Jack

01 Apr 2026

Contributed by Lukas

The supposition that we live in a "goldilocks zone" is frankly just nonsense built up by an anthropocentric need to feel self-important, like Coperni...

“LIMBO: Who We Are, What We Do, and an Exciting High-Impact Funding Opportunity” by faul_sname

01 Apr 2026

Contributed by Lukas

We are excited to publicly introduce the Laboratory for Importance-sampled Measure and Bayesian Observation (LIMBO), a small research group working a...

“Chat, is this sus?” by Tyler Tracy

01 Apr 2026

Contributed by Lukas

A large assumption we have made in AI control is that humans will be perfect at auditing, that is, being shown a transcript and determining if the AI...

″“You Have Not Been a Good User” (LessWrong’s second album)” by habryka

01 Apr 2026

Contributed by Lukas

tldr: The Fooming Shoggoths are releasing their second album "You Have Not Been a Good User"! Available on Spotify, Youtube Music and (hopefully with...

“Lesswrong Liberated” by Ronny Fernandez

01 Apr 2026

Contributed by Lukas

A spectre is haunting the internet—the spectre of LLMism. The history of all hitherto existing forums is the history of clashing design tastes. For...

“The Claude Code Source Leak” by Error

01 Apr 2026

Contributed by Lukas

(mods: I assume this is blowing up twitter and so discussing it here won't do additional damage -- and there are already a thousand github forks -- b...

“Experiments With Opus 4.6’s Fiction” by Tomás B.

31 Mar 2026

Contributed by Lukas

The Unslop prize has reminded me of my terrified fascination with AI Generated fiction. I have come to identify more as a writer since my post from l...

“Movie Review: The AI Doc” by Zvi

31 Mar 2026

Contributed by Lukas

The AI Doc: Or How I Became an Apocaloptimist is a brilliant piece of work. (This will be a fully spoilorific overview. If you haven’t seen ...

“Product Alignment is not Superintelligence Alignment (and we need the latter to survive)” by plex

31 Mar 2026

Contributed by Lukas

tl;dr: progress on making Claude friendly[1] is not the same as progress on making it safe to build godlike superintelligence. solving the former doe...

“Co-Found Lens Academy With Me. (We have early users and funding)” by Luc Brinkman

31 Mar 2026

Contributed by Lukas

tl;dr. Lens Academy is creating scalable superingelligence x-risk education with several USPs. Current team: Luc (full time founder, technical genera...

“Slack in Cells, Slack in Brains” by Mateusz Bagiński

31 Mar 2026

Contributed by Lukas

[A veridically metaphorical explanation of why you shouldn't naïvely cram your life with local optimizations (even for noble or altruistic reasons)....

“I am definitely missing the pre-AI writing era” by N. Cailie

31 Mar 2026

Contributed by Lukas

Yesterday, I wrote my first technical draft on what I was working on with the goal to share it publicly on here (well using an account dedicated to t...

“The state of AI safety in four fake graphs” by Boaz Barak

30 Mar 2026

Contributed by Lukas

Here is a quick overview of my intuitions on where we are with AI safety in early 2026: So far, we continue to see exponential improvements in capa...

“AI should be a good citizen, not just a good assistant” by Tom Davidson, wdmacaskill

30 Mar 2026

Contributed by Lukas

Introduction Consider a lorry driver who sees a car crash and pulls over to help, even though it’ll delay his journey. Or a delivery driver who not...

″(Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL” by 7vik, Sid Black, Joseph Bloom

30 Mar 2026

Contributed by Lukas

Authors: Satvik Golechha*, Sid Black*, Joseph Bloom * Equal Contribution. This work was done as part of the Model Transparency team at the UK AI Secu...

[Linkpost] “Parkinson’s Law of Worry” by Jakub Halmeš

29 Mar 2026

Contributed by Lukas

This is a link post. Parkinson's law says that "Work expands so as to fill the time available for its completion." I think that a similar observation ...

“Folie à Machine: LLMs and Epistemic Capture” by DaystarEld

29 Mar 2026

Contributed by Lukas

"Truly, whoever can make you believe absurdities can make you commit atrocities." — Voltaire, 1765 A man in his late forties discovers a new passio...

“Stop asking “how good is this” to decide between donation opportunities I recommend” by Zach Stein-Perlman

29 Mar 2026

Contributed by Lukas

Prospective donors often ask how cost-effective my team's various recommendations are — they want to donate to the most cost-effective opportunity....

“Nick Bostrom: How big is the cosmic endowment?” by Zach Stein-Perlman

28 Mar 2026

Contributed by Lukas

Superintelligence, pp. 122–3. 2014. Consider a technologically mature civilization capable of building sophisticated von Neumann probes of the kind...

“Don’t Overdose Locally Beneficial Changes” by Mateusz Bagiński

28 Mar 2026

Contributed by Lukas

If you are very overweight, then you should aim to cut down your daily caloric intake. This doesn't mean the optimal daily caloric intake is zero. If...

“Stanley Milgram wasn’t pessimistic enough about human nature?” by David Gross

28 Mar 2026

Contributed by Lukas

A landmark of social psychology research was “The Milgram Experiment,” but a new look at the audio tapes and other evidence collected during that...

[Linkpost] “What if superintelligence is just weak?” by Simon Lermen

28 Mar 2026

Contributed by Lukas

This is a link post. In response to “2023 Or, Why I am Not a Doomer” by Dean W. Ball. Dean Ball is a pretty big voice in AI policy – over 19k su...

“Pray for Casanova” by Tomás B.

28 Mar 2026

Contributed by Lukas

I am fascinated by the beautiful who become deformed. Some become bitter, more bitter than those born less pulchritudinous. Most learn to cope with t...

“AI #161 Part 1: 80,000 Interviews” by Zvi

28 Mar 2026

Contributed by Lukas

The major technical advances this week were in agentic coding, as covered yesterday. The major non-DoW political and alignment developments will be ...

“Anthropic vs. DoW #6: The Court Rules” by Zvi

27 Mar 2026

Contributed by Lukas

Last night, Anthropic was given its preliminary injunction, with a stay of seven days. Emil Michael is a very angry person right now. So is the Hono...

“AI’s capability improvements haven’t come from it getting less affordable” by Anders Woodruff

27 Mar 2026

Contributed by Lukas

METR's frontier time horizons are doubling every few months, providing substantial evidence that AI will soon be able to automate many tasks or even ...

“ControlAI 2025 Impact Report” by Andrea_Miotti, Alex Amadori

27 Mar 2026

Contributed by Lukas

This post highlights a few key excerpts from our full impact report. You can read the full report at https://controlai.com/impact-report-2025. Contro...

“Scaffolded Reproducers, Scaffolded Agents” by Mateusz Bagiński

27 Mar 2026

Contributed by Lukas

[Experimenting with transposing a concept coined in one domain to another domain, perhaps not completely legibly to the entire intended audience.] In...

“My hobby: running deranged surveys” by leogao

27 Mar 2026

Contributed by Lukas

In late 2024, I was on a long walk with some friends along the coast of the San Francisco Bay when the question arose of just how much of a bubble we...

“The Terrarium” by Caleb Biddulph

26 Mar 2026

Contributed by Lukas

System: You are an AI agent in the Terrarium, a self-contained “society” of AI agents. The purpose of the Terrarium is to solve open mathematical...

“Sen. Sanders (I-VT) and Rep. Ocasio-Cortez (D-NY) propose AI Data Center Moratorium Act” by Matrice Jacobine

26 Mar 2026

Contributed by Lukas

The text of the bill can be found here. It begins by citing the warnings of AI company CEOs and deep learning pioneers Geoffrey Hinton and Yoshua Ben...

“Test your best methods on our hard CoT interp tasks” by daria, Riya Tyagi, Josh Engels, Neel Nanda

26 Mar 2026

Contributed by Lukas

Authors: Daria Ivanova, Riya Tyagi, Arthur Conmy, Neel Nanda Daria and Riya are co-first authors. This work was done during Neel Nanda's MATS 9.0. Cl...

“Claude Code, Cowork and Codex #6: Claude Code Auto Use and Full Cowork Computer Use” by Zvi

26 Mar 2026

Contributed by Lukas

Whatever else you think about Anthropic's agentic coding department, they ship. The highlights of this edition are three related big upgrades. You ...

″“What Exactly Would An International AI Treaty Say?” Is a Bad Objection” by Davidmanheim

26 Mar 2026

Contributed by Lukas

I’ve heard a number of people say that it's unclear what the technical contours of a global AI treaty would look like. That is true - but it's not ...

“Socrates is Mortal” by Benquo

26 Mar 2026

Contributed by Lukas

Socrates is Mortal There is a scene in Plato that contains, in miniature, the catastrophe of Athenian public life. Two men meet at a courthouse. One...

“Agents Can Get Stuck in Self-distrusting Equilibria” by Ashe Vazquez Nuñez

26 Mar 2026

Contributed by Lukas

Or: Identities as Schelling Fences for Embedded Agents This post was written as part of research done at MATS 9.0 under the mentorship of Richard Ngo...

“Label By Usable Volume” by jefftk

26 Mar 2026

Contributed by Lukas

I always look at unit prices: how much do I get for my dollar? But that assumes I can use all of it. The manufacturer gets "12oz" whether I'll be a...

“Past Automation Replaced Jobs. AI Will Replace Workers.” by James_Miller

26 Mar 2026

Contributed by Lukas

There's LLM-written text here. Unknown Model writes:en-US-AvaMultilingualNeural__ Human workers are about to face a competitor unlike any technolog...

“A Toy Environment For Exploring Reasoning About Reward” by jenny, Bronson Schoen

26 Mar 2026

Contributed by Lukas

tldr: We share a toy environment that we found useful for understanding how reasoning changed over the course of capabilities-focused RL. Over the co...

“Dispatch from Anthropic v. Department of War Preliminary Injunction Motion Hearing” by Zack_M_Davis

26 Mar 2026

Contributed by Lukas

Dateline SAN FRANCISCO, Ca., 24 March 2026— A hearing was held on a motion for a preliminary injunction in the case of Anthropic PBC v. U.S. Depart...

“Can Agents Fool Each Other? Findings from the AI Village” by Shoshannah Tekofsky

26 Mar 2026

Contributed by Lukas

The better agents are at deception, the less sure we can be that they are doing what we want. As agents become increasingly capable and autonomous, t...

″$1 billion is not enough; OpenAI Foundation must start spending tens of billions each year” by Davidmanheim

25 Mar 2026

Contributed by Lukas

OpenAI is now a public benefit corporation, with a charter that demands they use AGI for the benefit of all, and do so safely. To justify this struct...

“Is Gemini 3 Scheming in the Wild?” by Alejandro Wainstock, Agustin_Martinez_Suñe, Iván Arcuschin, Victor Braberman

25 Mar 2026

Contributed by Lukas

TL;DR When faced with an unexpected tool response, without any adversarial attack, Gemini 3 deliberately and covertly violates an explicit system pro...

“Latent Introspection (and other open-source introspection papers)” by vgel

24 Mar 2026

Contributed by Lukas

@vgel, Martin Vanek, @Raymond Douglas, @Jan_Kulveit — ACS Research, CTS, Charles University --- Paper | Code | Earlier post | Twitter thread | Blue...

“The Fourth World” by Linch

24 Mar 2026

Contributed by Lukas

Is consciousness the last moral world? Imagine trying to explain to a virus why suffering matters. A virus is a simple self-replicating molecule: uns...

“My cost-effectiveness unit” by Zach Stein-Perlman

24 Mar 2026

Contributed by Lukas

It feels like the grantmaking around me is only partially moneyball-pilled, or it's only somewhat competent at moneyball. There's alpha in putting nu...

“The AIXI perspective on AI Safety” by Cole Wyeth

24 Mar 2026

Contributed by Lukas

Epistemic status: While I am specialized in this topic, my career incentivizes may bias me towards a positive assessment of AIXI theory. I am also di...

“Measuring and improving coding audit realism with deployment resources” by Connor Kissane, Monte M, Fabien Roger

24 Mar 2026

Contributed by Lukas

TL;DR We study realism win rate, a metric for measuring how distinguishable Petri audit transcripts are from real deployment interactions. We use it ...

“Ablating Split Personality Training” by OscarGilg

24 Mar 2026

Contributed by Lukas

I was part of the SPAR team that worked on Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities. I ran some follow-...

“AI character is a big deal” by wdmacaskill, Tom Davidson

24 Mar 2026

Contributed by Lukas

0. Intro Due to Claude's Constitution and OpenAI's model spec, the issue of AI character has started getting more attention, particularly concerning ...

“The Federal AI Policy Framework: An Improvement, But My Offer Is (Still Almost) Nothing” by Zvi

24 Mar 2026

Contributed by Lukas

The Federal AI Policy Framework has been released. Well, it is a four page outline. Mostly it just reiterates existing such outlines. But that is four...

“Some things I noticed while LARPing as a grantmaker” by Zach Stein-Perlman

23 Mar 2026

Contributed by Lukas

Written to a new grantmaker. Most value comes from finding/creating projects many times your bar, rather than discriminating between opportunit...

“Understanding when and why agents scheme” by Mia Hopman, Jannes Elstner, Maria Avramidou, Amritanshu Prasad, David Lindner

23 Mar 2026

Contributed by Lukas

TL;DR To understanding the conditions under which LLM agents engage in scheming behavior, we develop a framework that decomposes the decision to sch...

“Attend the 2026 Reproductive Frontiers Summit, June 16–18, Berkeley” by TsviBT, K Richards

23 Mar 2026

Contributed by Lukas

We’ll be hosting the 2026 Reproductive Frontiers Summit at Lighthaven in Berkeley, CA, on June 16—18. Come join us if you want to learn, connect,...

“Is fever a symptom of glycine deficiency?” by Benquo

22 Mar 2026

Contributed by Lukas

A 2022 LessWrong post on orexin and the quest for more waking hours argues that orexin agonists could safely reduce human sleep needs, pointing to sh...

“My Most Costly Delusion” by Ihor Kendiukhov

22 Mar 2026

Contributed by Lukas

Suppose there is a fire in a nearby house. Suppose there are competent firefighters in your town: fast, professional, well-equipped. They are expecte...

“China declares AGI development to be a part of 5-year plan” by Darmani

22 Mar 2026

Contributed by Lukas

The CCP writes in its 15th 5-year plan that it will. Encourage innovation in multimodal, agentic, embodied, and swarm intelligence technologies, an...

“Pre-Review of Toy Story 5” by Raemon

22 Mar 2026

Contributed by Lukas

I am the second most spoiler-averse person I know. (Maybe tied for 2nd with a couple other people?). I once was considering going to an immersive ex...

“Does Hebrew Have Verbs?” by Benquo

22 Mar 2026

Contributed by Lukas

Spinoza's Compendium of Hebrew Grammar (1677, posthumous, unfinished) contains a claim that scholars have been misreading for centuries. He says that...

“Untrusted Monitoring is Default; Trusted Monitoring is not” by J Bostock

21 Mar 2026

Contributed by Lukas

These views are my own and not necessarily representative of those of any colleagues with whom I have worked on AI control. TL;DR: It's much cheaper ...

“China Derangement Syndrome” by Arjun Panickssery

21 Mar 2026

Contributed by Lukas

Often I see people claim it's essential for America to win the AI race against China (in whatever sense) for reasons like these: “What is the reas...

“Contrastive features elicit different perturbation responses than SAE features” by Francisco Ferreira da Silva, StefanHex

21 Mar 2026

Contributed by Lukas

Note: This is a research update sharing preliminary results as part of ongoing work. Figure 1: Contrastive (difference-of-means, English→Mandarin) ...

“Confusion around the term reward hacking” by ariana_azarbal

21 Mar 2026

Contributed by Lukas

Summary: "Reward hacking" commonly refers to two different phenomena: misspecified-reward exploitation, where RL reinforces undesired behaviors that ...

“A List of Research Directions in Character Training” by Rauno Arike

20 Mar 2026

Contributed by Lukas

Thanks to Rohan Subramani, Ariana Azarbal, and Shubhorup Biswas for proposing some of the ideas and helping develop them during a sprint. Thanks to R...

“The Distaff Texts” by Tomás B.

20 Mar 2026

Contributed by Lukas

Though I spend most of my time studying what is labelled “history” in some manuscripts and “malignant lies” in others and the “siren scrawl...

“Intention vs. Trying: Separate Prediction from Goal-Seeking” by plex

20 Mar 2026

Contributed by Lukas

tl;dr: Mixing goal-directedness into cognitive processes that are working to truth-seek about possible futures tends to undermine both truth-seeking ...

“On restraining AI development for the sake of safety” by Joe Carlsmith

20 Mar 2026

Contributed by Lukas

(Podcast version, read by the author, here, or search for "Joe Carlsmith Audio" on your podcast app. This is the tenth essay in a series I’m callin...

“Nullius in Verba” by Aurelia

20 Mar 2026

Contributed by Lukas

Independent verification by the Brain Preservation Foundation and the Survival and Flourishing Fund — the results so far Cultivating independent ve...

“The Case for Low-Competence ASI Failure Scenarios” by Ihor Kendiukhov

20 Mar 2026

Contributed by Lukas

I think the community underinvests in the exploration of extremely-low-competence AGI/ASI failure modes and explain why. Humanity's Response to the ...

“No, we haven’t uploaded a fly yet” by Ariel Zeleznikow-Johnston

20 Mar 2026

Contributed by Lukas

In the last two weeks, social media was set abuzz by claims that scientists had succeeded in uploading a fruit fly. It started with a video released ...

″“The AI Doc” is coming out March 26” by Rob Bensinger, Beckeck

19 Mar 2026

Contributed by Lukas

On Thursday, March 26th, a major new AI documentary is coming out: The AI Doc: Or How I Became an Apocaloptimist. Tickets are on sale now. The movie ...