AI 2040: Plan A report - Daniel Kokotajlo & Thomas Larsen

episode

Machine Learning Street Talk (MLST) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from Machine Learning Street Talk (MLST)

VAEs Are Energy-Based Models? [Dr. Jeff Beck] 46 min · 25 Jan 2026

Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta] 53 min · 23 Jan 2026

Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION] 42 min · 23 Jan 2026

Adversarial Examples and Data Modelling - Andrew Ilyas (MIT) 1h 28m · 22 Aug 2024

All episodes from Machine Learning Street Talk (MLST) →

Report any issue

Source

Recorded08 Sep 2026

Publisheranchor.fm ↗

Handlerec:20296204

Show notes

Could slowing AI development make superintelligence safer? Daniel Kokotajlo and Thomas Larsen of the AI Futures Project join Tim Scarfe to examine AI 2040: Plan A, a proposal to buy time before AI exceeds human control.SPONSOR:---Cyber Fund built the Monastery to help founders ship products that were impossible a year ago.Apply now: https://cyber.fund---After revisiting AI 2027 and the limits of forecasting, they ask what happens when AI can automate research and sustain an economy without human workers. Tim challenges the case for general models and asks whether intelligence alone explains power. Plan A proposes an initial pause to build safety infrastructure, then cautious development up to the strongest AI that can still be reliably controlled. The discussion tests the distinction between control and alignment, the case for public AI research, and whether the US and China could enforce a slowdown. It ends with the evidence that would change their forecasts.---TIMESTAMPS:00:00:00 AI 2040: a slower route to superintelligence00:01:34 Sponsor: Cyber Fund00:02:12 From OpenAI to AI 202700:06:58 Forecasts, war games and self-fulfilling prophecies00:17:44 Why AI sceptics are changing their minds00:23:04 When AI can replace its own researchers00:28:45 Could an AI economy grow without human workers?00:37:32 One general model or a society of specialists?00:47:43 Brains, machines and collective intelligence00:56:12 Plan A: buy time at the controllable frontier01:00:02 Why control buys time but cannot replace alignment01:06:36 Why AI research should be public01:10:32 Can the US and China enforce an AI slowdown?01:19:04 Why AI policy debates miss the technology01:21:56 Is AI normal technology? The remaining disagreementMany thanks to James Wilken-Smith for helping with show research. ---REFERENCES:other:[00:00:01] AI 2040: Plan Ahttps://ai-2040.com/[00:03:27] AI 2027https://ai-2027.com/[00:13:47] Scenario Scrutiny for AI Policyhttps://blog.aifutures.org/p/scenario-scrutiny-for-ai-policy[00:33:11] The 2028 Global Intelligence Crisishttps://www.citriniresearch.com/p/2028gic[01:00:40] Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidenthttps://www.redwoodresearch.org/research/hugging-face-incident[01:09:21] The Hugging Face incident and the road aheadhttps://openai.com/index/hugging-face-incident-and-the-road-ahead/[01:22:01] AI as Normal Technologyhttps://www.normaltech.ai/p/ai-as-normal-technology[01:22:51] Common Ground between AI 2027 & AI as Normal Technologyhttps://asteriskmag.substack.com/p/common-ground-between-ai-2027-andperson:[00:19:43] Geoffrey Hintonhttps://www.cs.toronto.edu/~hinton/[00:20:07] Ryan Greenblatthttps://www.lesswrong.com/users/ryan_greenblatt[00:26:06] Elon Muskhttps://www.tesla.com/elon-musktool:[00:21:46] ARC-AGI-3https://arcprize.org/arc-agi/3[00:21:53] AlphaGo and Move 37https://deepmind.google/research/alphago/[00:39:41] Claudehttps://claude.com/product/overview[00:39:58] NVIDIA H100 GPUhttps://www.nvidia.com/en-us/data-center/h100/paper:[00:24:42] Training AI Scientists to Replicate Researchhttps://arxiv.org/abs/2608.13331v1[01:27:19] Validity of the single processor approach to achieving large scale computing capabilitieshttps://www.cs.cmu.edu/~18742/papers/Amdahl1967.pdfbook:[00:28:52] Bullshit Jobs: A Theoryhttps://www.simonandschuster.com/books/Bullshit-Jobs/David-Graeber/9781501143335organization:[01:05:09] Redwood Researchhttps://www.redwoodresearch.org/---RESCRIPT: https://app.rescript.info/public/share/33d1a58fa8f307ae7dfd504d4fdaa9d5

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.