80,000 Hours Podcast

Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

16 Apr 2026

Audio

Description

Hundreds of prominent AI scientists and other notable figures signed a statement in 2023 saying that mitigating the risk of extinction from AI should be a global priority. At 80,000 Hours, we’ve considered risks from AI to be the world’s most pressing problem since 2016. But what led us to this conclusion? Could AI really cause human extinction? We’re not certain, but we think the risk is worth taking very seriously. In particular, as companies create increasingly powerful AI systems, there’s a concerning chance that:These AI systems may develop dangerous long-term goals we don’t want.To pursue these goals, they may seek power and undermine the safeguards meant to contain them.They may even aim to disempower humanity and potentially cause our extinction.This article is written by Cody Fenwick and Zershaaneh Qureshi, and narrated by Zershaaneh Qureshi. It discusses why future AI systems could disempower humanity, what current AI research reveals about behaviours like power-seeking and deception, and how you can help mitigate the dangers.You can see the original article — packed with graphs, images, footnotes, and further resources — on the 80,000 Hours website: https://80000hours.org/problem-profiles/risks-from-power-seeking-ai/ Chapters:Risks from power-seeking AI systems (00:01:00)Introduction (00:01:17)Summary (00:03:09)Why are the risks from power-seeking AI a pressing world problem? (00:04:04)Section 1: Humans will likely build advanced AI systems with long-term goals (00:05:43)Section 2: AIs with long-term goals may be inclined to seek power (00:11:32)Section 3: These power-seeking AI systems could successfully disempower humanity (00:26:26)Section 4. People might create power-seeking AI systems without enough safeguards, despite the risks (00:38:34)Section 5: Work on this problem is neglected and tractable (00:47:37)Section 6: What are the arguments against working on this problem? (00:59:20)Section 7: How you can help (01:25:07)Thank you for listening (01:28:56)Audio editing: Dominic ArmstrongProduction: Zershaaneh Qureshi, Elizabeth Cox, and Katy Moore

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes

🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Other episodes from 80,000 Hours Podcast

Transcribed and ready to explore now

Can AIs already start 'rogue deployments' inside AI companies? (Landmark new METR report)

20 May 2026

80,000 Hours Podcast

I Know How to Build Safe Superintelligence | Yoshua Bengio, the most-cited AI researcher

07 May 2026

80,000 Hours Podcast

"AI doesn't work" – the story behind the stat that misled millions

28 Apr 2026

80,000 Hours Podcast

How scary is Claude Mythos? 303 pages in 21 minutes

10 Apr 2026

80,000 Hours Podcast

Is there a case against Anthropic? And: The Meta leaks are worse than you think.

03 Apr 2026

80,000 Hours Podcast

80,000 Hours Podcast

Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

This episode hasn't been transcribed yet

Other episodes from 80,000 Hours Podcast

Can AIs already start 'rogue deployments' inside AI companies? (Landmark new METR report)

I Know How to Build Safe Superintelligence | Yoshua Bengio, the most-cited AI researcher

"AI doesn't work" – the story behind the stat that misled millions

How scary is Claude Mythos? 303 pages in 21 minutes

Is there a case against Anthropic? And: The Meta leaks are worse than you think.

AI designs genomes from scratch & outperforms virologists at lab work. What could go wrong? | Dr Richard Moulange, CLTR

Sign in to Audioscrape

Share this moment