Debate with a former OpenAI Research Team Lead — Prof. Kenneth Stanley

Prof. Kenneth Stanley is a former Research Science Manager at OpenAI leading the Open-Endedness Team in 2020-2022. Before that, he was a Professor of Computer Ccience at the University of Central Florida and the head of Core AI Research at Uber. He coauthored Why Greatness Cannot Be Planned: The Myth of the Objective, which argues that as soon as you create an objective, then you ruin your ability to reach it.In this episode, I debate Ken’s claim that superintelligent AI *won’t* be guided by goals, and then we compare our views on AI doom.00:00 Introduction00:45 Ken’s Role at OpenAI01:53 “Open-Endedness” and “Divergence”9:32 Open-Endedness of Evolution21:16 Human Innovation and Tech Trees36:03 Objectives vs. Open Endedness47:14 The Concept of Optimization Processes57:22 What’s Your P(Doom)™01:11:01 Interestingness and the Future01:20:14 Human Intelligence vs. Superintelligence01:37:51 Instrumental Convergence01:55:58 Mitigating AI Risks02:04:02 The Role of Institutional Checks02:13:05 Exploring AI's Curiosity and Human Survival02:20:51 Recapping the Debate02:29:45 Final ThoughtsSHOW NOTESKen’s home page: https://www.kenstanley.net/Ken’s Wikipedia: https://en.wikipedia.org/wiki/Kenneth_StanleyKen’s Twitter: https://x.com/kenneth0stanleyKen’s PicBreeder paper: https://wiki.santafe.edu/images/1/1e/Secretan_ecj11.pdfKen's book, Why Greatness Cannot Be Planned: The Myth of the Objective: https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237The Rocket Alignment Problem by Eliezer Yudkowsky: https://intelligence.org/2018/10/03/rocket-alignment/---Lethal Intelligence Guide, the ultimate animated video introduction to AI x-risk – https://www.youtube.com/watch?v=9CUFbqh16FgPauseAI, the volunteer organization I’m part of — https://pauseai.info/Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!---Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe

There are no comments yet.

Please log in to write the first comment.