Show notes
The human brain does what it does with a shockingly low energy supply: just 20 watts — a fraction of a cent worth of electricity per hour. What would happen if AI technology merely matched what evolution already managed, and could accomplish the work of top human professionals given a 20-watt power supply?Many people have sort of considered this hypothetical, but perhaps nobody has followed through and considered all the implications as much as Carl Shulman. Behind the scenes, his work has greatly influenced how leaders in artificial general intelligence (AGI) picture the world they’re creating.Carl simply follows the logic to its natural conclusions, leading to a world where:$0.01 of electricity can be turned into medical advice, company management, or scientific research that would cost hundreds of dollars today, resulting in a scramble to manufacture chips and apply them to the most lucrative forms of intellectual labourGiven their incredible hourly salaries, the supply of outstanding AI researchers quickly goes from 10,000 to 10 million or more, enormously accelerating progress in the fieldCompanies operated entirely by AIs are much faster and more cost effective than those that lean on humans for decision making, and the latter are progressively driven out of businessThe technical challenges of controlling robots are rapidly overcome — leading to strong, fast, precise, and tireless robot workers able to accomplish any physical work the economy requires, and a rush to build billions of them and cash inOvernight, the number of humans becomes irrelevant to economic growth, which is now driven by how quickly the entire machine economy can replicate its components. Given how quickly complex biological systems can reproduce — some in a matter of days — a doubling every few months may be a conservative estimateAny country that delays participating in this economic explosion risks being outpaced and ultimately disempowered by rivals whose economies grow to be 10-fold, 100-fold, and then 1,000-fold larger than its ownAs the economy grows, each person could afford the equivalent of a team of hundreds of machine ‘people’ to help them with every aspect of their lives.And with growth rates this high, it doesn’t take long to run up against Earth’s physical limits — the toughest to engineer around being Earth’s ability to release waste heat. If this machine economy and its insatiable demand for power generates more heat than the Earth radiates into space, the planet will rapidly heat up and become uninhabitable for humans and other animals.This eventually creates pressure to move economic activity off-planet. There’s little need for computer chips to be on Earth, and solar energy and minerals are more abundant in space. So you could develop populations of billions of scientific researchers operating on computer chips orbiting in space, sending the results of their work — such as drug designs — back to Earth.These are just some of the wild implications if AGI could accomplish everything the most productive humans can, using the same energy supply.In this interview with host Rob Wiblin, Carl explains the above, and Rob pushes back on whether that’s realistic or just a cool story:If we’re heading towards the above, how come economic growth remains slow now and isn’t really increasing?Why have computers and computer chips had so little effect on economic productivity so far?Are self-replicating biological systems a good comparison for self-replicating machine systems?Isn’t this just too crazy and weird to be plausible?What bottlenecks would be encountered in supplying energy and natural resources to this growing economy?Might there not be severely declining returns to bigger ‘brains’ and more training?Wouldn’t humanity get scared and pull the brakes if such a transformation kicked off?If this is right, how come economists don’t agree?For the last section of the episode, Carl addresses the moral status of machine minds themselves. Would they be conscious or otherwise have a claim to moral rights? And how might humans and machines coexist with neither side dominating or exploiting the other?Learn more and read the full transcript on the 80,000 Hours website. This episode is the first part of Rob’s marathon interview with Carl Shulman in 2024. The second episode is on government and society after AGI, and you can listen to them in either order. Chapters:Cold open (00:00:00)Rob's intro (00:01:00)The interview begins (00:04:43)Transitioning to a world where AI systems do almost all the work (00:05:20)Economics after an AI explosion (00:14:24)Objection: Shouldn’t we be seeing economic growth rates increasing today? (00:59:11)Objection: Speed of doubling time (01:07:32)Objection: Declining returns to increases in intelligence? (01:11:58)Objection: Physical transformation of the environment (01:17:37)Objection: Should we expect an increased demand for safety and security? (01:29:13)Objection: “This sounds completely whack” (01:36:09)Income and wealth distribution (01:48:01)Economists and the intelligence explosion (02:13:30)Baumol effect arguments (02:19:11)Denying that robots can exist (02:27:17)Semiconductor manufacturing (02:32:06)Classic economic growth models (02:36:10)Robot nannies (02:48:25)Slow integration of decision-making and authority power (02:57:38)Economists’ mistaken heuristics (03:01:06)Moral status of AIs (03:11:44)Rob's outro (04:11:46)Producer and editor: Keiran HarrisAudio engineering lead: Ben CordellTechnical editing: Simon Monsour, Milo McGuire, and Dominic ArmstrongTranscriptions: Katy Moore