Just a year ago we released a two part episode titled An Overview of Technical AI Alignment with Rohin Shah. That conversation provided details on the views of central AI alignment research organizations and many of the ongoing research efforts for designing safe and aligned systems. Much has happened in the past twelve months, so we've invited Rohin — along with fellow researcher Buck Shlegeris — back for a follow-up conversation. Today's episode focuses especially on the state of current research efforts for beneficial AI, as well as Buck's and Rohin's thoughts about the varying approaches and the difficulties we still face. This podcast thus serves as a non-exhaustive overview of how the field of AI alignment has updated and how thinking is progressing. Topics discussed in this episode include: -Rohin's and Buck's optimism and pessimism about different approaches to aligned AI -Traditional arguments for AI as an x-risk -Modeling agents as expected utility maximizers -Ambitious value learning and specification learning/narrow value learning -Agency and optimization -Robustness -Scaling to superhuman abilities -Universality -Impact regularization -Causal models, oracles, and decision theory -Discontinuous and continuous takeoff scenarios -Probability of AI-induced existential risk -Timelines for AGI -Information hazards You can find the page for this podcast here: https://futureoflife.org/2020/04/15/an-overview-of-technical-ai-alignment-in-2018-and-2019-with-buck-shlegeris-and-rohin-shah/ Timestamps: 0:00 Intro 3:48 Traditional arguments for AI as an existential risk 5:40 What is AI alignment? 7:30 Back to a basic analysis of AI as an existential risk 18:25 Can we model agents in ways other than as expected utility maximizers? 19:34 Is it skillful to try and model human preferences as a utility function? 27:09 Suggestions for alternatives to modeling humans with utility functions 40:30 Agency and optimization 45:55 Embedded decision theory 48:30 More on value learning 49:58 What is robustness and why does it matter? 01:13:00 Scaling to superhuman abilities 01:26:13 Universality 01:33:40 Impact regularization 01:40:34 Causal models, oracles, and decision theory 01:43:05 Forecasting as well as discontinuous and continuous takeoff scenarios 01:53:18 What is the probability of AI-induced existential risk? 02:00:53 Likelihood of continuous and discontinuous take off scenarios 02:08:08 What would you both do if you had more power and resources? 02:12:38 AI timelines 02:14:00 Information hazards 02:19:19 Where to follow Buck and Rohin and learn more This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.