How Researchers Test AI for Hidden Goals — Apollo Research

episode

Machine Learning Street Talk (MLST) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from Machine Learning Street Talk (MLST)

VAEs Are Energy-Based Models? [Dr. Jeff Beck] 46 min · 25 Jan 2026

Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta] 53 min · 23 Jan 2026

Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION] 42 min · 23 Jan 2026

Adversarial Examples and Data Modelling - Andrew Ilyas (MIT) 1h 28m · 22 Aug 2024

All episodes from Machine Learning Street Talk (MLST) →

Report any issue

Source

Recorded31 Jul 2026

Publisheranchor.fm ↗

Handlerec:19594830

Show notes

Can an AI do the right thing for the wrong reason? Tim Scarfe speaks with Apollo Research’s Alexander Meinke, Axel Højmark and Jérémy Scheurer about Measuring Reward-Seeking via Contrastive Belief Updates, their new research with OpenAI.The panel asks how models infer what graders reward, why good behaviour can come from the wrong reason, and whether that difference can be measured. The conversation moves through promise-breaking, grader awareness, reward hacking, scheming, opaque reasoning and corrigibility, then turns to a detailed walkthrough of the contrastive-belief method and what its results do and do not show. The o3 results discussed here concern an intermediate checkpoint without safety training.This episode was made in partnership with Apollo Research. MLST retained full editorial control.ReferenceApollo Research: https://www.apolloresearch.ai/---TIMESTAMPS:00:00:00 Cold Open00:02:12 Right Things, Wrong Reasons00:12:47 Grader Awareness00:26:22 Legibility00:32:35 What To Call It00:35:58 Intelligence, Agency, Anthropomorphism00:45:16 Apollo’s Mission00:48:54 The End of the Exponential00:55:45 The Paper01:16:34 Closing Reflection---REFERENCES:tool:[00:00:08] Claude Fablehttps://www.anthropic.com/claude/fable[00:12:50] AlphaGo Zerohttps://deepmind.google/blog/alphago-zero-starting-from-scratch/[00:44:30] AlphaFold 3https://deepmind.google/science/alphafold/paper:[00:01:02] Measuring Reward-Seeking via Contrastive Belief Updateshttps://arxiv.org/abs/2607.18966[00:16:19] Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationshttps://transformer-circuits.pub/2026/nla/[00:26:48] Stress Testing Deliberative Alignment for Anti-Scheming Traininghttps://arxiv.org/abs/2509.15541[00:35:33] Shortcut learning in deep neural networkshttps://arxiv.org/abs/2004.07780[00:53:49] Measuring AI Ability to Complete Long Software Taskshttps://arxiv.org/abs/2503.14499[00:59:52] Modifying LLM Beliefs with Synthetic Document Finetuninghttps://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/[01:10:44] Alignment Faking in Large Language Modelshttps://arxiv.org/abs/2412.14093[01:13:55] Natural Emergent Misalignment from Reward Hackinghttps://www.anthropic.com/research/emergent-misalignment-reward-hackingother:[00:10:14] We Need a Science of Scheminghttps://www.apolloresearch.ai/science/science-of-scheming/[00:32:56] CoastRunners reward hacking examplehttps://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/organization:[01:06:07] Redwood Researchhttps://www.redwoodresearch.org/---ReScript: https://app.rescript.info/share/718ab68e18cfa3b9b800da6b3290fd42

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.