“Held-out Monitors Sometimes Degrade, Even When Not Trained Against” by Joey Yudelson

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded29 Jul 2026

Publisherfeeds.type3.audio ↗

Handlerec:19565375

Show notes

Thanks to Dennis Akar, Rauno Arike, Shubhorup Biswas, Claude Fable, Max Heitmann, Vladimir Ivanov, and Jordan Taylor (alphabetical order) for helpful comments on this draft and on the research so far. Thanks to Rohan Subramani and Rhys Ward for high-level comments and discussion. Based on project proposals from Max Heitmann, Jordan Taylor, and Joshua Clymer. This work was done while at Aether Research. Code available here, metrics & run info available here. TL;DR: Held-out evals / monitors / probes would be really nice to have, but the “held-out-ness” is easier claimed than guaranteed. We measure a generalized form of “feedback spillover” and show that training against an LLM monitor can sometimes degrade a deception probe, and vice versa. Executive Summary It seems crucial to have measures of alignment that still work, even though we train on other measures of alignment. Whether we get this by default is an open question.We run preliminary experiments on a suite of probes and LLM monitors, and report the following: Training against one proxy can produce reward hacking policies that are less suspicious.Proxies also become worse at discriminating hacks from non-hacks, even when not trained against.We can observe the [...] ---Outline:(01:01) Executive Summary(02:10) Introduction(03:40) Experimental Setup(05:47) Our Proxies(07:46) Results(07:49) Training against one proxy sometimes produces hacking policies that are less suspicious in general(09:42) Proxies also degrade, even when not trained against(12:16) Which proxies have correlated degradation?(16:00) Discussion(18:21) Caveats(19:41) Future Work The original text contained 18 footnotes which were omitted from this narration. --- First published: July 29th, 2026 Source: https://www.lesswrong.com/posts/APkFfRp2AicL9RqvT/held-out-monitors-sometimes-degrade-even-when-not-trained --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.