“Untrusted advice for AI control: Short, strong advice significantly uplifts weak LLMs” by Caleb Biddulph, Adam Kaufman

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded28 Jul 2026

Publisherfeeds.type3.audio ↗

Handlerec:19525141

Show notes

TL;DR: We introduce the untrusted advice protocol, in which a trusted executor LLM takes every action and an untrusted advisor LLM can only send it short hints. Even with as few as 4 characters per step, this advice recovers a substantial fraction of the capability gap between the two models. Because the untrusted LLM's influence flows through such a narrow, monitorable channel, we argue that this achieves near-maximal safety in our BashArena setting. We also discuss the general concept of information bottlenecks and their benefits for interpretability, security, and cost. In SWE-bench Verified, a strong, untrusted LLM advising a weak, trusted LLM every step can significantly improve the latter's performance, even when we limit the length of the advice. See the more detailed version of this figure later in this post. In high-stakes AI control, we want to safely use a highly capable but untrusted model (U) that might secretly attempt a misaligned, catastrophic action. To do this, we create protocols that call U alongside a less capable, trusted model (T). Typically, T takes an auxiliary role in these protocols: for example, T might monitor U's actions and alert a human if they are suspicious enough (trusted monitoring), or [...] ---Outline:(04:51) Experiments(05:26) Main experiment: how does limiting advice length affect performance?(09:43) Reducing U's bit usage(10:49) Counting bits using LLM surprisal(13:37) Making U select from finite options(14:17) Why don't we red-team this protocol?(16:56) Is studying maximally safe protocols worth the safety tax?(19:05) Types of restrictions on U's advice(21:09) Information bottlenecks provide other advantages(21:40) Interpretability(23:50) Security(24:14) Cost(25:03) Conclusion(26:16) Appendix: more ways to implement information bottlenecks(26:23) Amortizing U's influence with pre-deployment work(28:14) Interpolating between T and U(28:53) Bottlenecking updates to T's weights(31:04) Appendix: colluding instances of U could defeat untrusted advice(33:10) Appendix: how to measure surprisal(37:59) Appendix: selecting advice from a menu(40:31) Appendix: best-of-n protocol(42:24) Appendix: advising less frequently The original text contained 26 footnotes which were omitted from this narration. --- First published: July 27th, 2026 Source: https://www.lesswrong.com/posts/jLkRCK35ri2btEHMF/untrusted-advice-for-ai-control-short-strong-advice --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.