“Reasoning Models Struggle to Control Their Chains of Thought” by Yueh Han Chen, robert mccarthy, Bruce W. Lee, Tomek Korbak

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded06 Mar 2026

Publisherfeeds.type3.audio ↗

Handlerec:16766452

Show notes

Authors: Yueh-Han Chen, Robert McCarthy, Bruce W. Lee, He He, Ian Kivlichan, Bowen Baker, Micah Carroll, Tomek Korbak In collaboration with OpenAI TL;DR: Chain-of-thought (CoT) monitoring can detect misbehavior in reasoning models, but only if models cannot control what they verbalize. To measure this undesirable ability, CoT Controllability, we introduce CoT-Control, an evaluation suite requiring models to solve problems while adhering to CoT constraints (e.g., reasoning about genetics without using "chromosome"). OpenAI just announced that they will start tracking CoT controllability in future models, starting from GPT-5.4. We find that reasoning models have significantly lower CoT controllability than output controllability; Claude Sonnet 4.5 controls its CoT only 2.7% of the time, versus 61.9% for final outputs. CoT controllability increases with model size but decreases with more RL training, test-time compute, and problem difficulty. These limitations persist under incentives to evade monitoring and adversarially optimized prompts, though models show slightly higher controllability when told they are being monitored. Our results suggest CoT controllability is currently unlikely to undermine CoT monitoring, but the underlying mechanism remains poorly understood. We recommend that frontier labs track CoT controllability in future models. Paper | Twitter Thread | OpenAI Blog Post | CoT-Control Eval [...] ---Outline:(02:04) Research Summary(03:26) Reasoning models rarely succeed at controlling their reasoning(03:41) CoT controllability decreases over the course of reasoning training(04:17) Larger models show higher CoT controllability, while increased inference-time reasoning effort reduces it(04:35) Situational awareness non-trivially increases CoT controllability(04:56) en-US-AvaMultilingualNeural__ Bar graph comparing controllability scores across AI models under three awareness conditions: Baseline, Monitor Awareness, and Eval Awareness.(05:11) Adversarially optimizing prompts for high controllability poorly transfers to new instruction types(05:44) CoT controllability tends to decrease as task difficulty increases. CoTs are less controllable when they are necessary to answer a question correctly.(05:56) en-US-AvaMultilingualNeural__ Two bar graphs showing controllability scores for Claude Sonnet versions across different difficulty levels and CoT necessity.(06:08) Conclusion(06:42) Future Work --- First published: March 5th, 2026 Source: https://www.lesswrong.com/posts/qTH9QrwJjPDWKQfDG/reasoning-models-struggle-to-control-their-chains-of-thought --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.