“Consistency Training while Mitigating Obfuscation via Rate Matching” by Sohaib Imran, Prakhar Gupta, Jannes Elstner, David Africa

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded02 Jul 2026

Publisherfeeds.type3.audio ↗

Handlerec:19149640

Show notes

Sohaib Imran, Prakhar Gupta, Jannes Elstner, David Demitri Africa Links: Paper | Code TL;DR. Models condition their behavior on extraneous input features in undesirable ways — for example, on evaluation-likeness (resulting in evaluation gaming), or on the user's preferred answer (resulting in sycophancy). Consistency training teaches a model to behave the same whether or not an extraneous feature/cue is present in the input. Existing methods do this by fine-tuning LLMs to generate responses (BCT) or internal activations (ACT) that would be generated under a clean input, when given an input with a cue. We show this generalises to obfuscation: after BCT, models verbalise cues far less, even when they are influenced by the cues. We introduce a new method, Rate Matching Consistency Training (RMCT), which instead uses RL to match the rate of outcomes across inputs, leaving the rest of the response unconstrained. On the sycophancy benchmark of s Chua et al. (2025), RMCT matches or exceeds BCT at reducing bias-following while leaving verbalisation of the biasing cue largely intact. We think outcome-consistency training of this kind is one of the more natural tools available against evaluation gaming, and below we discuss that [...] ---Outline:(02:10) Extraneous features(05:15) Rate Matching Consistency Training(09:28) Summary of results(14:22) Limitations(15:27) The case for applying Consistency Training to mitigate evaluation gaming(19:09) Why RMCT for mitigating eval gaming?(21:04) A few other pathologies that RMCT is particularly suited to mitigate(21:10) Propensity gaps across modalities/languages/domains/formats.(21:36) Preference for self-generated text.(22:23) Incorporating Consistency Training into the alignment training stack(23:53) Call for feedback The original text contained 8 footnotes which were omitted from this narration. --- First published: July 1st, 2026 Source: https://www.lesswrong.com/posts/z2zYWnbGjcs6ypbsj/consistency-training-while-mitigating-obfuscation-via-rate --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.