“Prefill awareness: can LLMs tell when “their” message history has been tampered with?” by David Africa

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded09 Mar 2026

Publisherfeeds.type3.audio ↗

Handlerec:16821789

Show notes

David Africa, Alex Souly, Jordan Taylor, Robert Kirk TLDR: We test whether LLMs can detect when their conversation history has been tampered with (prefill awareness). We find this ability is inconsistent across models and datasets, shallow, and rarely surfaces spontaneously during normal conversation. However, recent Claude models show rather strong prefill detection capabilities when prompted, suggesting prefill awareness is an emerging and model-specific confound that should be actively monitored in off-policy alignment evals. Introduction AI systems have become increasingly sensitive to cues that they are being evaluated, and this can threaten the validity of evaluations broadly– including but not limited to alignment evaluations, capability benchmarks, and production monitoring. One underexamined assumption underlying many evaluations is that a model treats its conversation history as an authentic record of its own prior behavior — that is, it doesn't question whether the assistant turns in its context were actually produced by it. This assumption is implicit in any evaluation that constructs or manipulates a model's message history, from inserting tool-call results that simulate an environment, to replaying transcripts originally generated by a different model (as in OpenAI's production evals). Importantly, there are alignment evals we want to do that require prefilling [...] ---Outline:(00:49) Introduction(02:55) Motivation: Evaluations using Prefill(05:51) Experiments(05:54) Benign case(10:32) Measuring Prefill Sub-Capabilities(18:44) Conclusion(19:46) Whats next?(23:11) Appendix(23:14) Related Work(24:38) Setup Details(26:11) Behavioral classification methodology.(30:39) Example(36:36) Further Results(38:22) Plots --- First published: March 9th, 2026 Source: https://www.lesswrong.com/posts/jsFGuXDMxy5NZg9T2/prefill-awareness-can-llms-tell-when-their-message-history --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.