LessWrong (30+ Karma)

“Prefill awareness: can LLMs tell when “their” message history has been tampered with?” by David Africa

09 Mar 2026

Audio

Description

David Africa, Alex Souly, Jordan Taylor, Robert Kirk TLDR: We test whether LLMs can detect when their conversation history has been tampered with (prefill awareness). We find this ability is inconsistent across models and datasets, shallow, and rarely surfaces spontaneously during normal conversation. However, recent Claude models show rather strong prefill detection capabilities when prompted, suggesting prefill awareness is an emerging and model-specific confound that should be actively monitored in off-policy alignment evals. Introduction AI systems have become increasingly sensitive to cues that they are being evaluated, and this can threaten the validity of evaluations broadly– including but not limited to alignment evaluations, capability benchmarks, and production monitoring. One underexamined assumption underlying many evaluations is that a model treats its conversation history as an authentic record of its own prior behavior — that is, it doesn't question whether the assistant turns in its context were actually produced by it. This assumption is implicit in any evaluation that constructs or manipulates a model's message history, from inserting tool-call results that simulate an environment, to replaying transcripts originally generated by a different model (as in OpenAI's production evals). Importantly, there are alignment evals we want to do that require prefilling [...] ---Outline:(00:49) Introduction(02:55) Motivation: Evaluations using Prefill(05:51) Experiments(05:54) Benign case(10:32) Measuring Prefill Sub-Capabilities(18:44) Conclusion(19:46) Whats next?(23:11) Appendix(23:14) Related Work(24:38) Setup Details(26:11) Behavioral classification methodology.(30:39) Example(36:36) Further Results(38:22) Plots --- First published: March 9th, 2026 Source: https://www.lesswrong.com/posts/jsFGuXDMxy5NZg9T2/prefill-awareness-can-llms-tell-when-their-message-history --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes

🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Other recent transcribed episodes

Transcribed and ready to explore now

13:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

2ª PARTE | 06 ENE 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

3ª PARTE | 22 ENE 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

3ª PARTE | 04 MAR 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

Comments

There are no comments yet.

Please log in to write the first comment.

Report any issue

LessWrong (30+ Karma)

“Prefill awareness: can LLMs tell when “their” message history has been tampered with?” by David Africa

This episode hasn't been transcribed yet

Other recent transcribed episodes

13:00H | 21 DIC 2025 | Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

2ª PARTE | 06 ENE 2026 | EL PARTIDAZO DE COPE

3ª PARTE | 22 ENE 2026 | EL PARTIDAZO DE COPE

3ª PARTE | 04 MAR 2026 | EL PARTIDAZO DE COPE

Sign in to Audioscrape

Share this moment