Nicholas Andresen
👤 SpeakerAppearances Over Time
Podcast Appearances
Try maintaining a secret code across that.
Have weaker models check whether they can actually follow each step.
If Claude Jr.
can't understand what Claude Sr.
is saying, maybe Claude Sr.
is hiding something.
M-O-O-O-M.
Claude's being weird again.
None of these are proven robust.
But each one makes hiding harder.
Reduce the selection pressure.
Make hiding less rewarding.
One approach.
Sever the pathways that transmit the pressure.
Split reasoning and output generation into separate models, so the pressure to look good accumulates only in the part that talks to users, the reasoning model never directly learns to prettify itself because nobody's judging its appearance, though indirect pressure remains.
Another.
Add countervailing pressure toward honesty.
OpenAI's recent confessions at work creates a separate output, a confession, trained purely on honest self-report.
In the regular output, pressure may favor hiding.
In the confession, pressure favors truth.