Matt Freeman
👤 SpeakerAppearances Over Time
Podcast Appearances
We ran all the tests and they were great.
They're internal and you can't see them.
And we don't make them public, but just trust us.
Like, like I, at least that's like honorable.
Yeah, so we sort of talked about two different things.
I just want to clarify it sort of for my own sake, that there is the fact that this event will be in the training data at some later date of the next version, and then there's just what's happening right now, and maybe at a more meta level, like...
If a sufficiently weird, crazy thing happens, the model is probably going to default to at least seriously considering the idea that this is an eval scenario.
I mean, it's fairly common that people will tell one of the models, any of them, some current event, and it'll be like, no, that's not true.
No one would be so stupid as to.
I mean, and maybe, I don't know, this is just maybe a disagree.
Like, don't you kind of want it to be asking that question?
Don't you kind of want it to be considering like, you know, am I being judged by God metaphorically for what I'm about to do?
Because, because it's an eval.
Is God or the simulators watching me?