Trenton Bricken

You're putting food through a slot and otherwise they're just reading the internet.

You don't even necessarily know what they're reading.

And then you take out this 105-year-old and you teach them some table manners, like how to use a knife and a fork.

And that's it.

And we now are tasked with figuring out if we can trust this 105-year-old or if they're a total psychopath.

And it's like, what did they read on the internet?

What beliefs did they form?

What are their underlying goals?

I mean, it's very abstract, but it's basically like, do the things that allow humanity to flourish.

Easy.

No, so hard to define.

Incredibly hard to define.

Yeah.

I mean, there's a fun thought experiment first posed by Yudkowsky, I think, where you tell the super intelligent AI, hey, all of humanity has got together and thought really hard about what we want, what's the best for society.

And we've written it down and put it in this envelope, but you're not allowed to open the envelope.

And so what that means is that – but do what's in the envelope.

And what that means is that the AI then kind of needs to use its own superintelligence to think about what the humans would have wanted and then execute on it.

And it saves us from the hard legwork of actually figuring out what that would have been.

Well, but now you just put that in the training data, so –

So now it's going to be like, oh, I know you're faking.