Sholto Douglas

the loss function for these models is unsupervised.

You don't have to like come up with these bespoke things that you keep out of the training data.

You know, is there a way you can do a benchmark that's also unsupervised where, I don't know, another LLM is rating it in some way or something like that?

And maybe the answer is like, well, if you could do this, like reinforcement learning would work because then you have this like unsupervised

On the other thread, going back to the Sherlock Holmes thing,

If it's all associations all the way down, this is a sort of like naive dinner party question.

If I just like met you, I'm working on AI.

But okay, does that mean we should be less worried about super intelligence?

Because there's not this sense in which it's like Sherlock Holmes plus plus.

It'll still need to just like find these associations, like humans find associations and like, you know what I mean?

It's not just like it sees a frame of the world and it's like figured out all the laws of physics.

But the recursive self-improvement would still have to be them.

Like if intelligence is fundamentally about these associations, like the improvement is just them getting better at association.

There's not like another thing that's happening.

And so then it seems like you might disagree with the intuition that, well, they can't be that much more powerful if they're just doing it.

Should we talk about the intelligence explosion then?

So the reason I'm interested in discussing this is, with you guys in particular, is the models we have of the intelligence explosion so far come from economists, which is fine, but I think we can do better because

In the model of the intelligent explosion, what happens is you replace the AI researchers and then there's like a bunch of automated AI researchers who can speed up progress, make more AI researchers, make further progress.

And so I feel like if that's the metric or that's the mechanism,

We should just ask the AI researchers about whether they think this is plausible.