Demis Hassabis

And I think if we were talking five plus years ago, I would have said to you, maybe we need an additional algorithmic breakthrough in order to do that, maybe more like the brain works.

And I think that's still true if we want explicit, abstract concepts, neat concepts.

But it seems that these systems can implicitly learn that.

Another really interesting, I think, unexpected thing was that these systems have some sort of grounding.

Even though they don't experience the world multimodally, or at least until more recently when we have the multimodal models.

And that's surprising that the amount of information and models that can be built up just from language.

And I think that I have some hypotheses about why that is.

I think we get some grounding through the RLHF feedback systems because obviously the human raters are by definition grounded people.

We're grounded in reality.

So our feedback's also grounded.

So perhaps there's some grounding coming in through there.

And also maybe language contains more grounding if you're able to ingest all of it.

than we perhaps thought, or linguists perhaps thought before.

So it's actually some very interesting philosophical questions that I think we haven't, people haven't even really scratched the surface of yet, looking at the advances that have been made.

You know, it's quite interesting to think about where it's going to go next.

But in terms of your question of large models, I think we've got to push scaling as hard as we can.

And that's what we're doing here.

And it's an empirical question whether that will hit an asymptote or a brick wall.

And there are different people that argue about that.

But actually, I think we should just test it.