Demis Hassabis

AlphaZero has quite a decent model, but the top human players have a much richer, much more accurate model than of Go or chess.

So that allows them to make world-class decisions on a very small amount of search.

So I think there's a sort of trade-off there.

If you improve the models, then I think your search can be more efficient and therefore you can get further with your search.

Well, of course, that's why we, you know, we pioneered and DeepMind sort of famous for using games as a proving ground, partly because obviously it's efficient to research in that domain.

But the other reason is obviously it's, you know, extremely easy to specify a reward function, winning the game or improving the score or something like that sort of built into most games.

So that is one of the challenges of real-world systems is how does one define the right objective function, the right reward function, and the right goals and specify them in a general way, but they're specific enough and actually points the system in the right direction.

And for real-world problems, that can be a lot harder.

But actually, if you think about it in even scientific problems, there are usually ways that you can specify the goal that you're after.

Yeah.

Well, look, I think it's different because our brains are not built for doing Monte Carlo tree search, right?

It's just not the way our organic brains would work.

So I think that in order to compensate for that, you know, people like Einstein have come up, you know, their brains have using their intuition and, you know, we maybe come to what intuition is, but they use their sort of knowledge and their experience to build intuition.

extremely, in Einstein's case, extremely accurate models of physics, including these sort of mental simulations.

I think if you read about Einstein and how he came up with things, he used to visualize and sort of really kind of feel what these physical systems should be like, not just the mathematics of it, but have a really intuitive feel for what they would be like in reality.

And that allowed him to think these sort of very outlandish thoughts at the time.

So I think that it's the sophistication of the world models that we're building, which then, if you imagine your world model can get you to a certain node in a tree that you're searching, and then you just do a little bit of search around that node, that leaf node, and that gets you to these original places.

But obviously, if your model and your judgment on that model is very, very good, then you can pick which leaf nodes you should sort of expand with search much more accurately.

So therefore, overall, you do a lot less search.

I mean, there's no way that any human could do a kind of brute force search over any kind of significant space.