Max Tegmark

Because we know that if you even want to build a super simple neural network, tell apart cat pictures and dog pictures, right?

That you can do that very, very well now.

But if you think about it a little bit, you convince yourself it must be impossible because if I have one megapixel, even if each pixel is just black or white, there's two to the power of one million possible images, which is way more than there are atoms in our universe.

And then for each one of those, I have to assign a number, which is the probability that it's a dog.

So an arbitrary function of images is a list of...

more numbers than there are atoms in our universe.

So clearly I can't store that under the hood of my GPU or my computer.

Yet it somehow works.

So what does that mean?

Well, it means that out of all of the problems that you could try to solve with a neural network, almost all of them are impossible to solve with a reasonably sized one.

But then what we showed in our paper was that the fraction of all the problems that you could possibly pose that we actually care about given the laws of physics is also an infinitesimally tiny little part.

And amazingly, they're basically the same part.

Yeah, but you could say maybe where the world was created for us, but I have a more modest interpretation, which is that instead evolution endowed us with neural networks precisely for that reason.

Because this particular architecture, as opposed to the one in your laptop, is very, very well designed.

adapted to solving the kind of problems that nature kept presenting our ancestors with, right?

So it makes sense that why do we have a brain in the first place?

It's to be able to make predictions about the future and so on.

So if we had a sucky system, which could never solve it, it wouldn't have evolved.

So this is, I think, a very beautiful fact.

We also realize that there's been earlier work on