Andrej Karpathy

you can basically prompt these neural nets in arbitrary ways and you can ask them to solve problems and they will.

So you can just tell them, you can make it look like you're trying to solve some kind of a mathematical problem and they will continue what they think is the solution based on what they've seen on the internet.

And very often those solutions look very remarkably consistent, look correct potentially even.

I would say I'm definitely on... I'm much more hesitant with the analogies to the brain than I think you would see potentially in the field.

And I kind of feel like certainly the way neural networks started is everything stemmed from inspiration by the brain.

But at the end of the day, the artifacts that you get after training, they are arrived at by a very different optimization process than the optimization process that gave rise to the brain.

And so I think...

I kind of think of it as a very complicated alien artifact.

It's something different.

The neural nets that we're training.

They are a complicated alien artifact.

I do not make analogies to the brain because I think the optimization process that gave rise to it is very different from the brain.

So there was no multi-agent self-play kind of setup and evolution.

It was an optimization that is basically what amounts to a compression objective on a massive amount of data.

That said, evolution has found that it is very useful to predict and have a predictive model in the brain.

And so I think our brain utilizes something that looks like that as a part of it.

but it has a lot more, you know, gadgets and gizmos and value functions and ancient nuclei that are all trying to like make it survive and reproduce and everything else.

Certainly, I would say it's an extremely remarkable story that I'm only briefly learning about recently.

All the way from... Actually, you almost have to start at the formation of Earth and all of its conditions and the entire solar system and how everything is arranged with Jupiter and Moon and the habitable zone and everything.

And then you have an active Earth that's turning over material.