Dr. Fei-Fei Li

So that pattern is learned.

So that pattern is learned.

And once you learn enough in a big, huge neural network, your ability to predict

And once you learn enough in a big, huge neural network, your ability to predict

the next word when you're given a word is really, really quite amazing, amazingly high to the point that it can converse more or less like a human.

the next word when you're given a word is really, really quite amazing, amazingly high to the point that it can converse more or less like a human.

And because in the training data, it has so much knowledge, whether it's chemistry or movie reviews or geopolitical facts, it has memorized all of them.

And because in the training data, it has so much knowledge, whether it's chemistry or movie reviews or geopolitical facts, it has memorized all of them.

and so it can give out very very good answers so those are the things we know we know how the algorithm works we know it needs training we know that it's learning and predicting pattern what we don't know is that because these models are huge there are billions and billions hundreds of billions of parameters and then

and so it can give out very very good answers so those are the things we know we know how the algorithm works we know it needs training we know that it's learning and predicting pattern what we don't know is that because these models are huge there are billions and billions hundreds of billions of parameters and then

inside these models there are these little nodes, each one of them have a little mathematical function that connects to each other.

inside these models there are these little nodes, each one of them have a little mathematical function that connects to each other.

So how do we know exactly how these billions and billions of parameters learn the pattern and where is the pattern stored and why sometimes it hallucinates a pattern versus

So how do we know exactly how these billions and billions of parameters learn the pattern and where is the pattern stored and why sometimes it hallucinates a pattern versus

it gives out a correct answer.

it gives out a correct answer.

There is no not yet precise mathematical explanation.

There is no not yet precise mathematical explanation.

We don't know at the level of, there's no equation that can tell us, oh, I know exactly why at this moment the chat GPT gives you the word, how are you versus how is he?

We don't know at the level of, there's no equation that can tell us, oh, I know exactly why at this moment the chat GPT gives you the word, how are you versus how is he?