Dr. Fei-Fei Li

You just give that.

You just give that.

the the documents and that's called self-supervised learning so whether it's supervised with additional labels or supervised without additional label is self-supervised it starts with data now data goes into the algorithm and the algorithm has to have an objective to learn

the the documents and that's called self-supervised learning so whether it's supervised with additional labels or supervised without additional label is self-supervised it starts with data now data goes into the algorithm and the algorithm has to have an objective to learn

Typically, in the language model, the objective is to predict the next syllabus as accurately as the training data shows you.

Typically, in the language model, the objective is to predict the next syllabus as accurately as the training data shows you.

In the case of images with cat labels, for example, is to predict an image that has a cat with the right label cat instead of the wrong label microwave.

In the case of images with cat labels, for example, is to predict an image that has a cat with the right label cat instead of the wrong label microwave.

And then because it has this objective, during training, if it makes a mistake,

And then because it has this objective, during training, if it makes a mistake,

you know, if I didn't predict the next word right or if I labeled the cat wrong, it goes back and iterates and updates its parameters based on the mistake.

you know, if I didn't predict the next word right or if I labeled the cat wrong, it goes back and iterates and updates its parameters based on the mistake.

It has some mathematical rules or learning rules to update.

It has some mathematical rules or learning rules to update.

And then it just keeps doing that till it, you know, when humans ask it to stop or it no longer updates, you know, whatever stop criteria.

And then it just keeps doing that till it, you know, when humans ask it to stop or it no longer updates, you know, whatever stop criteria.

And then you're left with a...

And then you're left with a...

ginormous neural network that's already trained by ginormous amount of data.

ginormous neural network that's already trained by ginormous amount of data.