Ilya Sutskever

It is so narrow, artificial narrow intelligence.

So in response, as a reaction to this, some people said,

Well, this is not good.

It is so narrow.

What we need is general AI.

General AI, an AI that can just do all the things.

The second, and that term just got a lot of traction.

The second thing that got a lot of traction is pre-training.

Specifically, the recipe of pre-training.

I think the way people do RL now is maybe undoing the conceptual imprint of pre-training.

But pre-training had the property.

You do more pre-training, and the model gets better at everything, more or less uniformly.

General AI.

Pre-training gives AGI.

But the thing that happened with AGI and pre-training is that in some sense they overshot the target.

Because if you think about the term AGI, you will realize, and especially in the context of pre-training, you will realize that a human being is not an AGI.

Because a human being, yes, there is definitely a foundation of skills.

A human being...

A human being lacks a huge amount of knowledge.

Instead, we rely on continual learning.