Ilya Sutskever

We've discussed the value function business earlier.

And maybe once people get good at value functions, they will be using their resources more productively.

And if you find a whole other way of training models, you could say, is this scaling or is it just using your resources?

I think it becomes a little bit ambiguous in a sense that when people were in the age of research back then, it was like people say, hey, let's try this and this and this.

Let's try that and that and that.

Oh, look, something interesting is happening.

And I think there will be a return to that.

Yeah.

Yeah.

So the discussion about value function, I think it was interesting.

I want to emphasize that I think the value function is something like, it's going to make our realm more efficient.

And I think that makes a difference.

But I think that anything you can do with a value function, you can do without, just more slowly.

The thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people.

And it's super obvious.

That seems like a very fundamental thing.

Yeah.

So you know, you could actually wonder.

One possible explanation for the human sample efficiency that needs to be considered is evolution.

And evolution has given us a small amount of the most useful information possible.