Richard Sutton

But they're also a way of putting in lots of

knowledge.

And so this is an interesting question.

It's a sociological or industry question.

Will they reach the limits of the data and be superseded by things that can get more data just from experience rather than from

from people.

In some ways, it's a classic case of the bitter lesson.

The more human knowledge we put into the large language models, the better they can do.

And so it feels good.

And yet, one, well, I in particular expect there to be systems that can learn from experience, which could well perform much, much better and be much more scalable, in which case it will be another instance of the bitter lesson that the things that used human knowledge were eventually superseded by things that just trained from experience and computation.

Well, in every case of the bitter lesson, you know, you could start with human knowledge.

Right.

And then do the scalable things.

Yeah.

That's always the case.

And there's never any reason why that has to be bad.

Right.

But in fact, and in practice, it has always turned out to be bad.

Because people get locked into the human knowledge approach and they psychologically, or, you know, now I'm speculating why it is, but this is what has always happened.

Yeah.