Richard Sutton

What we want, I think, to quote Alan Turing, what we want is a machine that can learn from experience.

Right.

Where experience is the things that actually happen in your life.

You do things, you see what happens, and that's what you learn from.

The large language models learn from something else.

They learn from here's a situation and here's what a person did.

And implicitly, the suggestion is you should do what the person did.

No, I agree that it's the large language model perspective.

I don't think it's a good perspective.

Yeah, curious why.

So, to be a prior for something, there has to be a real thing.

I mean, a prior bit of knowledge should be the basis for actual knowledge.

What is actual knowledge?

There's no definition of actual knowledge in that large language framework.

What makes an action a good action to take?

You recognize the value, the need for continual learning.

Right.

So if you need to learn continually, continually means learning during normal interaction with the world.

Yeah.

And so then there must be some way during the normal interaction to tell what's right.