Sergey Levine

And that means that during that movement, you're doing less processing and you kind of batch it up in advance.

But you're not like entirely an open loop.

It's not like you're playing back a tape recorder.

You are actually reacting as you go.

You're just reacting at a different level of abstraction, a more basic level of abstraction.

And, again, this comes back to representations.

Figure out which representations are sufficient for kind of planning in advance and then enrolling, which representations require a tight feedback loop.

And for that tight feedback loop, like, what are you doing feedback on?

Like, you know, if I'm driving a vehicle, maybe I'm doing feedback on the position of the lane marker so that I stay straight.

And then at a lower frequency, I sort of gauge where I am in traffic.

Yeah.

So the key here is prior knowledge.

Yeah.

So, in order to effectively learn from your own experience, it turns out that it's really, really important to already know something about what you're doing.

Otherwise, it takes far too long.

It's just like it takes a person when they're a child a very long time to learn very basic things, to learn to write for the first time, for example.

Once you already have some knowledge, then you can learn new things very quickly.

Training the models with supervised learning now is to build out that foundation that provides the prior knowledge so they can figure things out much more quickly later.

And, again, this is not a new idea.

This is exactly what we've seen with LLMs, right?