John Collison

If you have a purely end-to-end system, let's look at the simulator.

Now, what do you do?

You're then constrained to just go from pixels

That's all you can run the system on, right?

And it's a very high-dimensional space, so it's a hard problem to generate everything.

But even if you solve that, it just becomes incredibly inefficient.

to run it in the full way of pixels to trajectories and simulation for training or for evaluation.

So this is when intermediate representations come in.

There are some intermediate representations in the world in this task, you know, in the physical world, we know are correct.

They're not sufficient, but they're not generality limiting.

There's an object here, there's a concept of a road, there's signs, there's speed limits.

So this is where augmenting that learned representation, those learned embeddings from the encoder-decoder with that more

structured representation is what we do.

And we find that this kind of gives us additional knobs to simulate in that space, just pixels to trajectories.

It allows us to have additional safety validation layers in real time.

And it also gives us additional mechanisms to specify the reward function for evaluation of the critic or for training.

So this is again, like we've gone kind of full circle of it.

Is it intent?

Yes, it is.

Yes.