Andrej Karpathy

So it's not just about the likelihood term coming up from the data itself telling you about what you are observing, but also the prior term of where are the likely things to see and how do they likely move and so on.

Yeah, driving is really hard.

Because it has to do with the predictions of all these other agents and the theory of mind and, you know, what they're going to do.

And are they looking at you?

Are they, where are they looking?

Where are they thinking?

There's a lot that goes there at the, at the full tail of, you know, the, the expansion of the knives that we have to be comfortable with it eventually.

Yeah.

The final problems are of that form.

I don't think those are the problems that are very common.

I think eventually they're important, but it's like really in the tail end.

Well, basically, the sensor is extremely powerful, but you still need to process that information.

And so going from brightnesses of these pixel values to, hey, here are the three-dimensional world is extremely hard.

And that's what the neural networks are fundamentally doing.

And so the difficulty really is in just doing an extremely good job of engineering the entire pipeline.

the entire data engine, having the capacity to train these neural nets, having the ability to evaluate the system and iterate on it.

So I would say just doing this in production at scale is like the hard part.

It's an execution problem.

Yeah, for the neural net specifically, just making sure everything fits into the chip on the car.

And you have a finite budget of flops that you can perform and memory bandwidth and other constraints.