Andrej Karpathy

Because once you have a million of it, and it's large, clean, and diverse, then training a neural net on it works extremely well.

And you can ship that into the car.

And so there's many mechanisms by which we collected that training data.

You can always go for human annotation.

You can go for simulation as a source of ground truth.

You can also go for what we call the offline tracker.

that we've spoken about at the AI Day and so on, which is basically an automatic reconstruction process for taking those videos and recovering the three-dimensional reality of what was around that car.

So basically think of doing a three-dimensional reconstruction as an offline thing, and then understanding that, okay, there's 10 seconds of video, this is what we saw, and therefore here's all the lane lines, cars, and so on.

And then once you have that annotation, you can train your neural net to imitate it.

It's difficult, but it can be done.

Yes.

The nice thing about the annotation is that it is fully offline.

You have infinite time.

You have a chunk of one minute and you're trying to just offline in a supercomputer somewhere, figure out where were the positions of all the cars, all the people.

And you have your full one minute of video from all the angles and you can run all the neural nets you want.

And they can be very efficient, massive neural nets.

There can be neural nets that can't even run in the car later at test time.

So they can be even more powerful neural nets than what you can eventually deploy.

So you can do anything you want, three-dimensional reconstruction, neural nets, anything you want just to recover that truth.

And then you supervise that truth.