Andrej Karpathy

You're changing the loss functions of what the neural net is trying to do, how it's trying to predict things.

But basically the data sets and the architectures of the neural net.

And

So in the case of the autopilot, a lot of the data sets had to do with, for example, detection of objects and lane line markings and traffic lights and so on.

So you accumulate massive data sets of, here's an example, here's the desired label, and then here's roughly what the algorithm should look like, and that's a convolutional neural net.

So the specification of the architecture is like a hint as to what the algorithm should roughly look like.

And then the fill-in-the-blanks process of optimization is the training process.

And then you take your neural net that was trained, it gives all the right answers on your dataset, and you deploy it.

Yeah, very much so.

Yeah.

On a high level, I would say if you look at the software running in the autopilot, I gave a number of talks on this topic.

I would say originally a lot of it was written in software 1.0.

Imagine lots of C++, right?

And then gradually, there was a tiny neural net that was, for example, predicting, given a single image, is there a traffic light or not, or is there a landline marking or not?

And this neural net didn't have too much to do in the scope of the software.

It was making tiny predictions on individual little images.

And then the rest of the system stitched it up.

So, okay, we don't have just a single camera, we have eight cameras.

We actually have eight cameras over time.

And so what do you do with these predictions?