Andrej Karpathy

it's going to be accumulating training sets and data sets and crafting these objectives by which you train these neural nets.

And at some point, there's going to be a compilation process from the data sets and the objective and the architecture specification into the binary, which is really just the neural net weights and the forward pass of the neural net.

And then you can deploy that binary.

And so I was talking about that sort of transition, and that's what the post is about.

And I saw this play out in a lot of fields, autopilot being one of them, but also just simple image classification.

People thought originally in the 80s and so on that they would write the algorithm for detecting a dog in an image.

And they had all these ideas about how the brain does it.

And first we detect corners, and then we detect lines, and then we stitched them up.

And they were really going at it.

They were thinking about how they're going to write the algorithm.

And this is not the way you build it.

And there was a smooth transition where, okay, first we thought we were going to build everything.

Then we were building the features, so like hog features and things like that, that detect these little statistical patterns from image patches.

And then there was a little bit of learning on top of it, like a support vector machine or binary classifier for cat versus dog and images on top of the features.

So we wrote the features, but we trained...

the last layer, sort of the classifier.

And then people are like, actually, let's not even design the features because we can't.

Honestly, we're not very good at it.

So let's also learn the features.

And then you end up with basically a convolutional neural net where you're learning most of it.