Geoffrey Hinton

even though it's never seen one before.

So it's self-teaching.

Let me carry on with my explanation of how neural networks work.

And I'm going to do it by saying how I would design one by hand.

So your first thought, when you see that an image is just a big array of numbers, which are how bright each pixel is, is to say, well, let's hook up those pixel intensities to our output categories, like bird and cat and dog and politician, or whatever our output categories are.

And that won't work.

And the reason is, if you think about what does the brightness of one pixel tell you about whether it's a bird or not?

Well, it doesn't tell you anything, because birds can be black and birds can be white.

There's all sorts of other things that can be black and white.

So the brightness of a pixel doesn't tell you anything.

So what can you derive from those numbers that you have in the image that describe the image?

Well, the first thing you can derive, which is what the brain does, is you can recognize when there's little bits of edge present.

So suppose I take a little column of three pixels, and I have a neuron that looks at those three pixels, a brain cell, and has big positive weights to those three pixels.

So when those pixels are bright, the neuron gets very excited.

Now, that would recognize a little streak of white that was vertical.

But now suppose that next to it

There's another column of three pixels.

So the first column was on the left, and the second column was on the right.

And I give the neuron big negative connection strengths to those pixels.

So you can think of the neuron as getting votes from the pixels.