Geoffrey Hinton

So for the three pixels on the left, the votes it gets are big positive numbers times big positive intensities, so great big votes.

Now, from the three pixels in the right-hand column, it's got negative weights.

So if those pixels are bright, it'll get a big brightness times a big negative weight, so it'll get a lot of negative votes, and then all cancel out.

So if the column of pixels on the left is the same brightness as the column of pixels on the right, the positive votes it gets from the left-hand column will cancel the negative votes it gets from the right-hand column, and it'll get zero net input, and it'll just stay quiet.

But if the pixels on the left are bright and the pixels on the right are dim, the negative votes will be multiplied by small intensity numbers, and the positive votes will be multiplied by big intensity numbers.

And so the neuron get lots of input and get very excited and say, I found the thing I like.

And the thing it likes is an edge which is brighter on the left than on the right.

So we do know how to make a neuron, if we handwire it like that, pick up on the fact that there's an edge at a particular location in the image that's brighter on one side than the other side.

Now, what the brain does, roughly speaking, a lot of neuroscientists will be horrified by me saying this, but very roughly speaking, what the brain does is, in the early stages of visual cortex, which is where you recognize objects, it has lots and lots of neurons that pick up on edges at different orientations in different positions and at different scales.

So it has thousands of different positions and dozens of different orientations and several different scales.

And it has to have edge detectors for each combination of those.

So it has like a gazillion little edge detectors.

well, including some big edge detectors.

So a cloud, for example, has a big, soft, fuzzy edge.

And you need a different neuron for detecting that than what you'd need for detecting, say, the tail of a mouse disappearing around a corner in the distance, which is a very fine thing.

And you need an edge detector that was very sharp and saw very small things.

So, first stage, we have all these edge detectors.

I mean, you can understand it without dealing with color yet.

That's what the first layer of neurons will do.

They'll look at the pixels and they'll detect little bits of edge.