Geoffrey Hinton

Now, in the next layer of neurons, what I would do is I'd make a neuron that maybe detects three little bits of edge that all line up with one another.

and slope gently down towards the right.

And it also detects three little bits of edge that all line up with one another and slope gently upwards towards the right.

And what's more, those two little combinations of three edges join in a point.

So I think you can imagine some edges sloping down to the right, some edges sloping up to the right, and joining in a point.

And I have a neuron that detects that.

And we know how to build that now.

You just give it the right connections to the edge-detecting neurons.

And maybe you give it some negative connections to neurons to detect edges in different orientations, so it doesn't just go off anyway.

It's suppressed by those.

Now, that you might think of as something that's detecting a potential beak of a bird.

If that guy gets active, it could be all sorts of things.

It could be an arrowhead.

It could be all sorts of things.

But one thing it might be is the beak of a bird.

So now you're beginning to get some evidence that's kind of relevant to whether or not it might be a bird.

So in the second layer of neurons, I'd have lots of things to detect possible beaks all over the place.

I might also have things that detect a little combination of edges.

that form a circle, an approximate circle.

And I'd have detectors for those all over the place.