Geoffrey Hinton

The next thing I'm going to do is maybe, because we're sort of running out of patience at this point, I'm going to have a final layer that has neurons that say cat, dog, bird, politician, whatever.

And in that final layer, I'll take the neuron that says bird, and I'll hook it up to the things that detect birds' heads.

But I'll also hook it up to other things in the third layer that detect things like birds' feet or the tips of birds' wings.

And so now, my sort of output neuron for bird, when that gets active, the neural net is saying it's a bird.

If it sees a bird's foot and a possible bird's head and a possible tip of the wing of a bird, it'll get lots of input and say, hey, I think it's a bird.

So I think you can now understand how I might try and design that by hand.

And I think you can see there's huge problems in that.

I need an awful lot of detectors.

I need to cover this whole space of positions and orientations and scales.

I need to decide what features to extract.

I mean, I just made up the idea of getting a beak and then a bird's head.

There may be much better things to go after.

What's more, I want to detect lots of different objects.

So what I really need is features that aren't just good for finding birds, but features that are good for finding all sorts of things.

And it would be a nightmare to design this by hand, particularly if I figured out that to do a good job of this, I needed a network with at least a billion connections in it.

So I have to by hand design the strengths of these billion connections.

And that'll take a long time.

Then we say, well, okay, a network like that, maybe it could recognize birds if it had the right connection strengths in it.

But where am I going to get those connection strengths from?

Because I sure as hell don't want to put them in by hand.