Geoffrey Hinton

I don't even want to tell my graduate students to put them in.

That's absolutely what they're there for, but you need about 10 million of them for this.

Can you imagine the grants you'd have to write to support 10 million graduate students?

So, here's an idea.

that initially seems really dumb, but will get you the idea of what we're going to do.

We're going to start with random connection strengths.

Some will be positive numbers, some will be negative numbers.

And so the features in these layers I've been talking about, we call them hidden layers, the features in those layers will be just random features.

And if we put in an image of a bird and look at how the output neurons get activated, the output neurons for cat and dog and bird and politician will all get activated a tiny bit and all about equally, because the connection strength is just random.

So that's no good.

But we could now ask the following question.

Suppose I took one of those connection strengths, one of those billion connection strengths,

And I said, OK, I know this is an image of a bird.

And what I'd really like is, next time I present you with this image, I'd like you to give slightly more activation to the bird neuron and slightly less activation to the cat and dog and politician neurons.

And the question is, how should I change this connection strength?

Well, I could do an experiment.

If I'm not very theoretical and don't know much math, I'd do an experiment.

I would say, let's increase the connection strength a little bit and see what happens.

Does it get better at saying bird?

And if it gets better at saying bird, I say, okay, I'll keep that mutation to the connection strength.