Geoffrey Hinton

I told you, don't go there yet.

No, not exactly.

This is a light bulb moment, though.

So for many years, the people who believed in neural networks knew how to change the very last layer of connection strengths, which you call weights, the ones that are going into the output units, the connection strengths going from the last layer of features into the bird neuron,

We knew how to change those, but we didn't understand how to get forces operating on those hidden neurons, the ones that detected a bird's head, for example.

Backpropagation showed us how to get forces acting on those, so then we could change the incoming weights of those, and that was a eureka moment.

Many different people had that eureka moment at different times.

Okay, the early 1970s,

There was someone in Finland who had it, I think, in his master's thesis.

And then in probably the late 70s, someone called Paul Warpus at Harvard had the idea.

In fact, some control theorists there called Bryson and Ho had had the idea for doing things like controlling spacecraft.

So when you land a spacecraft on the moon, you're using something very like backpropagation, but it's in a linear system.

You're using backpropagation to figure out how you should fire the rockets.

That's a large part of it, yes.

The other thing we didn't have is, back in the 70s, people didn't show that when you applied this in multilayer networks, what you get is very interesting representations.

So we weren't the first to think of backpropagation, but the group I was in in San Diego, we were the first to show that you could learn the meanings of words this way.

You could show the string of words and by trying to predict the next word, you could learn how to assign features to words that captured the meaning of the word.

And that's what got it published in Nature.

Okay, it's a good question.

You're not getting it quite right.