Geoffrey Hinton

The basic idea is that macroscopic things like a word correspond to big patterns of neural activity in the brain.

Similar words correspond to similar patterns of neuron activity.

So the idea is Tuesday and Wednesday will correspond to very similar patterns of neuron activity, where you can think of each neuron as a feature.

Better to call it a microfeature.

But when the neuron gets active, it says, this has that microfeature.

So if I say cat to you, all sorts of microfeatures will get active, like it's animate, it's furry, it's got whiskers, it might be a pet, it's a predator, all those things.

If I say dog, a lot of the same things will get active, like it's a predator, it might be a pet, but some different things, obviously.

So the idea is underlying these symbols that we manipulate, there's much more complicated microscopic goings-on that the symbols kind of are associated with.

And that's where all the action really is.

And if you really want to explain what goes on when we think or when we do analogies, you have to understand what's going on at this microscopic level.

And that's the neural network level.

Yes, there's a lot of that goes on.

Probably the easiest way to get into it is by thinking of a task that seems very natural, which is take an image.

Let's say it's a gray-level image.

So it's got a whole bunch of pixels, little areas of uniform brightness, that have different intensity levels.

So as far as the computer's concerned, that's just a big array of numbers.

And now imagine the task is...

you want to say whether there's a bird in the image or not, or rather whether the prominent thing in the image is a bird.

And people tried for many, many years, like half a century, to write programs that would do that, and they didn't really succeed.

And the problem is, if you think what a bird looks like in an image, well, it might be an ostrich up close in your face, or it might be a seagull in the far distance.