Tom Griffiths

But if a concept's a point in space, then where do we go from there?

How do we learn what those concepts are, and how do we work out what the consequences are?

And the answer to that came from neural networks.

So people had been thinking about neural networks since the 1940s.

initial work by McCulloch and Pitts, which was translating the idea of a Boolean circuit, the kinds of logical structures that George Boole had thought about.

They came up with a way of expressing Boolean circuits in terms of operations between neurons.

You could connect neurons up in such a way that they could represent logical ands and ors and nots and so on.

And from that, you could build complex neural circuits.

Marvin Minsky had actually, as a doctoral student at Princeton, his PhD thesis was all about neural networks.

And he actually built a neural network that could learn in the basement of the psychology department at Harvard using bits that he was able to scrounge from places.

And then decided to give up on that whole thread of research because he thought it would be impossible to build a neural network that was large enough to learn anything interesting.

So with his adjustable resistors and these things that he was using for building his neural networks, there was a very clear constraint on the size of the model he could build.

And he did some calculations.

He's like, oh, in order to learn anything interesting, it would have to be ridiculously large.

And so he abandons that and then Frank Rosenblatt who was a psychology PhD at Cornell

in his PhD, got into building computers.

He actually came up with a device for tabulating data electronically as part of his PhD thesis and started to think, oh, hey, this thing could maybe help us understand how brains work, wanted to understand how brains work, and then came up with the first provably effective learning algorithm for simple neural networks, something called the perceptron, which is a sort of

a neural network which has one layer of adjustable weights in the original version that he had, although he subsequently explored versions that had multiple layers of adjustable weights.

And so you can think about what a perceptron does, what a neural network does, as you've got your inputs that go into that neural network.

So you've got a set of units that represent different dimensions of the input, and you've got an output that comes from that neural network.