Tom Griffiths

But what it's doing is it's taking a vector of values that goes in and it's transforming it into another vector of values.

So you can think about a neural network as precisely as a way of computing with spaces.

It takes a point in one space and then it transforms it to a point in another space.

And so as you add more layers into that neural network, then the kind of computation that it's able to do

gets more complicated, now you're able to do multi-step computations, where you're transforming point from this space into the point in this space, and a point in that space, and a point in another space, and so on.

And so you're able to solve potentially more complex kinds of problems.

And so that approach had hit a stumbling block, because after Rosenblatt had shown, OK, we can have a learning algorithm,

his learning algorithm was sort of only provably worked for neural networks with one layer of adjustable weights.

And so Marvin Minsky, working with Seymour Papert, wrote a book that said, in fact, those neural networks with one layer of adjustable weights are fundamentally limited.

They are, because they're limited to

being able to compute things that are linear functions that are local functions of their input, then they're not going to be able to solve interesting kinds of problems.

And they sort of characterized a bunch of interesting problems that the models couldn't solve.

And so people got a little bit less interested in neural networks.

And so, you know, the computer scientists who sort of got excited about this drifted away.

And another bunch of psychologists, David Rumelhart, working with Jeff Hinton, who is a postdoc in that group, Jay McClellan, they started exploring

these models and then came up with learning algorithms for multilayer neural networks that could then solve these more complex problems.

And then as a consequence, if you fast forward another 40 years, we get to our modern AI systems.

And so the big thing that changed over those 40 years was just being able to use these models on much larger scales.

So deeper neural networks trained on more data with many more internal connections and so on.

In many ways, you can kind of think about that as, you know, Minsky there in the basement of the psychology department saying this is never going to work because you'd have to run it at ridiculous scales.