Tom Griffiths

Really, you can see the current reality is people just accepted that ridiculously large neural networks are what you need and are sort of willing to pay for having those and the sort of technologies there to be able to create them.

One thing that I really like about that story though is that the key insight that made it possible to train these multilayer neural networks actually comes from Leibniz.

The piece of math that you need to derive the back propagation algorithm, which was actually

named by Rosenblatt.

He said it had a back propagation algorithm, it just didn't work quite well.

So the back propagation algorithm that Rommelhart and Hinton developed was using the chain rule, which was a piece of the calculus that we can trace back to .

Yeah, so the way that Ma set this up, so we have these three levels, right?

The computational level, it's about the abstract problem being solved in the ideal solution.

The algorithmic level, which is about the sort of actual processes that approximate that solution and the implementation level of how that's realized instead of something physical.

That tells us that

we're never going to have just one theory of how the mind works.

Because you can have theories at those different levels, which are all correct as long as they're compatible with one another.

So we can say, oh, at the computational level, logic and probability theory really give us a good answer for what it is that minds and brains should be doing.

And then at the algorithmic level, you can say,

an artificial neural network gives us a way of approximating the solutions to those systems.

Then at the implementation level, we have a story about, well, this is how that structure could be realized in something which is either cells in a brain or pieces of silicon.

At those levels, as long as those theories are compatible, everyone can be right.

I think that's really important because there's a long history of people wanting to have a

single explanation for things, right?

Where you're like, oh, this is the theory.