Yoshua Bengio

But I don't think that's going to be nearly enough.

I think we need some fairly drastic changes in the way that we're considering learning

To achieve the goal that these learners actually understand in a deep way the environment in which they are, you know, observing and acting.

Right, right.

Oh, I see what you mean.

I'm with you there.

So I agree that in order to build neural nets with the kind of broad knowledge of the world that typical adult humans have, probably the kind of computing power we have now is going to be insufficient.

So the good news is there are hardware companies building neural net chips, and so it's going to get better.

However, the good news in a way, which is also a bad news, is that even our state-of-the-art deep learning methods fail to learn models that understand even very simple environments like some grid worlds that we have built.

Even these fairly simple environments, I mean, of course, if you train them with enough examples, eventually they get it.

But it's just like, instead of what humans might need just dozens of examples, these things will need millions, right?

For very, very, very simple tasks.

And so I think there's an opportunity for academics who don't have the kind of computing power that, say, Google has to do really important and exciting research to advance the state of the art in training frameworks, learning models, agent learning in even simple environments that are synthetic, that seem trivial, but yet current machine learning fails on.

And how do you think those goals can be addressed?

Right.

So first of all, I believe that one reason why the classical expert systems approach failed is because a lot of the knowledge we have, so you talked about common sense, intuition...

There's a lot of knowledge like this, which is not consciously accessible.

There are lots of decisions we're taking that we can't really explain, even if sometimes we make up a story.

And that knowledge is also necessary for machines to take good decisions.

And that knowledge is hard to codify in expert systems, rule-based systems, and classical AI formalism.