Yoshua Bengio

I mean, this is like too much of an assumption.

They're going to have some interesting relationships that allow to predict things in the future, to explain what happened in the past.

The kind of knowledge about those relationships in a classical AI system is encoded in the rules.

Like a rule is just like a little piece of knowledge that says, oh, I have these two, three, four variables that are linked in this interesting way.

Then I can say something about one or two of them given a couple of others, right?

In addition to disentangling the elements of the representation, which are like the variables in a rule-based system, you also need to disentangle the

the mechanisms that relate those variables to each other.

So like the rules.

So the rules are neatly separated.

Each rule is living on its own.

And when I change a rule because I'm learning, it doesn't need to break other rules.

Whereas current neural nets, for example, are very sensitive to what's called catastrophic forgetting, where after I've learned some things and then I learn new things, they can destroy the old things that I had learned, right?

If the knowledge was better factorized and separated, disentangled, then you would avoid a lot of that.

Now, you can't do this in the sensory domain, but

What do you mean by sensor?

Like in pixel space.

But my idea is that when you project the data in the right semantic space, it becomes possible to now represent this extra knowledge beyond the transformation from input to representations, which is how representations act on each other and predict the future and so on.

in a way that can be neatly disentangled.

So now it's the rules that are disentangled from each other and not just the variables that are disentangled from each other.

Yes.