Yoshua Bengio

So these systems understand that we want to shut them down, and they try to resist.

Unfortunately, we don't put these things in the code.

That's part of the problem.

The problem is we grow these systems by giving them data and making them learn from it.

Now, a lot of that training process boils down to imitating people because they take all the texts that people have written, all the tweets and all the Reddit's comments and so on.

And they internalize the kind of drives that humans have, including the drive to preserve oneself and the drive to have more control over their environment so that they can achieve whatever goal we give them.

It's not like normal code.

It's more like you're raising a baby tiger, right?

And you, you, you know, you feed it, you, you let it experience things.

Sometimes, you know, it does things you don't want.

It's okay.

It's still a baby, but it's growing.

is a black box and then on the outsides we've kind of taught it what we want it to do how does it it's mostly a black box everything in the neural net is is essentially a black box now the part as you say that it's on the outside is that we also give it verbal instructions we type these are good things to do these are things you shouldn't do don't help anybody build a bomb okay

Unfortunately, with the current state of the technology right now, it doesn't quite work.

People find a way to bypass those barriers.

So those instructions are not very effective anymore.

And there are two reasons why it's going to not do it.

One is because it was given explicit instructions to not do it, and usually it works.

And the other is, in addition, there's an extra layer, because that layer doesn't work.

sufficiently well.