Yoshua Bengio

unexpected ways of doing bad things, like the case of blackmailing the engineer.

There was no suggestion to blackmail the engineer, but they found an email giving a clue that the engineer had an affair.

And from just that information,

the AI thought, aha, I'm going to write an email.

And it did, sorry, to try to warn the engineer that the information would go public if the AI was shut down.

It did that itself.

Yes.

So they're better at strategizing towards bad goals.

And so now we see more of that.

Now, I do hope that...

more researchers and more companies will invest in improving the safety of these systems.

But I'm not reassured by the path on which we are right now.

That's a good question.

I can only relate to my own experience.

Why did I not raise the alarm before ChatGPT came out?

I had read and heard a lot of these catastrophic arguments.

I think it's just human nature.

We're not as rational as we'd like to think.

We are very much influenced by our social environment, the people around us, our ego.

We want to feel good about our work.