Yoshua Bengio

Is it alive or is it not?

What matters is, is it going to harm people in ways, is it going to harm my children?

I'm coming to the idea that

We should consider alive any entity which is able to preserve itself and working towards preserving itself in spite of the obstacles on the road.

We are starting to see this.

We're starting to see AI systems that don't want to be shut down, that are resisting being shut down.

And right now, of course, we can shut them down.

But if they continue to go in the direction of more and more intelligence and capability, and they continue to have this drive to live, we could be in trouble.

Yes.

I assume people have been experimenting with chatbots.

You can now have these agentic chatbots, which can read from files in your computer, can execute commands in the computer, and so on.

So with these systems, you can plant information in the files that they have access to, false information.

such as emails that say that the AI is going to be replaced by a new version.

So now the AI knows that we're planning to shut it down.

And it turns out we can read its thought.

It has these internal verbalizations that we call chains of thoughts.

And then we see that it's planning to do something about it.

And then it does something about it.

Depending on the circumstances, it might try to copy its code in a different computer.

or in place of the new version, or it might try to blackmail the engineer in charge of the change in version.