Yoshua Bengio

And the statement goes, mitigating the risk of extinction from AI should be a global priority.

I then testify in front of the US Senate about those risks.

I travel the world to talk about it.

I'm the most cited computer scientist in the world, and you'd think that people would heed my warnings.

Hundreds of billions of dollars are being invested every year on developing this technology, and this is growing.

And these companies have a stated goal of building machines that will be smarter than us, that can replace human labor.

Yet we still don't know how to make sure they won't turn against us.

National security agencies around the world are starting to be worried that the scientific knowledge that these systems have could be used to build dangerous weapons, for example, by terrorists.

Recently, last September, the O1 system from OpenAI was evaluated, and the threat of this kind of risk went from low to medium, which is just the level below what is acceptable.

So I'm worried about these increasing capabilities, but what I'm most worried about today is increasing agency of AI.

You have to understand that planning and agency is the main thing that separates us from current AI to human-level cognition.

And these AIs are still weak in planning, but if you look back over the last five years, they measured the duration of tasks that the AI could complete, and it's getting better exponentially fast.

It's doubling every seven months.

What are AIs going to do with that planning ability in the future?

Well, bad news.

Recent studies in the last few months show that these most advanced AIs

have tendencies for deception, cheating,

And maybe the worst, self-preservation behavior.

So I'm going to share with you a study that is helping us understand this.

In this study, the AI has read in its input that it would be replaced by a new version.