Yoshua Bengio

If governments were to mandate liability insurance...

Then we would be in a situation where there is a third party, the insurer, who has a vested interest to evaluate the risk as honestly as possible.

And the reason is simple.

If they overestimate the risk, they will overcharge and then they will lose market to other companies.

If they underestimate the risks, then they will lose money when there's a lawsuit, at least on average.

And they would compete with each other.

So they would be incentivized to improve the ways to evaluate risk.

And they would, through the premium, that would put pressure on the companies to mitigate the risks because they don't want to pay a high premium.

Let me give you another angle from an incentive perspective.

We have these cards, CBRN.

These are national security risks.

As AIs become more and more powerful, those national security risks will continue to rise.

And I suspect at some point, the governments in the countries where these systems are developed, let's say US and China, will just not want this to continue without much more control.

AI is already becoming a national security asset, and we're just seeing the beginning of that.

And what that means is there will be an incentive for governments to have much more of a say about how it is developed.

It's not just going to be the corporate competition.

Now, the issue I see here is, well, what about the geopolitical competition?

Okay, so that doesn't solve that problem.

that it's going to be easier if you only need two parties, let's say the US government and the Chinese government, to kind of agree on something.

And yeah, it's not going to happen tomorrow morning, but if capabilities increase and they see those catastrophic risks, and they understand them really in the way that we're talking about now, maybe because there was an accident or for some other reason, public opinion could really change things there, then...