Stuart Russell

So one in a million for a nuclear meltdown.

Extinction is much worse.

Yeah.

So if you said one in a billion, right, then you'd expect one extinction per billion years.

There's a background.

So one of the ways people work out these risk levels is also to look at the background.

But other ways of going extinct would include, you know, giant asteroid crashes into the Earth.

And you can roughly calculate what those probabilities are.

We can look at how many extinction-level events have happened in the past, and, you know, maybe it's half a dozen.

So maybe it's like a one in 500 million year event, right?

So somewhere in that range, right?

Somewhere between one in 10 million, which is the best nuclear power plants, and one in 500 million or one in a billion, which is the background risk from giant asteroids.

So let's say we settle on 100 million.

One in 100 million chance per year.

Well, what is it according to the CEOs?

25%.

So they're off by a factor of multiple millions, right?

So they need to make the AI systems millions of times safer.

One in four, yeah.

Yeah.