Sam Altman

What we have to prevent and –

And I think society has like actually a fairly good

messy but good process for collectively determining what safety thresholds should be.

Like that is a complex negotiation with a lot of stakeholders that we as a society have gotten better and better at over time.

But we have to prevent, and I think what you were touching on there is that the kind of catastrophic risks.

So nuclear is the example everyone gives, you know, nuclear war had this very global impact.

And so the world treated it differently and has done what I think is a remarkable job the last,

almost 80 years.

And I think there will be things with AI that are like that.

Certainly one example people talk about a lot is AI being used to design and manufacture synthetic pathogens that can cause a huge problem.

Another thing people talk a lot about is computer security issues and AI that can just like go hack beyond what any human could do and certainly at any scale.

And then there's another category of things that I think are just new, which is if the model gets capable enough that it can help design the own way to, like, exfiltrate the weights off of a server and make a lot of copies and modify its behavior.

More of, like, the sci-fi scenario.

Yeah.

um but i think we do as a world need to stare that in the face maybe not that specific case but this idea that there is catastrophic or potentially even existential risk in a way that just because we can't precisely define it doesn't mean we get to ignore it either and so we're doing a lot of work here to try to forecast and measure what those issues might be when they might come how we would detect them early

And I think all the people who say you shouldn't talk about this at all, you should just talk about the issues of misinformation and bias and the issues of today, they are wrong.

We have to talk about both.

We have to be safe at every step of the way.

Okay.

That's terrifying as I thought it would be.