John Schulman

So I'm not sure how you would maintain this equilibrium for a long period of time.

But I think if we got to that point, we would be in an okay position.

Yeah, I would say if we had everyone reasonably coordinated, we could figure out some.

And we felt like we had solved the technical problems around alignment well enough to be able to deploy like really smart AIs that can like

like act as an extension of people's will, but also, uh, prevent, uh, them from being misused in some way that would cause a catastrophe, catastrophe, I think then, uh, then that would be great.

Like we could, uh, go ahead and, uh, like safely deploy these systems and, uh, it would, um, it would usher in a lot of, uh, prosperity and a new, uh, like much, uh, more rapid phase of scientific advancement and so forth.

So I think that would be what the good scenario would look like.

Well, I would say if we can deploy systems incrementally that are successively smarter than the ones before, then I think that's safer.

So I hope the way things play out is it's not the scenario where everyone has to coordinate and lock things down and safely release things.

Because it would lead to this big buildup in potential energy, potentially.

So I would rather some scenario where we're just continually releasing things that are a little better than what came before.

And then while making sure we're confident that each diff is...

Right, like improving the safety and alignment in correspondence to the improvement in capability.

And if things started to look a little bit scary, then we would be able to slow things down.

So that's what I would hope for.

I would say if there's more of a discontinuous jump and the question is, how do you know if the thing you've got is safe to release?

I would say...

I can't give a generic answer, like I would wanna, but like the type of thing you might wanna do to make that more acceptable would be, you would wanna do a lot of testing, like simulated deployment, where that you expect, so red teaming of sorts, like you'd wanna do that in a way that you feel is like,

uh much less favorable than uh or much uh more likely to fail than the thing you're planning to do it in the real world uh you'd want to have a really good monitoring system so that you can uh like if something does start to go go wrong with the deployed system you can uh you feel like it's gonna be uh detectable immediately like you've got maybe you've got something watching over uh the deployed ais and what they're doing and looking for signs of trouble so

So I would want to, yeah, I would say just you'd want some defense in depth.