Rob Wiblin

Like, do you have like auditors in there being like, what are you working on?

What are you working on to like all the team leads in the companies and like, you know, checking off that they have that it's 50% safety.

I can imagine stuff like that.

I think it would require like extremely technically deep regulators that like we just don't really have right now, I think.

Yeah, I think that's a possibility.

I don't think it's the most likely way it fails.

On my views, I think the most likely way it fails is that they don't go super hard on it.

But I think it's also plausible that they're just trying to get the AIs to help with alignment and the AIs are just like,

misaligned and the control procedures and other things are like ineffective and so they just deliberately only help with further AI R&D and don't help with alignment and safety and biodefense and like all these other things you'd want them to help with.

I would hope that at that stage the transparency regime is strong enough that that fact

is broadcast really widely.

And then that could inspire a change in policy that causes us to slow down.

But then in that world, it's a bad world even if we do slow down a lot because we're just on our own.

We have to do this stuff without the AI's help because we can't get them to help us.

But I'm actually reasonably bullish about control techniques getting...

early AIs that are not super galaxy brain super intelligences to be helpful for a range of stuff that they're good at.

Yeah, I do think anything that requires a large corporation to be super discontinuous in something it's doing is facing big headwinds as a plan.

So I would hope that they're sort of smoothly increasing the amount of internal inference compute that is going towards safety.

as the AIs get better and better so that the jump doesn't have to be huge at that final stage.

And that is something that if we could elicit honest reports without creating perverse incentives, that's something I'd want to know about.