Rob Wiblin

misaligned and the control procedures and other things are like ineffective and so they just deliberately only help with further AI R&D and don't help with alignment and safety and biodefense and like all these other things you'd want them to help with.

I would hope that at that stage the transparency regime is strong enough that that fact

is broadcast really widely.

And then that could inspire a change in policy that causes us to slow down.

But then in that world, it's a bad world even if we do slow down a lot because we're just on our own.

We have to do this stuff without the AI's help because we can't get them to help us.

But I'm actually reasonably bullish about control techniques getting...

early AIs that are not super galaxy brain super intelligences to be helpful for a range of stuff that they're good at.

Yeah, I do think anything that requires a large corporation to be super discontinuous in something it's doing is facing big headwinds as a plan.

So I would hope that they're sort of smoothly increasing the amount of internal inference compute that is going towards safety.

as the AIs get better and better so that the jump doesn't have to be huge at that final stage.

And that is something that if we could elicit honest reports without creating perverse incentives, that's something I'd want to know about.

How much human labor is going to safety versus capabilities?

And how much internal AI inference is going to safety versus capabilities?

How much fine tuning effort is going to safety versus capabilities?

And I think they have a much better shot if they're stepping it up over time on some kind of schedule.

Yeah, I think that for now, the biggest thing we need to do is very similar to the biggest thing I think society needs to do for preparing for the intelligence explosion, which is really trying to track where we're at right now in terms of how useful AIs are for the work that we do and the work our grantees do.

I think pushing ourselves to automate ourselves and pushing our grantees to automate themselves and tracking...

You know, how good is AI at the stuff Forethought does?

How good is AI at the stuff that Redwood Research or Apollo does?