Rob Wiblin

is broadcast really widely.

And then that could inspire a change in policy that causes us to slow down.

But then in that world, it's a bad world even if we do slow down a lot because we're just on our own.

We have to do this stuff without the AI's help because we can't get them to help us.

But I'm actually reasonably bullish about control techniques getting...

early AIs that are not super galaxy brain super intelligences to be helpful for a range of stuff that they're good at.

Yeah, I do think anything that requires a large corporation to be super discontinuous in something it's doing is facing big headwinds as a plan.

So I would hope that they're sort of smoothly increasing the amount of internal inference compute that is going towards safety.

as the AIs get better and better so that the jump doesn't have to be huge at that final stage.

And that is something that if we could elicit honest reports without creating perverse incentives, that's something I'd want to know about.

How much human labor is going to safety versus capabilities?

And how much internal AI inference is going to safety versus capabilities?

How much fine tuning effort is going to safety versus capabilities?

And I think they have a much better shot if they're stepping it up over time on some kind of schedule.

Yeah, I think that for now, the biggest thing we need to do is very similar to the biggest thing I think society needs to do for preparing for the intelligence explosion, which is really trying to track where we're at right now in terms of how useful AIs are for the work that we do and the work our grantees do.

I think pushing ourselves to automate ourselves and pushing our grantees to automate themselves and tracking...

You know, how good is AI at the stuff Forethought does?

How good is AI at the stuff that Redwood Research or Apollo does?

How good is AI at the stuff that our policy grantees do?

And I think that that is just like one thing is just like just socializing within ourselves like that.