Ryan Kidd

Your stock price is going to plummet.

What do you do?

Do you revert to an older system?

That's safer, probably.

So I think, yeah, we should definitely be tracking this stuff.

And I wouldn't say that we are in the clear by a long shot.

I would say that we are in a better world, by my estimation, than Bostrom and Meary predicted 10-something years ago.

But I don't know.

They would say I'm very wrong about that.

But I don't know.

I think that it's useful that we can get some work out of these things that looks like it is actually quite likely to accelerate AI safety work.

It's a very good question.

And I'll preface by saying that all safety work is capabilities work.

Fundamentally, people like to distinguish these things in terms of like, oh, capabilities work is about the engine.

It's about making the plane go faster.

And safety work is about the directionality.

But as you've pointed out, ROHF, which was intended as safety work to help the directionality steer it to where you want to go, also made people realize, oh, wait, this thing is useful.

I can actually hop in this plane now because it's going to land where I want.

which made them want to make the engine go faster so they could get there faster, right?

And that whole feedback loop started.