Ajeya Cotra

So let's imagine that via whatever mechanism society does get a heads up that we are starting to see the early stages of an intelligence explosion.

What would we do with that heads up?

So the challenge we have is AI is becoming much smarter very quickly.

And we feel very nervous about that.

And the opportunity that's created is that, well, we have a lot more labor and we have much smarter potential researchers than we did before.

So why don't we turn that new resource towards solving this problem that I guess at the moment we don't really know how to fix.

It's a little bit, I guess,

I think some people who are not too worried about AI, they look at society as a whole or they look at history and they say, well, technology has enabled us to do all kinds of more destructive things, but we don't particularly feel like we're in a more precarious situation now or at much greater personal risk now than in 1900 or in 1800 because advances in destructive technology have been offset by advances in safety increasing technology.

And on balance, probably things have gotten safer.

And so the idea is, well, can we potentially, it's like going to be a vertiginous time, but perhaps we could pull off the same trick in this crunch time period.

Yeah, it's interesting that you say that this approach has often been dismissed.

Because I feel it's very in vogue now.

I hear about this proposal every couple of days.

Someone presents it or I read something about it in one guise or another.

I guess one reason why years in the past it might have felt unpopular is people were mostly focused on the issue of misaligned AI.

They were concerned about

an AI that has it in for you and would like to take over if it had the opportunity.

And that's maybe the worst application of this out of all of them, because you're asking the AI to align itself, but you don't know whether it's assisting you or trying to undermine you.

And so, I mean, you could try to make that work.

People have suggested proposals where you could try to get useful, honest work out of an AI that...