Rob, Luisa, and the 80000 Hours team

Let's talk, though, about this sort of paradoxical element to all of this, which is that we're imagining that the AIs that we're operating, that we're interacting with, they really do have our best interests at heart, for the sake of argument.

That's the scenario we're picturing.

How is it that that wouldn't kind of protect us to a great extent from going off in these negative directions, that they would help us see that things are going badly?

they would be trying to anticipate these negative outcomes and then warning us, you shouldn't be absorbing this kind of culture.

We should be trying to like, you should vote in this direction because that will help to protect your political rights in the long term.

I see.

So you can imagine a scenario where I'm nervous that gradually my rights or my influence over the government is going to be gradually eroded.

But I guess by that point, we were imagining that people, I guess like some actors within society or some people who have like greater influence over the government than me, they might well have made it such that all of the most powerful AIs that I have access to

are not really going to help me to unwind their power, to weaken them, to figure out how would we organize to do this, or give me the ideal advice that would help me to gain power at their expense.

And I guess this is kind of a dynamic that we see in general, that very often people or organizations or individuals who have a lot of power, they figure out ways of entrenching it by doing all kinds of things to interfere with the people who might try to take back power from them.

And this would just be another instance of that very common historical pattern.

I see.

I think people will be able to tell there's a lot of moving pieces in this picture.

There's a lot of different...

mechanisms by which potentially humans could be losing influence over the direction of civilization and intelligent life.

I think there's still a lot of work, I guess, to be done exploring them and figuring out which of them are most powerful and which of them potentially could have countervailing forces that control them.

What are some, I guess, specific narratives or scenarios, ways that this could potentially play out for people to picture in their head as they're trying to think about how gradual disempowerment would look?

We have an episode from earlier in the year with Tom Davidson about the possibility of a human-driven coup assisted by AI.

For people, I guess, who haven't listened to that and won't go back and listen to it, do you want to explain a little bit why it's more plausible in this post-AGI world for a group of human beings to seize power in a way that's very difficult to reverse?

Yeah.