Rob, Luisa, and the 80000 Hours team

Which group do I want to side with?

Do I want to side with, you know, China, or do I want to side with the values that I expect the AIs to have?

I guess, depending on your, like...

guesses about what kind of moral attitudes or how much value the AIs will be able to produce, you might end up saying, well, I would rather side with at least some group of AIs over my fellow humans.

I guess that's one possibility or one way that things could go that actually makes it more likely that disempowerment could occur.

Yeah.

I think one reaction you've had to the paper is people saying, are you really saying that we want humans to remain, to have hard power, to be, like, the people on which the decisions bottom out forever?

The AIs are going to be getting more and more capable relative to us.

Even if we try to augment ourselves one way or the other, surely at some point we have to give it up and say, we've done what we can.

We think that we've aligned the AIs or we think we've given them good goals and it's time for us to take our hands off the reins.

Do you agree with that?

Yeah, I feel like it does complicate, because you say we want them to be aligned with human values.

And I think that's like fine as a shorthand and as a first past, but I think it like really does complicate things that people disagree as much as they do about like what the future should look like.

And also while we, I guess, different people with different moral philosophies or different like perspectives on the world, I guess, different religions, different spiritual values, they often kind of agree about like what they would like to be the next step, you

They'd like people to be more educated, healthier, more empowered.

But as there's more capability to optimize for exactly what Buddhists might think is the optimal thing, exactly what adherents of Islam or Christianity or different secular moral perspectives might want, then they might come radically apart such that you could just find yourself... There'll be some humans you're like, yes, I would love them to have more power.

But many of the other humans, I would like them not to have power because I think what they're going to do is terrible or useless.

And there'll be like some AIs that are aligned with you and some that are not.

It's going to be like quite a fierce potential, you know, fight.

I see.