Scott Alexander

I'm not sure, but I think...

Like here's a simple model.

Let's say like nine people out of 10 just don't actually care and would be fine with the factory farm equivalent for the AIs going on into the future.

But maybe like one out of 10 do care and would like lobby hard for good like living conditions for the robots and stuff.

well, if you expand the circle of people who have power enough, then it's going to include a bunch of people in the second category, and then there'll be some big negotiation.

And those people will advocate for, like, you know... So, like, I do think that one simple intervention is just the same stuff we were talking about previously.

Like, expand the circle of power to...

larger groups, then it's more likely that people will care about this.

That's an argument for the Singleton stuff, by the way.

That's right.

That's right.

like not just a moral argument, but also just like an epistemic prediction.

Like if it's true that some of those super weapons are possible and some of these like private moral atrocities are possible, then even if you have like eight different power centers, it's going to be like in their collective interest to come to some sort of bargain with each other to like prevent more power centers from arising and doing crazy stuff.

Similar to how nuclear non-proliferation is sort of like whatever set of countries have nukes, it's like in their collective interest to like stop lots of other countries from getting nukes, you know?

And you can't tell anyone that you've agreed to this.

Great question.

Yeah, so I don't know.

Let me try to reason aloud here.

So my wife and I talked about it for a while, and we also talked with some friends and got some legal advice.

One of the filters that we had to pass through was even noticing this stuff in the first place.