David Duvenaud

And if there's like some unified government, then they might be able to say like, hey, our value system is such that we want to keep the monkeys alive.

And so we're going to like have a protected forest.

Um, if there's no such like unified polity or whatever, and it's just like a piecemeal growth thing, then you do expect that at some point there's somebody living right next to your land and then they have more kids and they want to expand onto your land.

And so that particular person actually does have a big incentive to, you know, take your last bit of land or something like that.

So in a sense, this makes me think that the only stable good outcomes involve some sort of like strong global coordination, which is also very scary because, you know, if you get global coordination wrong, then you end up locked into bad scenarios as well.

Yeah, yeah.

So I totally agree that I think...

It's easy to make the case that in the long run, some sort of global coordination is stable.

Once you have a lot of agency and coordination, you can use it to get more.

The question is, will our values survive the chaos in the generation of that global power or whatever it is?

Basically, there's this window where right now humans have a lot of influence, our institutions and cultures kind of serve us, but we don't have very good coordination ability.

then at some point we're going to have like diminishing influence and diminishing sort of like cultural cache and all those things at the same time as our ability to coordinate is going to go up.

So the question is sort of like, what's going to happen first?

Are we going to be all competed and then later the machines coordinate or do we get to be part of that like global coordination process?

Yeah, exactly.

That's exactly what I have in mind when I'm saying that our ability to coordinate will increase.

And then the question is sort of, will human influence survive long enough to strongly influence that process?

Yeah, so the sort of only thing we can do is to introduce a new fitness landscape where sort of who the government deserves worthy of UBI is the new like fitness function that is going to be just optimized by natural selection, right?

Yeah, absolutely.

So if we just allow unfettered reproduction like we do today, then it's unstable in the long run.