David Duvenaud

thinking about the world in a way that you think is valuable or something like that.

So that question of how sort of morally valuable this giant machine economy shoggoth is kind of dominates that question of what positive sub-dynamics still exist.

Totally.

And I think you're getting at a big open empirical question, which is sort of like, what is stable at the top?

Like, should we expect there to be some big like global government that forms and then consolidates power and like lasts forever?

Or is it going to be the case that just like the Roman Empire, like it became more and more powerful, but then there was like new religions that formed inside and there was sort of this like

cultural competitions, and all these reasons why every time an empire falls, it eventually ossifies and some sort of internal competition makes it fall.

Because if the case is that we're all heading towards some global government and it's going to be really hard to change, then we need to be investing in ways that we can, like,

steer this or slow it down or make room for pluralism or something like that.

But if we think that the default at the scales we care about is just that it's hard to coordinate, there's always going to be runaway competition eating any surplus, then we'll want to invest in like, no, no, how do we actually slow that process down and make sure that we can preserve these non-competitive values?

Sure, yeah.

And one thing I want to say is, I mentioned a fixed pie, but the pie itself might be growing.

And it might be growing exponentially as the solar system is colonized.

But the point is that it's zero-sum in the sense that humans don't contribute to the pie growing faster or smaller in the way that economic activity does make the pie grow today.

So it might not be a fixed pie, but it's still the case that it's zero-sum when you compete with the other people for your share.

Right, right.

When we actually start to colonize the solar system or galaxies or something like that, there is going to be still a selection pressure for the fastest possible growth.

But this happens to be wasteful in a way that, like, just because of the loss of thermodynamics, the slower you use the resources of, like, just, like, neg entropy and...

the sort of more compute you get, then the more control and presumably the more value you can get out of the universe.

So that's pretty far out there, but one concrete way in which we can expect unfettered competition to destroy value.